Variance

Theme from  Batman TV Show

Batman, Batman, Batman
Da da da da da da da da da da da da da da da da da
Batman!

No, Not Batman! Flatman!

In 2006, my firm, Cambridge Systematics, was undertaking a study for the Massachusetts Governor’s Highway Safety Board, GHSB.  The GHSB was submitting  that study to the National Transportation Safety Board.  It required that the result be confirmed  by a certified statistician.  At the time, my former colleague, Kevin Tierney, and I were “chosen” as the potential certified statisticians.  Since it required attesting that the sampling plan for the survey in the memo was correct and since Kevin Tierney was a national expert ( although not certified) on survey design, the required attestation fell to him.  I joked with Kevin at the time that it was a good thing that  it did not require my signature because I was only certifiable, while he could claim to be certified. I am still certifiable (as insane) , but  I am not certified as a statistician, if there is such a thing. But I would like to think that I know something about statistics.

In Flatland, which was written in 1884, an analogy was proposed that if you limit your perception to a Flat, two-dimensional, plane then you will have a tough time explaining things such as spheres, etc.  In other words, perception matters.  Einstein’s Theory of General Relativity, in 1916,  explained the same concept in mathematical terms.  There can be an absolute, (i.e. the speed of light) and our perception of such things as length, weight, time etc., depend on the relationship to that absolute. Despite NOT being a statistician, I would like to use statistics, in the same manner as Flatland did, to make an analogy

If a group thinks that there is an absolute, i.e. the mean, and that group accepts no deviations, i.e. variance, from that absolute truth, then the group has a mean that is greater than 0, but variance of the group is zero. ( This probably also makes all other statistical moments of the distribution, such as the skew or the kurtosis, also zero).  What this defines is a single point. If there are any observations away from this point, then those observations can NOT be members of the group whose variance is zero.  In addition to many other things, distributions with zero variance are inherently unstable, that is they are NOT resilient.  If the truth, e.g. mean, is contradicted, for example, if the group absolutely believes that the World would end on December 21, 2012 when the Mayan Calendar Cycle ended, then when the world did not end, it does not require that the group be dissolved.  In a normal distribution the mean, median, and mode are equal.  If the variance is 0, then 100% of the group, every member, has the mean as their value.  In a uniform normal distribution, the variance is 1.0  The mean, median and mode are still equal, but only 40% of  members of that group are at that mean.  It is thus possible to be a member of the group and NOT have a value that is equal to the mean.  If the mean changes, is found not to be TRUE,  then the group does not have to dissolve.

A problem with a normal distribution is that it allows any observation to be less than zero.  In observations of real data, for example income, what is often required is a distribution that allows only non-zero values.  The exponential distribution is such a distribution, but it has the problem that the closer you get to zero, the larger the number of observations are expected at zero.  If the number of observations at zero should also be zero, but the distribution is still expected to be normal,  then an exponentially modified Gaussian ( normal) distribution is often used.  This distribution can be defined by three attributes: the mean, µ; the variance, σ; and the exponential parameter, λ.  Unlike a pure normal distribution, this distribution can be skewed.  The closer the distribution is to normal, the closer the skew is to zero.  But if the distribution tends towards the exponential, the maximum value of the skew is 0.31. 

So by analogy, a group should not have a variance of zero, belief in an absolute, and accept no deviation.  A normal group can be skewed, but that skew should not exceed 0.31.  If the skew is greater than zero, then there is no way that the variance can also be zero. If Flatland used mathematics to show that not everything could be explained by a flat plane, then statistics  says you can not accept a skew and expect a variance of zero.  Now you know, and knowing is half the battle! Go Flatman!

Winning III

 

Tubthumping

I get knocked down (we'll be singing)
But I get up again (pissing the night away)
You are never gonna keep me down (when we're winning)

Getting up again IS winning.

To err is human, to forgive is divine.  Humans can aspire to the divine, but we probably will not achieve it.  But we are human and we can, and will, err, get knocked down.  What is improbable is that humans will never err, get knocked down.  That is NOT winning.  Getting up again is winning.

Error

 

Oops I Did It Again.

Oops, I did it again to your heart
Got lost in this game, oh baby
Oops, you think that I'm sent from above
I'm not that innocent

Being human is not being innocent.

To err is human, To forgive is divine. Being divine is being innocent. Since I am human, as are you, I expect there that will be errors. The Standard Error is expressed mathematically as Standard Error= σ /√n, where σ is a statistical measure of the distribution called the variance, and √n is the square root of the number of observations, n. In the real-world n has to be less than infinity so √n also has to be less than infinity. If the variance is real and not zero, then the standard error can NOT be zero. So when we say there is an error, we are saying that the variance has to be greater than 0.

Absolute truth means that there is NO error. If there is no error then the variance must be zero. If there is error then the variance has to be greater than zero. For a given variance, one way to reduce the error is to increase the number of observations. That is why when pollsters want to decrease their error, they increase the number of observations.

One would think that if there is an absolute there can be no error. Albert Einstein’s Theory of General Relativity answered this apparent contradiction. (Actually Jesus addressed the same thing in that whole “Render unto Caesar” gig.). There can be an absolute truth, e.g. the speed of light, and yet depending on your frame of reference, things like length, weight, time, all depend on your relationship to that absolute. The variance is greater than zero. No matter how many observations you make, if there is an absolute truth, only one observation should be enough and  the variance is zero. In a group, if there is no variance, then the mean of the group is equal to the median. However the mean can be equal to the median any time the variance is greater than than zero if the skew is also zero. In fact a normal distribution is any distribution in which the mean and the median are equal. A uniform normal distribution is one in which the variance is 1. If the variance is 1, then the Standard Error has to be greater than zero, even if there is only one observation:  Standard Error = 1/ √1. You can make the error smaller if you increase the sample size, increase n, but mathematics says that you can never can  eliminate the error. You’re not that innocent.

Ideas

 

You May Be Right

You may be right
I may be crazy
But it just may be a lunatic you’re looking for.

Can’t someone be both right and crazy?

I have proposed a number of things. They might all be right. But apparently not everyone is looking for a lunatic.

My career has been in traffic engineering, particularly travel demand forecasting. One suggested finding is how impedance  on a traffic link ( e.g. a road) increases as traffic volume  increases. In trying to address this issue, since impedance appears to be a function of  and travel time,  I proposed equations for reliability and for  travel time, only as a mean of improving the equation for  impedance. However I have been reluctant to network in my career and my reputation,  except with those whom I have worked, is admittedly non-existent. The blog posts for reliability https://dbeagan.blogspot.com/2022/02/reliability.html, travel time https://dbeagan.blogspot.com/2022/02/traffic-flow.html and impedance https://dbeagan.blogspot.com/2022/02/time-and-reliabilty.html were taken from  papers submitted to a peer reviewed conference but rejected.

Since I have been semi-retired, locked down, and bored, and I have spent most of my career analyzing data, I looked at income and wealth data. It suggested that the distribution of wealth and income in the United States is not only anecdotally skewed, it is statistically skewed, not only over time, but compared to other nations, https://dbeagan.blogspot.com/2021/07/inequality.html, https://dbeagan.blogspot.com/2022/04/distribution-of-wealth.html This information has been the subject of my blog posts and has been shared with other, but I have no standing with those others, so I can understand why it has been apparently been ignored.

I have looked the income data and suggested that the tax code is responsible for a decline in growth and an increase in inequity https://dbeagan.blogspot.com/2020/06/taxman.html and suggested a revision to the tax code, https://dbeagan.blogspot.com/2021/12/georgia-techfight-song-im-ramblinwreck.html. I have looked at the Consumer Price Index and suggested that international trading and the use of the US dollar in international trading is responsible for much of the systemic long-term inflation https://dbeagan.blogspot.com/2018/08/the-happening-riding-high-on-top-of.html. I also suggested that inflation should be divided into  currency inflation and supply-demand inflation and treated separately https://dbeagan.blogspot.com/2022/04/inflation-v.html. And while I have suggested this to others in these fields, I am not an economist, and since my traffic engineering findings have not been accepted, I am hardly surprised that my economic suggestions have not been pursued.

I have looked at human behavior and suggested that it can be example by three characteristics. https://dbeagan.blogspot.com/2021/05/framework-for-human-behavior-ii.html But again since my background is NOT in social sciences, I have no standing.

Supreme Court II

 

Session One

As soon as I draw, get sent to Allah
Bilinguist don, I kill with the tongue, I'm Atilla the Hun
I'm Genghis Khan, I'm a genius spawn

I pillage your village for fun, an egregious con
A syllable gun, real as they come, Long Beach Saddam!
Slaughterhouse equals swine flu, are South flyin' 

Egregious, “Strip-Search Sammy”?  Really? Really? Really? 

I have suggested to the Presidential Commission on the Supreme Court that Justices should serve staggered terms of 16 years. Then except for a death in office, each President can expect to make only two nominations during his term. That means that during the one term of the Orange Menace, it should be expected that only two Justices  should have been nominated to the Supreme Court. We might still have gotten Justices Neil “NOT Merrick Garland” Gorsuch and Brett “Frat Boy” Kavanaugh, but at least we might have been spared Justice Amy “Handmaiden” Coney Barrett.

Also Supreme Court decisions should not reflect just a simple majority. Up until the 1940s, opinions of the Supreme Court were by consensus. The recent majority opinions are a historical aberration. If the opinions are by consensus, then they are by definition supermajority decision. But a consensus as a supermajority can be blocked by a single individual. The most important decisions for the United States, wars, treaties, amendments, etc, are made by supermajorities that are NOT 100%. Shouldn’t the opinions of the Supreme Court also be by a supermajority. Otherwise they are opinions of a simple majority of the SCOTUS,  but might reflect a minority of the People they have sworn to represent. This appears to be the case in the draft opinion on abortions. Despite the fact that the majority of the county appears to favor abortions in some cases, the majority of the SCOTUS appears poised to opine that the heck with that majority, their opinion is the Law. It is the Law. It is not Justice.

“Egregious”, Justice “Strip-Search Sammy” Alito? It would nice if you knew what that word means. It literally means illustrious, standing out from the flock. Your usage is its more common ironic use. Even if you intended the ironic meaning, even if you share that opinion with four other justices, it is not a supermajority of the Supreme Court. Only supermajority opinions are considered to be those of the People.

I am old enough to remember that when John F. Kennedy ran for President,  he had to convince Southern Baptists that his Catholic beliefs would not take precedence over his responsibility as President. Justice Alito can have his beliefs and the “conservative” Justices endorsed by the Federalist Society can also share those beliefs. However their beliefs are not those of the People that they have sworn to represent.

 

 

 

 

 

Hope

 

Go The Distance

And I won't look back, I can go the distance
And I'll stay on track, no I won't accept defeat
It's an uphill slope
But I won't lose hope, 'till I go the distance
And my journey is complete

If there is any chance of victory, then go the distance.

The Washington Generals, also known as the Boston Shamrocks, the New Jersey Reds, the Baltimore Rockets, the Atlantic City Seagulls, the New York Nationals, the International Elite, the Global Select and the World All-Stars were the barnstorming opponents in each Harlem Globetrotters exhibition “game”.  They are supposed to lose.  Winning against the Globetrotters is like killing Santa Claus. 

The Generals have “lost” more than 15,000 games against the Globetrotters but they have won somewhere between 3 and 6 times. The records are spotty, but they did record wins in 1954, 1958 and 1971.  Therefore the “odds” are more than 3 in 15,000, but they are not zero.  A Tortoise can seldom beat a Hare, but as in Aesop’s Fable, sometimes it happens.  The Generals play on even though the odds are small, they are not zero. Even when the game is rigged, they won’t lose hope.  They have gone the distance.

 

Elections Matter

 

Trust Me 

Trust in me, baby, give me time, gimme time, um gimme time.
I heard somebody say, oh, "The older the grape,
Sweeter the wine, sweeter the wine." 

When you elect someone to represent you, you are placing your trust in them. 

The United States is a republic (and tell Ben Franklin, we are planning to keep it).  Elections are to select those individuals who will represent you in the actions required by the People of that republic.  I will not try to ague what your position should be when electing those individuals,  but I will remind you that those who seeking election may be lying to you when they are stating their own positions. I have suggested that individuals can be characterized by three attributes: Individual vs. Team; Nature vs. Nurture, and Fact vs. Fiction.  

The first attribute is whether User Optimal or System (i.e. Group) Optimal decisions will be chosen.  If a person prefers User Optimization, you should elected someone whose System Optimal preferences are close to your User Optimal preferences.  Selecting someone who has User Optimal preferences means that your representative might seek to advance his own interest rather than your interest.  They may lie that this is their System Optimal position in order for you to elect them. 

The Nature vs Nurture attribute has to do with who you consider to be in the group, excluded because of their nature, or included if they are nurtured to change their nature.  A person may state that they are excluding the same individuals from the group that you are excluding, but they may be lying about who they are really excluding ( for example, they may be excluding you from their group). 

The Fact vs. Fiction attribute has to do with your preference for facts.  Some people may believe in astrology, the earth is flat, WWE wrestling is real, etc.  A person seeking your vote may pretend to accept your beliefs when they really believe those beliefs are folly. 

Political parties used to vet candidates for you.  Primaries mean that you are vetting candidates for yourself.  A political party may be lying in their vetting.  My rule of thumb is that if someone had distinguished military service, then they have demonstrated their Team (Group) position.  For example Dwight Eisenhower, John McCain.  The heir of a wealthy family is less likely to seek to advance his own interests ( in the belief that he already has it all, he probably does not need more). For example,  the Roosevelts,  the Kennedys, etc. Storytellers are proven to, and trained in, tricking people.  My rule of thumb is to assume that actors, authors, TV personalities, etc. are lying until proven otherwise. My eldest son is a visual effects artist at Warner Brothers Media, so sorry Ryan, I guess this means I don't trust you.

If you are lied to then, regardless of your position, your representative may not be representing your position.  That hurts not only you, but those with other positions as well as you.  I need you to trust those who you elect.