Theme
from Batman TV Show
Batman, Batman, Batman
Da da da da da da da da da da da da da da da da da
Batman!
No, Not Batman!
Flatman!
In 2006, my firm, Cambridge Systematics, was undertaking a
study for the Massachusetts Governor’s Highway Safety Board, GHSB. The GHSB was submitting that study to the National Transportation Safety Board. It required that the result be confirmed by a certified statistician. At the time, my former colleague, Kevin Tierney,
and I were “chosen” as the potential certified statisticians. Since it required attesting that the sampling
plan for the survey in the memo was correct and since Kevin Tierney was a national
expert ( although not certified) on survey design, the required attestation
fell to him. I joked with Kevin at the
time that it was a good thing that it did
not require my signature because I was only certifiable, while he could claim
to be certified. I am still certifiable (as insane) , but I am not certified
as a statistician, if there is such a thing. But I would like to think that I know
something about statistics.
In Flatland, which was written in 1884, an analogy was proposed
that if you limit your perception to a Flat, two-dimensional, plane then you
will have a tough time explaining things such as spheres, etc. In other words, perception matters. Einstein’s Theory of General Relativity, in
1916, explained the same concept in mathematical
terms. There can be an absolute, (i.e.
the speed of light) and our perception of such things as length, weight, time
etc., depend on the relationship to that absolute. Despite NOT being a statistician,
I would like to use statistics, in the same manner as Flatland did, to make an analogy
If a group thinks that there is an absolute, i.e. the
mean, and that group accepts no deviations, i.e. variance, from that absolute
truth, then the group has a mean that is greater
than 0, but variance of the group is zero. ( This probably also makes all
other statistical moments of the distribution, such as the skew or the kurtosis,
also zero). What this defines is a single
point. If there are any observations away from this point, then those
observations can NOT be members of the group
whose variance is zero. In addition to
many other things, distributions with
zero variance are inherently unstable, that is they are NOT resilient. If the truth, e.g. mean, is contradicted, for
example, if the group absolutely believes that the World would end on December
21, 2012 when the Mayan Calendar Cycle ended, then when the world did not
end, it does not require that the group be dissolved. In a normal distribution the mean, median, and
mode are equal. If the variance is 0,
then 100% of the group, every member, has the mean as their value. In a uniform normal distribution, the variance is 1.0 The mean, median and mode are still equal,
but only 40% of members of that group are
at that mean. It is thus possible to be
a member of the group and NOT have a value that is equal to the mean. If the mean changes, is found not to be TRUE,
then the group does not have to
dissolve.
A problem with a normal distribution is that it allows any
observation to be less than zero. In observations
of real data, for example income, what is often required is a distribution that
allows only non-zero values. The exponential
distribution is such a distribution, but it has the problem that the closer you
get to zero, the larger the number of observations are expected at zero. If the number of observations at zero should also
be zero, but the distribution is still expected to be normal, then an exponentially modified Gaussian ( normal) distribution is often
used. This distribution can be defined
by three attributes: the mean, µ; the variance, σ; and the exponential
parameter, λ. Unlike a pure normal distribution, this distribution can be skewed. The closer
the distribution is to normal, the closer the skew is to zero.
But if the distribution tends towards the exponential, the maximum value
of the skew is 0.31.
So by analogy, a group should not have a variance of zero,
belief in an absolute, and accept no deviation.
A normal group can be skewed, but that skew should not exceed 0.31. If the skew is greater than zero, then there
is no way that the variance can also be zero. If Flatland used mathematics to show
that not everything could be explained by a flat plane, then statistics says you can not accept a skew and expect a
variance of zero. Now you know, and knowing
is half the battle! Go Flatman!
