Misery

 

Outlaw Blues

Ain't it hard to stumble
And land in some funny lagoon
Ain't it hard to stumble
And land in some muddy lagoon
Especially when it's nine below zero
And three o'clock in the afternoon

That’s nine below a relative zero, regardless of the time!

Being below a relative zero is cold.  Being below an absolute zero is impossible.  So is annual income a relative amount? or an absolute amount?  Mr. Micawber in David Copperfield said “Annual income twenty pounds, annual expenditure nineteen, nineteen, and six, result happiness. Annual income twenty pounds, annual expenditure twenty pounds, ought, and six, result misery.”  If we want to include both misery and happiness, income must therefore be measured on a relative scale.

The US Census reports on annual income, that has been adjusted for inflation, expressing it in 2021 US Dollars from 1968 to 2022. (actually the years are 1967 to 2021).  It also reports the income limits for various percentiles.  This allows the median and mean income to be calculated for each year which is reported.

When those reported incomes are fit to a hyperbolic tangent function, it is almost a perfect match to the distribution of income in 1968 (adjusted for inflation to 2021 US Dollars).  However a hyperbolic tangent requires both negative incomes and negative percents.  If the income and percentages are viewed as absolutes, then the income should not fall below $0, the perctages should not be negatives,  and the fit should be to a random curve which is adjusted from an exponential distribution. The distribution of income in 1968 did not follow this random curve.  However the policies in the intervening years have resulted in a distribution of income that is a better fit to a random curve, but it is also is more skewed to upper incomes and has increased the misery of those who are nearer, or below, zero income.

Disaggregation II

 

Breaking Up Is Hard to Do

They say that breaking up is hard to do
Now I know
I know that it's true
Don't say that this is the end
Instead of breaking up I wish that we were making up again

Disaggregating is breaking up.

I am returning to the original purpose of this blog, which was to keep from repeating my self to colleagues, and instead referring them to a link to a blog post. So while the subject probably does not interest the vast majority of you, here it goes.

1.  Define disaggregation. The unique primary key fields in the Freight Analysis Framework, FAF, are Origin, Destination, Commodity, and Mode. Disaggregation is the opposite of aggregation. Each of these reported primary key fields are an aggregation.

• Origins and destinations are reported at level of FAF regions. Domestically these are states or portions of states, and they can be disaggregated to county, districts, firms, etc. However, I do not believe there is an interest in disaggregating International FAF regions into countries. (e. g. Europe into Germany, France, etc.).
• Commodities are in the Standard Classification of Transported Goods at the two-digit level, SCTG2 system and while these could be either disaggregated into another commodity classification system, e.g. Standard Transportation Commodity Code, STCC, or be disaggregated within the hierarchical SCTG system, e. g. SCTG5,  I do not believe that this is what is meant by disaggregation.
• Similarly, modes have been aggregated, and can be disaggregated ( e.g. Water into Great Lakers, Inland barges, Deep Sea ships), but I do not believe that this is the intent.

IMHO disaggregation is intended to mean only DOMESTIC origins and destinations into smaller zones. 

2. Why are you disaggregating. IMHO, disaggregation is done of the trip table for use in analysis. And disaggregation is also done of the trip table for use in traffic assignment. When it is done for the trip table to be used in analysis, such as the preparation of Freight Plans, assignment may, or may not, also be done. When it is done only for assignment, to prevent “lumpy loading,” i.e. over assigning links, and the link volumes, and not the trip table itself, may be analyzed. So disaggregation does NOT mean assignment. In fact, the assignment generally is done of vehicles that carry freight while the FAF is the contents of those vehicles. These are NOT the same. 

3. What does disaggregation include? To STEM-ers like myself, disaggregation is simply the opposite of aggregation, which is if you compress a matrix to aggregate it, then you expand a matrix to disaggregate it. This means that if the matrix has II/IE/EI/EE, and I is the internal geography to be disaggregated and E is the external geography not to be disaggregated, then EE is NOT disaggregated/expanded. However non-STEM-ers often include EE pass through as part of disaggregation. It is possible to identify EE pass through, it is simply subarea extraction, but it needs a trip table AND a network, unlike matrix expansion which only needs a trip table. So, does disaggregation include, or exclude, EE pass through? 

4.  Where do you get the disaggregation factors? If you have decided to disaggregate Domestic Origins and Destinations, it is important to understand the reported Domestic Origins and Destinations in the FAF are sometimes POEs, i.e. Ports of Entry for imports and Ports of Exit for exports, not origins and destinations.  In the FAF, actual origins and destinations are correlated with the economy of the shippers and receivers, but the POE is correlated with the carrier(s). There are different sources for economic disaggregation factors, and carrier disaggregation factors. You can infer what kind of a POE it is based on the reported foreign and domestic modes. 

5. How finely can you disaggregate? The FAF is an expansion of a sample, e.g. the Commodity Flow Survey, CFS, and additions that are Out Of Scope, OOS, with respect to the CFS. The expansion of a survey is for statistically significant zones. To protect privacy of the survey respondents, the original survey and the expansion factors are not reported. Beside even if the surveys were reported, they would NOT be statistically significant for smaller geographies. Additionally, for OOS there might not even be surveys and the whole concept of statistically significant does not apply. But you can come up with most probable disaggregation factors even if you can not come up with exact disaggregation factors. When disaggregating on the most probable you may be effectively betting on the favorite but that does not preclude the possibility that the factors associated with the long shot should have been used. Because these are most probable, not exact, Disaggregation Factors, I am uncomfortable disaggregating, inferring, too finely. 

6. How different should disaggregation factors be? The FAF is an asymmetrical table, e.g. the tons of grain from Iowa to New York City does NOT equal the tons of grain from New York City to Iowa. The FAF is asymmetrical across every primary key field, i.e. origin/destination/commodity/mode. Thus, ideally disaggregation factors should also be different by origin/destination/commodity/mode. But this is the ideal. The more dimensions that can be included the better, but some dimensions, and not others, is better than nothing. 

7. How should a successful disaggregation be judged? Is the goal to match the original table or to match some secondary source? What are those secondary sources and what is the error between them and the original FAF? The difference between the disaggregated FAF and the secondary source might in fact only be error between the original FAF and the secondary source. Matching the original if the disaggregation factors sum to 100% is easy, but this may not match the secondary source. 

In conclusion

·        What is meant by disaggregation? Does it include only II/IE/EI or does it also includes EE passthrough?

·        What is the deliverable? Is it the disaggregation process and a set of default disaggregation factors, or does it also include the process by which to create new disaggregation factors.

·        If the process of creating new disaggregation factors relies on public data sources, what is to be done if the format, contents, link location  or processing of the public sources which are not under the consultants control change  (e.g. US Army Corps of Engineers, USACE, Waterborne Commerce Statistics Center, WCSC, changed the format, contents, and location of the reporting that is usable with FAF3 (2007), FAF4 (2012) and FAF5 (2017).  The number of FAF regions is different in FAF3 from those in  FAF4 or FAF5. The number of regions stayed the same between FAF4 and FAF5, but three counties switched regions. Also, the numeric code associated with Rhode Island as a FAF region changed from 440 to 441. (not disputing either change in the FAF, only pointing out that it did change, and this has impacted the software). The counties not served by active rail tracks or navigable water appears to be no longer reported by Oak Ridge National Laboratory, ORNL, Center for Transportation Analysis, CTA.

Disaggregation

 

Pave the World

I swear I'm in fourth gear,
My rollers down, trail of smears.
Asphalt, aggregate,
No more trees, pavement.

If you don’t like aggregate, then how do you feel about disaggregation.

I am returning to the original purpose of this blog, which was to keep from repeating my self to colleagues, and instead referring them to a link to a blog post. So while the subject probably does not interest the vast majority of you, here it goes.

Disaggregation

Disaggregation is merely the opposite of aggregation. Matrices might be aggregated; in which case they can of course be disaggregated. There are also cells in the matrix which might be required to be aggregated, or otherwise identified, for example cells that pass through the area to be “disaggregated.” A knowledge of which cells pass through the area to be disaggregated, for example a network and/or its routing table which identifies which cells of the matrix use links of the network that are located in the area being disaggregated, then also needs to be provided. The application of disaggregation factors, without identifying which links in the disaggregated network should be used, is called matrix expansion. The filtering of cells in the matrix that use links in the disaggregation area is variously called subarea extraction, windowing, or select link analysis. For purposes of this discussion, disaggregation is taken to mean that only occurring without a network and includes only cells that travel to, from, or within the study area which is being disaggregated. This is also known as I-I, I-E and E-I trips, where I are the internal cells to be disaggregated and E are the external cells that are not to be disaggregated. Cells that are E-E, which begin outside of the area to be disaggregated, end outside of the area to be disaggregated, but also use links within the area to be disaggregated, i.e. E-E Pass-through, use a different method than matrix expansion and its disaggregation factors.

Disaggregation, i.e. matrix expansion, requires the identification of how the matrix was aggregated in the first place. This is not always known, or even knowable. The matrix might not have been aggregated. For example, the Federal Highway Administration’s, FHWA’s, Freight Analysis Framework, FAF, is developed from the Commodity Flow Survey, CFS and methods applied to shipments that are outside the scope of the Commodity Flow Survey. Those CFS Out Of Scope, OOS, flows, were not aggregated. The Commodity Flow Survey flows were aggregated by applying expansion factors to surveys. However the surveys themselves are not disclosed to protect the confidentiality of the survey respondents. Those surveys also do not represent 100% of the flows. Their expansion factors only apply to the groupings of the surveys which are statically significant. There are no expansion factors for grouping of cells which were not statistically significant. The absence of aggregation factors  means that it is not possible to develop the guaranteed disaggregation factors. However that does not mean that it is not possible to develop most probable disaggregation factors.

The concept of the most probable is common in transportation planning.

·        Frataring, often known as IPF, Iterative Proportional Fitting.

·        the Gravity Model.

·        the Logit Mode Split Model; and

·        Origin Destination Matrix Estimation, ODME, etc.

are all based on finding the most probable trip table. Disaggregation itself is an intermediate, meso, state between individual, micro, states and the aggregated, macro, state. The desired matrix is an intermediate, thus the mathematical concept of the most probable, the mesostate which contains the greatest number of microstates, can be used. However using the most probable disaggregation requires honoring the concept of the most probable disaggregation.

Dimensions Being Disaggregated.

Theoretically any of the indices, dimensions, primary key fields of the matrix, can be used in disaggregation. For example, the FAF has four primary key fields:  the origin, the destination, the commodity, and the mode. While it is possible to disaggregate commodities in the FAF, which are currently reported as Standard Classification of Transported Goods at the two-digit level, SCTG2, by a greater number of SCTG digits, this is not often done (using a different Commodity classification system would be CROSSWALKING, not DISAGGREGATING). While the modes reported in the FAF can be disaggregated (e.g. Water can be disaggregated into Inland Barges, Great Lakers, and Deep Sea vessels) the most common application of disaggregation is for the origins and destinations, geography, from the reported states and regions to smaller zones, for example to county-equivalents in those states and regions.

Correlation is not causation

Disaggregation can be on any characteristics of the dimensions of the matrix. For example, a region with two counties can be disaggregated 50% to each county for both origins and destinations. However it is not expected that this disaggregation factor will be correlated with the number of those smaller zones. For example, it might be more likely that origins will be correlated with employment and that destinations will be correlated with population. If the two counties had respectively 70% and 30% of the employment and 60% and 40% of the population, those characteristics might have been used as disaggregation factors instead. However those factors do not consider the commodity or mode. It may not be possible to develop or obtain factors that are CAUSING the flow. It is more likely that information showing CORRELATED factors. In this case, care should be taken that the correlation makes sense and is not a spurious correlation.

Perspective Matters

The nature of the matrix being disaggregated may have information that can be used in creating disaggregation factors. For example, in the transportation of freight, there are three functions: shipper, receiver, and carrier. It is possible for two or more of those functions to be combined: e.g. Dole may carry goods that it receives in trucks which it owns. Wal-Mart may carry goods in trucks which it owns, and additionally serve as both the shipper and the receiver, and that flow may not even be reported if it is an internal business operation. The source of the matrix may provide information on the dimensions and the disaggregation factors to be used.

As noted, the FAF is derived from the Commodity Flow Survey of shippers. The dimension is an origin, but the Domestic Origin is the origin of the shipper for domestic trade, while the Foreign Origin is the origin of the shipper for import trade. The Domestic Origin for imports, and the Domestic Destination for exports, is more properly a Port Of Entry/Exit, POE, which is a carrier transfer point at which no economic activity of the shipper or receiver is likely to occur.

Because the flows are as reported by the shipper, the mode used by the carrier may not be known, only the rate that was charged by the carrier. Thus a mode in the FAF is Air and Truck where the shipper pays a rate that is typical of air, but the carrier might choose operationally to use a truck. Because the flow is as reported by the shipper, the commodity is always known which is why the SCTG2 system, which does not include Freight All Kind, FAK, is used, while the carrier may not know the commodity and use the Standard Transportation Classification Code commodity system which does include FAK. Similarly the FAF does not include the packaging of the commodity in containers, while containers may be an important reporting by carriers.

In carrier focused matrices, the origins may include both carrier and shipper origins and the destinations may include both carrier and receiver destinations. A source of economic activity from an origin may only be pertinent for shipper-receiver matrices, such as the FAF, not carrier matrices such as Transearch. The first origin and the last destination may also reflect the economic activity of a customer, or a carrier operations while intermediate stops as reported in modal databases, are almost always carrier operations/activity.

Disaggregation Factors should be unique.

The matrix is probably not symmetric and has unique flows in each cell, and thus the disaggregation factors should also be expected to be unique. This is the ideal. It may be that disaggregation factors can be created for several dimensions, but not for all dimensions. In which case something is better than nothing. For example, import trucks and trains are reported at Border Crossing Port of Entries. The commodity contents of those trucks and trains are not reported and the export flows are not reported. It can be assumed that the relative share for vehicles is the same as the relative share for the commodities in those vehicles, and the relative share for imports is the same as the relative share for exports. While this is hardly unique, it is preferable to making no assumptions at all.

Disaggregation factors should sum to 100%.

Disaggregation factors must sum to 100% for each dimension for each cell that is being disaggregated. Thus if regions are being disaggregated to counties, the disaggregation factors for those counties should sum to 100% for each region. This means that the absolute number of shipments from each county is not needed to create disaggregation factors, only the relative number of shipments, activities, for each county in that region.

The number of disaggregations, mesostates, should be kept as small possible.

The most probable disaggregation factors depend on having as many microstates as possible. The number of microstates will not change. Increasing the number of disaggregation factors for each cell increases the number of mesostates but decreases the number of microstates in each mesostate.

The most probable is not a  guarantee. For example in the game of craps, the most probable roll over a very large number of rolls is a seven. On the next roll the total may be a two, but this does not change the fact that seven was still the next most probable roll. To ensure that the most probable event has meaning, the number of events, i.e. the number of microstates in a mesostate, should be kept as high as possible which means that the number of mesostates should be as low as possible.

Activities should correlate with a disaggregation factor.

Economic activity may apply to all goods transported from a shipper to a receiver. However only those economic activities that also use a mode should be related to any disaggregation of those modal flows. Information is available on which counties are not served by active rail tracks or navigable water. This does not mean that they are served by rail or water carriers, only that if the tracks or navigable water do not exist, then service by carriers could not be offered. Such information is not available for other modes and as a default all economic activity may be considered in creating disaggregation factors. While there is no information on trucks, it is assumed that trucks serve all economic activities.

Disaggregation factors in the tool that I developed

Domestic economic activity

For the domestic trade type, the origin is the domestic origin, and the destination is the domestic destination. The disaggregation factors should also be by commodity and by mode. As noted above, the Oak Ridge National Laboratory’s Center of Transportation Analysis has identified counties that are NOT servicable by rail or water. And this information can be used in developing relative disaggregation factors. The economic activity information is not generally available by commodity but is available by employment in an economic sector, which is generally the industry by the three-digit North American Industrial Classification System, NAICS3. Economic Input - Output models have Make and Use tables which relate economic sectors with commodities. The Bureau of Economic Analysis maintains Make and Use tables and crosswalks can be developed between its commodity and economic sectors,  and NAICS3 industries and SCTG2 commodities. The BEA Make and Use tables are for average percentages of value by economic sector. These averages are just that and a firm may be more or less productive than the average. Additionally the share of employment may not equate to the share of economic value. And the share of the economic value may not equate to the share of tonnages. However if these are assumed, it is possible to develop the economic activity by commodity for origin (shipper/Make) and destination ( receiver/Use.)

Import carrier activity

The economic activity of imports to the domestic destination is the same as the domestic economic activity above. The domestic origin for imports is a Port of Entry. For truck and rail, the Bureau of Transportation Statistics, BTS, Transborder flows, can identify the relative share of carrier activity at each truck and train border crossing in a FAF region. For water, the US Army Corps of Engineers Waterborne Commerce Statistics Center reports import tonnages by commodity. It uses the Public Group commodity classification codes, but those can be crosswalked to SCTG2 commodities. The relative share by water Port POE for each commodity for each FAF region can be calculated from this information. There is no information on the domestic mode which is used at these POEs. For all other foreign modes there is no information that can be used to develop disaggregation factors, so the POE is merely relabeled with its foreign mode and domestic region.

 Export carrier activity

The economic activity of exports from the domestic origin is the same as the domestic economic activity above. The domestic destination for exports however is a Port of Exit, POE. For truck and rail, the Bureau of Transportation Statistics, BTS, Transborder flows, can identify the relative share of carrier import activity at each truck and train border crossing and it is assumed that carrier export activity has the same share. For water, the US Army Corps of Engineers Waterborne Commerce Statistics Center, reports export tonnages by commodity by port waterway. It uses the Public Group commodity classification codes, but these can be crosswalked to SCTG2 commodities as used by the FAF. The relative share by water Port POE for each commodity for each region can be calculated from this information. There is no information on the domestic mode that is used at these POEs. For all other foreign modes there is no information that can be used to develop disaggregation factors, so the POE is merely relabeled with its foreign mode and domestic region.

Complexity

 

Just My Imagination (Running Away with Me)

But it was just my imagination
Runnin' away with me
It was just my imagination
Runnin' away with me

Reality is just one component of a complex number that requires an imagination.

A complex number, x, has a real component and an imaginary component, a+bi, where a is the coeffcint of the real component and b is the coeffcient of the imaginary component, i.  This can also be expressed as re^iθ, where r is the real component and θ is the angle to the imaginary axis.  It is thus proposed that x, the relationship to the absolute, is complex, and has both a real and an imaginary component.

For example, the Probability Density Function, PDF, of an exponential distribution, e.g. the formula for radioactive decay, is λe^-λx, where λ is the decay rate and x >0.  This has a Cumulative Distribution Function, CDF, which is the integral of the PDF, of 1-e^-λx.  This can be translated along the x-axis by an amount µ as PDF = λe^-λ(x-µ) which makes the CDF =1-e^-λ(x-µ).  The median is when the CDF=50%.  Thus 0.5=1-e^-λ(x-µ) requires that the median occurs when x-µ=-ln(0.5)/λ.  If µ=0, which means that x is a relationship to absolute zero, the median is -ln(0.5)/λ. But for an exponential distribution with a µ of zero, the median has previously been defined as 1/λ.  To resolve this apparent paradox, where the median must be both -ln(0.5)/λ and 1/λ, it is proposed that the median is a complex number and that 1/λ is only the real component of that complex number, -ln(0.5)/λ.  This means that -ln(0.5)/λ =(1/λ)e^iθ,  and that the angle θ, in radians, of the imaginary axis, is thus ln(-ln(0.5))=-0.36651i. Also, according to Euler’s Formula, iθ=ln(cos(θ)+isin(θ)), which means that (1/λ)e^-0.36651i= a*cos(θ)+i*b*sin(θ). This means that a decay rate, λ, defines the complex number, the angle of the imaginary axis, and the translation along the x-axis of the complex number x.

Coincidence

 

It’s All Over Now, Baby Blue

The highway is for gamblers, better use your sense
Take what you have gathered from coincidence
The empty-handed painter from your streets
Is drawing crazy patterns on your sheets
This sky, too, is folding under you
And it's all over now, baby blue

Coincidence is God’s sense of humor

I have been musing on Pythagoras’ Theorem.  The Transportation Modeling Improvement Program included a post from someone who wanted to know whether he should get a Professional Engineering, PE, license.  In responding I was going to quote from the movie the Wizard of Oz https://www.youtube.com/watch?v=uCOxU2rKLas. When given a diploma, e.g. a PE, the Scarecrow  quotes Pythagoras’ Theorem, and to me that is the coincidence.  

The Scarecrow had to already know that Theorem.  The Wizard's diploma only said that he could be trusted to know it.  Getting a PE does not add to your wisdom.  It does serve as a shorthand that others have tested your wisdom and it can be trusted.    The reality is that you had to already know Pythagoras’ Theorem BEFORE you get your “Doctor of Thinkology, Th.D.”,…. or a PE.  A PE says that your judgement has been judged by others.  It may be that you will someday be in a position where you are not known, and your opinion may have to be quickly judged.  In that case a PE may serve as that proof. Getting a PE to me is like giving Chicken Soup to a dead man.  Will it help?  It couldn’t hurt!

Climate Change III

 

Sinnerman

Well, I run to the river
It was boilin', I run to the sea
It was boilin', I run to the sea
It was boilin', all on that day
So I ran to the Lord
I said Lord, hide me
Please hide me
Please help me, all on that day
He said, hide?
Where were you?
When you oughta have been prayin'
I said Lord, Lord
Hear me prayin', Lord, Lord
Hear me prayin', Lord, Lord
Hear me prayin', all on that day
Sinnerman, you oughta be prayin'
Oughta be prayin', sinnerman
Oughta be prayin', all on that day

When the hurricane and earthquake hits SoCal
and the wildfire hits Maui and Canada, where you gonna run to?

Are you looking for someplace to hide from Climate Change?  Good luck with that!

Cordinate Transformation

 

Reflections

Through the mirror of my mind
Time after time
I see reflections of you and me
Reflections of
The way life used to be
Reflections of
The love you took from me

But can you reflect an absolute?

Mathematically you can translate a function with respect to an x-axis as g(x)=f(x-µ), where µ is the new location.  You can rotate a function with respect to an x-axis as g(x)=sin(θ)*f(x) where θ is the angle between the rotated and the original x-axis.  You can reflect a function with respect to an x-axis as g(x)=-f(x). But if x is the relationship to an absolute then that can restrict the ability to translate, rotate, or reflect a function of x, f(x). Translation, rotation, and reflection together can be called cordinate transformation

The simplest one to explain is the translation.  There is an absolute zero temperature.  It is measured in degrees Kelvin.  There is a 0° Kelvin but by definition you can not have a temperature below 0 degrees Kelvin.  It is more convenient for us to use a translated temperature scale where the origin, zero, has been translated by 273.15 degrees Kelvin, to the freezing point of water.  You can observe a temperature of -273° Celsius but you can’t observe a temperature of -274° Celsius because that would be below absolute zero.

Rotating a function in one direction is equivalent to rotating a function in the opposite direction by 2*π because the period of the sine function is 2π.  This has an impact on Euler’s Formula e^ix=cos(x)+isin(x) which is a rotation of the imaginary axis by x degrees.  If x is the relationship to the absolute, then x can not be less than zero because you can’t rotate the imaginary axis by something less than absolute zero.  That is why e^ix cannot be defined as a real number, or else it would create the paradox which would require both that cos(-x)= cos (x) and cos(-x) = -cos(x). 

This same restriction applies to the reflection.  If the origin of zero has been translated to µ, then you can reflect that function between µ and 0, but you can not reflect that function below 0 or else it would be a reflection below absolute zero.

Since a random number has the parameters µ and σ, and the variance, σ², defines the range of that random number, a random number must be µ>3σ or else the random number could be below absolute zero.  

Strategy v. Tactics

 

Climb Every Mountain

Climb every mountain
Ford every stream
Follow every mountain (every mountain)
Don't you ever give up, no ohh
Climb every mountain (every mountain)
There's a brighter day on the other side
Follow every rainbow
'Till you find your dream

It is NOT winning every battle!

Winning every battle might seem like a winning strategy, but it is not.  Bad strategies include Pyrrhic victories and "Winning the battle but losing the war". Winning a battle, being a good field general, a great tactician, does not mean that you would be a good strategist.  Frank Robinson might have been a great player and great manager, but the more likely examples are Ted Williams who was a great player/tactician, but a lousy manager/strategist and Joe Torre who was a lousy player/tactician but a great coach/strategist.  Dynasties are more likely when a great player is teamed with a great coach,  e.g. Tom Brady and Bill Belichick, than one without the other.

Winning every battle is a good strategy only if you prevent the other side in the battle from ever contending again.  But then you have to go all scorched earth on this, a la Rome v. Carthage. For example, while Dunkirk may have been a tactical loss for the British, destroying the British Army might have only emboldened the small ship captains who particpated in the miracle at Dunkirk and other civilians who would have ultimately defeated Hitler.  I think that this is a lesson that Vladmir Putin is learning in his ill-fated invasion of Ukraine.  The fact that the leader of Ukraine is a ordinary former TV comedian, and not a bully, only reinforces the fact that bullying/dominating alone does not always work.

Sometimes sacrificing a tactical victory is the better strategy.  In the 2004 American League Baseball Championship Series, Tim Wakefield was scheduled to be the Game 4 starting pitcher. But when Bronson Arroyo was knocked out of Game 3 after only two innings and the relievers were taking a beating in the game that the Sox would go on to lose 19-8, Wakefield sacrificed his chance to start Game 4 and volunteered to "take one for the team." He ended up pitching 3 1/3 innings in Game 3, allowing manager Terry Francona to keep the rest of the Red Sox bullpen rested. Relief pictchers Keith Foulke and Mike Timlin didn't have to pitch at all that day, and Alan Embree faced only four batters. When the next two games of the series went extra innings, it was crucial to have rested relievers, and Francona and his Red Sox teammates were quick to point out that without Wake's selfless act, the comeback would not have been possible.  In other words, Wakefield knew that a strategic series victory for the team was more important than his own tactical win in Game 4. And in doing so, he climbed the mountain and found every Red Sox fan’s dream.

Absolutely Nothing

 

I’ve Got Plenty of Nuttin’

Oh, I got plenty o' nuttin'
An' nuttin's plenty fo' me
I got my gal, I got my song
Got Hebben de whole day long
No use complainin'
Got my gal,
Got my Lawd,
Got my song!

In Other Words, NOT absolutely nuttin’

Porgy in this song from Porgy and Bess, states that he has nothing on the scale of plenty.  But plenty excludes Heaven, his Lord, his gal, his song, and many other things. Plenty thus must be measured on a relative scale, not an absolute scale.  It is like saying that the temperature is below zero outside. On the absolute Kelvin scale it can by definition never be below zero.  Saying that a temperature is below zero on the Centigrade scale only means that zero on that scale has been relatively set as the freezing point of water. 

Saying that there is a temperature of absolute zero must therefore mean that at the opposite end there is an absolute temperature.  The fact that we have never observed an absolute temperature does not mean that there isn’t one.  There are absolutes that we can observe such as light.  Einstein’s Theory of Relativity says that the speed of light is an absolute that can be approached, but not exceeded.  An exponential equation, including radioactive decay, measures the relationship to an absolute that can be approached, but not exceeded.  The question becomes how many absolutes are there?  An answer is that there is only one absolute, which can be perceived differently, but each perception is only one aspect of a single absolute. If there is only one absolute then if x is measured on an absolute scale then the integral of any function of x, f(x), should not be which ∫_-∞^∞ f(x) dx which implies the existent of two absolutes, infinity and its negative, but instead should be ∫₀^∞ f(x)dx, which implies the existence of only one absolute, infinity and its absence, zero.

This has an implication on the computation of the Cumulative Distribution Function, CDF, for all distributions.  The Probability Density Function, PDF, is defined without consideration of positive or negative x, but the CDF, the integral of that PDF, is then also only defined for x>0, i.e. x less than zero is undefined.  If the PDF is a normal random logistic function, its PDF is 1/(4s)*sech²(0/2s), and its CDF is ½* tanh(0/2s)+1/2.  A physicist might say that if s=0 and 0/0 is defined by its limit as 1, then the PDF is 0 but the CDF is ½, but a mathematician might say that if s=0 and 0/0 is undefined, then both the PDF and CDF are undefined.

Free Speech?

 

I Swear

And I swear by the moon and the stars in the sky
I'll be there
I swear like the shadow that's by your side
I'll be there

I swear to tell the truth, the whole truth, and nothing but the truth.

Not A truth. Not what you BELIEVE to be the truth. Not YOUR truth, but THE truth. And that is because a trial is about establishing CERTAINTY. You are not asked for your opinion on the evidence. You are not asked for your opinion of the witnesses. You are not asked for your opinion on the officers of the court who are charged with protecting the witness and seeing that the evidence is being fairly presented. You are not asked for your opinion on the jury who will be ruling on that evidence. Your opinions may be free speech, but they could also be viewed as witness tampering, interfering with a fair trial, or jury intimidation. If you wish, then you are free to express your opinions under oath, on the stand, in front of that jury.

Shouting “Fire” in a crowded theater when there is no fire, using the phase “Your money or your life” during a robbery, or as King Henry II famously ranted leading to the murder of Thomas Becket, “Who will rid me of this troublesome priest” are all speech. But they led to actions that are crimes.

Free speech is protected because the Constitution has rightfully judged that the state can not determine what is the truth. However whether it is true or not, …and there is cetainly the possiblilty that that the speech was a lie, that does not absolve you of the consequences of that speech.

King Henry II might have only uttered words, not committed the murder, and might not even intended the murder, but his contemporaries and history have judged him to have been wrong. King Henry publicly did penance for his words. Just as the state can not prevent your speech because you might be telling the truth, you also can not state with certainty that what you said was THE truth. That certainty is why there is a trial.

Don't Dwell in the Past

 

Dog Days Are Over

Run fast for your mother, run fast for your father
Run for your children, for your sisters and brothers
Leave all your love and your longing behind
You can't carry it with you if you want to survive
The dog days are over
The dog days are done
Can you hear the horses?
'Cause here they come

The worst is behind you.

The dog days or dog days of summer are the hot, sultry days of summer. They were historically the period following the rising of the star system Sirius (known as the "Dog Star"), which was connected with heat, drought, sudden thunderstorms, lethargy, fever, mad dogs, and bad luck. They are now taken to be the hottest, most uncomfortable part of summer. If you have survived those days, then cooler, better days are ahead. The dog days will come again next year, but the important thing to remember is that you have survived. Do things in the days to come that ensure that you will survive the dog days next year. You have a whole year to get ready.

Competition

 

All I Really Want To Do

Now I ain't lookin' to compete with you
Beat on, cheat on, mistreat you
Simplify you, classify you
Deny, defy, mystify you
All I really want to do
Is baby, be friends with you
Baby, be friends with you

Reality is NOT a competition.  No wagering please!

Reality is random.  Fair betting requires random events.  Therefore there should be betting on reality, correct?  Not exactly.  There is betting by individuals, but collectively there should be no betting.  Betting requires a zero sum game: i.e. winners AND losers.  Reality is NOT a zero sum game, there is expected to be at least a change, and hopefully growth, from the current reality. So betting individually may make sense, but betting collectively does not make sense.  If there is change, and again hopefully growth, it should be shared by both winners AND losers, because in a fair game, the winners in the past could be losers in the current contest.

If the total value of a system increases, then the median should increase.  However it is possible that the increase could go only to the current winners, or a subset of the current winners. In this case, the mean might increase while the median decreases, or stays the same.  The problem is that popular wisdom is that the average is the mean, because that is easiest to compute, i.e. system total divided by individuals in the system. Actually, average is the centrality of the normal, which is by definition the median, where half of the individuals have more than the median and half have less than the median. If the mean is the median, as it is in normal distributions, that popular wisdom isn’t a problem.  But if there is a gap between the mean and the median, it indicates that the situation is NOT normal.  If the gap between the median and the mean is increasing, this also indicates that the situation is becoming more ABnormal.

I wish I could say that the gap has been decreasing, but based on US Census reports, the gap has been increasing. 

This is not only because people have confused the mean and the median  https://dbeagan.blogspot.com/2018/08/wonderful-world-dont-know-much-about.html , inflation has masked the impact, and the tax code has exacerbated the effect, https://dbeagan.blogspot.com/2020/06/taxman.html.

Competition should be between groups, not between individuals within groups. Let’s not compete. Let’s be friends.

 

Art v. Science

 

It’s So Easy

People tell me love's for fools.
So, here I go, breaking all the rules.
It seems so easy (Seems so easy, seems so easy)
Oh so doggone easy (Doggone easy, doggone easy)

Breaking rules isn’t easy, but it is necessary.

“Art is in the eye of the beholder”.  “There are rules”.  And this describes the apparent conflict between the arts and the sciences.  Artists are individuals, breakers of rules.  Scientists are also individuals, discoverers of rules.  Those of you who are old enough to remember the classic TV show The Prisoner with Patrick McGoohan, playing Number 6, might remember the phrase.  “I am not a number.  I am a free man.”  Those of you who also remember the finale will remember that the elusive Number One was shown also to be Patrick McGoohan.  The finale seemed to imply that I am a free man because I am a number.  The problem is that man is an individual animal, but man is also a group animal, a member of a group.

The phrase is “All for One, and One for All”.  In an ideal world, All, the group, supports the individual, One and the individual, One, supports the group, All.  Or as I learned in Cub Scouts, “The Cub Scout helps the Pack go. The Pack helps the Cub Scout grow”.  A group, such as the United States in its Constitution, “Promote(s) the Progress of Science and useful Arts.”  It does so by promoting those who discover new rules and those who break old rules. To do so you don’t force old rules on individuals because those rules might be wrong or incomplete.  You also don’t prevent individuals from breaking old rules, again because those rules might be wrong or incomplete.  There should be no conflict between arts and sciences. They both have the same end goal, the making of new rules, that improve on, and may replace, old rules.

Normal III

 

A Wonderful Guy

I'm as corny as Kansas in August,
I'm as normal as blueberry pie.
No more a smart little girl with no heart,
I have found me a wonderful guy!

One Blueberry Pie distribution coming right up!

In a normal distribution, the mean is equal to the median. (Or as Donald Trump Junior so famously put it “That’s how math works”).  However the mean is not always equal to the median in many distributions.  This is often only because the negative numbers are not reported because of where the y-axis is being set.  For example saying that temperatures are 40 below zero is only because the temperature is being measured on a relative scale, not on an absolute scale.  If an absolute scale had been used, then the median should be equal to the mean for a normal distribution.  Stating the median and the mean with a with a relative rather than an absolute zero, allows the calculation of the variance, if negative values had been allowed

NAR, the National Association of Realtors, has reported the mean ($639,000) and median ($396,500) prices for existing home sales.  Since the mean is not equal to the median this is hardly normal. It is suggested that this might be because negative sales prices are obviously not reported.  But if they could be reported, then if this was a normal distribution then no values would be observed for the mean minus 3 times the square root of the variance.  This requires that the variance must be the mean²/9=45.37 million.  It has already been reported that 50% of the homes sell for less than $396,500. If this were also a normal distribution, then 68% of the existing homes should sell for less than $852,000;  95% of the existing homes should sell for less than $1,065,000 and 100% of the existing homes should sell for less than  $1,278,000.  If there are existing homes selling for more than $1.3 million, then that also indicates that the existing home sales prices are not normal.

Pro-Choice II

 

My Way

And now the end is here
And so I face that final curtain
My friend I'll make it clear
I'll state my case, of which I'm certain
I've lived a life that's full
I traveled each and every highway
And more, much more
I did it, I did it my way

Yes, but did you also let me do it my way, or did you insist that I do it your way?

Those who are Pro-Life think they are going to heaven.  If they are Pro-Life but also Anti-Choice, then I am afraid that they have violated the First of the Ten Judeo-Christian Commandments, “I am the Lord thy God . Thou shalt have no other gods before me.”  By being Anti-Choice, they have confused a discontinuity with the absolute.  They are saying that Good is equal to Evil, that they have chosen Good, and everyone else should have no choice/should also choose Good.  Unfortunately the math does not work that way.

An exponential distribution has a discontinuity.  Traditionally this is zero, but it can be shifted to any discontinuity, µ.  While the exponential distribution is not defined before that discontinuity, it can be reflected at that discontinuity. This is implicitly saying that if the distribution at the discountinuity has a value of 144%, ln(2)%, certainty, then there is a pseudo mean, because of the discontinuity, at µ+1/λ.  That is interesting but if you smooth out the discontinuity to a normal distribution, and say that at most you can be 100% certain, then a factoring of a normal distribution is appropriate. But by saying it must be 100% certain, you are also saying that there is a Cumulative Distribution Function, CDF, of that factoring, which has almost the shape as the CDF of the exponential distribution with a discontinuity.  But the problem is that before the discontinuity, the exponential distribution implies a negative absolute while the factored continuous normal distribution has only one absolute. 

If you say that there is a choice, then you should only have a 50% chance of making that choice at the discontinuity. However by also saying that choices made before the discontinuity should not be considered, are treated as if they are zero, the Cumulative Distribution Function is of course substantially lower, and it takes until well after the discontinuity to reach 100%.  If you are Pro-Life AND Pro-Choice, then your CDF will reach 100% much sooner. If you are Anti-Choice, then you are also saying that there are two absolutes, not one.

So if someone is Pro-Life AND Pro-Choice, then they are keeping the Commandments, not breaking them.