Case Study 1: Abraham Wald and the Art of Seeing What Isn't There

The Man

Abraham Wald was born in 1902 in what is now Cluj-Napoca, Romania. He was a mathematician of extraordinary ability — his doctoral work in geometry was recognized as exceptional — but as a Jew in interwar Europe, he faced severe barriers to academic employment. He emigrated to the United States in 1938, just before the annexation of Austria closed the last escape routes.

At Columbia University, Wald joined the Statistical Research Group (SRG), a classified think tank applying mathematical methods to military problems during World War II. The SRG included several future Nobel laureates and was one of the most concentrated pools of mathematical talent ever assembled for a single purpose.

The Problem

The bomber survivability problem was among the most consequential the SRG tackled. American bombers flying over Europe were being shot down at rates that threatened the entire strategic bombing campaign. Adding armor could improve survival rates, but every pound of armor reduced payload capacity, fuel efficiency, and maneuverability. The armor had to go where it would do the most good.

The Data

The military collected data on every returning bomber's damage — a meticulous catalog of bullet holes, shrapnel damage, and structural failures. The data showed clear patterns:

The military's interpretation: the fuselage and the "rest of plane" sections took the heaviest fire and needed the most armor.

The Insight

Wald's analysis inverted the conclusion. He recognized that the data came from a non-random sample: planes that survived their missions. The planes that were hit in the engines didn't return — they were shot down over enemy territory. The low engine hit rate among survivors wasn't evidence that engines were less targeted; it was evidence that engine hits were more fatal.

Formally, Wald modeled the problem as a missing-data problem. If a plane can be hit anywhere with roughly equal probability, then the differences in hit rates on returning planes must reflect differences in survivability, not in targeting. The areas with fewer hits on returning planes are the areas where hits are fatal — because planes hit in those areas didn't return to be counted.

The Broader Mathematical Contribution

Wald didn't just solve the bomber problem. He developed a general mathematical framework for decision-making under missing data — what would later become foundational to sequential analysis and statistical decision theory. His work recognized that the pattern of what's missing from a dataset can be as informative as what's present — a principle that applies far beyond military applications.

The Legacy

Wald died in a plane crash in 1950, at age 48, during a lecture tour in India. His bomber analysis wasn't widely known outside military circles until decades later, when it became a canonical example in statistics education and decision theory.

The irony of his death in a plane crash — given his work on making planes safer — has been noted by many writers. But the deeper legacy is the cognitive skill his work exemplifies: the ability to see the selection process that shapes the evidence, and to ask what the missing evidence would reveal.

Discussion Questions

1. What made Wald able to see what the military analysts couldn't? Was it mathematical skill, fresh perspective (as an outsider), or something else?
2. Apply Wald's framework to a non-military problem: what would "armoring the engines" look like in your field?
3. The military initially had the data right but the interpretation wrong. What does this tell us about the relationship between data and interpretation?
4. Is Wald's insight applicable to AI/machine learning? How might training data exhibit survivorship bias?

References

• Wald, A. (1943). "A Method of Estimating Plane Vulnerability Based on Damage of Survivors." Statistical Research Group, Columbia University. Declassified. (Tier 1)
• Mangel, M. & Samaniego, F. J. (1984). "Abraham Wald's Work on Aircraft Survivability." Journal of the American Statistical Association, 79(386), 259–267. (Tier 1)
• Ellenberg, J. (2014). How Not to Be Wrong: The Power of Mathematical Thinking. Penguin. Chapter on survivorship bias. (Tier 1)