Where Do You Add Your Armor?

It was decided that the armor should be added to those respective areas.

Someone spoke up and said that perhaps it would be a good idea to get some input from the Statistical Research Group (SRG). The SRG was a highly classified program akin to the Manhattan Project, but instead of making weapons they used equations to increase the effectiveness of the military machine. Jacob Wolfowitz for example concluded the special mixtures of ammunition favored by the ordinance sergeants were in fact not nearly as effective as an entire belt of Armor Piercing Incendiary rounds.

One of the young mathematicians that had joined the effort was a man by the name of Abraham Wald. Mr.Wald was already starting to get recognized for his contributions to the war effort, due to his ability to approach a problem differently. It was a popular running joke at the time that Mr. Wald’s work was so highly classified that the secretaries were under strict orders to snatch the paper out of his hand as soon as he had finished writing on it.

Mr. Wald was brought in to give his opinion on the data collected. The other researchers believed that this was an extra unnecessary step as the data shows where we should add the armor. What could this young man possibly add to the very apparent conclusion? Well, what happened next was the stuff of legend. Mr. Wald looked at the information, went to the diagram, and stated that they should add the armor to the spots where there were no bullet holes, specifically the engines and the cockpit.

The researchers scoffed, after all, what sense does that make? There were no holes in those areas. Mr. Wald pointed out a significant detail that the researchers had overlooked; they were making their decision based solely on the aircraft that made it back. Mr. Wald postulated that the “missing bullet holes” were in the engine and cockpit area of the planes that did not make it back. The places where there were holes on the returning planes indicated that the plane could sustain damage in those areas and make it back to base. The “missing bullet holes” area represented critical spots that if hit, resulted in a catastrophic failure and therefore the plane did not return. Thousands of airmen owe a debt of gratitude to a second opinion. This story became a textbook example of the cognitive bias known as survivorship bias.

Today, we cannot make our systems more resilient by simply armoring everything. Economic and operational constraints do not allow for it. One could even argue that adding unnecessary redundancy adds to the complexity and can inject risk; think Charles Perrow’s theory of systems or normal accidents. While making decisions regarding redundancy, understand your tolerable risk. Understand your susceptibility to cognitive biases and decide based on critical thinking. Don’t be afraid to get a second opinion and remember to always think about where you are putting your armor.