The Sneaky Mental Trap of Survivorship Bias (and Why It Distorts Reality)
- Ariel K
- Oct 4, 2023
- 3 min read
Survivorship bias is an insidious logical error that distorts our judgement by focusing solely on what succeeded and ignoring what failed or perished. This dangerous cognitive blindspot skews our understanding of the world in fields from business to history.
Here we explore this persistence mental trap and just how much reality it obscures.
What is Survivorship Bias?
Survivorship bias happens when we draw false conclusions by analysing only the “survivors” who made it through some selection process or trial. We miss crucial lessons by ignoring those who did not survive or were excluded at various stages due to attributes we may not see.
For example, studying successful entrepreneurs alone yields limited insight if we ignore the far greater number whose ventures did not survive. The survivors may exhibit traits like persistence, passion, and luck while many unsuccessful founders may have been equally diligent and talented but faced different circumstances. Focusing only on survivors paints a distorted picture.
Survivorship bias has misled analysis across many domains:
Military plane damage
During WWII, analysts looked at returned aircraft to identify where added armor was needed. However, the bullet holes they tallied revealed the areas planes could be hit and still survive, not the vulnerable unseen places that caused crashing.
The Military got it wrong
Statistician Abraham Wald was asked to help the Allied forces determine where added armor should be placed on aircraft to better protect them against enemy fire. The military had conducted surveys of returning planes, tallying and mapping where bullet holes were concentrated based on the idea that additional armor should go on the most heavily damaged areas.
Statistician Abraham Wald got it right
However, Wald recognized that this analysis suffered from survivorship bias. The holes showing where planes were getting hit only reflected areas that planes could sustain damage yet still survive the flight home. The surveys overlooked the important factor of planes that had been shot down and crashed rather than survived with damage.
Wald concluded that the military should reinforce the areas showing fewer hits in the surveys of surviving planes. Why? Because the absence of holes indicated the planes that were hit in those spots did not make it back. The areas with little to no damage were actually the most vulnerable parts that caused planes to crash when hit. The survivors' bullet patterns were misleading.
Through this example, Wald demonstrated how survivorship bias can lead to false conclusions. The surviving aircraft presented a limited picture based on their ability to withstand certain amounts of damage. But the planes that went down revealed key insights on structural vulnerabilities that survivor-only analysis missed.
His work on mitigating survivorship bias had a pivotal impact on subsequent statistical methodology and theory. This case became a canonical example of survivorship bias across many disciplines beyond aeronautics, entering statistics textbooks and research literature to illustrate the prevalence of distortion from analysing only survivors. It remains a powerful reminder today on how accounting for losses is equally important as studying successes.
Avoiding Survivorship Bias
While subtle, survivorship bias can lead to dangerously wrong assumptions in areas from investment choices to company culture to public policy. Here are some tips for detecting and avoiding survivorship bias:
Consider what and who is missing from the analysis - Are certain groups excluded? Can you broaden the sample?
Question selection criteria and filters - How were successes chosen? Which selection pressures or criteria may have influenced the sample?
Recognize your own cognitive biases - We instinctively gravitate to success stories. Make an effort to intentionally expose failures.
Seek dissenting perspectives - Talk to people with contrasting vantage points to gain a fuller picture of reality.
Comments