Blind Sided: The Survivorship Bias

What it is?

The old story goes that during World War 2 a number of the United States military officials realised that their planes were returning home from bombing runs suffering moderate to severe bullet damage at specific points on the aircraft, such as the wingtips, tails and the fuselage. The military boffins acted quickly to improve the armour of the affected areas. Makes a lot of sense right? If your planes are being shot to shit you want to improve protection to the aircraft and pilots to maximise mission efficacy. We will struggle to believe that the vast majority of people don’t agree with this point of view at first.

However, here is the kicker. This point of view is completely wrong and damaging to most industries of which athletic development is one of many.

Example bullet damage on a surviving aircraft.

Then Abraham Wald, an absolute genius, had the idea that rather than protect the area’s most heavily afflicted by bullet damage they should increase the armour of areas that suffered the least amount of damage. At first glance, this seems stupid. But, the reasoning is that the US was only observing planes that made it home. The planes could clearly survive the wingtips, fuselage and tails peppered with bullets. The planes that survived were not hit in the vital areas, such as the engines and cockpits. By originally focusing solely on the surviving planes, the reality was wrapped into this bias that could have had disastrous consequences. They did not look at where they had failed and where they could make the biggest improvements and learn the most meaningful lessons.

Why do we need to avoid it?

This phenomenon causes us to be obsessed with victories and ignore the lessons of failure. In swimming, it’s a poor programme who happen to have, by chance, an exceptional athlete. An athlete that would be good no matter where they learned to swim and compete. An exceptional athlete will survive a poor programme. This creates a survivorship bias because it reinforces that the programme is better than it is, that the coaches are more skilled than they are and further masks the failings of the programme with the exceptional athlete. There is no learning experience because the exceptional athlete fuels this bias. To access the true effectiveness of a swimming programme the entire athlete population must be considered, from Learn To Swim right the way up to the elite level. We aren’t suggesting if a programme has only 1 exception athlete that that programme is shit, it’s just an example.

Tiger Woods is a prime example, everyone talks about him as the poster boy for early specialisation and he may be. But what about the thousands, likely millions of others who didn’t make it by following a similar path. Tiger Woods is a plane that returned home. How many more incredible athletes have we let slip through our fingers because we were so focused on the survivors?

Think of it this way. Say we communicated with every climber after they successfully summited Everest and returned home. We could conclude that climbing Everest is completely safe with zero risks because everyone we’ve spoken to is still alive after climbing the mountain. This is, of course, a ridiculous conclusion but a great example of survivorship bias.

How we can avoid it!

Coaches must be reflective in nature, we must be able to take a step back and analyse our knowledge, our skills and our limitations. What are we doing that could be better? Where is the evidence disproving our methods and are we open to change? Is our practice the best it can be to maximise our athlete’s potential? Because that’s why we do this, the athletes in front of us. Yeah, we all have ambitions but using athletes to further our own careers is despicable. Any coach bias is damaging to athlete potential and by focusing on the surviving athletes in a programme we lose sight of what really matters and what makes the biggest differences.

History is written by the victors – Sir Winston Churchill