Survivorship bias is a type of logical error that occurs when a person draws a conclusion from a set of data that includes only surviving observations or participants, disregarding those that did not make it to the end of the process. This bias, which is a form of selection bias, can lead to incorrect conclusions because the results may not accurately represent the entire population or the full set of possible outcomes.
For example, when researchers focus only on successful companies while ignoring the many that have failed, they overlook important lessons that could be learned from those failures. They might incorrectly conclude that certain strategies or characteristics guarantee success when, in fact, they do not.
In finance, investment funds are often scrutinized for this error. Studies examining the performance persistence of funds must account for all funds initially considered, not just those that prevail.
In research, longitudinal studies, like those addressing mental health during the COVID-19 pandemic, may inadvertently exclude participants who drop out, possibly skewing results. The controversial 2005 paper “Why Most Published Research Findings Are False” highlights a number of research issues, including survivorship bias, by demonstrating how many published medical research papers contain results that are not repeatable.
This cognitive bias can significantly skew the understanding of success factors because it omits the insights from non-surviving entities, which are just as critical for a well-rounded analysis. Researchers and analysts must be wary of this bias to ensure their conclusions reflect a comprehensive view of all relevant data.
Military Impact
During World War II, the American military examined returning planes covered in bullet holes. The initial thought was to reinforce the areas that showed the most damage. However, this approach neglected to consider the aircraft that did not return from combat.
Statistician Abraham Wald took the problem of survivorship bias in the military’s analysis and turned it on its head. He proposed that the armour on aircraft should not be distributed based on where bullet holes were found on surviving aircraft.
Instead, Wald theorized that the sections without bullet holes on the returning planes, such as engines and cockpits, were likely the areas that, if hit, would result in a plane being shot down. His counterintuitive approach led to the strategic placement of armour in areas critical to an aircraft’s survivability in battle.
Implications in Finance
When evaluating mutual fund performance, survivorship bias can inflate the perceived success of funds. Funds that have been successful are more likely to remain, whereas funds that have performed poorly and were liquidated or merged tend not to be included in performance studies.
This absence of data of the defunct funds skews the analysis, making the average return of mutual funds appear higher than it might truly be.
Market Strategies and Risks
Market strategies often rely on historical performance data. Survivorship bias can lead to strategies that seem robust only because they are based on the surviving entities, overlooking those that failed.
This biased view does not accurately incorporate the risks and can lead to overconfidence in diversification strategies. For example, strategies that worked for the remaining components of a small-cap index could ignore those that have gone bankrupt, thus misrepresenting the level of risk involved.
Stock Market Interpretations
Survivorship bias also affects the interpretation of stock market data. When analysts conduct backtesting, the missing data from failed companies leads to selection bias. They only see the successes, not the failures, which could demonstrate a misleading correlation between variables being tested.
This influences investment decisions and false confidence in certain stocks or market sectors. Data sources that account for such biases provide a more accurate view of the market, aiding in more informed decision-making.
Success and High Achievers
When survivorship bias is at play, the stories of successful individuals often become the blueprint for ambition. Consider college dropouts like Mark Zuckerberg, Steve Jobs, and Bill Gates; their narratives reinforce the idea that dropping out can lead to immense wealth and influence.
Because of their status as billionaires and founders of transformative companies, their stories are often cherry-picked to represent the entrepreneurial spirit. However, this overlooks the countless individuals who took similar risks but did not reach the same heights, thereby warping the public’s understanding of realistic success.
Each of the aforementioned figures — Zuckerberg, Jobs, and Gates — left college early to build what would become incredibly prosperous enterprises. Mark Zuckerberg created Facebook, a company that redefined social interaction. Steve Jobs, known for his role in making Apple a synonym for innovation, directed the development of products that would change the way people interact with technology. Bill Gates, through Microsoft, significantly shaped the personal computer revolution.
While these stories are inspiring, they are exceptional cases of college dropouts who made it big. They often overshadow the reality that most successful entrepreneurs do complete their higher education, and the majority of rich business owners worked their way up with a formal college degree.
Survivorship Bias in Everyday Life
In decision-making, individuals frequently look to successful outcomes when evaluating choices, disregarding those who have not succeeded. For instance, when a person hears about a successful entrepreneur who dropped out of university, they may downplay the value of formal education, not considering the many who followed a similar path but did not succeed.
In the context of work, survivorship bias can cause professionals to overestimate what behaviors lead to success. Within corporate environments, employees might attribute promotions or accolades to specific actions or characteristics observed in those who have succeeded, ignoring the possibly crucial impact of luck or other factors.
Influence on Society and Culture
Survivorship bias also greatly shapes society’s narrative around success and failure. Societal views on buildings and architecture, for instance, may be influenced by the ones that last through the ages and are admired for their enduring nature, while less durable structures, no longer standing, are seldom considered. This can skew the perception of historical building techniques and materials.
Furthermore, culture tends to celebrate high achievers in various fields, creating the illusion that such heights are easily attainable with enough determination. This bias can impact young people as they make critical life choices, such as deciding whether to pursue work immediately or invest in a university education. They may end up making decisions based on exceptional stories rather than comprehensive data.
Addressing and Avoiding Survivorship Bias
Researchers and analysts can address this bias by including all relevant data points in their studies. This involves tracking and analyzing both the survivors and non-survivors to get a holistic view of the data. For example, in assessing mutual funds, they would consider the performance of all funds, not just the ones that have lasted the longest.
When testing hypotheses, scientists should rigorously attempt to account for all variables that could impact their outcome. Hidden variables can lead to a misinterpretation of results if not accounted for properly. Regular peer review and replication of studies can help in detecting any overlooked aspects that might contribute to survivorship bias.
To avoid survivorship bias, they should:
- Understand its occurrence: recognize where and when survivorship bias is likely to occur.
- Do comprehensive data collection: gather complete datasets that include both ‘survivors’ and ‘non-survivors’.
- Use longitudinal studies: conduct longitudinal studies to monitor subjects over extended periods.
- Ensure transparent reporting: document and disclose the selection criteria for subjects or datasets.
- Conduct sensitivity analysis: test how different scenarios affect the findings of a study to ensure robustness.
By implementing these strategies, one can minimize the risk of making decisions based on incomplete information, thus enhancing the credibility and reliability of their conclusions.
References:
- Amin, G. S., & Kat, H. M. (2003). Welcome to the dark side: Hedge fund attrition and survivorship bias over the period 1994–2001. The Journal of Alternative Investments, 6(1), 57-73
- Czeisler, M., Wiley, J., Czeisler, C., Rajaratnam, S., & Howard, M. (2021). Uncovering Survivorship Bias in Longitudinal Mental Health Surveys During the COVID-19 Pandemic. medRxiv : the preprint server for health sciences, 2021.01.28.21250694.
- Elton; Gruber; Blake (1996). Survivorship Bias and Mutual Fund Performance. Review of Financial Studies. 9 (4): 1097–1120. doi:10.1093/rfs/9.4.1097
- Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Med. 2 (8): e124. doi:10.1371/journal.pmed.0020124
- Mumm, Susan (2010). Women and Philanthropic Cultures, in Women, Gender and Religious Cultures in Britain, 1800-1940, Eds Sue Morgan and Jacqueline deVries . London: Routledge.
- Taleb, Nassim Nicholas (2010). The Black Swan: The Impact of the Highly Improbable (2nd ed.). New York: Random House. ISBN 9780679604181
- Wald, Abraham. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Statistical Research Group, Columbia University. CRC 432