Chapter 9 Quiz: Survivorship Bias — The Most Dangerous Lie Statistics Tell
Q1. In Abraham Wald's WWII analysis, the military's initial conclusion was to armor the wings and fuselage because that's where returning planes showed bullet damage. Wald's correction was to armor the engines instead. Why?
a) Engines are more expensive to repair than wings b) The engines were never hit by enemy fire c) The planes that took engine hits never returned — making engine damage the most dangerous but invisible in the data d) Wings could be more easily armored than engines
Show Answer
**c) The planes that took engine hits never returned — making engine damage the most dangerous but invisible in the data** Wald recognized that the returning planes were not a random sample of hit planes — they were the planes that survived hits. The pattern of hits on returning planes showed where hits were *survivable*. The absence of engine damage on returning planes meant that engine damage was likely *fatal*, causing those planes to crash and never return. The most important data was in the missing observations.Q2. Survivorship bias is defined as:
a) The tendency to remember successes more vividly than failures b) The error of drawing conclusions from a sample filtered by success without accounting for the filter c) The statistical phenomenon where extreme observations return to average d) The tendency to attribute survival to skill rather than luck
Show Answer
**b) The error of drawing conclusions from a sample filtered by success without accounting for the filter** Survivorship bias is a systematic error in what data you examine, not primarily a memory bias (option a) or a statistical phenomenon (option c). It occurs specifically because the population you can observe has been filtered — only certain outcomes are visible, and the filter distorts what the visible outcomes mean.Q3. Nadia follows a successful YouTuber's advice precisely but doesn't achieve comparable success. The survivorship bias explanation for this outcome is:
a) She didn't follow the advice correctly b) The advice might describe what worked for one successful person but tells us nothing about whether it works for the broader population who tried it c) YouTube's algorithm changed after the book was written d) Success in content creation requires talent that cannot be replicated through strategy
Show Answer
**b) The advice might describe what worked for one successful person but tells us nothing about whether it works for the broader population who tried it** The survivorship-biased advice describes the experience of someone who succeeded. Without knowing how many people followed the same advice and didn't succeed, you cannot evaluate whether the advice causes success or is simply correlated with success in the filtered sample of survivors. Option (a) assumes the advice was right and Nadia erred; survivorship bias suggests the advice's validity is itself in question.Q4. Mutual fund survivorship bias inflates apparent fund performance because:
a) Successful fund managers take on more risk than they report b) Funds that perform poorly are closed or merged, leaving databases with only the better-performing survivors c) Fund managers selectively report only their best years d) Survivorship bias actually deflates fund performance, making funds look worse than they are
Show Answer
**b) Funds that perform poorly are closed or merged, leaving databases with only the better-performing survivors** When underperforming funds are closed, they disappear from databases. The remaining funds are, by definition, those that performed well enough to survive. Calculating average returns across this filtered sample overstates the returns that an investor would have experienced randomly selecting a fund at the beginning of the period, because the failed funds they might have selected are no longer in the database.Q5. The "invisible graveyard" metaphor refers to:
a) The historical record of all investments that lost money b) The uncounted population of people who tried the same strategies as successful people but failed and are not visible in success-focused data c) The psychological damage caused by repeated failure d) Companies that went bankrupt before achieving success
Show Answer
**b) The uncounted population of people who tried the same strategies as successful people but failed and are not visible in success-focused data** The invisible graveyard is the crucial missing data in survivorship-biased analysis — everyone who attempted what the survivors attempted but didn't succeed. They are invisible because their failures don't generate books, podcasts, conference talks, or case studies. Training yourself to ask about this invisible graveyard is the primary antidote to survivorship bias.Q6. The Dale-Krueger research on elite university outcomes found that:
a) Attending an elite university produces substantially higher lifetime earnings for all student types b) For most students, the long-run income difference between attending an elite vs. non-elite university largely disappears when you control for the student's characteristics at admission c) Elite university graduates earn more because of their superior educational experience d) Elite universities improve outcomes primarily through their alumni networks
Show Answer
**b) For most students, the long-run income difference between attending an elite vs. non-elite university largely disappears when you control for the student's characteristics at admission** Dale and Krueger found that comparing students who were admitted to elite universities and attended them to students who were admitted and chose to attend other schools (controlling for the selection effect) largely eliminated the apparent income premium. The raw comparisons that show elite university benefits reflect selection bias — the students attending elite universities were already exceptional before enrollment.Q7. Which of the following best illustrates "asking about the denominator"?
a) "How much money did this investment return?" b) "Out of how many companies funded in this sector, how many returned 10x or more?" c) "How many employees does this successful company have?" d) "What was the founder's educational background?"
Show Answer
**b) "Out of how many companies funded in this sector, how many returned 10x or more?"** "Asking about the denominator" means asking about the full reference class from which the successful case was drawn. A 10x return sounds impressive until you learn it was one out of 100 investments, making it a 1% success rate. Option (a) focuses on the outcome without the reference class; options (c) and (d) are facts about the successful case, not about the population it came from.Q8. Why does the chapter describe survivorship bias as "the most dangerous lie statistics tell"?
a) Because it uses deliberately false data to mislead b) Because it is the most common form of bias in published research c) Because it misleads using true facts presented from a perspective that omits the most important data (the non-survivors) d) Because it is impossible to correct for once identified
Show Answer
**c) Because it misleads using true facts presented from a perspective that omits the most important data (the non-survivors)** Survivorship bias doesn't require false data — it misleads with true facts. Tyler Ash really did post daily. Returning planes really were hit in the wings. These facts are accurate. But the perspective from which they're presented — seeing only the survivors — changes what the facts mean. This combination of true facts and systematically misleading conclusions is what makes survivorship bias particularly hard to detect and correct.Q9. The SPIVA report (S&P Dow Jones Indices) found that over 15 years, approximately what fraction of large-cap active equity funds underperformed the S&P 500 index?
a) ~30% b) ~55% c) ~75% d) ~92%
Show Answer
**d) ~92%** The SPIVA reports consistently find that the overwhelming majority of actively managed funds underperform passive index benchmarks over long periods. The 2022 report found approximately 92% underperformance over 15 years for large-cap US equity funds, even after correcting for survivorship bias (closed funds are included in the analysis). This finding is central to the case for passive investing.Q10. A centenarian is interviewed and attributes their long life to: drinking one glass of red wine every evening, sleeping 8 hours nightly, and never worrying too much. The survivorship bias critique of this advice is:
a) The centenarian is probably lying about their habits b) Longevity is determined by genetics, not lifestyle c) We don't know how many people followed the same habits and died at 70 — the advice comes from a filtered survivor sample, not from a study of the general population d) One centenarian is an insufficient sample size for health recommendations
Show Answer
**c) We don't know how many people followed the same habits and died at 70 — the advice comes from a filtered survivor sample, not from a study of the general population** This is the classic survivorship bias critique. The centenarian may genuinely have these habits. But we're hearing from a survivor — someone who lived to be over 100. We have no data about people who had the same habits and died at 70, 80, or 90. Without that comparison, you cannot determine whether the habits contributed to the centenarian's longevity or are simply activities they happen to do. Option (d) is also correct as a separate statistical concern but is not the survivorship bias argument.Q11. When analyzing startup advice from successful founders, which of the following represents the survivorship bias correction approach?
a) Follow only the advice that multiple successful founders agree on b) Look specifically for research that includes failed founders with similar strategies, not just successful ones c) Discount advice from founders who had unusual advantages like prior funding d) Focus only on founders from the past 5 years, since older advice is less relevant
Show Answer
**b) Look specifically for research that includes failed founders with similar strategies, not just successful ones** The survivorship correction requires including the non-survivors — the founders who tried similar strategies and failed. Research that only draws on successful founders (option a, finding agreement among survivors) remains survivorship-biased. Options (c) and (d) address different concerns and don't correct for survivorship.Q12. Priya notices that everyone she admires in her target industry went to a small set of prestigious universities. Before concluding she needs to attend one, the survivorship bias question she should ask is:
a) "Are those universities accepting applications right now?" b) "Of the many people who graduated from those universities and entered this industry, how many achieved the outcomes I admire?" c) "Which specific courses at those universities are most relevant?" d) "How do alumni from those universities rate their experience?"
Show Answer
**b) "Of the many people who graduated from those universities and entered this industry, how many achieved the outcomes I admire?"** This is "asking about the denominator" — not just "who are the successful people I can see?" but "what fraction of all people from those universities (or who followed similar paths) achieved these outcomes?" The people Priya admires are a selected sample from a larger population of graduates with diverse outcomes. Without knowing the full distribution, she can't determine whether the university caused the success.Q13. The Lindy Effect (the idea that things that have survived a long time will survive longer) explicitly uses survivorship. This is:
a) Always a fallacy because survivorship bias always produces incorrect conclusions b) A valid use of survival information — things that have survived longer have demonstrated robustness, which is genuinely predictive c) Only valid for physical objects, not ideas or institutions d) The same logical error as the gambler's fallacy
Show Answer
**b) A valid use of survival information — things that have survived longer have demonstrated robustness, which is genuinely predictive** The Lindy Effect is a case where the survivorship information is actually informative: a book that has been continuously read for 500 years has demonstrated that it contains something worth reading across many different contexts. This is different from the errors described in this chapter because the survival itself is the evidence — it's not being used to incorrectly infer that all attempts would have succeeded. Survivorship bias is an error in a specific logical move (inferring general truths from filtered samples), not in all uses of survival data.Q14. Which statement about survivorship bias in the self-help literature is most accurate?
a) Self-help books are generally fraudulent because their authors are intentionally hiding failure data b) Self-help advice is worthless because it comes from people who succeeded c) Self-help books accurately describe what worked for the author but may not account for the broader distribution of outcomes among people who followed similar approaches d) Survivorship bias only affects quantitative advice, not qualitative mindset advice
Show Answer
**c) Self-help books accurately describe what worked for the author but may not account for the broader distribution of outcomes among people who followed similar approaches** This is the calibrated view. Self-help authors are generally honest — they describe their genuine experience. The issue is not dishonesty but incomplete information: they cannot know what happened to the much larger group of people who followed similar approaches and had different outcomes. The advice can be useful input to your decision-making; it just needs to be weighted by uncertainty about whether it generalizes.Q15. The most important practical habit for fighting survivorship bias in everyday decision-making is:
a) Always consulting academic research before making decisions b) Trusting only randomized controlled trials and meta-analyses c) Actively imagining and seeking information about the population of people who tried the same approach and failed — the invisible graveyard d) Focusing on base rates for the general population rather than specific success stories