Exercises: The Plausible Story Problem

Difficulty Guide: - ⭐ Foundational | ⭐⭐ Intermediate | ⭐⭐⭐ Challenging | ⭐⭐⭐⭐ Advanced/Research

Part A: Conceptual Understanding ⭐

A.1. Define the "narrative fallacy" in your own words. Why is it a structural vulnerability rather than just a bad habit?

A.2. Explain the difference between explanation and prediction. Why does the chapter argue that prediction is a stronger test of understanding than explanation?

A.3. What is the conjunction fallacy, and how does it demonstrate the power of narrative coherence over probability?

A.4. What is the "explanatory depth illusion"? Give an example from your own experience where you believed you understood something deeply but discovered your understanding was actually shallow.

A.5. Explain the concept of "underdetermination." Why does it make the alternative narrative test possible?

A.6. What is "narrative survivorship" (the combination of survivorship bias + plausible story problem)? Why is the combination more dangerous than either failure mode alone?

Part B: Applied Analysis ⭐⭐

B.1. Choose a major event from the last decade (an election, a corporate failure, a pandemic response, a market crash). Apply the alternative narrative test: construct the dominant explanation, then construct an equally plausible alternative using the same evidence.

B.2. Find a "just-so story" in your field — an explanation that sounds plausible but has never been tested prospectively. Apply the prediction test: could this explanation have predicted the phenomenon before it was observed?

B.3. The chapter discusses the Beltway sniper profile. Identify another case where a professional narrative (criminal profile, medical diagnosis, strategic analysis, policy explanation) was confident, coherent, and wrong. Analyze what made the narrative compelling despite being incorrect.

B.4. Compare the plausible story problem in criminal profiling with the plausible story problem in medical diagnosis. What structural features do they share? What makes each domain particularly vulnerable?

B.5. Choose a business success story (Apple, Amazon, Tesla, Netflix, etc.) and apply the alternative narrative test. Can you construct an equally plausible "they succeeded despite their approach, not because of it" narrative?

Part C: Research Design Challenges ⭐⭐–⭐⭐⭐

C.1. Design a study to measure the "explanation-prediction gap" in your field: compare experts' explanatory confidence with their predictive accuracy. What would you measure? What challenges would you face?

C.2. Propose a de-biasing intervention for a professional context where the plausible story problem is active (criminal investigation, medical diagnosis, investment analysis). How would you test whether the intervention works?

C.3. Tetlock found that the best predictors ("foxes") were humble, eclectic, and willing to update their views, while the worst predictors ("hedgehogs") had strong narratives and high confidence. Design a selection process for hiring analysts that favors foxes over hedgehogs.

Part D: Synthesis & Critical Thinking ⭐⭐⭐

D.1. Is this entire book subject to the plausible story problem? The book constructs a narrative about how knowledge fails — is this narrative itself underdetermined by the evidence? Evaluate using the alternative narrative test.

D.2. The chapter argues that narrative coherence "feels identical to truth." Can you think of any way to distinguish the feeling of genuine understanding from the feeling of narrative satisfaction? Or are they truly indistinguishable?

D.3. Compare the plausible story problem with unfalsifiability (Chapter 3). Both involve explanations that are not well-constrained by evidence. What's the difference? Can a plausible story be falsifiable? Can an unfalsifiable claim be a plausible story?

D.4. The confabulation research suggests that the narrative-generating machinery operates independently of actual causal understanding. If this is true, what are the implications for expertise in any narrative-dependent field (history, journalism, strategic consulting, psychotherapy)?

Part M: Mixed Practice (Interleaved) ⭐⭐–⭐⭐⭐

M.1. (From Chapter 4) The streetlight effect determines WHAT gets measured. The plausible story problem determines HOW the measurements are interpreted. Trace this interaction for a specific case.

M.2. (From Chapter 5) The antidepressant publication bias case showed that survivorship-biased evidence made drugs appear more effective. Apply the plausible story lens: how does the narrative of "antidepressants work" shape the interpretation of individual patient experiences?

M.3. (From Chapter 2) When an authority tells a plausible story, both the authority cascade and the narrative fallacy operate simultaneously. Identify a case where the two reinforced each other.

M.4. (Integration) Return to your Epistemic Audit. Apply all six failure modes (lifecycle, authority cascade, unfalsifiability, streetlight effect, survivorship bias, plausible story) to your field. Which combination is most active?

Part E: Research & Extension ⭐⭐⭐⭐

E.1. Read Tetlock's Expert Political Judgment (or a summary) and write a 1,000-word analysis of the fox/hedgehog distinction in the context of the plausible story problem.

E.2. Investigate the accuracy of criminal profiling by reviewing the research literature. How does profiling performance compare to other forms of expert prediction? Write a 1,500-word analysis.

Solutions

Selected solutions in appendices/answers-to-selected.md.