Key Takeaways: Red Flags
The Big Idea
Parts I-IV diagnosed the failure modes and showed them operating in real fields. This chapter translates that diagnosis into a practical tool: 15 questions that detect the structural conditions sustaining wrong consensuses. The tool doesn't prove claims wrong — only evidence can do that — but it identifies where skepticism is warranted and where investigation should focus.
The 15 Diagnostic Questions (Quick Reference)
| # | Question | Failure Mode Detected |
|---|---|---|
| 1 | Who funded this? | Incentive structures (Ch.11) |
| 2 | Independently replicated? | Replication problem (Ch.10) |
| 3 | What would disprove this? | Unfalsifiability (Ch.3) |
| 4 | Who benefits? | Incentives (Ch.11) + sunk cost (Ch.9) |
| 5 | How old is the core evidence? | Authority cascade (Ch.2) + anchoring (Ch.7) |
| 6 | Precision or accuracy? | Precision without accuracy (Ch.12) |
| 7 | What happens to dissenters? | Consensus enforcement (Ch.14) + outsider problem (Ch.18) |
| 8 | One source or many? | Authority cascade (Ch.2) + citation amplification |
| 9 | Effect size meaningful? | Precision (Ch.12) + streetlight (Ch.4) |
| 10 | Works outside the lab? | Imported error (Ch.8) + streetlight (Ch.4) |
| 11 | Simpler explanation? | Plausible story (Ch.6) + complexity hiding (Ch.15) |
| 12 | Prediction track record? | Unfalsifiability (Ch.3) + revision myth (Ch.20) |
| 13 | Field's own history? | Revision myth (Ch.20) |
| 14 | Outsiders saying differently? | Outsider problem (Ch.18) + consensus enforcement (Ch.14) |
| 15 | How would we know if wrong? | Error visibility (Ch.27) + crisis threshold (Ch.19) |
Interpretation
- 0-2 red flags: Probably sound
- 3-5 red flags: Caution warranted
- 6-9 red flags: Significant concern
- 10+ red flags: Deep skepticism warranted
Calibration
The dietary fat hypothesis (circa 1990) scored ~8 red flags and turned out to be substantially wrong. The pre-2008 financial consensus scored ~12 red flags and was catastrophically wrong.
Epistemic Audit — Chapter 31 Addition
Apply all 15 questions to your field's core consensus claim. Score, document, and identify the highest-risk dimensions.