Further Reading: Red Flags

Tier 1: Verified Sources

Ioannidis, John P. A. "Why Most Published Research Findings Are False." PLoS Medicine 2, no. 8 (2005): e124. The landmark paper that crystallized the structural problems in biomedical research — the very problems that the Red Flag Scorecard is designed to detect. Ioannidis showed that small studies, weak effects, flexible designs, financial interests, and hot topics all increase the probability that a published finding is false. Essential background for understanding why the scorecard's questions work.

Tetlock, Philip E. Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press, 2005. Tetlock's systematic evaluation of expert predictions — demonstrating that most experts predict poorly and that the most confident experts are often the least accurate. Directly relevant to Question 12 (prediction track record) and the general principle that structural screening is needed because expert confidence is not a reliable guide.

Goldacre, Ben. Bad Science. Fourth Estate, 2008. An accessible guide to identifying flawed science in media and marketing. Goldacre's diagnostic approach — asking who funded the study, whether it's been replicated, whether the effect size is meaningful — overlaps significantly with the Red Flag Scorecard and provides numerous worked examples.

Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. While this book focuses on individual cognitive biases rather than institutional failure modes, Kahneman's work on overconfidence, anchoring, and the illusion of validity provides the psychological foundation for understanding why experts can be wrong and confident simultaneously — the problem the scorecard is designed to address.

Ritchie, Stuart. Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth. Metropolitan Books, 2020. A comprehensive account of the structural problems in scientific research — fraud, bias, negligence, and hype — with detailed examples. Ritchie's analysis aligns closely with the failure modes in this book and provides additional cases that could be scored using the Red Flag Scorecard.

Tier 2: Attributed Claims

The concept of structured checklists for evaluating claims has precedents in critical appraisal tools used in evidence-based medicine (e.g., the CASP checklists for evaluating clinical studies) and in intelligence analysis (e.g., the Analysis of Competing Hypotheses method developed by Richards Heuer at the CIA).

The "traffic light" scoring system used in the Red Flag Scorecard is adapted from risk assessment frameworks used in engineering, finance, and healthcare quality improvement.

The worked example scoring of the dietary fat hypothesis draws on evidence documented extensively throughout this book, particularly in Chapters 2, 5, 9, 11, 14, and 26.

  1. Start with Goldacre (Bad Science) — for accessible practice in identifying flawed claims
  2. Then Ioannidis (2005) — for the structural argument about why published research is often wrong
  3. Then Tetlock (Expert Political Judgment) — for the evidence on expert prediction failure
  4. Then Ritchie (Science Fictions) — for the comprehensive account of structural problems
  5. Then Kahneman (Thinking, Fast and Slow) — for the cognitive foundations