Exercises: How Incentive Structures Manufacture Error

Difficulty Guide: - ⭐ Foundational | ⭐⭐ Intermediate | ⭐⭐⭐ Challenging | ⭐⭐⭐⭐ Advanced/Research

Part A: Conceptual Understanding ⭐

A.1. Explain the distinction between incentive misalignment and corruption. Why does the distinction matter for designing solutions?

A.2. What is the "normalization of deviance"? Give an example from a field not discussed in the chapter.

A.3. Apply the five-stage incentive map (funding → research → evaluation → publication → dissemination) to describe how bias compounds across stages.

A.4. What is the "issuer-pays" model in credit rating, and why does it produce systematically biased ratings?

A.5. Explain the game theory of the ratings agency problem. Why is the "rate favorably" strategy dominant for each individual agency?

A.6. The chapter argues that the pharmaceutical incentive structure produces systematic overestimation of drug efficacy without fraud. Trace the mechanism.

Part B: Applied Analysis ⭐⭐

B.1. Apply the five-question Incentive Audit to your own field. Map each stage (funding, research, evaluation, publication, dissemination) and identify the misalignments.

B.2. The Challenger disaster resulted from incentive misalignment at every organizational level. Map the incentives for each actor (NASA management, Thiokol management, engineers, Congress). How did the misalignments interact?

B.3. Compare the pharmaceutical funding bias with the think tank model. What structural features do they share? What's different?

B.4. The opioid epidemic resulted from incentive misalignment across the pharmaceutical, medical, and regulatory systems. Map the specific incentives for each actor (Purdue Pharma, physicians, regulators, patients).

B.5. Identify a current example of "normalization of deviance" in your field or organization. What standards have drifted? What incentives drove the drift?

Part C: Research Design Challenges ⭐⭐–⭐⭐⭐

C.1. Design a pharmaceutical trial system that separates funding from research conduct. How would you maintain quality while eliminating the funder's ability to influence results?

C.2. Propose a peer review reform that addresses the four incentive problems identified in this chapter (unpaid labor, anonymity without accountability, confirmation bias, speed pressure).

C.3. Design a "rating agency for science" — an institution that would rate the reliability of published findings. What incentive structure would prevent this agency from falling into the same traps as financial rating agencies?

Part D: Synthesis & Critical Thinking ⭐⭐⭐

D.1. Is incentive misalignment the "master" failure mode that explains all the others? (Sunk cost is an incentive problem; publication bias is an incentive problem; the replication crisis is an incentive problem.) Or are there failure modes that operate independently of incentives?

D.2. The chapter proposes five redesign principles. For each, identify a potential unintended consequence. (Every incentive change creates new incentives.)

D.3. The tobacco industry's "doubt is our product" strategy has been applied to climate change, sugar, chemicals, and technology. Is there a way to inoculate the public against manufactured doubt without destroying legitimate scientific uncertainty?

D.4. Apply the incentive analysis to this book. What incentive structures might bias its conclusions? (Hint: a book about failure modes has an incentive to find failure modes everywhere.)

Part M: Mixed Practice (Interleaved) ⭐⭐–⭐⭐⭐

M.1. (From Chapter 4) Goodhart's Law + incentive misalignment: when hospital quality metrics become high-stakes incentives, how does the resulting misalignment compound?

M.2. (From Chapter 5) Publication bias is both a survivorship bias problem (Ch.5) and an incentive problem (Ch.11). Trace both mechanisms and show how they reinforce each other.

M.3. (From Chapter 9) Sunk cost maintains wrong answers; incentive misalignment generates wrong answers. How do these two persistence mechanisms interact?

M.4. (Integration) You now have eleven diagnostic lenses. Which three are most active in your field?

Part E: Research & Extension ⭐⭐⭐⭐

E.1. Read Diane Vaughan's The Challenger Launch Decision. Write a 1,500-word analysis of the normalization of deviance using this chapter's incentive framework.

E.2. Investigate the incentive structure of a knowledge-producing institution you know well. Map the five stages and identify the misalignments. Write a 1,500-word analysis.

Solutions

Selected solutions in appendices/answers-to-selected.md.