Quiz: How Incentive Structures Manufacture Error
Target: 70% or higher to proceed confidently.
Section 1: Multiple Choice (1 point each)
1. The key distinction in this chapter is between: - A) Good incentives and bad incentives - B) Incentive misalignment (structural) and corruption (individual) - C) Public and private funding - D) Academic and industry research
Answer
**B)** Misalignment is structural (honest people produce wrong outcomes); corruption is individual (dishonest people produce wrong outcomes). The solutions differ. *Reference:* Section 11.12. The Challenger disaster was caused primarily by: - A) A manufacturing defect in the O-rings - B) Astronaut error - C) Incentive structures that elevated schedule pressure over engineering judgment at every organizational level - D) Sabotage
Answer
**C)** The incentive structure systematically elevated schedule → safety at every level. *Reference:* Section 11.1, 11.53. The "normalization of deviance" describes: - A) How organizations gradually redefine acceptable risk when incentives favor operations over safety - B) How deviant behavior becomes normal in society - C) How statistical normalization is applied to data - D) How new employees adapt to organizational culture
Answer
**A)** Vaughan's concept: each tolerated deviation shifts the definition of "acceptable." *Reference:* Section 11.54. The financial rating agency problem is an example of: - A) Individual corruption by analysts - B) The "issuer-pays" model creating systematic incentive to overrate - C) Inadequate mathematical models - D) Government regulation failure
Answer
**B)** When the entity being rated pays for the rating, the incentive to rate favorably is structural. *Reference:* Section 11.45. The "doubt is our product" strategy involves: - A) Producing genuinely uncertain findings - B) Funding research designed to create enough conflicting evidence that the science appears "unsettled" - C) Manufacturing physical products that cause doubt - D) Questioning all scientific conclusions equally
Answer
**B)** The tobacco template: fund research to create doubt, not to find truth. *Reference:* Section 11.66. The five-stage incentive map covers: - A) Hypothesis, experiment, analysis, conclusion, publication - B) Funding, research, evaluation, publication, dissemination - C) Question, data, analysis, interpretation, action - D) Planning, execution, review, revision, deployment
Answer
**B)** Each stage has its own incentive structure, and biases compound across stages. *Reference:* Section 11.3Section 2: True/False with Justification (1 point each)
7. "If we eliminate corruption in science, the incentive misalignment problem will be solved."
Answer
**False.** Corruption and incentive misalignment are distinct problems requiring distinct solutions. Eliminating corruption (enforcing rules against fraud) does nothing about misalignment (redesigning structures so that rational behavior produces correct outcomes).8. "Industry-funded research is always unreliable."
Answer
**False.** Industry funding creates a structural bias, but structural safeguards (pre-registration, independent conduct, transparent reporting) can mitigate the bias. The diagnostic question is not "Who funded this?" but "Does the funding structure create systematic incentives toward specific conclusions?"Section 3: Short Answer (2 points each)
9. Explain how the five-stage incentive cascade compounds bias from funding through dissemination. Use a specific example.
Sample Answer
Example: dietary supplement. Stage 1 (funding): supplement company funds the trial — incentive toward positive results. Stage 2 (research): researchers use small sample, multiple outcome measures — researcher degrees of freedom. Stage 3 (evaluation): reviewers from the supplement field — confirmation bias. Stage 4 (publication): journal selects for novelty and significance — publication bias. Stage 5 (media): "Breakthrough supplement!" headline — sensationalism bias. Each stage adds its own distortion; the cumulative effect is far larger than any single stage's bias.Section 4: Applied Scenario (3 points)
10. A social media company commissions a study on whether its platform affects teen mental health. The study is conducted by researchers at a respected university, funded by a grant from the company. The results show "no significant negative effect." Apply the incentive analysis framework. What questions should you ask before accepting the finding?
Sample Answer
Questions: (1) Did the company have input on the study design? (comparison group, outcome measures, sample selection). (2) Were the results pre-registered, or could the researchers have tested multiple outcomes and reported the most favorable? (3) Is this one of multiple studies the company commissioned? (publication bias — were negative results suppressed?). (4) Do the researchers have ongoing funding relationships with the company? (career incentive to maintain the relationship). (5) Who selected the researchers? (the company chose researchers it expected would produce favorable results). (6) Was the raw data shared for independent analysis? Even if every individual involved was honest, the incentive structure (company selects researchers → company funds research → company benefits from positive results) creates a systematic bias. The finding may be correct — but the structural incentives mean it should be weighted less heavily than an independently funded, pre-registered study with open data.Scoring & Next Steps
| Score | Assessment | Recommended Action |
|---|---|---|
| < 50% | Needs review | Re-read 11.1–11.3 |
| 50–70% | Partial | Review the Challenger and ratings agency cases |
| 70–85% | Solid | Ready to proceed |
| > 85% | Strong | Proceed to Chapter 12 |