Quiz: Building Better Knowledge Systems

Q: The distinction between self-correcting and self-protecting institutions is determined by: (a) What the institution says about itself (b) Structural features — incentives, feedback loops, outsider access — that determine whether errors are treated as information (self-correcting) or as threats (self-protecting) (c) The intelligence of the institution's leaders (d) The institution's age

(b) The distinction is structural, not intentional. Every institution believes it self-corrects. The actual capacity for self-correction is determined by structural features — not by stated values, mission statements, or good intentions.

Q: Principle 2 (Incentive Alignment) is identified as the most important design principle because: (a) It's the cheapest to implement (b) Incentives determine behavior, and behavior determines outcomes — regardless of stated values, formal structures, or individual virtue (c) Other principles don't matter (d) It's the easiest to measure

(b) Every field autopsy in Part IV identified incentive misalignment as a core driver of error persistence. When incentives reward error-producing behavior (novelty over accuracy, confirmation over replication, confidence over calibration), the other principles are undermined.

Q: The "self-correction illusion" is dangerous because: (a) Self-correction never happens (b) It creates the belief that existing mechanisms are adequate — reducing the perceived urgency of building the structural supports that genuine self-correction requires — while actual correction is slow, incomplete, and usually crisis-driven (c) It makes institutions overconfident (d) It costs too much money

(b) The illusion is partially true — fields do correct. But they correct slowly, incompletely, at enormous cost, and usually only when forced by crisis. The illusion that this process is automatic and adequate is what prevents the structural investment needed to make it better.

Q: "Correction fragility" refers to: (a) Corrections being permanent once established (b) Self-correcting structures being easy to erode through incentive drift, culture erosion, budget pressure, and normalization of deviance — requiring continuous investment to maintain (c) Corrections being too aggressive (d) Institutions being too fragile to survive

(b) Self-correction is not a state to be achieved and then maintained by inertia. It is a practice requiring continuous investment. Each of the four erosion mechanisms can gradually undermine self-correcting structures without any single dramatic failure.

Q: Principle 6 (Correction Celebration) addresses which failure mode? (a) The streetlight effect (b) The revision myth (Chapter 20) and sunk cost of consensus (Chapter 9) — by making error correction visible, valued, and prestigious rather than stigmatized (c) The outsider problem (d) Capital-sustained error

(b) Most fields celebrate discovery and ignore (or stigmatize) correction. This asymmetry removes the incentive for error correction and sustains the revision myth — the illusion that corrections happen naturally. Celebrating correction changes what is valued.

Q: Principle 5 (Measurement Validity Audits) addresses the risk that: (a) Measurements are too expensive (b) Metrics that were once valid proxies become targets (Goodhart's Law), cease to measure what matters, and distort behavior — requiring periodic audit and sunset of corrupted metrics (c) Too many things are measured (d) Measurement technology is outdated

(b) The body count in Vietnam, test scores in education, VaR in finance — all were metrics that became targets and ceased to measure what mattered. Regular audits asking "is this metric still a valid proxy?" prevent the streetlight effect from calcifying.

Q: The Institutional Reform Template has six steps. The correct sequence is: (a) Design → Diagnose → Implement → Hope (b) Diagnose (Checklist) → Map (Tool-Failure Mode Matrix) → Prioritize → Design → Disseminate (Dissent Strategy) → Monitor (c) Monitor → Design → Diagnose (d) Implement everything simultaneously

(b) The template follows the logic of this book: first diagnose the problem (using Part V's diagnostic tools), then identify solutions (using Chapter 34's tools), then design implementation (using Chapter 33's dissent strategy), then monitor (with feedback loops and overcorrection detection).

Q: Principle 3 (Structural Outsider Access) is important because: (a) Outsiders are always right (b) The history of knowledge correction is the history of outsiders — Marshall, Wegener, Hinton, the Innocence Project — and institutions that exclude outsiders protect their consensus from challenge, right or wrong (c) Outsiders have more funding (d) Outsiders are more educated

(b) Outsiders are not always right. But correct challenges frequently come from outside the field, and institutions that are impermeable to outside challenge cannot be corrected by evidence generated outside their boundaries.

Q: The chapter argues that self-correction is "not the natural state of institutions." This means: (a) Institutions cannot self-correct (b) The default tendency of institutions is toward self-protection — defending existing positions, rewarding consensus maintenance, and suppressing challenges — and self-correction requires deliberate structural investment to counter this default (c) Natural institutions are better (d) Only new institutions can self-correct

(b) Self-protection is the path of least resistance because it serves the interests of incumbents (career, reputation, institutional stability). Self-correction requires structural features that push against these interests — and those features must be deliberately built and maintained.

Q: The Epistemic Audit, accumulated through 37 chapters, produces: (a) A casual opinion about your field (b) A professional-grade assessment that identifies failure modes, scores claims and institutional health, designs dissent and teaching strategies, inventories correction mechanisms, and produces a specific institutional reform proposal (c) A research paper (d) A grade for your field

(b) The Epistemic Audit is the progressive project of the entire book — each chapter has added a diagnostic lens, building toward a comprehensive assessment that can serve as the foundation for genuine institutional reform.

Quiz: Building Better Knowledge Systems

Q1. The distinction between self-correcting and self-protecting institutions is determined by:

(a) What the institution says about itself (b) Structural features — incentives, feedback loops, outsider access — that determine whether errors are treated as information (self-correcting) or as threats (self-protecting) (c) The intelligence of the institution's leaders (d) The institution's age

Answer

**(b)** The distinction is structural, not intentional. Every institution believes it self-corrects. The actual capacity for self-correction is determined by structural features — not by stated values, mission statements, or good intentions.

Q2. Principle 2 (Incentive Alignment) is identified as the most important design principle because:

(a) It's the cheapest to implement (b) Incentives determine behavior, and behavior determines outcomes — regardless of stated values, formal structures, or individual virtue (c) Other principles don't matter (d) It's the easiest to measure

Answer

**(b)** Every field autopsy in Part IV identified incentive misalignment as a core driver of error persistence. When incentives reward error-producing behavior (novelty over accuracy, confirmation over replication, confidence over calibration), the other principles are undermined.

Q3. The "self-correction illusion" is dangerous because:

(a) Self-correction never happens (b) It creates the belief that existing mechanisms are adequate — reducing the perceived urgency of building the structural supports that genuine self-correction requires — while actual correction is slow, incomplete, and usually crisis-driven (c) It makes institutions overconfident (d) It costs too much money

Answer

**(b)** The illusion is partially true — fields do correct. But they correct slowly, incompletely, at enormous cost, and usually only when forced by crisis. The illusion that this process is automatic and adequate is what prevents the structural investment needed to make it better.

Q4. "Correction fragility" refers to:

(a) Corrections being permanent once established (b) Self-correcting structures being easy to erode through incentive drift, culture erosion, budget pressure, and normalization of deviance — requiring continuous investment to maintain (c) Corrections being too aggressive (d) Institutions being too fragile to survive

Answer

**(b)** Self-correction is not a state to be achieved and then maintained by inertia. It is a practice requiring continuous investment. Each of the four erosion mechanisms can gradually undermine self-correcting structures without any single dramatic failure.

Q5. Principle 6 (Correction Celebration) addresses which failure mode?

(a) The streetlight effect (b) The revision myth (Chapter 20) and sunk cost of consensus (Chapter 9) — by making error correction visible, valued, and prestigious rather than stigmatized (c) The outsider problem (d) Capital-sustained error

Answer

**(b)** Most fields celebrate discovery and ignore (or stigmatize) correction. This asymmetry removes the incentive for error correction and sustains the revision myth — the illusion that corrections happen naturally. Celebrating correction changes what is valued.

Q6. Principle 5 (Measurement Validity Audits) addresses the risk that:

(a) Measurements are too expensive (b) Metrics that were once valid proxies become targets (Goodhart's Law), cease to measure what matters, and distort behavior — requiring periodic audit and sunset of corrupted metrics (c) Too many things are measured (d) Measurement technology is outdated

Answer

**(b)** The body count in Vietnam, test scores in education, VaR in finance — all were metrics that became targets and ceased to measure what mattered. Regular audits asking "is this metric still a valid proxy?" prevent the streetlight effect from calcifying.

Q7. The Institutional Reform Template has six steps. The correct sequence is:

(a) Design → Diagnose → Implement → Hope (b) Diagnose (Checklist) → Map (Tool-Failure Mode Matrix) → Prioritize → Design → Disseminate (Dissent Strategy) → Monitor (c) Monitor → Design → Diagnose (d) Implement everything simultaneously

Answer

**(b)** The template follows the logic of this book: first diagnose the problem (using Part V's diagnostic tools), then identify solutions (using Chapter 34's tools), then design implementation (using Chapter 33's dissent strategy), then monitor (with feedback loops and overcorrection detection).

Q8. Principle 3 (Structural Outsider Access) is important because:

(a) Outsiders are always right (b) The history of knowledge correction is the history of outsiders — Marshall, Wegener, Hinton, the Innocence Project — and institutions that exclude outsiders protect their consensus from challenge, right or wrong (c) Outsiders have more funding (d) Outsiders are more educated

Answer

**(b)** Outsiders are not always right. But correct challenges frequently come from outside the field, and institutions that are impermeable to outside challenge cannot be corrected by evidence generated outside their boundaries.

Q9. The chapter argues that self-correction is "not the natural state of institutions." This means:

(a) Institutions cannot self-correct (b) The default tendency of institutions is toward self-protection — defending existing positions, rewarding consensus maintenance, and suppressing challenges — and self-correction requires deliberate structural investment to counter this default (c) Natural institutions are better (d) Only new institutions can self-correct

Answer

**(b)** Self-protection is the path of least resistance because it serves the interests of incumbents (career, reputation, institutional stability). Self-correction requires structural features that push against these interests — and those features must be deliberately built and maintained.

Q10. The Epistemic Audit, accumulated through 37 chapters, produces:

(a) A casual opinion about your field (b) A professional-grade assessment that identifies failure modes, scores claims and institutional health, designs dissent and teaching strategies, inventories correction mechanisms, and produces a specific institutional reform proposal (c) A research paper (d) A grade for your field

Answer

**(b)** The Epistemic Audit is the progressive project of the entire book — each chapter has added a diagnostic lens, building toward a comprehensive assessment that can serve as the foundation for genuine institutional reform.

Scoring Guide

9-10 correct: Excellent. You have the complete toolkit.
7-8 correct: Good. Review the relationship between design principles and failure modes.
5-6 correct: Fair. Revisit the self-correction vs. self-protection distinction and the fragility analysis.
Below 5: Re-read the chapter with attention to why each principle is included — trace each one back to the failure mode it addresses.