Quiz: The Humility Chapter

Q: "The feeling of being wrong is identical to the feeling of being right" means: (a) People enjoy being wrong (b) The subjective experience of holding a belief does not change based on whether the belief is correct — confidence is a feeling, not a signal of accuracy (c) There is no difference between right and wrong (d) All beliefs are equally valid

(b) You feel the same level of certainty whether you are right or wrong. The difference only becomes apparent when you discover the error — which may never happen if the field lacks error-detection mechanisms.

Q: "Metacognitive blindness" refers to: (a) Being unable to read (b) The inability to assess the accuracy of your own cognition through self-examination — requiring external tools for error detection (c) Forgetting facts (d) Not caring about accuracy

(b) You have thoughts, and you have feelings of confidence about those thoughts. But the confidence is not a reliable signal of accuracy. External tools (calibration exercises, diagnostic questions, peer challenge) are needed because introspection cannot detect its own errors.

Q: In the 90% confidence interval exercise, most people get approximately: (a) 9-10 out of 10 correct (well calibrated) (b) 3-5 out of 10 correct (systematically overconfident) (c) 0-1 out of 10 correct (systematically underconfident) (d) Exactly 5 out of 10 (random)

(b) 3-5 correct out of 10. People's 90% confidence intervals are too narrow — they believe they know more precisely than they actually do. This systematic overconfidence is one of the most robust findings in cognitive psychology.

Q: Epistemic humility is best distinguished from epistemic nihilism by: (a) Epistemic humility trusts everything; nihilism trusts nothing (b) Epistemic humility combines confidence in *methods* with humility about *conclusions*; nihilism rejects both methods and conclusions (c) They are the same thing (d) Epistemic humility applies only to science

(b) Humility is active and productive — it uses reliable methods while remaining open to revision. Nihilism is passive and paralytic — it abandons the pursuit of truth entirely. The book argues for humility, not nihilism.

Q: The "What Would Change Your Mind?" test (Exercise 3) detects: (a) Whether you are smart (b) Whether your beliefs are falsifiable — if you cannot specify conditions for updating, your beliefs are structurally immune to correction (Chapter 3) (c) Whether you have read enough (d) Whether your beliefs are popular

(b) The inability to specify what would change your mind is a red flag — not because the belief is necessarily wrong, but because unfalsifiable beliefs cannot be corrected even when correction is warranted.

Q: The chapter argues that calibration is: (a) An inborn trait that cannot be changed (b) A skill that improves with deliberate practice — regular confidence audits, pre-mortems, and surprise journals all improve calibration over time (c) Only relevant to scientists (d) Impossible to achieve

(b) Tetlock's research on superforecasters demonstrates that calibration improves with practice, feedback, and specific cognitive habits. It is not a personality trait — it is a skill.

Q: The "surprise journal" practice improves calibration because: (a) It records successes (b) Surprises are evidence that your model of the world was incomplete — tracking patterns of surprise reveals your blind spots (c) Writing is therapeutic (d) It replaces other practices

(b) Each surprise indicates a place where your predictions were wrong, which means your model was incomplete or incorrect. Patterns of surprise reveal systematic blind spots — the specific types of errors your thinking is prone to.

Q: The chapter warns that if nothing surprises you, this is: (a) Evidence that you are well-calibrated (b) The most concerning signal — it means either that your model is perfect (extremely unlikely) or that you are not paying attention to disconfirming evidence (extremely likely) (c) Normal and healthy (d) Evidence of intelligence

(b) An absence of surprise almost always indicates insensitivity to disconfirming evidence rather than a perfect model. The most dangerous state is the one where your beliefs are never challenged by experience.

Q: Barry Marshall's approach to H. pylori exemplified epistemic humility because: (a) He was uncertain about everything (b) He was confident in his *method* (the clinical experiment would demonstrate whether the bacterium caused disease) while being open to the *result* (the experiment might have shown he was wrong) (c) He had no confidence (d) He ignored the evidence

(b) Marshall held the productive combination: confidence in method, humility about conclusions. He didn't know he was right — he knew that the evidence would tell him, and he was willing to accept either outcome.

Q: The chapter's Project Checkpoint asks you to audit *your own audit*. This is important because: (a) The audit is always correct (b) Your Epistemic Audit is subject to the same failure modes that you've been diagnosing in your field — your Red Flag scores might be wrong, your Checklist scores might be biased, and your own position inside the field might distort your assessment (c) The audit doesn't matter (d) All audits are biased

(b) The humility framework applies to everything in this book — including the reader's use of the book's tools. If you've been applying the Red Flag Scorecard and the Epistemic Health Checklist from inside a field, your position within that field may have biased your assessments.

Quiz: The Humility Chapter

Q1. "The feeling of being wrong is identical to the feeling of being right" means:

(a) People enjoy being wrong (b) The subjective experience of holding a belief does not change based on whether the belief is correct — confidence is a feeling, not a signal of accuracy (c) There is no difference between right and wrong (d) All beliefs are equally valid

Answer

**(b)** You feel the same level of certainty whether you are right or wrong. The difference only becomes apparent when you discover the error — which may never happen if the field lacks error-detection mechanisms.

Q2. "Metacognitive blindness" refers to:

(a) Being unable to read (b) The inability to assess the accuracy of your own cognition through self-examination — requiring external tools for error detection (c) Forgetting facts (d) Not caring about accuracy

Answer

**(b)** You have thoughts, and you have feelings of confidence about those thoughts. But the confidence is not a reliable signal of accuracy. External tools (calibration exercises, diagnostic questions, peer challenge) are needed because introspection cannot detect its own errors.

Q3. In the 90% confidence interval exercise, most people get approximately:

(a) 9-10 out of 10 correct (well calibrated) (b) 3-5 out of 10 correct (systematically overconfident) (c) 0-1 out of 10 correct (systematically underconfident) (d) Exactly 5 out of 10 (random)

Answer

**(b)** 3-5 correct out of 10. People's 90% confidence intervals are too narrow — they believe they know more precisely than they actually do. This systematic overconfidence is one of the most robust findings in cognitive psychology.

Q4. Epistemic humility is best distinguished from epistemic nihilism by:

(a) Epistemic humility trusts everything; nihilism trusts nothing (b) Epistemic humility combines confidence in methods with humility about conclusions; nihilism rejects both methods and conclusions (c) They are the same thing (d) Epistemic humility applies only to science

Answer

**(b)** Humility is active and productive — it uses reliable methods while remaining open to revision. Nihilism is passive and paralytic — it abandons the pursuit of truth entirely. The book argues for humility, not nihilism.

Q5. The "What Would Change Your Mind?" test (Exercise 3) detects:

(a) Whether you are smart (b) Whether your beliefs are falsifiable — if you cannot specify conditions for updating, your beliefs are structurally immune to correction (Chapter 3) (c) Whether you have read enough (d) Whether your beliefs are popular

Answer

**(b)** The inability to specify what would change your mind is a red flag — not because the belief is necessarily wrong, but because unfalsifiable beliefs cannot be corrected even when correction is warranted.

Q6. The chapter argues that calibration is:

(a) An inborn trait that cannot be changed (b) A skill that improves with deliberate practice — regular confidence audits, pre-mortems, and surprise journals all improve calibration over time (c) Only relevant to scientists (d) Impossible to achieve

Answer

**(b)** Tetlock's research on superforecasters demonstrates that calibration improves with practice, feedback, and specific cognitive habits. It is not a personality trait — it is a skill.

Q7. The "surprise journal" practice improves calibration because:

(a) It records successes (b) Surprises are evidence that your model of the world was incomplete — tracking patterns of surprise reveals your blind spots (c) Writing is therapeutic (d) It replaces other practices

Answer

**(b)** Each surprise indicates a place where your predictions were wrong, which means your model was incomplete or incorrect. Patterns of surprise reveal systematic blind spots — the specific types of errors your thinking is prone to.

Q8. The chapter warns that if nothing surprises you, this is:

(a) Evidence that you are well-calibrated (b) The most concerning signal — it means either that your model is perfect (extremely unlikely) or that you are not paying attention to disconfirming evidence (extremely likely) (c) Normal and healthy (d) Evidence of intelligence

Answer

**(b)** An absence of surprise almost always indicates insensitivity to disconfirming evidence rather than a perfect model. The most dangerous state is the one where your beliefs are never challenged by experience.

Q9. Barry Marshall's approach to H. pylori exemplified epistemic humility because:

(a) He was uncertain about everything (b) He was confident in his method (the clinical experiment would demonstrate whether the bacterium caused disease) while being open to the result (the experiment might have shown he was wrong) (c) He had no confidence (d) He ignored the evidence

Answer

**(b)** Marshall held the productive combination: confidence in method, humility about conclusions. He didn't know he was right — he knew that the evidence would tell him, and he was willing to accept either outcome.

Q10. The chapter's Project Checkpoint asks you to audit your own audit. This is important because:

(a) The audit is always correct (b) Your Epistemic Audit is subject to the same failure modes that you've been diagnosing in your field — your Red Flag scores might be wrong, your Checklist scores might be biased, and your own position inside the field might distort your assessment (c) The audit doesn't matter (d) All audits are biased

Answer

**(b)** The humility framework applies to everything in this book — including the reader's use of the book's tools. If you've been applying the Red Flag Scorecard and the Epistemic Health Checklist from inside a field, your position within that field may have biased your assessments.

Scoring Guide

9-10 correct: Excellent. You understand both the concept and its self-application.
7-8 correct: Good. Review the distinction between humility and nihilism.
5-6 correct: Fair. Revisit the calibration exercises and the "feeling of being wrong" analysis.
Below 5: Re-read the chapter — and notice whether your reaction to scoring poorly is defensive (protecting your self-image) or curious (interested in the gap between confidence and accuracy). The reaction itself is data.