Chapter 10 Quiz: Self-Assessment

Instructions: Answer each question without looking back at the chapter. After completing all questions, check your answers against the key at the bottom. If you score below 70%, revisit the relevant sections before moving on to Chapter 11.

Multiple Choice

Q1. Bayes' theorem tells you how to:

a) Calculate the frequency of an event in a large sample b) Update your belief about a hypothesis when you encounter new evidence c) Determine whether a hypothesis is true or false with certainty d) Eliminate prior beliefs to achieve objective analysis

Q2. In Bayes' theorem, the prior probability refers to:

a) The probability of the evidence occurring b) Your belief about the hypothesis before encountering new evidence c) The probability of the evidence given that the hypothesis is true d) Your final belief after all evidence has been considered

Q3. The posterior probability is:

a) The probability of the hypothesis before seeing evidence b) The probability of the evidence regardless of the hypothesis c) Your updated belief about the hypothesis after incorporating new evidence d) The probability that the evidence is misleading

Q4. Base rate neglect occurs when a reasoner:

a) Fails to collect enough evidence before making a judgment b) Focuses on specific evidence while ignoring the prior probability of the condition c) Updates too slowly in response to new evidence d) Confuses correlation with causation

Q5. A mammogram has 90% sensitivity and 91% specificity. In a population where 1% of women have breast cancer, what is the approximate probability that a woman with a positive mammogram actually has cancer?

a) About 90% b) About 50% c) About 9% d) About 1%

Q6. The prosecutor's fallacy involves confusing:

a) The probability of guilt with the probability of innocence b) The probability of the evidence given innocence with the probability of innocence given the evidence c) The probability of a false positive with the probability of a true positive d) The prior probability with the posterior probability

Q7. In Turing's codebreaking work at Bletchley Park, Bayesian reasoning was used to:

a) Design the Enigma machine b) Systematically narrow down possible Enigma settings by accumulating evidence from intercepted messages c) Prove mathematically that the Enigma code was unbreakable d) Generate random settings for Allied encryption machines

Q8. Paul Graham's Bayesian spam filter differed from rule-based filters because:

a) It used a fixed list of spam keywords b) It learned which words were associated with spam by analyzing classified messages and updated continuously c) It blocked all emails from unknown senders d) It required human operators to classify each email manually

Q9. The frequentist interpretation of probability holds that:

a) Probability is a property of minds -- a degree of belief in a proposition b) Probability is a property of the physical world -- the long-run frequency of events in repeated trials c) Probability is meaningless and should not be used in scientific reasoning d) Probability is always exactly 0 or 1

Q10. The Bayesian interpretation of probability holds that:

a) Probability is only meaningful for repeatable events b) Probability is a measure of physical randomness c) Probability represents a rational agent's degree of belief in a proposition given available evidence d) Probability is arbitrary and cannot be constrained by evidence

Q11. According to the chapter, the reproducibility crisis in science can be understood as a Bayesian problem because:

a) Scientists are using the wrong statistical software b) When the prior probability of hypotheses being true is low, even standard statistical methods produce many false positives c) Bayesian methods are never used in scientific research d) Scientists do not collect enough data

Q12. The immune system performs Bayesian updating through:

a) Conscious decision-making by immune cells b) Affinity maturation, where B cells producing better-fitting antibodies proliferate and those producing ineffective ones die off c) Direct communication between immune cells and the brain d) Random generation of antibodies with no selection process

Q13. Which of the following is true about priors in Bayesian reasoning?

a) Priors are always biases that should be eliminated b) Priors are irrelevant once evidence is collected c) Priors represent existing knowledge and are a feature, not a bug, of rational reasoning d) All rational agents should have the same priors

Q14. One reason Bayesian reasoning keeps being forgotten and rediscovered is:

a) The mathematics is incorrect and keeps being corrected b) Each generation of scientists discovers new evidence against it c) Institutional inertia, the seductiveness of simpler frequentist methods, and the cognitive difficulty of updating all contribute to periodic forgetting d) It only works in certain domains and fails in others

Q15. The chapter's threshold concept -- "Priors Are Not Bias" -- means:

a) Starting with no beliefs is the most objective approach b) Prior beliefs should never be changed c) Starting with prior beliefs and updating them with evidence is not bias but optimal rationality; objectivity means updating honestly, not starting with a blank slate d) All priors are equally valid regardless of evidence

Q16. In the HIV testing example, a test with 99.5% sensitivity and 99.5% specificity applied to a low-risk population (0.1% prevalence) yields a positive predictive value of approximately:

a) 99.5% b) 50% c) 17% d) 0.1%

Q17. Conditional probability refers to:

a) The probability of an event occurring regardless of any other events b) The probability of one event occurring given that another event has occurred c) The probability of two independent events both occurring d) The probability that a condition is met in a random sample

Q18. Which of the following best describes the relationship between Bayesian reasoning and the explore/exploit tradeoff (Chapter 8)?

a) They are unrelated concepts from different fields b) Bayesian updating provides the mechanism for incorporating exploration results into exploitation decisions (e.g., Thompson sampling) c) Bayesian reasoning eliminates the need for exploration d) The explore/exploit tradeoff disproves Bayesian reasoning

Q19. A credence is:

a) A type of statistical test b) A rational agent's degree of belief in a proposition, expressed as a probability c) A proof that a hypothesis is true d) The frequency of an event in a large sample

Q20. The deepest lesson of Bayesian reasoning, as presented in the chapter, is that:

a) Certainty is the goal of rational inquiry b) The best possible beliefs are those held with 100% confidence c) Calibrated uncertainty -- beliefs proportional to evidence -- is the hallmark of rational thought, and learning is the revision of beliefs d) Evidence should be ignored in favor of strong prior convictions

Short Answer

Q21. In two to three sentences, explain why the same positive test result can mean very different things for different patients. Use the terms "prior" and "posterior" in your answer.

Q22. Describe the prosecutor's fallacy and explain why it is dangerous in a courtroom setting. Give a specific example.

Q23. Explain in your own words why the statement "objectivity means starting with no beliefs" is a misunderstanding, according to the Bayesian perspective.

Answer Key

Q1: b) Update your belief about a hypothesis when you encounter new evidence Section 10.2 -- Bayes' theorem provides the formal mechanism for belief updating.

Q2: b) Your belief about the hypothesis before encountering new evidence Section 10.2 -- The prior is your starting belief, before new evidence is incorporated.

Q3: c) Your updated belief about the hypothesis after incorporating new evidence Section 10.2 -- The posterior combines the prior with the new evidence via the likelihood.

Q4: b) Focuses on specific evidence while ignoring the prior probability of the condition Section 10.4 -- Base rate neglect is the failure to incorporate prevalence into reasoning.

Q5: c) About 9% Section 10.1 and 10.4 -- The low base rate (1%) means most positive results are false positives: 90 true positives out of 981 total positives.

Q6: b) The probability of the evidence given innocence with the probability of innocence given the evidence Section 10.5 -- These are two different conditional probabilities related by Bayes' theorem.

Q7: b) Systematically narrow down possible Enigma settings by accumulating evidence from intercepted messages Section 10.6 -- Turing's weight-of-evidence approach was Bayesian updating applied to codebreaking.

Q8: b) It learned which words were associated with spam by analyzing classified messages and updated continuously Section 10.7 -- The Bayesian filter used word frequencies from classified corpora and updated its model with each new message.

Q9: b) Probability is a property of the physical world -- the long-run frequency of events in repeated trials Section 10.8 -- Frequentists interpret probability as relative frequency in infinite trials.

Q10: c) Probability represents a rational agent's degree of belief in a proposition given available evidence Section 10.8 -- Bayesians interpret probability as credence -- degree of belief constrained by evidence.

Q11: b) When the prior probability of hypotheses being true is low, even standard statistical methods produce many false positives Section 10.7 -- Ioannidis's argument: low priors + imperfect tests = many false positives, the same structure as the mammogram problem.

Q12: b) Affinity maturation, where B cells producing better-fitting antibodies proliferate and those producing ineffective ones die off Section 10.7 -- The immune system updates its "beliefs" about the pathogen through selection on binding affinity.

Q13: c) Priors represent existing knowledge and are a feature, not a bug, of rational reasoning Section 10.8 -- The threshold concept: priors are information, not contamination.

Q14: c) Institutional inertia, the seductiveness of simpler frequentist methods, and the cognitive difficulty of updating all contribute to periodic forgetting Section 10.9 -- Multiple factors conspire to make Bayesian reasoning difficult to sustain institutionally.

Q15: c) Starting with prior beliefs and updating them with evidence is not bias but optimal rationality; objectivity means updating honestly, not starting with a blank slate Section 10.8 -- The blank slate is not objective; it is ignorant.

Q16: c) 17% Section 10.4 -- In a population of 100,000 with 0.1% prevalence: 100 true positives, ~500 false positives, PPV approximately 100/600 = 17%.

Q17: b) The probability of one event occurring given that another event has occurred Section 10.2 -- Conditional probability is the foundation on which Bayes' theorem is built.

Q18: b) Bayesian updating provides the mechanism for incorporating exploration results into exploitation decisions (e.g., Thompson sampling) Section 10.7 and Chapter 8 -- Thompson sampling is a Bayesian algorithm for the multi-armed bandit.

Q19: b) A rational agent's degree of belief in a proposition, expressed as a probability Section 10.2 -- Credence is the Bayesian term for degree of belief.

Q20: c) Calibrated uncertainty -- beliefs proportional to evidence -- is the hallmark of rational thought, and learning is the revision of beliefs Section 10.10 -- The deepest lesson: certainty is not the goal; calibrated uncertainty is.

Q21. Sample answer: Different patients have different prior probabilities of having the condition (based on risk factors, age, family history, and population prevalence). The posterior probability -- the updated belief after the test result -- depends on both the test's accuracy and the patient's prior. A positive result for a high-prior patient yields a much higher posterior than the same positive result for a low-prior patient.

Q22. Sample answer: The prosecutor's fallacy occurs when a prosecutor argues that the probability of the evidence given innocence (e.g., a DNA match probability of 1 in 10 million) is the same as the probability of innocence given the evidence. These are different quantities related by Bayes' theorem. For example, if a defendant was identified through a cold database search in a city of 10 million, the one-in-ten-million match probability means approximately one innocent person in the city would also match. Without additional evidence, the DNA match alone does not make guilt near-certain. The fallacy is dangerous because it can lead juries to convict on statistical misunderstandings.

Q23. Sample answer: A blank slate is not objective -- it is simply uninformed. A doctor who ignores disease prevalence, a juror who ignores context, or a scientist who ignores prior research is not being more objective; they are discarding relevant information. Objectivity in the Bayesian sense means making your prior beliefs explicit and updating them honestly in the face of new evidence -- not pretending you have no beliefs. Bias arises not from having priors but from refusing to revise them when the evidence warrants revision.