Chapter 6 Quiz: Self-Assessment

Chapter 6 Quiz: Self-Assessment

Instructions: Answer each question without looking back at the chapter. After completing all questions, check your answers against the key at the bottom. If you score below 70%, revisit the relevant sections before moving on to Chapter 7.

Multiple Choice

Q1. In signal detection theory, a false positive (Type I error) occurs when:

a) A real signal is correctly detected b) An observer fails to detect a signal that is present c) An observer reports a signal when none is present d) An observer correctly determines that no signal is present

Q2. The base rate in a signal detection context refers to:

a) The speed at which signals are transmitted b) The prevalence of the condition being tested for in the population c) The minimum strength a signal must have to be detected d) The ratio of true positives to false positives

Q3. A mammography test with 90% sensitivity and 91% specificity is applied to a population where 1% of women have breast cancer. If a woman tests positive, the probability she actually has cancer is approximately:

a) 90% b) 50% c) 9% d) 1%

Q4. The noise floor of a detection system refers to:

a) The maximum signal the system can detect b) The baseline level of noise below which signals cannot be distinguished from background c) The threshold at which the system triggers an alarm d) The proportion of false alarms in the system's output

Q5. An ROC curve plots:

a) Signal strength against noise strength b) Sensitivity against specificity c) True positive rate (hit rate) against false positive rate (false alarm rate) d) Base rate against detection threshold

Q6. According to the chapter, the detection threshold in a signal detection system is best described as:

a) A technical parameter that should be optimized mathematically b) A values question that reflects judgments about which errors are more tolerable c) A constant that is the same across all detection systems d) A measure of detector quality

Q7. Alarm fatigue occurs when:

a) Alarms malfunction due to overuse b) A high rate of false alarms causes operators to ignore or delay responding to all alarms, including real ones c) An alarm system runs out of battery power d) Operators become too skilled at distinguishing real alarms from false ones

Q8. The chapter identifies the human brain's bias toward Type I errors (false positives) as:

a) A design flaw that can be corrected through education b) An evolutionary adaptation that favored detecting threats even at the cost of many false alarms c) A recent development caused by information overload d) A bias unique to untrained observers

Q9. Apophenia is defined as:

a) The inability to detect signals in noisy data b) The perception of meaningful patterns in random, unrelated data c) A type of signal amplification technique d) The tendency to set detection thresholds too high

Q10. Overfitting in a statistical model is analogous to:

a) A detector with too low a noise floor b) A signal that is too strong relative to the noise c) A detector that has learned the noise patterns of its training data and mistakes them for signal d) A detector with perfect sensitivity and perfect specificity

Q11. According to the chapter, the most effective way to improve a detection system is:

a) Lower the detection threshold to catch more signals b) Raise the detection threshold to reduce false alarms c) Reduce the noise floor, increase signal strength, or build a better detector d) Increase the number of observers

Q12. In the criminal justice system, Blackstone's ratio ("better that ten guilty persons escape than that one innocent suffer") reflects a threshold setting that prioritizes:

a) High sensitivity (catching all criminals) b) High specificity (not convicting innocent people) c) Equal weight to false positives and false negatives d) The base rate of criminal activity

Q13. The chapter describes the Three Mile Island nuclear accident as primarily a failure of:

a) Equipment quality b) Operator training c) Signal detection -- operators could not distinguish critical signals from a flood of alarms d) Detection threshold calibration

Q14. When the chapter states "the tradeoff is inescapable," it means:

a) All detection systems are equally good b) For any given detector, improving sensitivity necessarily decreases specificity (and vice versa) c) There is no way to improve detection systems d) The costs of false positives always equal the costs of false negatives

True or False

Q15. Reducing the noise floor of a detection system is equivalent to amplifying the signal relative to the background, even though the signal itself has not changed.

Q16. A detection system with 99% sensitivity and 99% specificity will always produce reliable positive results, regardless of the base rate.

Q17. Eyewitness confidence is a strong predictor of eyewitness accuracy.

Q18. In signal detection theory, the area under the ROC curve (AUC) measures the overall discriminative ability of a detector, with a value of 0.5 representing random chance and 1.0 representing perfect discrimination.

Q19. The choice of detection threshold in a spam filter is a purely technical optimization problem with a single correct answer.

Q20. Aviation's hierarchical alarm system (advisory, caution, warning) represents a multi-threshold signal detection system.

Q21. The six foundational patterns of Part I (substrate independence, feedback loops, emergence, power laws, phase transitions, signal/noise) are independent concepts that do not interact with each other.

Q22. A system near a phase transition has a different effective signal-to-noise ratio than the same system far from a phase transition, even if the raw noise level is the same.

Short Answer

Q23. In two or three sentences, explain why base rate neglect is such a common and consequential error in medical diagnosis.

Q24. Name the four possible outcomes of any signal detection decision. Give an example of each in the context of airport security screening.

Q25. The chapter argues that the scientific method can be understood as "a noise-reduction system for human cognition." Explain what this means in one or two sentences.

Answer Key

Q1. c) An observer reports a signal when none is present.

Q2. b) The prevalence of the condition being tested for in the population.

Q3. c) 9%. With 10,000 women: 100 have cancer, 90 detected (true positives). 9,900 healthy, 891 falsely flagged (false positives). Total positives: 981. Probability of cancer given positive: 90/981 = approximately 9.2%.

Q4. b) The baseline level of noise below which signals cannot be distinguished from background.

Q5. c) True positive rate (hit rate) against false positive rate (false alarm rate).

Q6. b) A values question that reflects judgments about which errors are more tolerable.

Q7. b) A high rate of false alarms causes operators to ignore or delay responding to all alarms, including real ones.

Q8. b) An evolutionary adaptation that favored detecting threats even at the cost of many false alarms.

Q9. b) The perception of meaningful patterns in random, unrelated data.

Q10. c) A detector that has learned the noise patterns of its training data and mistakes them for signal.

Q11. c) Reduce the noise floor, increase signal strength, or build a better detector.

Q12. b) High specificity (not convicting innocent people).

Q13. c) Signal detection -- operators could not distinguish critical signals from a flood of alarms.

Q14. b) For any given detector, improving sensitivity necessarily decreases specificity (and vice versa).

Q15. True. The signal-to-noise ratio improves when the noise floor drops, even without any change to the signal itself.

Q16. False. When the base rate is very low, even a test with 99% sensitivity and 99% specificity will produce more false positives than true positives.

Q17. False. Research consistently shows that eyewitness confidence is poorly correlated with accuracy.

Q18. True. AUC of 0.5 means the detector performs no better than random, while AUC of 1.0 means perfect discrimination.

Q19. False. The threshold reflects values about which errors are more tolerable (missing an important email vs. seeing spam), not just technical performance.

Q20. True. Different severity levels correspond to different detection thresholds, with the most intrusive alarms reserved for the most dangerous conditions.

Q21. False. The six foundational patterns deeply interact: feedback loops require signal detection, emergence generates noise, power laws shape noise distributions, and phase transitions change signal-to-noise ratios.

Q22. True. Near a phase transition, critical fluctuations occur that may be signal (precursors to a state change) rather than noise, changing the effective signal-to-noise ratio.

Q23. Base rate neglect is common because people tend to focus on the test's accuracy (sensitivity and specificity) while ignoring the prevalence of the condition in the population. When a condition is rare, even a highly accurate test produces far more false positives than true positives, because the error rate applies to the much larger healthy population. Physicians and patients alike overestimate the probability of disease given a positive test, leading to unnecessary treatment and anxiety.

Q24. The four outcomes: Hit (correctly identifying a real threat -- detecting an actual weapon in a bag). Miss (failing to detect a real threat -- a weapon passes through undetected). False alarm (flagging something harmless as a threat -- pulling aside a bag because a water bottle looked suspicious on the X-ray). Correct rejection (correctly allowing a harmless bag through -- a bag with no prohibited items passes screening without being flagged).

Q25. The human brain is biased toward seeing patterns in noise (apophenia, Type I errors). The scientific method -- with its emphasis on controlled experiments, blinding, statistical testing, peer review, and replication -- provides external systems that compensate for this built-in bias, filtering out false patterns and retaining only those that survive rigorous testing.

Scoring Guide

18-25 correct: Excellent. You have a strong grasp of signal detection concepts. Proceed to Chapter 7.
14-17 correct: Good. Review the sections corresponding to any missed questions before proceeding.
10-13 correct: Revisit the chapter, focusing especially on Section 6.7 (the SDT framework) and Section 6.3 (base rate neglect).
Below 10: Re-read the chapter carefully before moving on. Consider working through the Part A exercises as well.