Quiz: Field Autopsy — Psychology

Q: What did the Reproducibility Project find? (a) All psychology studies replicate perfectly (b) Only 36% of 100 tested psychology studies replicated successfully with the original effect size (c) Psychology is entirely unreliable (d) 90% of studies replicated

(b) 36% replicated with the original effect size. Even with more generous criteria, fewer than half replicated.

Q: Why was Daryl Bem's precognition paper important for the replication crisis? (a) It proved precognition exists (b) It used the same methods as hundreds of other studies — if those methods could "prove" precognition, the methods themselves were suspect (c) It was the first paper ever rejected by a psychology journal (d) It demonstrated that p-hacking was impossible

(b) Bem's methods were standard social psychology methods. The absurdity of the conclusion forced the field to confront what standard methods could produce.

Q: Which of the following is NOT a questionable research practice (QRP)? (a) P-hacking (b) HARKing (c) Pre-registration (d) The file drawer problem

(c) Pre-registration is a correction to QRPs, not a QRP itself. It requires researchers to specify hypotheses and analyses before data collection.

Q: Why did social psychology experience the worst replication rates among psychology's subfields? (a) Social psychologists are less intelligent than other psychologists (b) Small effects in noisy domains, narrative appeal encouraging flashy findings, low methodological barriers, and celebrity culture (c) Social psychology has the fewest practitioners (d) Social psychology is the newest subfield

(b) Social psychology was structurally vulnerable: the phenomena were subtle and context-dependent, the findings were media-friendly, studies were cheap to run (enabling mass production), and the celebrity culture created personal-brand sunk costs.

Q: What is the WEIRD problem? (a) Psychology studies produce weird results (b) Most psychology research uses participants from Western, Educated, Industrialized, Rich, Democratic societies — approximately 12% of the world's population (c) Researchers select unusual participants (d) Psychology findings are considered strange by non-psychologists

(b) WEIRD populations may be psychological outliers, meaning findings may not generalize to the majority of humanity. This is survivorship bias at population scale.

Q: What is a "registered report"? (a) A report filed with the government before conducting research (b) A journal format where peer review occurs before data collection — the study is accepted based on design, and published regardless of results (c) A database of all registered psychologists (d) A report published only if results are significant

(b) Registered reports eliminate publication bias by design: the study is evaluated on its design, not its results. Over 300 journals now offer this format.

Q: The "meta-lesson" of psychology's crisis is: (a) Psychology should be abolished (b) If the researchers who study cognitive bias exhibited those biases in their own research, no field is immune — awareness does not protect against structural failure modes (c) Only psychologists can fix psychology (d) The replication crisis was exaggerated by the media

(b) This is Theme 7 proven empirically: understanding bias does not protect against it. Only structural safeguards (pre-registration, open data, independent replication) provide protection.

Q: Which psychology subfields emerged relatively unscathed from the replication crisis? (a) Social psychology and personality psychology (b) Perception, psychophysics, learning, and core cognitive psychology (c) Clinical psychology and counseling (d) Developmental psychology

(b) Subfields with high evidence clarity, well-standardized methods, and large robust effects replicated well. The crisis was concentrated in subfields with small, context-dependent effects and flexible methods.

Quiz: Field Autopsy — Psychology

Q1. What did the Reproducibility Project find?

(a) All psychology studies replicate perfectly (b) Only 36% of 100 tested psychology studies replicated successfully with the original effect size (c) Psychology is entirely unreliable (d) 90% of studies replicated

Answer

**(b)** 36% replicated with the original effect size. Even with more generous criteria, fewer than half replicated.

Q2. Why was Daryl Bem's precognition paper important for the replication crisis?

(a) It proved precognition exists (b) It used the same methods as hundreds of other studies — if those methods could "prove" precognition, the methods themselves were suspect (c) It was the first paper ever rejected by a psychology journal (d) It demonstrated that p-hacking was impossible

Answer

**(b)** Bem's methods were standard social psychology methods. The absurdity of the conclusion forced the field to confront what standard methods could produce.

Q3. Which of the following is NOT a questionable research practice (QRP)?

(a) P-hacking (b) HARKing (c) Pre-registration (d) The file drawer problem

Answer

**(c)** Pre-registration is a *correction* to QRPs, not a QRP itself. It requires researchers to specify hypotheses and analyses before data collection.

Q4. Why did social psychology experience the worst replication rates among psychology's subfields?

(a) Social psychologists are less intelligent than other psychologists (b) Small effects in noisy domains, narrative appeal encouraging flashy findings, low methodological barriers, and celebrity culture (c) Social psychology has the fewest practitioners (d) Social psychology is the newest subfield

Answer

**(b)** Social psychology was structurally vulnerable: the phenomena were subtle and context-dependent, the findings were media-friendly, studies were cheap to run (enabling mass production), and the celebrity culture created personal-brand sunk costs.

Q5. What is the WEIRD problem?

(a) Psychology studies produce weird results (b) Most psychology research uses participants from Western, Educated, Industrialized, Rich, Democratic societies — approximately 12% of the world's population (c) Researchers select unusual participants (d) Psychology findings are considered strange by non-psychologists

Answer

**(b)** WEIRD populations may be psychological outliers, meaning findings may not generalize to the majority of humanity. This is survivorship bias at population scale.

Q6. What is a "registered report"?

(a) A report filed with the government before conducting research (b) A journal format where peer review occurs before data collection — the study is accepted based on design, and published regardless of results (c) A database of all registered psychologists (d) A report published only if results are significant

Answer

**(b)** Registered reports eliminate publication bias by design: the study is evaluated on its design, not its results. Over 300 journals now offer this format.

Q7. According to the chapter, why did psychology correct faster and deeper than economics?

(a) Psychologists are more honest (b) Lower switching costs, lower defender power, higher alternative availability, and higher outsider access (c) Psychology had a more severe crisis (d) Psychology receives more government funding for reform

Answer

**(b)** The structural comparison is sharp: psychology had no trillion-dollar industry dependent on old methods (lower switching cost), no external institutional leverage for defenders (lower defender power), ready-made Open Science alternatives (higher alternative availability), and open publication norms allowing junior researchers to publish criticism (higher outsider access).

Q8. The "meta-lesson" of psychology's crisis is:

(a) Psychology should be abolished (b) If the researchers who study cognitive bias exhibited those biases in their own research, no field is immune — awareness does not protect against structural failure modes (c) Only psychologists can fix psychology (d) The replication crisis was exaggerated by the media

Answer

**(b)** This is Theme 7 proven empirically: understanding bias does not protect against it. Only structural safeguards (pre-registration, open data, independent replication) provide protection.

Q9. Which psychology subfields emerged relatively unscathed from the replication crisis?

(a) Social psychology and personality psychology (b) Perception, psychophysics, learning, and core cognitive psychology (c) Clinical psychology and counseling (d) Developmental psychology

Answer

**(b)** Subfields with high evidence clarity, well-standardized methods, and large robust effects replicated well. The crisis was concentrated in subfields with small, context-dependent effects and flexible methods.

Q10. The chapter identifies a risk that Open Science reforms could become "rebound orthodoxy." This means:

(a) The reforms will be reversed (b) The overcorrected position becomes a new consensus defended by the same mechanisms that defended the original flawed practices (c) Researchers will rebel against Open Science (d) The reforms are cosmetic

Answer

**(b)** When pre-registration, large samples, and open data become institutional requirements enforced through peer review and hiring (the same mechanisms that enforced QRPs), they acquire the self-reinforcing properties of any orthodoxy — potentially constraining legitimate research that doesn't fit the reformed template.

Scoring Guide

9-10 correct: Excellent understanding of psychology's crisis and its structural lessons.
7-8 correct: Good. Review sections on the Open Science reforms (25.4) and the overcorrection risk (25.5).
5-6 correct: Fair. Revisit sections on the root causes (25.2) and the Correction Speed Model (25.6).
Below 5: Re-read the chapter focusing on the structural analysis, not just the specific failed studies.