Chapter 3: Quiz
1. What made Daryl Bem's 2011 precognition paper significant for psychology as a field?
- A) It proved that precognition is real
- B) It demonstrated that standard psychological methods could produce evidence for impossible phenomena
- C) It was rejected by every journal that reviewed it
- D) It was later replicated by multiple independent labs
Answer: B. Bem's paper used standard methods (conventional statistics, p < .05 threshold, university participant pool) to find "evidence" for precognition. This demonstrated that the methods themselves were flawed, not that precognition was real.
2. The Open Science Collaboration (2015) attempted to replicate 100 psychology studies. What percentage produced statistically significant results in the same direction?
- A) 12%
- B) 36%
- C) 64%
- D) 89%
Answer: B. Only 36% of replicated studies produced significant results in the same direction. The average effect size was also about half the size reported in the originals.
3. P-hacking refers to:
- A) Hacking into psychology journal databases
- B) Analyzing data in multiple ways until a significant result is found, then reporting only that result
- C) Deliberately fabricating data to achieve p < .05
- D) Using a p-value threshold that is too lenient
Answer: B. P-hacking is the (often unintentional) practice of exploiting analytical flexibility to find significant results. It includes flexible stopping rules, flexible measures, and selective reporting.
4. HARKing stands for:
- A) Hypothesis Alignment and Research Knowledgebase
- B) Hypothesizing After the Results are Known
- C) Honest Analytical Research Knowledge
- D) High-Accuracy Replication Knowledge
Answer: B. HARKing means forming hypotheses after seeing the data, then writing the paper as if the hypotheses were pre-specified. It turns exploratory findings into seemingly confirmatory ones.
5. What happened when a pre-registered replication of ego depletion was conducted with 23 labs and over 2,000 participants?
- A) The effect replicated strongly
- B) The effect was found only in half the labs
- C) Essentially no ego depletion effect was found
- D) The effect was found to be twice as large as originally reported
Answer: C. The large-scale pre-registered replication found essentially no evidence for the ego depletion effect, undermining one of social psychology's most influential findings.
6. Publication bias means that:
- A) Only well-designed studies get published
- B) Journals preferentially publish significant results, creating a biased view of the evidence
- C) All published studies are fraudulent
- D) Researchers deliberately hide their best work
Answer: B. Publication bias is the systematic preference for statistically significant results, which means null findings go unpublished. The published literature overrepresents positive findings relative to the full body of research conducted.
7. The "winner's curse" in small-sample studies means:
- A) The researcher who publishes first gets all the credit
- B) Significant results from small studies tend to overestimate the true effect size
- C) Small samples always produce false results
- D) Only the largest study in a field is worth reading
Answer: B. In low-powered studies, a true effect can only reach significance if the observed effect is inflated by chance variation. This means published effect sizes from small studies are systematically larger than the real effect.
8. Pre-registration addresses which specific problem?
- A) Publication bias
- B) Small sample sizes
- C) P-hacking and HARKing (by committing to hypotheses and analyses before data collection)
- D) The Barnum effect
Answer: C. Pre-registration commits researchers to specific hypotheses and analysis plans before collecting data, eliminating the flexibility that enables p-hacking and HARKing.
9. Which of the following best describes the current status of the Stanford Prison Experiment?
- A) Fully replicated and confirmed
- B) Methodologically compromised: demand effects, selective behavior, small sample, and recently revealed archival problems
- C) Retracted from the published literature
- D) Confirmed by the Abu Ghraib prison abuses
Answer: B. The SPE has been criticized for researcher demand effects (Zimbardo encouraged guard aggression), selective participant behavior (not all guards became cruel), a tiny sample (N=24), and problems revealed by Le Texier's (2018) archival research. It has not been retracted but is widely considered methodologically compromised.
10. The chapter argues that the replication crisis makes psychology more trustworthy because:
- A) All pre-crisis findings have been debunked
- B) The field is now implementing reforms (pre-registration, open data, Registered Reports) that make new findings more reliable
- C) Only post-2015 studies should ever be cited
- D) The crisis proved that psychology was always a pseudoscience
Answer: B. The reforms catalyzed by the crisis — pre-registration, Registered Reports, open data, larger samples, multi-lab replications — have made the field more methodologically rigorous than it was before. A science that checks and corrects its work is more trustworthy than one that doesn't.