Chapter 4: Quiz
1. What is Step 1 of the Fact-Checker's Toolkit?
- A) Find the original study
- B) Check whether it's been replicated
- C) Pin down the specific claim being made
- D) Calculate the effect size
Answer: C. The first step is always to restate the claim as specifically as possible. Vague claims can't be evaluated.
2. A study finds that people who eat chocolate are 3% less likely to report feeling sad (p = .04, N = 15,000). This result is:
- A) Statistically significant AND practically important
- B) Statistically significant but with a very small effect size
- C) Not statistically significant
- D) A meta-analysis
Answer: B. With N = 15,000, even a tiny difference of 3% can reach statistical significance. The result is "real" in the statistical sense but practically trivial.
3. In the hierarchy of evidence, which source carries the MOST weight?
- A) A single large study
- B) A meta-analysis with publication bias corrections
- C) An anecdote from a famous psychologist
- D) A popular self-help book
Answer: B. Meta-analyses that account for publication bias provide the most reliable estimates of effect sizes because they combine multiple studies and correct for the file drawer problem.
4. The WEIRD problem refers to:
- A) Unusual findings that contradict common sense
- B) The overrepresentation of Western, Educated, Industrialized, Rich, and Democratic populations in psychology research
- C) Researchers who study unusual psychological phenomena
- D) The difficulty of replicating findings from one country in another
Answer: B. Henrich et al. (2010) found that 96% of psychology participants came from just 12% of the world's population, raising serious questions about generalizability.
5. A claim passes the "too good to be true" test if:
- A) It promises dramatic results from simple interventions
- B) It includes appropriate caveats and limitations
- C) It uses universal language like "everyone" and "always"
- D) It sounds like a perfect headline
Answer: B. Claims that include caveats, limitations, and conditional language ("for some people," "in some conditions") are more likely to be accurate than clean, dramatic, universal claims.
6. Step 8 of the toolkit asks "Who benefits from this claim being true?" This step is important because:
- A) Profitable claims are always false
- B) Understanding incentive structures helps you evaluate claims more clearly, even though profit doesn't equal falsehood
- C) Researchers never have financial interests
- D) Only the self-help industry has incentives to distort
Answer: B. Incentive awareness adds context, not cynicism. A profitable claim may still be true (exercise is genuinely beneficial AND profitable for the fitness industry), but profit motives warrant extra scrutiny.
7. Cohen's d = 0.2 is conventionally considered:
- A) A large effect
- B) A medium effect
- C) A small effect
- D) Not statistically significant
Answer: C. By convention: d = 0.2 is small, d = 0.5 is medium, d = 0.8 is large. Most psychology effects are in the small to medium range.
8. What should you conclude if a psychology claim has never been tested for replication?
- A) The claim is probably false
- B) The claim is probably true since nobody has disproved it
- C) The claim should be treated as preliminary — neither confirmed nor debunked
- D) The claim is based on a meta-analysis
Answer: C. An unreplicated finding is an untested finding. It may be true or false. The appropriate response is to treat it as preliminary and maintain uncertainty.
9. When applying the toolkit to the claim "people are left-brained or right-brained," the key finding is:
- A) The claim is supported by Sperry's Nobel Prize research
- B) fMRI studies find no evidence of hemispheric dominance in personality
- C) The claim is unresolved — experts disagree
- D) The effect size is small but real
Answer: B. Nielsen et al. (2013), using fMRI data from 1,011 participants, found no evidence that individuals are "left-brained" or "right-brained." The claim is debunked.
10. The toolkit commits to being applied to this book itself. Why is this important?
- A) It proves the book is always correct
- B) It demonstrates intellectual honesty and models the critical thinking skill the book teaches
- C) It is required by academic publishing standards
- D) It allows readers to sue the authors if they find errors
Answer: B. The toolkit's application to the book itself demonstrates that no source is exempt from scrutiny. This models the critical thinking skill and reinforces that the goal is evidence evaluation, not authority deference.