Chapter 4: Exercises

Comprehension Check

1. List the 9 steps of the Fact-Checker's Toolkit in order. For each step, write one sentence explaining why it matters.

2. Explain the difference between statistical significance and effect size. Why can a result be statistically significant but practically meaningless?

3. What is the hierarchy of evidence described in Step 3? Why is a meta-analysis generally more reliable than a single study?

4. What is the WEIRD problem in psychology research? How does it affect the generalizability of findings?

5. Explain the "too good to be true" test. What are the red flags that suggest a psychology claim has been oversimplified?

Application

6. Apply the full 9-step toolkit to the following claim: "Couples who share household chores equally are 50% less likely to divorce." Write one paragraph for each step. If you can't answer a step, explain what information you would need.

7. Apply the toolkit to: "Meditation rewires your brain."

8. Apply the toolkit to: "Birth order determines your personality — firstborns are leaders, middle children are peacemakers, youngest children are rebels."

9. Find a psychology-related post on social media (TikTok, Instagram, Twitter, LinkedIn — any platform). Screenshot or copy it. Then apply the 9-step toolkit. How many steps can you answer? What does the analysis reveal?

10. Find a self-help book in a bookstore or library. Read the back cover or first chapter. Identify three specific psychology claims the book makes. Apply Steps 1–3 to each claim. Does the book cite original sources? Are they studies, books, or anecdotes?

Critical Thinking

11. The toolkit includes "Who benefits from this claim being true?" (Step 8). Some people argue this is an ad hominem approach — attacking the motives of the claimant rather than the evidence. Is this a valid criticism? How does Step 8 differ from an ad hominem fallacy?

12. Step 9 asks whether a claim is "too good to be true." But some true findings ARE dramatic and surprising (e.g., antibiotics kill bacteria, vaccines prevent disease). How do you distinguish between a genuinely surprising finding and a too-good-to-be-true oversimplification?

13. The toolkit says meta-analyses are more reliable than single studies. But what if a meta-analysis includes studies affected by publication bias? Can you trust a meta-analysis of biased studies? What should you look for?

14. A friend argues: "I don't need a toolkit — I can just trust my gut about which psychology claims are true." Using what you've learned in Chapters 1–4, explain why "gut feeling" is not a reliable evidence evaluation method.

15. The chapter acknowledges that this book is subject to its own toolkit. What would it look like to apply the toolkit to the claims made in this book? What are the strengths and limitations of this book as a source of psychology information?

Fact-Check Portfolio

16. Apply the complete 9-step toolkit to TWO of your 10 selected claims. For each: - Write out the specific claim (Step 1) - Trace the original source (Step 2) - Note whether it's a single study or meta-analysis (Step 3) - Describe the sample (Step 4) - Search for replications (Step 5) - Note any effect sizes you can find (Step 6) - Look for expert commentary (Step 7) - Identify who benefits (Step 8) - Apply the TGTBT test (Step 9) - Assign a preliminary evidence rating: ✅ ⚠️ ❌ or 🔬