Chapter 3 Exercises

Chapter 3 Exercises

Exercise 1: Methodological Audit of a Published Study (Individual or Pair)

Task: Find a peer-reviewed attraction research paper published within the last five years. Use Google Scholar, your library's PsycINFO access, or JSTOR. Suitable journals include Journal of Personality and Social Psychology, Evolutionary Psychology, Personal Relationships, Archives of Sexual Behavior, and Social Psychological and Personality Science.

Once you have your paper, complete the following methodological checklist. For each item, provide a one-to-two sentence explanation based on what the paper reports.

Checklist: 1. Sample size and composition: How many participants? What are their demographics? Is the sample WEIRD? Is it exclusively students? 2. Research design: Is this experimental, correlational, observational, or survey-based? What causal claims (if any) does this design support? 3. Measurement approach: Does the study use self-report, behavioral, or physiological measures — or a combination? What are the stated limitations of these measures? 4. Effect size reporting: Does the paper report effect sizes (d, r, η², or similar)? If yes, are they small, medium, or large? If no, calculate one using means and standard deviations if they are provided. 5. Statistical significance: Does the paper distinguish statistical significance from practical significance, or does it conflate the two? 6. Pre-registration: Was the study pre-registered? You can check the Open Science Framework (osf.io) or look for a pre-registration statement in the paper. 7. Cultural generalizability: Does the paper make universal claims? Is there a discussion of cultural limits? What populations are not represented? 8. Funding source: Who funded the research? Is there any potential conflict of interest?

Write-up: A 300–500 word critical evaluation of the paper's methodology, identifying its two strongest methodological choices and its two most significant limitations.

Exercise 2: Calculating Cohen's d

Scenario: You are reviewing data from a small experimental study on attraction. Participants were randomly assigned to one of two conditions:

Condition A (Similarity): Participants read a profile describing a potential partner whose stated interests closely matched their own (n = 30, mean attraction rating = 7.4, SD = 1.6).
Condition B (Dissimilarity): Participants read a profile describing a potential partner whose stated interests differed substantially from their own (n = 30, mean attraction rating = 5.9, SD = 1.8).

Part A: Using the formula below, calculate Cohen's d for the difference between conditions.

$$d = \frac{M_1 - M_2}{SD_{pooled}}$$

where the pooled standard deviation is:

$$SD_{pooled} = \sqrt{\frac{(n_1 - 1)SD_1^2 + (n_2 - 1)SD_2^2}{n_1 + n_2 - 2}}$$

Part B: Interpret the effect size using Cohen's conventional benchmarks (small = 0.2, medium = 0.5, large = 0.8). Is this effect practically meaningful? Would you be confident in this result given the sample size?

Part C: The t-test for this comparison yields p = .03. A colleague says: "Great — it's significant, so similarity matters for attraction." Write a two-paragraph response that distinguishes statistical from practical significance and explains what additional information would be needed to evaluate the finding's importance.

Exercise 3: Designing a Cross-Cultural Attraction Study (Group Discussion, 3–4 Students)

Prompt: Imagine you have been awarded funding to design a cross-cultural study of romantic attraction norms in five countries: the United States, Japan, Kenya, Brazil, and Saudi Arabia. Your goal is to avoid the WEIRD bias while still producing comparable data across sites.

Discuss and address: 1. Sampling: How would you recruit participants in each country to achieve meaningful representation across education levels, income, and urban/rural settings? What practical obstacles would you face? 2. Measurement equivalence: How would you assess whether your attraction measures work similarly across countries? What would you do if they didn't? 3. Conceptual translation: The concept of "romantic attraction" itself may not map cleanly across cultural contexts. How would you investigate what attraction means in each culture before designing your instruments? 4. Research team composition: Who should conduct the research in each country? What are the ethical implications of an American research team conducting surveys in Kenya without Kenyan collaborators? 5. IRB complexity: You must obtain ethical review approval in five countries with different research ethics frameworks. What issues might arise, and how would you resolve them?

Deliverable: A 400–600 word design memo summarizing your group's decisions on the five issues above and identifying the most difficult trade-off you faced.

Exercise 4: Python Lab — Running the Chapter Code

Task: Open and run code/attraction_methods_demo.py. Read each section of the code before running it.

Part A: The script generates a synthetic dataset and runs a t-test comparing two experimental conditions. After running the script, answer: What was Cohen's d for the comparison? Is the effect "small," "medium," or "large" by conventional standards? How does the p-value relate to the effect size?

Part B: The script simulates a "file drawer" publication bias scenario. Describe what the simulation shows. How does the "published" effect size (based only on studies with p < .05) compare to the true effect size (based on all 20 simulated studies)?

Part C (Challenge): Modify the simulation to use 50 studies instead of 20, keeping the same true effect size. Does the distortion from publication bias become larger or smaller? Why? Write three sentences explaining your reasoning.

Part D (Discussion): If you were a meta-analyst trying to estimate the true effect of, say, eye contact on attraction ratings, and you suspected publication bias in the literature, what steps could you take to correct for it? Name at least two methods discussed in the chapter.