Chapter 40 Key Takeaways: Critical Thinking About Attraction Research

Meta-Analysis

A meta-analysis aggregates effect sizes from multiple independent studies to produce a more precise overall estimate. It is weighted by study precision (larger, more precise studies count more). Key outputs: the pooled effect size, its confidence interval, and the heterogeneity statistic I².

I² measures how much variation in effect sizes is due to true differences between studies (as opposed to sampling error). High I² (above 50–75%) means the studies are not measuring a single consistent effect — the aggregate should be interpreted with caution.

Forest plots visualize meta-analytic results: one row per study, a square for the effect size estimate sized by study weight, horizontal lines for confidence intervals, a diamond at bottom for the aggregate. Studies whose CI crosses zero did not individually reach significance.

Publication Bias

When null results are not published, meta-analytic estimates are inflated. Funnel plots test for this: an unbiased body of literature should produce a symmetrical funnel; asymmetry (missing studies in the lower-left) suggests the file drawer problem. Egger's test formally tests for funnel asymmetry.

P-Hacking and Researcher Degrees of Freedom

When researchers make multiple analytic choices (sample exclusions, covariate selection, outcome measure selection) and report only the choice that produced p < .05, the false positive rate inflates far beyond the nominal 5%. The simulation in code/meta_analysis_tools.py makes this concrete. Pre-registration — publicly committing to hypotheses, sample size, and analysis plan before data collection — is the structural solution.

Pre-Registration and Open Science

Pre-registered studies provide stronger evidence than unregistered ones because the hypothesis was committed to in advance. Registered Reports (accepted before data collection, published regardless of results) directly address publication bias. Open data and open materials enable verification and exact replication.

The WEIRD Problem at Scale

The Okafor-Reyes Global Attraction Project showed that some findings from the primarily Western literature generalize cross-culturally (warmth/responsiveness predicts attraction universally; contempt universally reduces it), while others do not (gender differences in status-cue preferences vary substantially by country).

Evaluating Press Releases

The seven-step protocol: (1) find the original study, (2) check the sample, (3) find the effect size, (4) check preregistration, (5) read the limitations section, (6) look for replication, (7) check causal language. Science can inform reflection and self-knowledge; it does not prescribe individual action.

The Science Literacy Principle

These skills extend beyond attraction research to all domains where scientific claims are made in public discourse. Calibrated skepticism — believing things in proportion to the evidence — is a civic skill as much as an academic one.