Key Takeaways: Analysis of Variance (ANOVA)

One-Sentence Summary

Analysis of variance (ANOVA) compares means across three or more groups by decomposing total data variability into between-group (explained) and within-group (unexplained) components, using the F-statistic ratio to determine whether group differences exceed what random variation alone would produce — followed by Tukey's HSD post-hoc tests to identify which specific groups differ while controlling the family-wise error rate.

Core Concepts at a Glance

The ANOVA Procedure

Step by Step

1. State hypotheses: - $H_0: \mu_1 = \mu_2 = \cdots = \mu_k$ (all group means equal) - $H_a$: Not all $\mu_i$ are equal (at least one group differs)

2. Check assumptions: - Independence (study design) - Normality within each group (histograms, QQ-plots, Shapiro-Wilk) - Equal variances (Levene's test, SD ratio $< 2$)

3. Compute the ANOVA table:

4. Find the p-value from the F-distribution with $df_1 = k-1$ and $df_2 = N-k$

5. Compute effect size: $\eta^2 = SS_B / SS_T$

6. If significant: run Tukey's HSD for pairwise comparisons

7. Interpret in context with descriptive statistics, F-statistic, p-value, effect size, and post-hoc results

Key Python Code

from scipy import stats
from statsmodels.stats.multicomp import pairwise_tukeyhsd
import numpy as np

# One-way ANOVA
F_stat, p_value = stats.f_oneway(group1, group2, group3)

# Check equal variances
stat, p_levene = stats.levene(group1, group2, group3)

# Effect size (eta-squared) — manual calculation
all_data = np.concatenate([group1, group2, group3])
grand_mean = np.mean(all_data)
ss_between = sum(len(g) * (np.mean(g) - grand_mean)**2
                 for g in [group1, group2, group3])
ss_total = np.sum((all_data - grand_mean)**2)
eta_squared = ss_between / ss_total

# Post-hoc: Tukey's HSD
data = np.concatenate([group1, group2, group3])
labels = ['G1']*len(group1) + ['G2']*len(group2) + ['G3']*len(group3)
tukey = pairwise_tukeyhsd(endog=data, groups=labels, alpha=0.05)
print(tukey)

Excel: Data Analysis ToolPak

1. Data tab → Data Analysis → Anova: Single Factor
2. Set Input Range to all data columns
3. Grouped By: Columns
4. Output includes ANOVA table with SS, df, MS, F, p-value, and $F_{\text{critical}}$

The Threshold Concept: Decomposing Variability

Total variation = Explained variation + Unexplained variation

$$SS_T = SS_B + SS_W$$

This is not just an ANOVA formula. It's a universal principle:

Getting this concept — really getting it — prepares you for regression, multiple regression, and the $R^2$ interpretation in Chapters 22-23.

The Multiple Comparisons Problem

ANOVA solves this by testing all groups in a single test with a single p-value, keeping the Type I error rate at exactly $\alpha$.

Key Formulas

Effect Size Benchmarks (Cohen, 1988)

Always interpret effect sizes in the context of your field — these benchmarks are starting points, not absolute standards.

Post-Hoc Tests: When and How

Assumptions Checklist

Common Mistakes

Reporting Template (APA Style)

"A one-way ANOVA revealed a statistically significant difference in [outcome variable] across the [k] [grouping variable] groups, $F(df_B, df_W) = [F\text{-value}]$, $p [= \text{or} <] [p\text{-value}]$, $\eta^2 = [value]$. Tukey's HSD post-hoc comparisons indicated that [specific group differences with means and adjusted p-values]."

Connections