Further Reading: Confidence Intervals: Estimating with Uncertainty

Books

For Deeper Understanding

Charles Wheelan, Naked Statistics: Stripping the Dread from the Data (2013) Wheelan's chapter on confidence intervals is one of the clearest popular-audience explanations available. He builds from polling data to medical studies, always emphasizing the "repeated sampling" interpretation that students find most difficult. His analogy of confidence intervals as "horseshoes" (close counts) is memorable and accurate. If the threshold concept in Section 12.3 still feels slippery, Wheelan's treatment will help solidify it.

Nate Silver, The Signal and the Noise: Why So Many Predictions Fail — but Some Don't (2012) Silver's discussion of polling methodology in Chapters 2 and 3 is the definitive popular account of margins of error in practice. He explains not just the statistical theory but the practical challenges: nonresponse bias, likely voter models, and the correlation between state-level polling errors that made 2012 and 2016 results surprising. Directly relevant to Case Study 2 and to anyone who wants to understand why "within the margin of error" doesn't mean "coin flip."

David Freedman, Robert Pisani, and Roger Purves, Statistics, 4th edition (2007) Chapter 21 provides perhaps the most careful textbook treatment of what confidence intervals mean and don't mean. Freedman was famous for his insistence on precise interpretation, and this chapter reflects that — every sentence is carefully worded to avoid the common "95% probability" misstatement. If you're struggling with the interpretation, Freedman's treatment is the gold standard.

Leonard Mlodinow, The Drunkard's Walk: How Randomness Rules Our Lives (2008) Previously recommended for the CLT (Chapter 11). Mlodinow's discussion of uncertainty in medical testing and legal contexts illustrates why confidence intervals matter in high-stakes decisions. His examples of courts misinterpreting statistical evidence connect to Professor Washington's work and to the ethical dimensions of uncertainty.

For the Mathematically Curious

George Casella and Roger Berger, Statistical Inference, 2nd edition (2002) Sections 9.2-9.3 provide the rigorous mathematical theory of confidence intervals, including pivotal quantities, the relationship between CIs and hypothesis tests, and the optimality properties of different interval procedures. Chapter 9 also covers the Bayesian approach to interval estimation (credible intervals), providing a formal comparison to the frequentist CIs in this chapter.

Larry Wasserman, All of Statistics: A Concise Course in Statistical Inference (2004) Chapter 6 covers confidence intervals with mathematical precision and conciseness. Wasserman's treatment of the relationship between confidence intervals and inverting hypothesis tests is particularly elegant. His discussion of the "bootstrap confidence interval" (Chapter 8) previews the simulation-based approach you'll encounter in Chapter 18.

Articles and Papers

Cumming, Geoff (2014). "The New Statistics: Why and How." Psychological Science, 25(1), 7-29. A landmark article arguing that psychology (and science more broadly) should move away from p-values and toward confidence intervals and effect sizes. Cumming demonstrates how CIs provide more information than binary "significant/not significant" decisions. This article influenced the American Statistical Association's 2016 statement on p-values and is directly relevant to the themes of Chapters 12-17.

Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E. J. (2014). "Robust misinterpretation of confidence intervals." Psychonomic Bulletin & Review, 21(5), 1157-1164. A research study showing that even statistics professors frequently misinterpret confidence intervals. The authors gave six statements about a CI (similar to the table in Section 12.3) and found that the majority of researchers, students, and professors endorsed incorrect interpretations. This paper demonstrates just how tricky the "95% confidence" concept is — and validates the extensive attention this chapter devotes to correct interpretation.

Brown, L. D., Cai, T. T., & DasGupta, A. (2001). "Interval estimation for a binomial proportion." Statistical Science, 16(2), 101-133. The definitive technical reference on confidence intervals for proportions. The authors show that the standard Wald interval ($\hat{p} \pm z^*\sqrt{\hat{p}(1-\hat{p})/n}$) — the one taught in this chapter — has poor coverage properties for small samples and extreme proportions. They recommend the Wilson interval as a better alternative. This is the paper behind the method='wilson' option in the statsmodels code in Section 12.7.

Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E. J. (2016). "The fallacy of placing confidence in confidence intervals." Psychonomic Bulletin & Review, 23(1), 103-123. A thought-provoking paper that argues even the "repeated sampling" interpretation of CIs is more nuanced than most textbooks acknowledge. The authors explore subtle cases where the repeated sampling interpretation breaks down and argue for Bayesian credible intervals as a more natural alternative. Advanced reading, but valuable for students who want to understand the philosophical foundations of inference.

Agresti, A., & Coull, B. A. (1998). "Approximate is better than 'exact' for interval estimation of binomial proportions." The American Statistician, 52(2), 119-126. Shows that the "exact" Clopper-Pearson confidence interval for proportions is unnecessarily conservative, and that the simpler Agresti-Coull (or Wilson) interval has better properties. A readable paper that demonstrates why seemingly simple statistical problems can have subtle solutions.

Online Resources

Interactive Tools

Seeing Theory — Frequentist Inference https://seeing-theory.brown.edu/frequentist-inference/ Brown University's beautiful interactive visualization. The "Confidence Interval" module lets you set the confidence level, draw samples, and watch confidence intervals appear — with misses highlighted in red. Spend 10 minutes here to internalize the repeated sampling interpretation. You can see, visually, that about 5% of 95% CIs miss the target. Previewed in Chapter 11's further reading; now you're ready to use it fully.

StatKey: Confidence Interval Simulation http://www.lock5stat.com/StatKey/ From the authors of Statistics: Unlocking the Power of Data. StatKey's confidence interval module lets you build CIs from real datasets, adjust the confidence level, and see the tradeoff between width and coverage. Also includes a bootstrap CI module (preview of Chapter 18). Web-based, no installation needed.

OnlineStatBook: Confidence Interval Simulation https://onlinestatbook.com/stat_sim/conf_interval/index.html An interactive simulation that constructs multiple confidence intervals and shows how many capture the true parameter. You can adjust the sample size, confidence level, and population shape to explore how these factors affect coverage. Particularly useful for understanding why small samples from skewed populations sometimes have actual coverage below the nominal level.

Wolfram Alpha: Confidence Interval Calculator https://www.wolframalpha.com/ Type "confidence interval for mean, xbar=128.3, s=18.6, n=120" and Wolfram Alpha returns the CI with full work shown. Useful for checking your hand calculations.

Video Resources

StatQuest with Josh Starmer: "Confidence Intervals" (YouTube) Josh Starmer's clear, step-by-step explanation of confidence intervals, with his signature directness and enthusiasm. He covers the construction, interpretation, and common misconceptions in about 15 minutes. He also has separate videos on the t-distribution and margin of error that complement Sections 12.4 and 12.8. Previewed in Chapter 11's further reading.

3Blue1Brown: "Why 'probability of 0' doesn't mean impossible" (YouTube) While not directly about confidence intervals, Grant Sanderson's treatment of continuous probability and the difference between probability and density is relevant to understanding why we can't assign a probability to a fixed parameter being in a specific interval. The distinction between "process probability" and "outcome probability" that runs through this chapter connects to Sanderson's broader philosophy of what probability means.

Khan Academy: "Confidence Intervals" (YouTube/khanacademy.org) Sal Khan provides a thorough walkthrough of CI construction for means and proportions, with multiple worked examples. Particularly good for students who want more practice with the formulas. His series covers the t-distribution, degrees of freedom, and sample size determination across several videos.

jbstatistics: "Introduction to Confidence Intervals" (YouTube) Jeremy Balka's concise, focused explanation covers the CI concept in about 12 minutes. His channel has separate videos on CIs for means, CIs for proportions, the t-distribution, and sample size determination — each targeted and efficient.

CrashCourse Statistics: "Confidence Intervals" (YouTube) A faster-paced, more visual introduction to CIs. Good for a quick review or as a complement to the textbook. About 10 minutes.

Software Documentation

SciPy: scipy.stats.t https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.t.html Documentation for the t-distribution functions used in this chapter. Key methods: .interval() for confidence intervals, .ppf() for critical values, .cdf() for cumulative probabilities. The interval() function is the fastest way to compute a CI in Python.

SciPy: scipy.stats.norm https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.norm.html The normal distribution functions used for proportion CIs. Key methods: .ppf(0.975) returns $z^* = 1.960$ for 95% confidence.

SciPy: scipy.stats.sem https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.sem.html Computes the standard error of the mean ($s/\sqrt{n}$) directly from an array. Convenient shortcut for the scale parameter of t.interval().

pandas: DataFrame.sem() https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sem.html Pandas' built-in standard error of the mean method. Equivalent to scipy.stats.sem() but available directly on DataFrames and Series.

statsmodels: proportion_confint https://www.statsmodels.org/stable/generated/statsmodels.stats.proportion.proportion_confint.html The proportion_confint() function computes confidence intervals for proportions using multiple methods: 'normal' (Wald, the method in this chapter), 'wilson' (recommended for small samples or extreme proportions), 'agresti_coull' (another improved method), and 'beta' (exact Clopper-Pearson). The Wilson interval is generally preferred for proportions — it has better coverage properties than the Wald interval, especially when $n$ is small or $\hat{p}$ is near 0 or 1.

Excel: CONFIDENCE.T() and CONFIDENCE.NORM() Microsoft's documentation for the confidence interval functions: - CONFIDENCE.T(alpha, standard_dev, size) — returns the margin of error using the t-distribution - CONFIDENCE.NORM(alpha, standard_dev, size) — returns the margin of error using the normal distribution - T.INV.2T(probability, deg_freedom) — returns the two-tailed t critical value - NORM.S.INV(probability) — returns the z critical value

Historical Notes

The People Behind the Confidence Interval

Jerzy Neyman (1894-1981) developed the modern theory of confidence intervals in a groundbreaking 1937 paper. Born in Poland, educated in Russia, and working in England, Neyman was the first to clearly articulate the repeated sampling interpretation — that "confidence" refers to the long-run performance of the method, not to any single interval. His formulation was controversial at the time; Ronald Fisher, his contemporary and rival, dismissed confidence intervals in favor of his own "fiducial" approach, leading to one of the most bitter feuds in the history of statistics.

Neyman's key insight was to shift the question from "What is the probability that this interval is correct?" to "What is the probability that my method produces a correct interval?" This subtle shift — from the specific to the general, from one outcome to the procedure — is exactly the threshold concept in Section 12.3.

William Sealy Gosset (1876-1937), publishing as "Student," developed the t-distribution in 1908 while working as a chemist at the Guinness brewery in Dublin. Gosset was trying to solve a practical problem: with small samples of barley, how could he determine whether one variety produced more extract than another? He realized that the standard normal distribution wasn't accurate for small samples and derived the exact distribution of $(\bar{x} - \mu) / (s/\sqrt{n})$ — what we now call the t-distribution. Ronald Fisher later refined the theory and introduced the concept of degrees of freedom.

Ronald Fisher (1890-1962) formalized the t-distribution theory and contributed the concept of degrees of freedom. Despite his dispute with Neyman over the interpretation of intervals, Fisher's work on the t-distribution and small-sample inference was essential to the practical application of confidence intervals.

The term "confidence interval" was coined by Neyman in his 1937 paper. Before that, statisticians used various informal methods to express uncertainty about estimates, but there was no rigorous framework. Neyman's formulation provided the mathematical foundation that made modern statistical inference possible.

What's Coming Next

Chapter 13 will introduce hypothesis testing — the other major pillar of statistical inference. You'll learn to:

• Formulate null and alternative hypotheses
• Calculate test statistics and p-values
• Make decisions based on statistical evidence
• Understand the deep connection between confidence intervals and hypothesis tests

Here's a preview of that connection: a 95% confidence interval contains exactly those values of the parameter that would not be rejected by a two-sided hypothesis test at the 5% significance level. In other words, the CI is the set of "plausible" values for the parameter — and the hypothesis test determines whether a specific claimed value is plausible. The two tools are two views of the same underlying mathematics.

Resources to preview: - StatQuest: "Hypothesis Testing" (YouTube) — Josh Starmer's explanation of the logic behind hypothesis tests - Seeing Theory — Hypothesis Testing (https://seeing-theory.brown.edu/frequentist-inference/) — interactive visualization of p-values and rejection regions - Wheelan, Naked Statistics, Chapter 9 — accessible introduction to hypothesis testing with real-world examples