Chapter 20: Further Reading

Essential Sources

1. Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin, Bayesian Data Analysis (CRC Press, 3rd edition, 2013)

The definitive graduate-level reference for Bayesian statistics. BDA3 covers everything from basic conjugate models through hierarchical modeling, MCMC, and model checking. Chapters 1-3 cover the foundations treated in this chapter (single-parameter models, multi-parameter models, prior selection) with greater mathematical depth and many more worked examples. Chapter 5 on hierarchical models is the best treatment in any textbook — it is the essential preparation for Chapter 21 of this book. The "Bayesian workflow" philosophy that pervades BDA3 (prior predictive checks, posterior predictive checks, model comparison) is the template for how applied Bayesian analysis should be conducted. The online appendices include R and Stan code for every example.

Reading guidance: Start with Chapters 1-3 for the theory behind this chapter. Then jump to Chapter 5 (hierarchical models) as preparation for Chapter 21. Chapters 10-12 on MCMC provide the computational foundation that Chapter 21 builds upon. If you read only one Bayesian textbook in your career, read this one.

2. Richard McElreath, Statistical Rethinking: A Bayesian Course with Examples in R and Stan (CRC Press, 2nd edition, 2020)

The most pedagogically effective Bayesian textbook ever written. McElreath has a gift for building intuition through simulation, and his emphasis on generative modeling (building the model before fitting it) aligns perfectly with the prior predictive check philosophy in this chapter. The first half (Chapters 1-8) covers linear models, multivariate models, and interactions from a thoroughly Bayesian perspective, with a conversational style that makes complex ideas accessible. The second half covers multilevel models, measurement error, mixture models, and Gaussian processes.

Reading guidance: Start with Chapters 1-4 for a gentler development of the ideas in this chapter. McElreath's treatment of prior selection (Chapter 4) is more extensive and practical than what space permits here. The associated YouTube lecture series (freely available) is among the best statistics teaching on the internet. If BDA3 is the reference you keep on your shelf, Statistical Rethinking is the one you actually read cover to cover.

3. John K. Kruschke, Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan (Academic Press, 2nd edition, 2015)

Subtitled "A Tutorial Introduction with R, JAGS, and Stan," this book is designed for researchers in the social and biomedical sciences who want to adopt Bayesian methods without a heavy mathematical prerequisite. Kruschke's treatment of the Beta-Binomial model (Chapters 5-6) is the most thorough elementary exposition available — he spends nearly 100 pages building the intuition that this chapter covers in a few sections. The book introduces the "ROPE" (Region of Practical Equivalence) framework for Bayesian hypothesis testing, which is a practical alternative to Bayes factors. The "puppies" diagrams (Kruschke's visualizations of prior-to-posterior updating) are memorable and effective.

Reading guidance: This book is most valuable if you found this chapter's pace too fast for the foundational concepts. Chapters 5-6 (Bayes' rule, the Beta-Binomial model) and Chapter 12 (Bayesian approaches to null-value assessment) directly expand on topics from this chapter. The ROPE framework in Chapter 12 offers a practical decision-making tool that complements the Bayes factor approach in Section 20.9.

4. Andrew Gelman, Aki Vehtari, Daniel Simpson, Charles C. Margossian, Bob Carpenter, Yuling Yao, Lauren Kennedy, Jonah Gabry, Paul-Christian Bürkner, and Martin Modrák, "Bayesian Workflow" (arXiv:2011.01808, 2020)

This paper codifies the modern Bayesian workflow as practiced by the Stan development team and the broader applied Bayesian community. The workflow is: (1) pick an initial model, (2) prior predictive simulation, (3) fit the model, (4) validate the computation (MCMC diagnostics), (5) posterior predictive checks, (6) model comparison, (7) iterate. The paper includes practical advice on what to do when things go wrong at each stage — non-convergence, poor posterior predictive checks, sensitivity to priors.

Reading guidance: Read this after finishing Chapter 21. The paper's advice on prior predictive checks (Section 3) and posterior predictive checks (Section 5) directly builds on the foundations from this chapter. The troubleshooting sections are invaluable for applied work — they cover the problems that textbooks often omit.

5. E.T. Jaynes, Probability Theory: The Logic of Science (Cambridge University Press, 2003)

The most philosophically rigorous defense of Bayesian probability as an extension of logic. Jaynes argues — with Cox's theorem as the foundation — that Bayesian inference is not one approach among many but the uniquely consistent method for reasoning under uncertainty. The book is dense, polemical, and occasionally pugilistic, but the core argument in Chapters 1-4 is compelling and beautifully constructed. The "robot" thought experiments (imagining a robot that must reason with uncertain information) strip away the philosophical baggage and reveal Bayesian inference as a set of engineering requirements.

Reading guidance: This is not a practical methods book — it is a foundational text. Read Chapters 1-4 for the logical foundations and the derivation of Bayes' theorem from desiderata of consistent reasoning. Skip Chapters 5-14 unless you are specifically interested in the philosophy of probability. Return to Chapters 15-20 (on model comparison and hypothesis testing) after completing Chapter 21. Jaynes is the strongest possible antidote to the "priors are subjective" objection — he argues that objectivity in inference comes not from avoiding assumptions but from being honest and consistent about them.