Further Reading: Chapter 31 — Marketing Analytics and Campaign Analysis

Books

"Hacking Growth" by Sean Ellis and Morgan Brown The definitive text on growth hacking and experimentation culture. Ellis coined the term "product-market fit" and founded GrowthHackers. The book covers A/B testing philosophy, channel prioritization, and building a data-driven growth function from scratch. Particularly strong on the organizational side — what it takes for experimentation to become a muscle, not a one-off project.

"Marketing Metrics: The Manager's Guide to Measuring Marketing Performance" by Paul W. Farris et al. A comprehensive reference for every marketing metric you will encounter in practice. Less narrative, more reference manual — exactly what you want when a CMO throws a term at you that was not in this chapter. Covers over 50 metrics with precise definitions, calculation methods, and interpretation guidance.

"Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing" by Ron Kohavi, Diane Tang, and Ya Xu Written by the engineers who built A/B testing systems at Microsoft, Google, and LinkedIn. This is the authoritative technical reference on experimentation at scale. Parts of it assume substantial statistical background, but Chapters 1–5 are accessible and cover the core principles — including an excellent treatment of the peeking problem and novelty effects — in detail that this chapter only introduced.

"Data-Driven Marketing: The 15 Metrics Everyone in Marketing Should Know" by Mark Jeffery Based on research from the Kellogg School of Management. Bridges the gap between marketing theory and measurement practice. Each chapter focuses on one metric, explains the business context, and works through a case study. Accessible for marketing professionals without deep statistical background.

"Conversion Optimization: The Art and Science of Converting Prospects to Customers" by Khalid Saleh and Ayat Shukairy Focused specifically on funnel analysis and conversion rate optimization. Covers user psychology, A/B testing methodology, and how to design experiments that produce actionable results. The authors founded Invesp, a CRO consultancy, and the book reflects hard-won practical experience.

Articles and Papers

"Online Experiments at Microsoft" — Kohavi et al., KDD 2007 The paper that launched the modern era of large-scale online experimentation. Readable for non-statisticians in its first half; it describes the infrastructure and cultural challenges as much as the statistics. Available free online.

"Seven Rules of Thumb for Web Site Experimenters" — Kohavi and Thomke, Harvard Business Review A condensed, accessible version of the experimentation principles from the book above. Excellent for sharing with non-technical colleagues who need to understand why A/B testing requires rigor.

"How Not to Run an A/B Test" — Evan Miller A widely-cited blog post that provides an intuitive simulation-based demonstration of the peeking problem. The interactive charts make the false positive inflation visceral in a way that statistical notation alone does not. Available at evanmiller.org.

"Is Marketing Attribution Broken?" — Tim Wilson An honest, practitioner-written examination of why marketing attribution is harder than vendors admit and what you can actually rely on. Available on the Analytics Demystified blog.

Online Tools and Calculators

Evan Miller's Sample Size Calculator (evanmiller.org/ab-testing/sample-size.html) An interactive tool for calculating A/B test sample sizes. Useful for cross-checking your Python calculations and for quick estimates in meetings.

Stan Shaw's Statistical Significance Calculator (checkmarket.com/sample-size-calculator/) Covers sample size calculation for both proportion tests (conversion rates) and continuous tests (average values). Includes a "minimum detectable effect" input that most generic calculators miss.

Google's Test My Site (web.dev/measure/) Not statistical in nature, but provides funnel context: page load speed directly correlates with funnel drop-off rates. If your "Product View → Add to Cart" step is underperforming versus benchmark, mobile page speed is one of the first things to check.

Python Libraries to Explore Further

pymc — Bayesian statistical modeling. For teams that want to move beyond frequentist p-values toward probability distributions over conversion rate lifts. The Bayesian approach naturally solves the peeking problem by treating posterior distributions rather than point estimates.

statsmodels — Python's most comprehensive statistical modeling library. Includes robust implementations of power analysis, proportion tests, and a range of regression models relevant to media mix modeling.

scipy.stats — You used this in the chapter. It is worth exploring the full documentation. The norm, chi2_contingency, ttest_ind, and mannwhitneyu functions cover the majority of statistical tests you will need for marketing work.

lifetimes — A Python library specifically for customer lifetime value modeling (BG/NBD and Gamma-Gamma models). When your LTV estimates need to be more sophisticated than AOV × Frequency × Lifespan, this library implements the academic-standard probabilistic models.

pyjanitor — A pandas extension that provides cleaner, more readable data transformation syntax. Particularly useful for the ETL work that precedes most marketing analytics (cleaning UTM parameters, standardizing channel names, etc.).

Courses and Certifications

Google Analytics 4 Certification (skillshop.google.com) Free. Covers the GA4 data model, UTM tracking, conversion events, and reporting. If you are working with web analytics data, this certification will make the data structures you encounter much more legible.

Udacity's A/B Testing course (free on YouTube and Udacity) A full course on A/B testing from Google engineers. Covers experimental design, statistical inference, and ethical considerations in experimentation. One of the best free resources on this specific topic.

Coursera: Marketing Analytics — University of Virginia Part of the Business Analytics Specialization. Covers customer lifetime value, attribution, and market mix modeling with hands-on exercises. More academic than this chapter but provides the theoretical grounding behind the practical tools.

Going Deeper: Media Mix Modeling

This chapter introduced budget allocation modeling at a simplified level. If your organization spends significant sums across multiple channels and wants a more rigorous approach, explore:

• Meridian (Google's open-source MMM library, released 2024): Bayesian media mix modeling with uncertainty quantification.
• Robyn (Meta's open-source MMM library): Ridge regression-based MMM with automated hyperparameter optimization.
• "Introduction to Media Mix Modeling" by Jin et al. (Google Research blog, 2017): The paper that forms the conceptual basis for most modern open-source MMM tools. Available on the Google Research blog.

MMM requires considerably more data (typically 2+ years of weekly data) and statistical sophistication than the tools in this chapter, but it provides the most defensible answer to the question "how much should we spend on each channel?"