Chapter 26: Further Reading

Backtesting Methodology

  • De Prado, M. L. (2018). Advances in Financial Machine Learning. Wiley. The single most important book for quantitative backtesting. Chapters on combinatorial purged cross-validation, the deflated Sharpe ratio, and backtesting pitfalls are essential reading. De Prado's framework for detecting false discoveries in backtesting is directly applicable to prediction market strategy evaluation.

  • De Prado, M. L. (2020). Machine Learning for Asset Managers. Cambridge University Press. A concise treatment of ML methods specifically designed for investment applications. Covers feature importance with substitution effects, portfolio optimization with ML inputs, and the dangers of backtest overfitting. More accessible than Advances but equally rigorous.

  • Bailey, D. H., Borwein, J. M., Lopez de Prado, M., & Zhu, Q. J. (2014). "The Probability of Backtest Overfitting." Journal of Computational Finance, 20(4). Introduces the probability of backtest overfitting (PBO) metric and the minimum backtest length (MBL) formula. Shows mathematically how the number of strategy trials inflates the likelihood of finding a spuriously profitable strategy. Essential for anyone who optimizes parameters on historical data.

  • Harvey, C. R., Liu, Y., & Zhu, H. (2016). "...and the Cross-Section of Expected Returns." Review of Financial Studies, 29(1), 5-68. Documents the multiple testing problem in financial research: with hundreds of factors tested, many "discoveries" are false. Proposes adjusting the significance threshold based on the number of tests conducted. Directly relevant to prediction market strategy research.

Event-Driven Backtesting

  • Halls-Moore, M. (2015). Successful Algorithmic Trading. QuantStart. Practical guide to building event-driven backtesting systems in Python. Covers the DataHandler-Strategy-Portfolio-ExecutionHandler architecture used in this chapter. Includes concrete code examples and discusses the transition from backtest to live trading.

  • Chan, E. P. (2013). Algorithmic Trading: Winning Strategies and Their Rationale. Wiley. Covers mean reversion and momentum strategies with detailed backtesting methodology. Particularly good on transaction cost modeling and the impact of realistic execution assumptions on strategy viability.

  • Chan, E. P. (2009). Quantitative Trading: How to Build Your Own Algorithmic Trading Business. Wiley. The precursor to Algorithmic Trading, focused on practical implementation. Covers data management, strategy evaluation, and the transition from research to production. Good introductory material before tackling De Prado's more advanced work.

Transaction Costs and Market Microstructure

  • Kissell, R., & Glantz, M. (2003). Optimal Trading Strategies: Quantitative Approaches for Managing Market Impact and Trading Risk. AMACOM. Comprehensive treatment of market impact models, including the square-root model used in this chapter. Covers temporary vs. permanent impact, optimal execution algorithms, and transaction cost analysis.

  • Almgren, R., & Chriss, N. (2001). "Optimal Execution of Portfolio Transactions." Journal of Risk, 3(2), 5-39. The foundational paper on optimal trade execution. Introduces the Almgren-Chriss framework for balancing market impact against timing risk. While focused on equities, the principles apply to any market with finite liquidity -- including prediction markets.

  • Gatheral, J. (2010). "No-Dynamic-Arbitrage and Market Impact." Quantitative Finance, 10(7), 749-759. Theoretical analysis of market impact models, showing that the square-root impact model is consistent with no-arbitrage constraints. Provides the mathematical foundation for the impact models used in backtesting.

Statistical Testing for Strategy Evaluation

  • White, H. (2000). "A Reality Check for Data Snooping." Econometrica, 68(5), 1097-1126. Introduces the "reality check" bootstrap procedure for testing whether the best strategy from a collection of strategies is genuinely significant after accounting for the multiple comparisons problem. A foundational paper for rigorous backtesting.

  • Hansen, P. R. (2005). "A Test for Superior Predictive Ability." Journal of Business & Economic Statistics, 23(4), 365-380. Extends White's reality check to the "superior predictive ability" (SPA) test, which has better power. Useful for comparing multiple prediction market strategies.

  • Diebold, F. X., & Mariano, R. S. (1995). "Comparing Predictive Accuracy." Journal of Business & Economic Statistics, 13(3), 253-263. The original Diebold-Mariano test for comparing forecast accuracy between two models. Essential for pairwise strategy comparison in backtesting.

Walk-Forward and Cross-Validation for Time Series

  • Arlot, S., & Celisse, A. (2010). "A Survey of Cross-Validation Procedures for Model Selection." Statistics Surveys, 4, 40-79. Comprehensive survey of cross-validation methods, including discussion of when standard k-fold CV is inappropriate (time series data) and alternatives like h-block CV and blocked cross-validation.

  • Bergmeir, C., Hyndman, R. J., & Koo, B. (2018). "A Note on the Validity of Cross-Validation for Evaluating Autoregressive Time Series Prediction." Computational Statistics & Data Analysis, 120, 70-83. Examines when standard cross-validation can be used with time series (when the model doesn't use lagged dependent variables) and when time-ordered splitting is required. Relevant for understanding when walk-forward analysis is strictly necessary.

Prediction Market-Specific References

  • Wolfers, J., & Zitzewitz, E. (2004). "Prediction Markets." Journal of Economic Perspectives, 18(2), 107-126. Foundational survey of prediction markets. Discusses the efficiency of prediction market prices and the types of biases (favourite-longshot, calibration) that create opportunities for systematic trading.

  • Manski, C. F. (2006). "Interpreting the Predictions of Prediction Markets." Economics Letters, 91(3), 425-429. Challenges the common assumption that prediction market prices equal probabilities. Shows that under risk aversion, prices can systematically deviate from probabilities. Relevant for backtesting strategies that assume price = probability.

  • Page, L. (2012). "'It Ain't Over Till It's Over': Yogi Berra Bias on Prediction Markets." Applied Economics, 44(28), 3633-3643. Documents a bias in prediction markets where prices are insufficiently extreme (too close to 50%) relative to the true probability. A backtestable hypothesis for mean-reversion-away-from-50% strategies.

Software and Tools

  • Backtrader Documentation. https://www.backtrader.com/docu/ A popular Python backtesting framework that implements the event-driven architecture discussed in this chapter. Good reference for production-quality backtesting code.

  • Zipline Documentation. https://zipline.ml4trading.io/ The backtesting engine originally developed by Quantopian. While designed for equities, its architecture is a useful reference for building prediction market backtesting systems.

  • pandas Documentation -- Rolling Operations. https://pandas.pydata.org/docs/reference/window.html Essential reference for implementing rolling statistics (moving averages, z-scores) that are core to many backtesting computations.

  • Chapter 14: Binary Outcome Trading Strategies -- The strategies that are evaluated using the backtesting framework built in this chapter.

  • Chapter 22: Statistical Modeling -- The statistical foundations (hypothesis testing, confidence intervals) applied to backtest evaluation.

  • Chapter 23: Machine Learning for Prediction Markets -- ML model evaluation methodology (temporal splits, cross-validation) that parallels backtesting walk-forward analysis.

  • Chapter 27: Feature Stores, Pipelines, and MLOps -- The operational infrastructure for deploying strategies that have passed backtesting into production.