Further Reading: The Frontier — Research Directions

LLMs and AI Forecasting

  1. Halawi, D., Shi, F., Chen, T., & Steinhardt, J. (2024). "Approaching Human-Level Forecasting with Language Models." arXiv:2402.18563. The most rigorous published benchmark of LLM forecasting performance. Tests GPT-4 against Metaculus community forecasts and superforecasters across 1,000+ questions. Introduces the multi-strategy prompting framework (base rate, adversarial, decomposition) and the geometric mean aggregation method. Essential reading for anyone building AI forecasting systems.

  2. Schoenegger, P., Park, P. S., Tetlock, P. E., & Tetlock, P. E. (2024). "AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy." Working Paper. Examines the hybrid human-AI forecasting approach, showing that LLM-assisted humans outperform both unassisted humans and standalone LLMs. Provides evidence that the optimal role for LLMs is as a "cognitive prosthesis" that surfaces base rates and counterarguments, rather than as a standalone forecaster.

  3. Zou, A., Xiao, K., Welbl, J., & Steinhardt, J. (2024). "ForecastBench: A Dynamic Benchmark for AI Forecasting Abilities." NeurIPS Datasets and Benchmarks Track. Introduces a continuously updated benchmark for evaluating AI forecasting systems, addressing the problem that static benchmarks become stale as LLMs are trained on their questions. ForecastBench uses only questions that resolve after the evaluation date, preventing data contamination.

Privacy-Preserving Computation

  1. Goldreich, O. (2009). Foundations of Cryptography: Volume 2 — Basic Applications. Cambridge University Press. The definitive theoretical reference for secure computation, including zero-knowledge proofs and multi-party computation. While dense, Chapters 4 (ZKPs) and 7 (MPC) provide the mathematical foundations needed to understand privacy-preserving prediction markets. Not for the faint of heart, but indispensable for serious implementers.

  2. Kosba, A., Miller, A., Shi, E., Wen, Z., & Papamanthou, C. (2016). "Hawk: The Blockchain Model of Cryptography and Privacy-Preserving Smart Contracts." IEEE Symposium on Security and Privacy, 839–858. Proposes a framework for privacy-preserving smart contracts using ZK-SNARKs. While not specifically about prediction markets, the Hawk model directly applies: trade data is hidden from the public blockchain while the market's aggregate state is publicly verifiable. The paper's formalization of "contractual security" (protection from the contract manager) is directly relevant.

  3. Galal, H. S., & Youssef, A. M. (2018). "Verifiable Sealed-Bid Auction on the Ethereum Blockchain." Financial Cryptography and Data Security, 265–278. Demonstrates a practical implementation of private sealed-bid auctions on Ethereum using Pedersen commitments and ZK proofs. The techniques directly transfer to prediction market order submission: traders commit to orders, a ZK proof validates the order's correctness, and the market clears without revealing individual orders.

Differential Privacy

  1. Dwork, C., & Roth, A. (2014). "The Algorithmic Foundations of Differential Privacy." Foundations and Trends in Theoretical Computer Science, 9(3–4), 211–407. The canonical survey of differential privacy theory. Covers the Laplace mechanism, composition theorems, the exponential mechanism, and applications. Sections on basic and advanced composition are directly relevant to prediction markets that repeatedly publish price updates. Freely available online.

  2. Hsu, J., Huang, Z., Roth, A., & Wu, Z. S. (2016). "Jointly Private Convex Programming." SODA, 580–599. Addresses the problem of privately solving optimization problems — directly relevant to computing market-clearing prices under differential privacy. Shows that convex optimization (which LMSR pricing involves) can be performed with differential privacy at a manageable utility cost.

Mechanism Design and Peer Prediction

  1. Prelec, D. (2004). "A Bayesian Truth Serum for Subjective Data." Science, 306(5695), 462–466. The foundational paper on Bayesian Truth Serum, the first mechanism that incentivizes truthful reporting without access to ground truth. The key insight — rewarding "surprisingly common" answers — has spawned an entire subfield of peer prediction research. Clear exposition accessible to non-specialists.

  2. Witkowski, J., & Parkes, D. C. (2012). "A Robust Bayesian Truth Serum for Small Populations." AAAI Conference on Artificial Intelligence. Extends BTS to work with small populations (as few as 3 agents), addressing a key limitation of the original mechanism. Directly relevant to prediction markets with thin participation, where peer prediction could supplement or replace market mechanisms.

  3. Sandholm, T. (2003). "Automated Mechanism Design: A New Application Area for Search Algorithms." CP, 19–36. Introduces the concept of automated mechanism design — using computational optimization to find mechanisms with desired properties rather than designing them by hand. While focused on auctions, the framework applies directly to prediction markets: optimize over parameterized cost functions to minimize market maker loss while maximizing information aggregation.

Causal Inference and Decision Markets

  1. Hanson, R. (1999). "Decision Markets." IEEE Intelligent Systems, 14(3), 16–19. The original proposal for decision markets — prediction markets where the decision-maker commits to choosing the policy that the market favors. Hanson argues that decision markets can aggregate information for policy decisions more effectively than voting or expert committees. A short, accessible paper that launched an active research program.

  2. Othman, A., & Sandholm, T. (2013). "Decision Rules and Decision Markets." AAMAS, 625–632. Formally analyzes the incentive properties of decision markets, showing that rational traders may have incentives to manipulate prices to influence the decision (the "strategic trader" problem). Proposes modified mechanisms that mitigate this problem under certain conditions.

Cross-Chain and Infrastructure

  1. Buterin, V. (2014). "A Next-Generation Smart Contract and Decentralized Application Platform." Ethereum White Paper. While primarily a platform paper, the Ethereum white paper includes a section on prediction markets as a key application of smart contracts. Understanding the constraints of on-chain computation (gas costs, block times, storage costs) is essential for designing practical decentralized prediction markets.

  2. Peterson, J., Krug, J., Zoltu, M., Williams, A. K., & Alexander, S. (2019). "Augur: A Decentralized Oracle and Prediction Market Platform." Working Paper. The technical white paper for Augur, the first major decentralized prediction market. Describes the oracle mechanism (REP token staking), the market creation process, and the economic incentives for honest resolution. A case study in the practical challenges of building trustless prediction infrastructure, including the oracle problem, low liquidity, and high gas costs.