Case Study 2: Monte Carlo Simulation of Prediction Market Returns

Overview

This case study uses Monte Carlo simulation to explore what happens when a trader makes 1,000 binary prediction market bets with varying edges over time. We will model outcomes using probability distributions, visualize profit-and-loss paths, demonstrate the Law of Large Numbers convergence, and show how variance affects short-term results versus long-term expectations. The goal is to build intuition about sample size, risk, and the practical difference between "having an edge" and "being profitable."

Scenario Setup

The Trader: An analytical prediction market participant who has developed a systematic strategy for identifying mispriced binary contracts. Based on historical backtesting, the trader believes they have a consistent edge.

The Strategy: - The trader identifies binary contracts where their estimated true probability q exceeds the market price p - They buy one contract on each identified opportunity - They make approximately 1,000 trades over the course of a year - All contracts are binary (pay $1 or $0) - Contract prices vary, but average around $0.45

Key Parameters:

Simulation Design

Model 1: Constant Edge

First, we simulate the simplest case: every trade has the same edge.

• Market price: $0.45 for every contract
• True probability: 0.52 for every contract
• Edge: $0.07 per contract
• Expected total profit: 0.07 * 1000 = $70

For each trade, the outcome is Bernoulli(0.52): the contract pays $1 with probability 0.52 and $0 with probability 0.48.

Profit per trade: $0.55 with probability 0.52, and -$0.45 with probability 0.48.

Model 2: Varying Edge

More realistically, different trades have different edges. We model this as:

• Market prices drawn from Beta(4.5, 5.5), giving a mean of 0.45 and moderate spread
• Edge for each trade drawn from a truncated distribution, averaging 0.07 but ranging from 0.02 to 0.15
• True probability = market price + edge

Model 3: Edge with Occasional Miscalibration

The most realistic model accounts for the fact that the trader's probability estimates are not perfect:

• True probabilities include noise around the trader's estimates
• Sometimes the trader overestimates their edge (actual true probability is lower than estimated)
• About 15% of trades have negative true edge despite the trader thinking they have positive edge

Results

Simulation 1: Constant Edge (1,000 Paths)

We simulate 1,000 independent paths, each consisting of 1,000 trades.

Key Results:

Observations:

1. The LLN works, but slowly. The mean observed profit ($70.12) is very close to the expected value ($70.00), confirming the LLN. But individual paths show enormous variation: from losing $65 to gaining $225.

2. Profitability is not guaranteed. Despite having a genuine 7% edge on every single trade, about 8% of simulated paths lost money over 1,000 trades. Having an edge does not mean you will always make money --- variance is real.

3. The standard deviation is large relative to the mean. The coefficient of variation (std/mean) is about 0.71, meaning the uncertainty is 71% of the expected profit. This highlights that 1,000 trades with a 7% edge is still not enough to make the outcome highly predictable.

Convergence Analysis

We examine how the running average profit per trade converges to the true EV as the number of trades increases:

The standard deviation of the average profit shrinks as $1/\sqrt{n}$, exactly as predicted by the Central Limit Theorem. At 10 trades, the standard deviation ($0.158) is more than twice the edge ($0.070), making the signal nearly invisible. By 500 trades, the standard deviation ($0.022) is well below the edge, and profitability is almost certain.

The practical lesson: With a 7% edge, you need roughly 100-250 trades before you can have high confidence in positive results. This maps directly to the CLT: you need $n$ large enough that $\frac{\text{edge}}{\sigma / \sqrt{n}} > 2$ (a 2-sigma signal), which gives $n > (2\sigma / \text{edge})^2$.

For our parameters: $\sigma \approx 0.499$ (std of single Bernoulli outcome), so $n > (2 \times 0.499 / 0.07)^2 = (14.26)^2 \approx 203$ trades.

Simulation 2: Varying Edge

With variable edges (averaging 7% but ranging from 2% to 15%):

The varying edge slightly increases variance (because some trades contribute more variance than others) and slightly decreases the probability of profitability, but the mean profit is essentially the same. The EV is determined by the average edge, not the distribution of edges.

Simulation 3: Edge with Miscalibration

When 15% of trades have negative true edge (the trader thinks q > p but actually q < p):

Miscalibration is costly. Even though only 15% of trades have negative edge, the mean profit drops by 41%. The trades the trader thinks are best (highest estimated edge) may actually be the worst (most overconfident). This underscores the importance of calibration --- having accurate probability estimates matters more than finding lots of trading opportunities.

Visualizing the P&L Paths

Cumulative Profit Over Time

The cumulative profit chart (see code output) shows 50 randomly selected paths out of 1,000. The visualization reveals:

1. Early divergence: In the first 100 trades, paths spread widely. Some paths are up $40 while others are down $30. At this stage, it is impossible to tell whether the trader has an edge.

2. Mid-run separation: Between trades 100-500, the positive drift becomes visible. Most paths trend upward, but a few stubborn paths remain near zero or slightly negative.

3. Late-stage convergence: By trade 1,000, the paths have formed a clear "fan" shape, mostly in positive territory. The median path is solidly profitable, but the range between the best and worst paths remains substantial.

Distribution of Final Profits

The histogram of final profits (after 1,000 trades) shows:

• An approximately normal distribution (as predicted by the CLT)
• Centered near $70 (the expected value)
• Standard deviation of about $50
• A small left tail extending into negative territory
• The bell curve shape confirms that the sum of many independent binary outcomes converges to a Gaussian

Running Average Convergence

The running average plot (average profit per trade over time) is perhaps the most instructive:

• At trade 10: wild fluctuations, many paths show negative average
• At trade 100: paths starting to cluster around $0.07 but still spread
• At trade 500: tight clustering around $0.07 with few outliers
• At trade 1000: nearly all paths have converged to within $0.03 of $0.07

This is the LLN in visual form: the running average converges to the true mean.

Risk Analysis

Maximum Drawdown

Even profitable strategies experience drawdowns --- periods where cumulative profit decreases from a previous high.

The median path experiences a maximum drawdown of $32 at some point during the 1,000 trades, even though it ends up with a $70 profit. The 95th percentile drawdown ($74) exceeds the expected total profit. This means a trader needs a bankroll substantially larger than their expected profit to survive the inevitable drawdowns.

Bankroll Requirements

Using the simulation results, we can estimate minimum bankroll sizes:

• Conservative (survive 99% of scenarios): Bankroll > $95 (maximum drawdown at 99th percentile)
• Moderate (survive 95% of scenarios): Bankroll > $74
• Aggressive (survive 90% of scenarios): Bankroll > $62

A common rule of thumb: your bankroll should be at least 2-3x your expected maximum drawdown. For this strategy, that suggests a minimum bankroll of $150-$225 to trade comfortably.

The Kelly Criterion Preview

The Kelly Criterion (which we will cover in detail in a later chapter) suggests the optimal fraction of your bankroll to bet on each trade:

$$f^* = \frac{q - p}{1 - p} = \frac{0.52 - 0.45}{0.55} = 0.127$$

This means bet about 12.7% of your bankroll on each trade at these odds. For a $500 bankroll, that is $63.50 per trade. Kelly is optimal for maximizing long-run growth rate but comes with significant variance. Many practitioners use "fractional Kelly" (e.g., half-Kelly) to reduce drawdowns at the cost of slower growth.

Sensitivity Analysis

How Edge Size Affects Outcomes (1,000 trades)

With only a 1% edge, you need nearly 10,000 trades for 95% confidence of profitability. With a 10% edge, fewer than 100 trades suffice. This table vividly illustrates why edge size matters so much more than trade volume.

How Price Level Affects Variance

Trades on contracts priced near $0.50 have the highest variance. Trades on extreme-priced contracts ($0.10 or $0.90) have lower variance. This suggests that, holding edge constant, trading extreme-priced contracts leads to more predictable outcomes --- but such contracts typically offer smaller absolute edges.

Practical Lessons

Lesson 1: Edge Is Everything, But Variance Is the Killer

A positive expected value is necessary but not sufficient for trading success. Variance determines whether you survive long enough for the LLN to work in your favor. Always consider both the edge AND the variance of your strategy.

Lesson 2: Sample Size Determines Confidence

The number of trades you need before you can confidently assess your strategy's profitability depends on the edge-to-variance ratio. With typical prediction market parameters, expect to need 200-500 trades before statistical significance emerges.

Lesson 3: Miscalibration Is More Dangerous Than Low Edge

A well-calibrated trader with a 5% edge outperforms a miscalibrated trader with a 10% estimated edge (but only 5% actual edge due to errors). Invest in improving your probability estimates before seeking more trading volume.

Lesson 4: Drawdowns Are Inevitable and Often Severe

Even the best strategies experience drawdowns that temporarily wipe out months of profits. Plan for drawdowns of 1-2x your expected total profit. Size your positions so that the worst plausible drawdown does not force you out of the market.

Lesson 5: The CLT Is Your Friend for Planning

Use the CLT to plan your trading. After n trades with average edge $\mu$ and standard deviation $\sigma$:

• Expected profit: $n\mu$
• Standard deviation of profit: $\sigma\sqrt{n}$
• 95% confidence interval: $n\mu \pm 1.96\sigma\sqrt{n}$

This lets you forecast ranges of outcomes and set appropriate expectations.

Python Implementation

The complete Python code for this case study is available in code/case-study-code.py. It includes:

• Full Monte Carlo simulation engine with configurable parameters
• Three model variants (constant edge, varying edge, miscalibrated edge)
• Visualization functions for P&L paths, convergence plots, and profit distributions
• Drawdown analysis and bankroll requirement calculations
• Sensitivity analysis across edge sizes and price levels
• All figures described in this case study can be regenerated by running the script

See the code file for the full implementation.