Exercises: Real-World Applications

Conceptual Exercises

Exercise 1: Domain Assessment

You are consulting for a large hospital network that wants to use prediction markets to forecast patient admission volumes, surgical outcomes, and equipment failure rates. For each of these three applications, assess: (a) whether information is genuinely distributed across the organization, (b) what resolution criteria would be appropriate, (c) who the ideal participants would be, and (d) what incentive structure you would recommend. Identify which application is most promising and explain why.

Exercise 2: Replication Market Design

Design a prediction market for predicting the replicability of machine learning research papers. Specifically: (a) How would you define "replication" for an ML paper? (Consider both narrow replication --- same data, same method --- and conceptual replication.) (b) Who should be allowed to participate? Should the original authors be included? (c) What market mechanism would you use, and why? (d) How would you handle the long time horizons involved in replication attempts?

Exercise 3: Pandemic Preparedness

Based on the lessons from COVID-19 prediction markets described in Section 40.4, design a standing pandemic preparedness forecasting system. Your design should address: (a) What questions should be continuously monitored even in non-pandemic times? (b) How would the system scale up during an emerging crisis? (c) How would you recruit and retain domain experts (epidemiologists, virologists)? (d) How would forecasts be communicated to policymakers?

Exercise 4: Corporate Market Failure Modes

A technology company launched an internal prediction market six months ago. Participation has declined by 70%, and the remaining traders are mostly from the engineering department. Diagnose potential causes and propose remedies for each. Consider at minimum: incentive structure, question design, organizational culture, platform usability, and management attitudes.

Exercise 5: Election Market Efficiency

PredictIt and Polymarket often show different prices for the same election outcome. Explain: (a) why persistent price differences can exist between these platforms, (b) whether these differences represent true arbitrage opportunities, (c) what specific frictions prevent price convergence, and (d) how you would evaluate which platform's prices are more accurate.

Exercise 6: Geopolitical Forecasting Ethics

The Good Judgment Project has shown that forecasting tournaments can outperform intelligence analysts. Discuss the ethical implications of: (a) using crowd forecasts to inform military decisions, (b) the potential for self-fulfilling prophecies when conflict probabilities are published, (c) whether participants have a moral responsibility for decisions made based on their forecasts, and (d) how to handle forecasts on events involving loss of life.

Exercise 7: LMSR Parameterization

For a new prediction market on quarterly sales figures at a mid-sized company (500 employees), you need to set the LMSR liquidity parameter $b$. (a) Calculate the maximum market maker loss for a binary market with $b = 50$, $b = 100$, and $b = 200$. (b) Explain the trade-off between larger and smaller values of $b$. (c) How would you determine the appropriate value empirically?

Exercise 8: Scientific Grant Markets

Propose a prediction market system for a national science funding agency. Markets should predict the scientific impact of proposed research projects. Address: (a) how to define and measure "scientific impact," (b) what time horizon to use, (c) how to handle the fact that unfunded projects cannot be evaluated for impact, and (d) whether market prices should influence funding decisions (and the circularity this creates).

Quantitative Exercises

Exercise 9: Brier Score Comparison

A replication market produces the following predictions for 10 studies, along with the actual outcomes (1 = replicated, 0 = did not replicate):

(a) Calculate the Brier score for the market and for the expert survey. (b) Calculate the calibration component and resolution component of each Brier score. (c) Which method is better calibrated? Which has better resolution? (d) Apply the extremizing transformation with $a = 2.0$ to the market prices and recalculate the Brier score. Does extremizing help?

Exercise 10: Fed Funds Probability Extraction

The current federal funds rate is 4.50%. The one-month fed funds futures contract is priced to imply a rate of 4.42%. (a) Calculate the implied probability of a 25bp rate cut. (b) If a two-month futures contract implies 4.35%, what does this tell you about the market's expectation for the second month? (c) Discuss the assumptions required for these calculations to be valid.

Exercise 11: Technology Adoption Forecasting

A prediction market on electric vehicle adoption produces the following estimates of the S-curve parameters: $K = 0.80$ (80% ultimate adoption), $r = 0.25$, $t_0 = 2028$. (a) Calculate the predicted adoption level for 2025, 2028, 2030, and 2035. (b) If a new market estimate changes $r$ to 0.35 but keeps other parameters the same, how does this affect the 2030 prediction? (c) At what year does the model predict 50% adoption under each scenario?

Exercise 12: LMSR Pricing

A binary prediction market uses LMSR with $b = 100$. Currently, 150 shares of "Yes" and 80 shares of "No" have been purchased. (a) Calculate the current price of a "Yes" share. (b) Calculate the cost of purchasing 10 additional "Yes" shares. (c) If a trader believes the true probability is 0.90, how many shares should they buy (assuming they want to move the price to their believed probability)?

Exercise 13: Extremizing Aggregation

You have forecasts from 5 forecasters for a geopolitical event: 0.35, 0.40, 0.55, 0.60, 0.70. (a) Calculate the simple average. (b) Apply the extremizing transformation with $a = 1.5$, $a = 2.0$, and $a = 3.0$. (c) If the true outcome is 1 (event occurs), which value of $a$ produces the best Brier score? (d) Discuss how you would choose $a$ in practice.

Exercise 14: Breakeven Inflation Calculation

A 10-year nominal Treasury yields 4.2%, and a 10-year TIPS yields 1.8%. (a) Calculate the breakeven inflation rate. (b) If the inflation risk premium is estimated at 0.3%, what is the market's true expected inflation? (c) A 5-year nominal Treasury yields 3.8% and a 5-year TIPS yields 1.5%. Calculate the 5-year, 5-year forward breakeven inflation rate. (d) Interpret the difference between the 5-year and 10-year breakeven rates.

Exercise 15: Signal Detection for Conflict

A conflict early warning system has detection probability $P_D = 0.80$ and false alarm rate $P_{FA} = 0.15$. The base rate of conflict in a given country-year is 0.05. (a) Calculate the positive predictive value (probability of actual conflict given an alarm). (b) Calculate the negative predictive value. (c) If you raise the detection threshold to reduce $P_{FA}$ to 0.05, $P_D$ drops to 0.60. Which setting is better if the cost of a missed conflict is 100x the cost of a false alarm? (d) Plot conceptually the ROC curve implied by these two operating points.

Programming Exercises

Exercise 16: Corporate PM Simulator

Write a Python program that simulates a corporate prediction market with the following features: - 100 employees across 5 departments - Each department has partial information about the outcome - Employees have varying levels of expertise (modeled as signal precision) - The market runs for 200 trading rounds - Compare the market's final price to (a) the average of all employees' signals and (b) the best department's average signal.

Exercise 17: Replication Market Backtester

Using the data from the Reproducibility Project: Psychology (or simulated data with similar characteristics), build a backtester for a replication prediction market. Your backtester should: - Simulate trading by participants with varying expertise - Track market prices over time - Calculate Brier scores at different points in the market's life - Compare to a baseline of simple expert surveys

Exercise 18: Pandemic Scenario Generator

Write a Python program that generates pandemic scenarios and tests a prediction market's ability to forecast key outcomes. The program should: - Model a basic SIR epidemic - Generate noisy observations available to different forecasters - Run a prediction market that aggregates forecaster signals - Evaluate the market's forecast accuracy against the true epidemic trajectory

Exercise 19: Geopolitical Event Classifier

Build a Python classifier that uses historical data to predict the type of geopolitical event most likely to occur in a given region. Features should include: - Economic indicators (GDP growth, unemployment, inflation) - Political indicators (regime type, years since last transition) - Social indicators (ethnic fractionalization, inequality measures) Compare a simple model's predictions against simulated prediction market prices.

Exercise 20: Multi-Domain Platform

Build a Python framework for a prediction market platform that supports multiple domains simultaneously. The platform should: - Support creating markets in different categories (corporate, scientific, geopolitical) - Implement LMSR pricing - Track user performance across categories - Generate leaderboards and calibration plots

Exercise 21: Policy Conditional Market

Implement a conditional prediction market for policy analysis. The market should: - Support trading on "outcome given policy A" and "outcome given policy B" simultaneously - Calculate the implied policy effect (difference between conditional markets) - Simulate scenarios where the policy effect is known and verify the market recovers it - Handle the correlation between the two conditional markets

Exercise 22: Forecast Aggregation Methods

Implement and compare at least four forecast aggregation methods: (a) Simple average (b) Median (c) Performance-weighted average (d) Extremized average Apply them to simulated forecasting data with 50 forecasters and 100 questions. Generate a comprehensive comparison table showing Brier scores, calibration, and resolution for each method.

Exercise 23: Market Manipulation Detection

Write a Python program that simulates a prediction market where one trader attempts to manipulate the price. Implement: - A normal market with 50 honest traders - A manipulation scenario where one trader has deep pockets and trades against the market's information - A detection algorithm that identifies suspicious trading patterns - An analysis of how manipulation affects final price accuracy

Exercise 24: Climate Derivative Pricer

Write a Python program that prices weather derivatives using the Ornstein-Uhlenbeck temperature model described in Section 40.8. Your program should: - Simulate daily temperatures using the model with seasonal mean and volatility - Price HDD and CDD futures for specific contract months - Calculate option prices on HDD/CDD using Monte Carlo simulation - Test the sensitivity of prices to model parameters

Exercise 25: End-to-End Application Builder

Design and implement a complete prediction market application for a domain of your choice (not one covered in the chapter). Your implementation should include: - Needs assessment documentation - Market mechanism implementation - Simulation of participant behavior - Accuracy evaluation against a baseline - A brief report on lessons learned This is an open-ended exercise; the domain could be education (predicting student outcomes), healthcare (predicting treatment efficacy), environment (predicting air quality), or any other domain where distributed information could be aggregated.