Chapter 33: Exercises

Section 33.2 --- Event Sourcing

Exercise 1: Event Schema Design Design a complete event schema for a prediction market that supports conditional markets (markets whose resolution depends on the outcome of another market). Define at least six event types, including events for creating, linking, and resolving conditional markets. For each event type, specify the data fields, their types, and validation rules.

Exercise 2: Temporal Query Given the EventStore and MarketAggregate classes from Section 33.2, write a function get_price_at(market_id: str, timestamp: float) -> Optional[float] that returns the last traded price of a market at any point in the past. The function should replay events up to the given timestamp and return the most recent trade price before that time. Handle the case where no trades have occurred before the given timestamp.

Exercise 3: Event Upcasting Suppose the OrderPlaced event originally had a flat structure:

{"trader_id": "abc", "price": 0.65, "quantity": 10, "side": "buy"}

In version 2, you want to add order_type (limit, market, stop) with a default of "limit", and nest the price/quantity under a "terms" key:

{"trader_id": "abc", "order_type": "limit", "terms": {"price": 0.65, "quantity": 10}, "side": "buy"}

Write an upcasting function that transforms version 1 events to version 2 format. Ensure the function is idempotent (applying it to already-upcasted events has no effect).

Exercise 4: Snapshot Optimization Modify the MarketAggregate.rebuild method to use snapshots. If a snapshot exists for the market, load the snapshot and only replay events after the snapshot's version. Write a function that determines the optimal snapshot frequency based on the rate of events for a market: markets with more than 100 events since the last snapshot should be snapshotted.

Exercise 5: Event Store Persistence The in-memory EventStore from Section 33.2 loses all data on restart. Implement a FileEventStore that persists events to a JSON Lines file (one JSON object per line). Implement both append (appends to file) and get_events (reads and filters from file). Ensure writes are atomic (use write-then-rename) to prevent corruption from crashes during writes.

Section 33.3 --- CQRS

Exercise 6: Portfolio Projection Implement a PortfolioProjection class that subscribes to the event store and maintains a read model of each trader's portfolio. The projection should track: (a) each trader's current positions by market and outcome, (b) each trader's realized PnL from completed trades, (c) each trader's open orders. Provide methods get_portfolio(trader_id), get_positions(trader_id), and get_open_orders(trader_id).

Exercise 7: Leaderboard Projection Build a LeaderboardProjection that ranks traders by total profit. The projection should update incrementally as new trade events arrive (do not recompute the entire leaderboard on each event). Provide a get_top(n) method that returns the top n traders by profit.

Exercise 8: Projection Rebuild Write a ProjectionRebuilder class that can rebuild any projection from scratch by replaying the entire event log. This is useful when you deploy a new projection or fix a bug in an existing one. The rebuilder should support progress tracking (e.g., "Rebuilt 45,000 of 120,000 events") and should be interruptible (can resume from where it left off).

Exercise 9: Write Model Validation Extend the CommandHandler to validate the following business rules before accepting an order: (a) the trader has sufficient funds to cover the order, (b) the order price is within the allowed range for the market, (c) the trader does not exceed a maximum position size per market. Write tests that verify each validation rule rejects invalid orders.

Exercise 10: Eventual Consistency Testing Write a test that demonstrates eventual consistency in CQRS. Submit an order via the command handler, then immediately query the order book projection. Show that under normal conditions the projection is updated before the function returns (synchronous subscribers), but design the test to also work with asynchronous subscribers where a small delay is expected.

Section 33.4 --- Database Optimization

Exercise 11: Index Analysis Given the following query patterns, design the optimal set of indexes. Justify each index and explain why you chose the column order: - Find all open orders for a specific market, sorted by price - Find all trades for a specific trader in the last 30 days - Find markets whose question contains a specific keyword - Find the top 10 markets by trading volume in the last 24 hours - Find all positions for a trader with net quantity greater than zero

Exercise 12: Query Optimization Rewrite the following inefficient query to use the indexes from Exercise 11:

SELECT m.question, COUNT(t.trade_id) as trade_count, SUM(t.quantity * t.price) as volume
FROM markets m
LEFT JOIN trades t ON m.market_id = t.market_id
WHERE m.status = 'open'
GROUP BY m.market_id, m.question
ORDER BY volume DESC
LIMIT 10;

Explain the execution plan and identify potential performance issues.

Exercise 13: Connection Pool Sizing A prediction market API handles 2,000 concurrent requests during peak hours. The average database query takes 3ms, and the average network latency between the API server and database is 0.5ms. (a) Calculate the minimum connection pool size. (b) If you have 4 API server instances, how should you distribute the pool? (c) What happens if you set the pool too small? Too large?

Exercise 14: Partition Strategy Design a partitioning strategy for the orders table that accounts for the following access patterns: (a) the matching engine reads only open orders for a specific market, (b) the portfolio view reads all orders for a specific trader, (c) historical analysis reads orders for a specific time range. Can a single partitioning strategy serve all three patterns efficiently? If not, what trade-offs would you make?

Section 33.5 --- Caching

Exercise 15: Cache Hit Rate Analysis A prediction market's market summary endpoint is called 10,000 times per minute. The summary changes (due to trades) approximately 50 times per minute. (a) With a 5-second TTL cache, what is the theoretical cache hit rate? (b) With event-driven invalidation (no TTL), what is the hit rate? (c) What is the cost in terms of stale data for each approach?

Exercise 16: Cache Warming Write a CacheWarmer class that pre-populates the cache with data for markets likely to be heavily accessed. The warmer should: (a) identify the top 50 markets by recent trading volume, (b) pre-load their market summaries, order books, and recent trade histories into the cache, (c) refresh the warm cache every 30 seconds for active markets. This should run as a background task.

Exercise 17: Cache Stampede Prevention When a cached item expires, multiple concurrent requests may all try to recompute it simultaneously (a "stampede"). Implement a stampede_protected_get function that uses a lock to ensure only one request computes the value while others wait for the result. Use Python's asyncio.Lock for the async implementation.

Section 33.6 --- Message Queues

Exercise 18: Priority Queue Ordering Extend the InMemoryMessageQueue to support fair scheduling: within the same priority level, messages should be processed in FIFO order. Messages in higher priority levels should always be processed before lower priority levels, but within a level, strict ordering must be maintained. Write tests that verify this behavior.

Exercise 19: Idempotent Message Processing Order matching messages must be processed exactly once. Implement an IdempotentHandler wrapper that deduplicates messages based on their message_id. The handler should maintain a set of processed message IDs (with a configurable retention period to prevent unbounded memory growth). Write tests showing that duplicate messages are ignored.

Exercise 20: Saga Pattern Implement a saga for the "place order" flow that coordinates: (a) reserving funds from the trader's account, (b) placing the order in the matching engine, (c) sending a confirmation notification. If any step fails, the saga should compensate (e.g., if matching fails, release the reserved funds). Define the saga as a state machine with explicit compensating actions.

Section 33.7 --- Horizontal Scaling

Exercise 21: Consistent Hashing Implement a consistent hashing ring with virtual nodes for distributing markets across matching engine shards. The ring should support: (a) adding and removing nodes with minimal key redistribution, (b) configurable number of virtual nodes per physical node, (c) reporting the percentage of keys that would be redistributed when a node is added or removed.

Exercise 22: Session Affinity Design a session management system that allows any API server to handle requests from any user. Use Redis for session storage. Implement create_session, get_session, update_session, and destroy_session methods. Include a session timeout of 30 minutes that resets on each access.

Exercise 23: Graceful Shutdown Implement a GracefulShutdown class that, when a SIGTERM is received: (a) stops accepting new requests, (b) waits for in-flight requests to complete (with a 30-second timeout), (c) drains any pending messages from local queues, (d) closes database connections cleanly, and (e) reports shutdown status.

Section 33.8 --- Monitoring

Exercise 24: Custom Dashboard Define a monitoring dashboard specification for a prediction market platform. The dashboard should include: (a) four panels for the four golden signals, (b) a panel for matching engine performance by market, (c) a panel for queue depths, and (d) a panel for cache hit rates. Specify the queries (using Prometheus-style PromQL syntax) for each panel.

Exercise 25: Anomaly Detection Implement a simple anomaly detector for order volume. The detector should: (a) maintain a rolling window of order counts per minute over the last hour, (b) calculate the mean and standard deviation, (c) flag as anomalous any minute where the count exceeds the mean by more than 3 standard deviations. Write it as a class that processes one data point at a time.

Exercise 26: SLO Monitoring Implement an SLOTracker that monitors compliance with the SLOs defined in Section 33.1.4. The tracker should: (a) calculate the current error budget (how much downtime/errors remain before the SLO is breached), (b) alert when more than 50% of the error budget is consumed, and (c) produce a weekly report summarizing SLO compliance.

Section 33.9 --- Security

Exercise 27: Rate Limiter Testing Write a comprehensive test suite for the TokenBucketRateLimiter. Test: (a) requests within the limit are allowed, (b) burst allowance works correctly, (c) rate recovery after a period of no requests, (d) the penalty box mechanism, (e) different rate limit tiers for different endpoint types.

Exercise 28: WAF Rules Define a set of Web Application Firewall rules specific to prediction markets. Each rule should specify: (a) what pattern it matches (regex or description), (b) what action to take (block, log, challenge), (c) the rationale. Include rules for SQL injection, XSS, bot detection, and market manipulation patterns (e.g., rapid order placement and cancellation).

Section 33.10 --- Disaster Recovery

Exercise 29: Chaos Engineering Design a chaos engineering experiment for the prediction market platform. Define: (a) the steady state hypothesis (what "normal" looks like), (b) the perturbation (what you will break --- e.g., kill a matching engine shard, introduce 500ms network latency to the database, fill the disk), (c) the expected behavior (how the system should respond), and (d) the abort conditions (when to stop the experiment to prevent real damage). Write a Python script that automates the experiment.

Section 33.11 --- Performance Testing

Exercise 30: Capacity Planning Using the CapacityPlanner class from Section 33.11, determine the infrastructure needed for the following scenario: A prediction market platform expects 100 active markets, with a peak of 20,000 requests per second during a major election. The matching engine must handle 2,000 orders per second. (a) Calculate the required infrastructure with a 2x headroom factor. (b) Estimate the monthly cost assuming cloud pricing ($0.05/core-hour, $0.01/GB-hour, $0.10/GB-month for storage). (c) Identify which component is the most expensive and suggest optimizations.