Chapter 18: Quiz - Game Outcome Prediction

Instructions

Choose the best answer for each question. Questions cover rating systems, feature engineering, model building, and evaluation for game prediction.

Section 1: Prediction Fundamentals (Questions 1-8)

Question 1

What is the approximate theoretical ceiling for game prediction accuracy in college football?

A) 95-98% B) 85-90% C) 75-78% D) 65-70%

Question 2

If a team has a predicted win probability of 72%, what is the implied point spread (using standard deviation of 13.5)?

A) Home -5.5 B) Home -7.8 C) Home -10.3 D) Home -12.0

Question 3

Why is accuracy alone insufficient for evaluating win probability models?

A) Accuracy doesn't account for model training time B) A model predicting 55% for all games would have high accuracy but poor calibration C) Accuracy is only valid for regression problems D) Football games have too many possible outcomes

Question 4

What does the Brier Score measure?

A) Classification accuracy B) Mean squared error of probability predictions C) Area under the ROC curve D) Average margin of error

Question 5

A perfect Brier Score is:

A) 1.0 B) 0.5 C) 0.0 D) -1.0

Question 6

The relationship between point spread and win probability follows what type of function?

A) Linear B) Logarithmic C) Sigmoid (logistic) D) Exponential

Question 7

What is the primary advantage of predicting point spreads over binary outcomes?

A) Spreads are easier to calculate B) Spread predictions contain more information about game competitiveness C) Binary predictions require more features D) Market spreads are always accurate

Question 8

In game prediction, what does "ATS" stand for?

A) Average Team Score B) Against The Spread C) Actual Touchdown Statistics D) Adjusted Team Strength

Section 2: Elo Rating Systems (Questions 9-15)

Question 9

In a standard Elo system, what happens to ratings after a game?

A) Winner gains points, loser loses points (equal amounts) B) Only the winner's rating changes C) Both teams' ratings move toward their average D) Ratings only change after upset victories

Question 10

What does the K-factor in an Elo system control?

A) The initial rating for new teams B) The maximum rating change per game C) The home field advantage D) The number of games considered

Question 11

If Team A (rated 1600) plays Team B (rated 1400), what is Team A's expected win probability using standard Elo formula?

A) About 60% B) About 76% C) About 85% D) About 92%

Question 12

Why do some Elo implementations include mean reversion between seasons?

A) To penalize teams that lose bowl games B) To account for roster turnover and regression to the mean C) To give new teams an advantage D) To make ratings converge faster

Question 13

What is the typical home field advantage in Elo points for college football?

A) 20-30 points B) 40-50 points C) 60-70 points D) 80-100 points

Question 14

How does margin of victory affect ratings in a standard margin-based Elo variant?

A) It doesn't - only wins and losses count B) Larger margins always produce proportionally larger rating changes C) Margin affects rating change with diminishing returns to prevent blowout distortion D) Only margins above 14 points affect ratings

Question 15

What problem does regularization (ridge regression) solve when calculating margin-based ratings?

A) Prevents overfitting when teams have few games B) Makes calculations faster C) Accounts for home field advantage D) Adjusts for opponent strength

Section 3: Feature Engineering (Questions 16-22)

Question 16

Which of the following features would cause data leakage in a game prediction model?

A) Home team's season win percentage through previous week B) Home team's final score in this game C) Home team's EPA per play from last 5 games D) Vegas spread for this game (if available before kickoff)

Question 17

What is the purpose of creating "differential features" (e.g., home_ppg - away_ppg)?

A) To reduce the number of features B) To capture relative team strength in a single variable C) To normalize features to a 0-1 scale D) To handle missing data

Question 18

Why might recent performance (last 5 games) be more predictive than season-long performance?

A) Recent data is always more accurate B) Recent performance captures current team form, injuries, and adjustments C) Season-long data is too noisy D) Recency bias affects model training

Question 19

What is strength of schedule (SOS) used for in feature engineering?

A) To identify the best teams B) To adjust raw statistics for opponent quality C) To determine playoff rankings D) To calculate home field advantage

Question 20

Which situational feature would you expect to have the LARGEST impact on prediction?

A) Time zone difference for away team B) Days since last game (rest advantage) C) Temperature at game time D) Stadium capacity

Question 21

Head-to-head features from previous matchups should:

A) Include all historical games between the teams B) Be weighted more heavily in the model C) Be used cautiously due to small sample sizes and roster changes D) Replace all other team strength features

Question 22

When engineering features for bowl game prediction specifically, what factor deserves special attention?

A) Conference matchups B) Layoff time and potential rust vs. rest C) Stadium location D) TV network broadcasting the game

Section 4: Model Building (Questions 23-28)

Question 23

Why is temporal splitting (train on past, test on future) required for game prediction?

A) Random splitting is computationally expensive B) To prevent using future information to predict past games C) To balance class distributions D) To ensure all teams appear in both sets

Question 24

When would you use TimeSeriesSplit instead of regular K-Fold cross-validation?

A) When features include time-dependent variables B) When the dataset is very large C) When you want faster training D) When there are categorical features

Question 25

What is the primary benefit of probability calibration for game prediction models?

A) Higher accuracy B) Faster predictions C) Probabilities that accurately reflect true outcome frequencies D) Smaller model file size

Question 26

In an ensemble that combines Elo and ML predictions, how might you determine optimal weights?

A) Always use equal weights B) Use the model with higher training accuracy C) Optimize weights to minimize validation set Brier score D) Use domain knowledge only

Question 27

When gradient boosting outperforms logistic regression for game prediction, what might this indicate?

A) The relationship between features and outcomes is non-linear B) The dataset is too small C) Features need better scaling D) The labels are imbalanced

Question 28

Which regularization technique is most important for game prediction when sample sizes are limited?

A) L1 (Lasso) B) L2 (Ridge) C) Dropout D) Early stopping

Section 5: Evaluation (Questions 29-35)

Question 29

A model achieves 65% accuracy while the home-team-always baseline achieves 58%. The model's improvement over baseline is:

A) 7 percentage points B) 12% relative improvement C) Both A and B D) Neither - this comparison is invalid

Question 30

Expected Calibration Error (ECE) measures:

A) How often predictions are correct B) The average difference between predicted probability and observed frequency C) The maximum prediction error D) Model training stability

Question 31

If a model predicts 70% probability for games where the actual win rate is 70%, the model is:

A) Highly accurate B) Well-calibrated C) Overconfident D) Underconfident

Question 32

Why is AUC-ROC useful for comparing game prediction models?

A) It's threshold-independent B) It accounts for home field advantage C) It penalizes calibration errors D) It measures spread prediction accuracy

Question 33

A model with 60% accuracy but excellent calibration is preferable for betting applications because:

A) 60% is above the typical breakeven point B) Calibrated probabilities allow proper bankroll management C) Accuracy doesn't matter for betting D) Higher accuracy models are always overfit

Question 34

Log loss (cross-entropy) heavily penalizes:

A) All incorrect predictions equally B) Predictions close to 0.5 C) Confident predictions that are wrong D) Correct predictions with low confidence

Question 35

When evaluating against Vegas spreads, what indicates a valuable prediction model?

A) Picking more winners than losers straight-up B) ATS accuracy above 52.4% (break-even with -110 juice) C) Lower RMSE than the spread for margin prediction D) Both B and C

Answer Key

Section 1: Prediction Fundamentals

  1. C - 75-78% (due to inherent game variance)
  2. B - Home -7.8 (using inverse normal CDF)
  3. B - A model predicting constant probabilities could be "accurate" but uninformative
  4. B - Mean squared error of probability predictions
  5. C - 0.0 (perfect prediction)
  6. C - Sigmoid (logistic curve)
  7. B - Spread predictions contain more information
  8. B - Against The Spread

Section 2: Elo Rating Systems

  1. A - Winner gains, loser loses equal amounts
  2. B - Maximum rating change per game
  3. B - About 76% (E = 1/(1+10^(-200/400)) ≈ 0.76)
  4. B - Account for roster turnover
  5. C - 60-70 points (~2.5-3 actual points)
  6. C - Diminishing returns to prevent distortion
  7. A - Prevents overfitting with few games

Section 3: Feature Engineering

  1. B - Final score is future information
  2. B - Captures relative strength in one variable
  3. B - Captures current form, injuries, adjustments
  4. B - Adjust statistics for opponent quality
  5. B - Rest advantage (days since last game)
  6. C - Use cautiously due to small samples
  7. B - Layoff time (rest vs rust)

Section 4: Model Building

  1. B - Prevent using future information
  2. A - When features are time-dependent
  3. C - Probabilities reflect true frequencies
  4. C - Optimize on validation Brier score
  5. A - Non-linear relationships exist
  6. B - L2 (Ridge) for stability with limited data

Section 5: Evaluation

  1. C - Both 7pp and ~12% relative improvement
  2. B - Average diff between predicted and observed
  3. B - Well-calibrated
  4. A - Threshold-independent
  5. B - Calibration enables proper bankroll management
  6. C - Confident wrong predictions
  7. D - Both ATS >52.4% and lower margin RMSE

Scoring Guide

  • 30-35 correct: Excellent! Ready for advanced prediction systems
  • 24-29 correct: Good understanding, review weak areas
  • 18-23 correct: Solid foundation, more practice needed
  • Below 18: Review chapter material before proceeding