Case Study 1: Evaluating Quarterback Performance Beyond Traditional Statistics

Introduction

Traditional quarterback statistics—completion percentage, yards, touchdowns, interceptions—have guided quarterback evaluation for decades. But these metrics fail to capture the full picture. A quarterback who completes 70% of his passes might be taking only safe checkdowns. Another who throws for 350 yards might have benefited from a single 80-yard catch-and-run.

This case study applies EPA and success rate analysis to compare three quarterbacks with similar traditional statistics but vastly different efficiency profiles. The analysis reveals why EPA-based evaluation has become essential for modern quarterback assessment.

Background

The Three Quarterbacks

We're comparing three starting quarterbacks from the same conference, all playing in comparable offensive systems with similar supporting casts.

Marcus Rivera (State University) - Traditional Stats: 68.5% completion, 3,245 yards, 28 TD, 8 INT - Passer Rating: 152.8

Jake Thompson (Tech Institute) - Traditional Stats: 64.2% completion, 3,512 yards, 31 TD, 11 INT - Passer Rating: 148.2

Chris Patterson (Valley College) - Traditional Stats: 71.3% completion, 2,985 yards, 24 TD, 6 INT - Passer Rating: 156.4

At first glance, all three quarterbacks appear similarly effective. Patterson has the highest efficiency ratings, Thompson leads in volume, and Rivera sits in the middle. But EPA analysis reveals a dramatically different story.

Data Collection

Play-by-Play Data

We collected all dropback plays (passes and sacks) for each quarterback across the 12-game regular season.

Volume Summary:

EPA Analysis

Total EPA and EPA Per Dropback

# EPA calculation results
qb_epa_summary = {
    'Rivera': {
        'total_epa': 78.5,
        'dropbacks': 442,
        'epa_per_dropback': 0.178,
        'pass_epa': 95.2,
        'sack_epa': -16.7
    },
    'Thompson': {
        'total_epa': 112.3,
        'dropbacks': 485,
        'epa_per_dropback': 0.232,
        'pass_epa': 124.8,
        'sack_epa': -12.5
    },
    'Patterson': {
        'total_epa': 52.1,
        'dropbacks': 378,
        'epa_per_dropback': 0.138,
        'pass_epa': 62.4,
        'sack_epa': -10.3
    }
}

Key Finding: Despite having the highest traditional efficiency metrics, Patterson ranks last in EPA per dropback. Thompson, despite more turnovers, generates the most value per play.

EPA Breakdown by Outcome

Let's examine where each quarterback's EPA comes from:

Marcus Rivera: | Outcome | Plays | Total EPA | EPA/Play | |---------|-------|-----------|----------| | Completions | 273 | +105.8 | +0.388 | | Incompletions | 125 | -18.5 | -0.148 | | Interceptions | 8 | -32.2 | -4.025 | | Sacks | 44 | -16.7 | -0.380 |

Jake Thompson: | Outcome | Plays | Total EPA | EPA/Play | |---------|-------|-----------|----------| | Completions | 290 | +142.5 | +0.491 | | Incompletions | 162 | -28.8 | -0.178 | | Interceptions | 11 | -43.2 | -3.927 | | Sacks | 33 | -12.5 | -0.379 |

Chris Patterson: | Outcome | Plays | Total EPA | EPA/Play | |---------|-------|-----------|----------| | Completions | 248 | +75.4 | +0.304 | | Incompletions | 100 | -13.5 | -0.135 | | Interceptions | 6 | -22.8 | -3.800 | | Sacks | 30 | -10.3 | -0.343 |

Insight: Thompson's completions generate significantly more EPA (+0.491 per completion) than Patterson's (+0.304). This explains the gap despite Thompson's lower completion percentage—his completions are worth more.

Depth of Target Analysis

EPA varies dramatically by throw depth. Let's see how each quarterback performs at different levels:

EPA Per Attempt by Throw Depth:

Key Finding: Thompson excels at medium and deep throws, generating significantly more EPA on throws of 10+ yards. Patterson's high completion percentage comes partly from an overreliance on short throws that generate less value.

Throw Distribution:

Patterson throws 73% of his passes at or behind the line of scrimmage or to short targets, while Thompson throws downfield more frequently.

Success Rate Analysis

Overall Success Rate

Patterson leads in success rate, confirming he's the most consistent. But consistency without value has limited utility.

Success Rate by Situation

By Down:

By Field Position:

Insight: Thompson actually excels in the red zone (54% success) despite his "riskier" style. Patterson's success rate drops in the red zone (48%), suggesting his short-passing game loses effectiveness when the field compresses.

Explosive Play Analysis

Explosive Pass Rate (15+ yards)

Thompson generates nearly twice as many explosive plays as Patterson. This explains how Thompson can have more turnovers yet still create more total value—his big plays more than offset the additional risk.

EPA from Explosive Plays

Thompson's game is built on explosive plays—they account for 76% of his total passing EPA.

Pressure Analysis

Performance Under Pressure

How do these quarterbacks perform when pressured?

EPA When Pressured:

Key Finding: Thompson maintains positive EPA even under pressure (+0.02), while both Rivera and Patterson struggle. This suggests Thompson's arm talent and decision-making translate even in adverse situations.

Turnover Analysis

Interception Context

Not all interceptions are equal. Let's examine the context:

Rivera's 8 Interceptions: - 3 in red zone (high cost) - 2 on third down (moderate cost) - 2 on first down (moderate cost) - 1 tipped at line (bad luck)

Thompson's 11 Interceptions: - 2 in red zone - 4 on third down - 3 on first down - 2 on Hail Mary attempts (minimal cost)

Patterson's 6 Interceptions: - 1 in red zone - 3 on third down - 2 on first down

EPA Lost to Interceptions:

Thompson's interceptions cost slightly less on average, partly because two came on Hail Mary situations where expected points were already low.

Comprehensive Efficiency Profile

The Four-Quadrant Analysis

Plotting EPA per dropback against success rate:

Thompson: High EPA, moderate success rate = Explosive style Rivera: Moderate EPA, moderate success rate = Balanced style Patterson: Lower EPA, high success rate = Grinding style

Variance Analysis

Standard deviation of EPA per game:

Thompson has the highest variance—his explosive style leads to bigger swings. Patterson is highly consistent but consistently producing less value.

Projection and Recommendations

Who is the Best Quarterback?

Based on EPA analysis, the ranking is:

1. Jake Thompson - Best EPA per dropback, highest total value
2. Marcus Rivera - Balanced efficiency, room for growth
3. Chris Patterson - Consistent but limited ceiling

Strategic Implications

For Thompson's Team: - Continue aggressive downfield approach - Work on reducing interceptions in standard situations - Leverage his pressure performance in critical moments

For Rivera's Team: - Increase deep attempts (his deep ball EPA is solid) - Better pass protection would amplify his production - His turnover management is already strong

For Patterson's Team: - Need to scheme more explosive plays - Current efficiency won't win against elite defenses - Consider whether style change is possible or necessary

Draft/Transfer Portal Evaluation

If evaluating for the next level:

Thompson: Projects as potential difference-maker. Arm talent + decision-making under pressure are elite traits. Interceptions may decrease with better scheme/weapons.

Rivera: Solid prospect with room for development. Needs to prove he can win games with his arm when required.

Patterson: Concerning profile. High completion percentage with low EPA suggests game manager ceiling. Would need to show ability to push the ball downfield.

Conclusions

This case study demonstrates why EPA analysis is essential for quarterback evaluation:

1. Traditional stats can be misleading - Patterson's higher completion percentage and passer rating masked his lower actual value production.

2. Context matters - Thompson's interceptions looked worse in raw numbers but were less costly on average due to situation.

3. Explosive plays drive value - Thompson's ability to generate big plays more than offset his additional turnovers.

4. Success rate measures consistency, EPA measures value - Patterson was consistent at producing less value; Thompson was inconsistent but produced more total value.

5. Depth of target matters - Completion percentage means little without knowing where the ball is going.

The quarterback who looks best traditionally isn't always the most valuable. EPA analysis provides the complete picture.

Code Implementation

The complete code for this analysis is available in code/case-study-code.py, including: - QuarterbackEPAAnalyzer class - EPA breakdown by outcome - Depth of target analysis - Pressure performance calculations - Comprehensive comparison framework

Discussion Questions

1. At what sample size would you be confident in EPA rankings? Is 400+ dropbacks sufficient?

2. How should teams balance EPA and success rate when choosing between quarterbacks?

3. Thompson's style is higher variance. In a playoff scenario, would you prefer the higher-floor Patterson or higher-ceiling Thompson?

4. How would you adjust this analysis if one quarterback played in a significantly harder conference?

5. What additional data would improve quarterback EPA analysis? (Hint: tracking data, receiver separation, time to throw)