Quiz: Defensive Analytics
Target: 70% or higher to proceed.
Section 1: Multiple Choice (1 point each)
1. Why is defensive analytics more challenging than offensive analytics?
- A) Defenses don't try as hard
- B) Individual attribution is difficult due to shared responsibility
- C) The NFL doesn't track defensive statistics
- D) Defenses only face bad offenses
Answer
**B)** Individual attribution is difficult due to shared responsibility *Explanation:* Defensive success involves 11 players working together, with outcomes often determined by the weakest link.2. What does defensive EPA allowed measure?
- A) Points scored by defense
- B) How many plays the defense runs
- C) The expected point value of plays allowed by the defense
- D) Defensive line weight
Answer
**C)** The expected point value of plays allowed by the defense *Explanation:* EPA allowed measures the average EPA generated by opponents against a defense. Lower (more negative) is better.3. What is a typical elite defensive EPA per play allowed?
- A) Greater than +0.10
- B) Less than -0.10
- C) Exactly 0
- D) Greater than +0.20
Answer
**B)** Less than -0.10 *Explanation:* Elite defenses limit opponent EPA to below -0.10 per play.4. What is the main limitation of sack rate as a pass rush metric?
- A) Sacks are too common
- B) Sacks are high variance and depend on coverage quality
- C) Sacks don't affect the game
- D) Sacks are only allowed in certain situations
Answer
**B)** Sacks are high variance and depend on coverage quality *Explanation:* Sacks are rare events (~1 per 30 dropbacks) and depend on how long coverage holds.5. What is "pressure rate" in defensive analysis?
- A) The rate of quarterback pressures including sacks, hits, and scrambles
- B) How much stress defenders feel
- C) The rate of fourth down attempts
- D) How quickly defenders move
Answer
**A)** The rate of quarterback pressures including sacks, hits, and scrambles *Explanation:* Pressure rate is a broader measure of pass rush disruption than sack rate alone.6. Why is turnover rate considered an unreliable metric?
- A) Turnovers don't help defenses
- B) Turnovers are largely random with low year-to-year correlation
- C) Turnovers are counted incorrectly
- D) Only bad defenses force turnovers
Answer
**B)** Turnovers are largely random with low year-to-year correlation *Explanation:* INT rates correlate around 0.25 year-to-year; fumble recovery is nearly random (~50%).7. What does "stuff rate" measure for a defense?
- A) Completion percentage allowed
- B) Percentage of runs stopped at or behind the line of scrimmage
- C) Number of interceptions
- D) Pass rush speed
Answer
**B)** Percentage of runs stopped at or behind the line of scrimmage *Explanation:* Defensive stuff rate measures the ability to stop runs for 0 or negative yards.8. Why should defensive metrics be opponent-adjusted?
- A) Some teams face harder opponents than others
- B) To make bad defenses look good
- C) The NFL requires it
- D) It makes calculations easier
Answer
**A)** Some teams face harder opponents than others *Explanation:* Raw stats are influenced by schedule strength; adjustment isolates true defensive quality.Section 2: True/False (1 point each)
9. A defense with a high interception rate is definitively a great defense.
Answer
**False** *Explanation:* Interceptions are high-variance events with significant luck components; they don't persist well year-to-year.10. Individual defensive player evaluation is possible from standard play-by-play data.
Answer
**False** *Explanation:* Individual attribution requires film charting (PFF, SIS) to determine assignments and responsibility.11. Yards after catch allowed is a good proxy for tackling quality.
Answer
**True** *Explanation:* YAC allowed reflects the defense's ability to tackle after the catch is made.12. A defense that allows low YPC but high explosive run rate is concerning.
Answer
**True** *Explanation:* This indicates inconsistent run defense that can be exploited for big plays despite average results.Section 3: Code Analysis (2 points each)
13. What does this code calculate?
passes.groupby('defteam').agg(
pressure_rate=('pressured', 'mean'),
clean_epa=lambda x: x[~x['pressured']]['epa'].mean(),
pressured_epa=lambda x: x[x['pressured']]['epa'].mean()
)
Answer
This calculates: - **Pressure rate**: Percentage of plays with pressure - **Clean pocket EPA**: Average EPA when no pressure - **Pressured EPA**: Average EPA when pressured This helps evaluate both pass rush effectiveness and the impact of pressure on offensive performance.14. What issue exists with this defensive coverage calculation?
coverage_quality = passes.groupby('defteam').agg(
comp_pct_allowed=('complete_pass', 'mean')
)
Answer
**Issues:** 1. Includes sacks in denominator (artificially lowers comp %) 2. Doesn't account for target quality (ADOT, receiver talent) 3. Team-level stat doesn't identify which defenders are good/bad 4. No context for scheme (zone vs man) **Better approach:**non_sacks = passes[passes['sack'] == 0]
coverage = non_sacks.groupby('defteam').agg(
comp_pct_allowed=('complete_pass', 'mean'),
yards_per_target=('yards_gained', 'mean'),
epa_allowed=('epa', 'mean')
)
Section 4: Short Answer (2 points each)
15. Explain why pass rush and coverage are interdependent in defensive evaluation.
Sample Answer
**Interdependence:** 1. **Coverage enables pass rush**: Good coverage gives rushers more time to get to QB 2. **Pass rush helps coverage**: Quick pressure means defenders don't have to cover as long 3. **Metrics conflate them**: A sack might result from great coverage OR great rush 4. **Weak link problem**: Either failing makes the other look bad **Example:** - Elite corner allows 0 catches → but if rush never arrives, eventually even he gives up completions - Great pass rusher gets 15 sacks → but may be because coverage held receivers Cannot evaluate either in isolation without accounting for the other.16. Why might two teams with similar EPA allowed have very different defensive qualities?
Sample Answer
**Reasons for similar EPA with different quality:** 1. **Opponent strength**: Team A faced weak offenses, Team B faced strong ones 2. **Turnover luck**: Team A got lucky INTs, Team B forced real stops 3. **Game script**: Team A had leads (opponents pass more = harder), Team B played from behind 4. **Schedule**: Team A played outdoors in bad weather, Team B in domes **Investigation needed:** - Opponent adjustment - Filter to neutral game scripts - Check turnover regression - Split by situation Similar EPA doesn't mean equal quality without controlling for context.Section 5: Application (3 points each)
17. Design an analysis to determine if a defense is "bend but don't break" style.
Sample Answer
**Bend but Don't Break Analysis:** "Bend but don't break" = allows yards but prevents points **Metrics to calculate:** 1. **Yards/play allowed** - should be high (the "bend") 2. **Red zone TD rate** - should be low (the "don't break") 3. **3rd down conversion** - should be low (getting off field) 4. **Points per drive** - should be low despite yards **Code approach:**bend_break = defense.agg(
ypp_allowed=('yards_gained', 'mean'), # High
rz_td_rate=red_zone_tds / rz_trips, # Low
third_conv=third_down_conversions, # Low
points_per_drive=points / drives # Low
)
# BBDB score
bbdb_score = (
ypp_allowed_rank + # Higher = more bending
(32 - rz_td_rank) + # Lower = less breaking
(32 - points_rank)
)
18. A team has the best sack rate in the league but below-average pass defense EPA. What might explain this and what would you investigate?
Sample Answer
**Possible Explanations:** 1. **Coverage issues**: Sacks coming but receivers still getting open 2. **Completions before pressure**: Coverage breaking before rush arrives 3. **QB mobility**: Sacking QBs who escape less, but giving up plays to scramblers 4. **Big plays**: Few plays allowed, but those allowed are explosive 5. **Opponent adjustment**: Facing elite passing offenses **Investigation:**# 1. EPA on non-sack plays
non_sack_epa = passes[passes['sack'] == 0].groupby('defteam')['epa'].mean()
# 2. Time to pressure (if available)
# Are sacks coming quickly or slowly?
# 3. Explosive pass rate
explosive = (passes['yards_gained'] >= 20).mean()
# 4. Deep pass defense
deep_comp = passes[passes['air_yards'] >= 15]['complete_pass'].mean()
# 5. Opponent-adjusted sack rate
# Are they sacking weak QBs?
Section 6: Critical Thinking (2 points)
19. Why is it problematic to evaluate individual cornerbacks using completion percentage allowed?
Sample Answer
**Problems with Comp % Allowed:** 1. **Target selection bias**: Bad corners get targeted more, good corners less - 0/20 targets = amazing or invisible? - 15/30 targets = bad or heavily targeted elite? 2. **No assignment data**: Don't know who was supposed to cover whom - Zone coverage shares responsibility - Blown assignment by safety affects corner's stats 3. **Small samples**: 50-80 targets per season - High variance in results - Lucky batted balls vs unlucky perfect throws 4. **Quality of targets**: Facing WR1 vs WR3 matters - Corner on Tyreek Hill vs corner on WR4 5. **Scheme effects**: Press vs off coverage affects numbers **What's needed:** - Target-adjusted metrics - Film-based grades (PFF) - EPA/target for magnitude - Sample size awareness20. What are the key limitations of current defensive analytics that future tracking data might solve?
Sample Answer
**Current Limitations:** 1. **Individual attribution**: Who failed on each play 2. **Coverage assessment**: Separation, positioning, technique 3. **Pass rush wins**: Block engagement tracking 4. **Scheme recognition**: What defense was called 5. **Pre-snap reads**: How well defense disguises **Tracking Data Solutions:** 1. **Separation metrics**: Distance from receiver at throw 2. **Closing speed**: How fast defenders react 3. **Coverage win rate**: Computer vision assessment 4. **Pressure probability**: Expected vs actual pressure 5. **Positioning**: Were defenders in correct spots **Emerging Applications:** - AWS Next Gen Stats coverage metrics - ESPN win rate for pass rushers - Completion probability over expectation - Expected YAC **Remaining Challenge:** Even with tracking, understanding intent (assignments, play calls) requires film study and coaching knowledge.Scoring
| Section | Points | Your Score |
|---|---|---|
| Multiple Choice (1-8) | 8 | ___ |
| True/False (9-12) | 4 | ___ |
| Code Analysis (13-14) | 4 | ___ |
| Short Answer (15-16) | 4 | ___ |
| Application (17-18) | 6 | ___ |
| Critical Thinking (19-20) | 4 | ___ |
| Total | 30 | ___ |
Passing Score: 21/30 (70%)