Case Study 2: Evaluating Shot-Making Ability - Separating Skill from Shot Selection

Introduction

One of the most valuable applications of shot quality models is separating two distinct skills: shot selection (taking high-quality shots) and shot-making (converting shots at rates above or below expectation). This case study demonstrates how to build and apply a comprehensive shot evaluation framework.

The Evaluation Challenge

Traditional Metrics Fall Short

Consider two players with identical statistics: - 18 PPG on 15 FGA - 45% FG, 36% 3PT

Traditional conclusion: Equally efficient scorers.

But what if: - Player A takes mostly wide-open catch-and-shoot threes - Player B creates contested pull-up jumpers off the dribble

The same percentages could represent very different levels of skill.

The Two-Dimensional Framework

Shot Quality (Selection): Are the shots easy or difficult? - Measured by expected FG% from model - Higher xFG% = easier shots

Shot-Making (Conversion): Does the player exceed expectations? - Measured by Actual FG% - Expected FG% - Positive = elite shot-maker

Building the Evaluation Model

Step 1: Define Expected FG% for Every Shot

Model inputs: - Shot distance - Defender distance - Shot type (catch-and-shoot, pull-up, etc.) - Shot clock - Touch time - Dribbles

Crucially excluded: - Shooter identity (to measure difficulty, not ability)

Step 2: Calculate Player-Level Metrics

For each player:

Shot Quality Index = Average(xFG% of all shots taken)
Shot-Making Index = Actual FG% - Shot Quality Index

Step 3: Create Evaluation Matrix

Case Analysis: 2022-23 Season Evaluation

Data Collection

• 500+ qualified players (100+ FGA)
• 150,000+ shot attempts
• Full tracking data for all shots

Top 20 Analysis

High Shot Quality, High Shot-Making (Elite Efficient)

Characteristics: - Elite rim finishers - Corner three specialists - Primary beneficiaries of playmaking - Often undervalued by traditional metrics

Low Shot Quality, High Shot-Making (Elite Shot Creators)

Characteristics: - Primary shot creators - High usage rates - Often face elite defenders - Generate difficult shots for themselves and others

High Shot Quality, Low Shot-Making (Underperformers)

Characteristics: - Getting good looks but missing - May indicate skill decline - Could be role mismatch - Potential trade candidates or development targets

Detailed Player Profile: Stephen Curry (Archetype Analysis)

Shot Distribution: | Shot Type | % of Shots | xFG% | Actual FG% | Diff | |-----------|------------|------|------------|------| | Catch-and-shoot 3 | 22% | 42.1% | 45.8% | +3.7% | | Pull-up 3 | 35% | 34.5% | 41.2% | +6.7% | | Rim (uncontested) | 15% | 72.5% | 78.3% | +5.8% | | Rim (contested) | 8% | 48.2% | 52.1% | +3.9% | | Mid-range | 12% | 42.8% | 49.5% | +6.7% | | Floater | 8% | 45.1% | 51.2% | +6.1% |

Analysis: - Overall xFG%: 43.2% (below average - takes difficult shots) - Overall Actual FG%: 48.9% (above average) - Shot-Making Index: +5.7% (elite)

Interpretation: Curry takes some of the most difficult shots in the league but makes them at an elite rate. His pull-up three differential (+6.7%) is historically exceptional.

Detailed Player Profile: Role Player Comparison

Player X (3-and-D Wing): | Metric | Value | |--------|-------| | xFG% | 49.8% | | Actual FG% | 48.2% | | Differential | -1.6% | | Shot Profile | 75% catch-and-shoot 3, 20% rim, 5% other |

Player Y (Versatile Scorer): | Metric | Value | |--------|-------| | xFG% | 45.2% | | Actual FG% | 46.8% | | Differential | +1.6% | | Shot Profile | 40% 3PT, 30% mid-range, 30% rim |

Traditional comparison: Player X shoots higher percentage (48.2% vs 46.8%) Shot quality comparison: Player Y is actually the better shot-maker (+1.6% vs -1.6%)

Implication: Player Y creates more value per shot attempt when adjusting for difficulty.

Team-Level Applications

Offensive System Evaluation

Team A (Motion Offense): | Metric | Value | League Rank | |--------|-------|-------------| | Team xFG% | 49.5% | 5th | | Actual FG% | 50.2% | 8th | | Shot-Making | +0.7% | 15th |

Interpretation: Elite shot creation, average shot-making. System generates quality looks.

Team B (ISO-Heavy): | Metric | Value | League Rank | |--------|-------|-------------| | Team xFG% | 45.8% | 25th | | Actual FG% | 47.5% | 18th | | Shot-Making | +1.7% | 5th |

Interpretation: Poor shot selection masked by elite shot-makers. Vulnerable if stars decline.

Lineup Optimization

Best Five-Man Lineup by Shot Quality: | Lineup | Minutes | xFG% | Actual FG% | Diff | |--------|---------|------|------------|------| | Starting 5 | 450 | 48.2% | 49.8% | +1.6% | | Bench Unit | 320 | 51.2% | 50.1% | -1.1% | | Death Lineup | 180 | 46.5% | 50.2% | +3.7% |

Analysis: - Bench unit gets easiest shots but underperforms - "Death Lineup" with difficult shots but elite conversion - Starting lineup balanced approach

Trade Evaluation Application

Acquiring Player Profile: - Traditional stats: 15 PPG, 45% FG - xFG%: 43.5% (takes difficult shots) - Actual FG%: 45.0% - Shot-Making: +1.5% (above average)

Projected Impact: - In new system with better shot creation: xFG% may increase to 47% - If shot-making holds: Actual FG% could reach 48.5% - Projected PPG increase: 15 to 17-18 PPG

Model Validation

Stability Testing

Year-over-year correlation of Shot-Making Index:

Conclusion: Shot-making ability is a stable, repeatable skill.

Predictive Power

Using Shot-Making Index to predict next-season efficiency: | Predictor | Next-Year FG% Correlation | |-----------|---------------------------| | Current FG% | 0.55 | | Shot-Making Index | 0.62 | | Combined Model | 0.71 |

Shot-making adds predictive value beyond raw shooting percentages.

Implementation Guidelines

For Player Evaluation

1. Calculate baseline xFG% using league-wide model
2. Compare to actual FG% to isolate shot-making skill
3. Examine shot distribution to understand role
4. Project to new situations using shot-making stability

For System Design

1. Identify shot-makers who exceed expectations
2. Create opportunities that match their profiles
3. Reduce difficult shots for players below expected
4. Optimize total expected points across lineup

For Development

1. Target improvement areas where player is below expected
2. Maintain strengths where player exceeds expected
3. Consider role changes for consistent underperformers
4. Track shot-making trends for early decline detection

Limitations

Model Constraints

1. Doesn't capture all difficulty factors: Fatigue, game context, mental pressure
2. Sample size requirements: Need 300+ shots for stable estimates
3. Context dependence: Role players may perform differently in larger roles
4. Defensive attention: Star treatment not fully captured

Interpretation Cautions

1. Shot creation value not measured: Taking difficult shots may create easy shots for others
2. Team effects: Shot quality depends on teammates
3. Role fit: Low shot-making may reflect role mismatch
4. Development potential: Young players may improve shot-making over time

Conclusion

Separating shot selection from shot-making provides crucial insights into player value that traditional statistics miss. Elite shot-makers who take difficult shots create unique value, while players who miss easy shots may be overvalued by conventional metrics.

The framework enables: - More accurate player evaluation - Better trade and free agency decisions - Optimized lineup construction - Targeted player development

Shot quality models are essential tools for modern basketball decision-making.

Discussion Questions

1. How should teams value a player who takes very difficult shots at average efficiency vs. a player who takes easy shots at above-average efficiency?

2. What additional factors might help improve shot difficulty estimation?

3. How might shot-making ability change with age? With role changes?

4. Should shot-making differential be incorporated into contract negotiations?

5. How can teams identify shot-makers before they have large NBA sample sizes?

Data Exercise

You have data on three free agent targets:

1. Calculate shot-making index for each player
2. Project their efficiency in your system (assume xFG% increases by 3% for all)
3. Calculate expected PPG in new role (assume same volume)
4. Which player provides best value per dollar?
5. What additional information would you want before signing?