Chapter 25: Key Takeaways

Quick Reference Summary

Text Processing Fundamentals

Essential Code Patterns

Text Cleaning:

import re

def clean_football_text(text: str) -> str:
    # Lowercase
    text = text.lower()
    # Remove special chars (keep hyphens)
    text = re.sub(r'[^\w\s\-]', ' ', text)
    # Normalize whitespace
    text = ' '.join(text.split())
    return text

Stat Extraction:

stat_pattern = re.compile(
    r'(\d+\.?\d*)\s*(yards?|tds?|catches|completions?)',
    re.IGNORECASE
)
stats = stat_pattern.findall(text)

Simple Sentiment:

def sentiment_score(text, positive_words, negative_words):
    words = text.lower().split()
    pos = sum(1 for w in words if w in positive_words)
    neg = sum(1 for w in words if w in negative_words)
    return pos / (pos + neg) if (pos + neg) > 0 else 0.5

Named Entity Recognition

Sentiment Analysis

Football-Specific Vocabulary:

Attribute Lexicons:

ATTRIBUTES = {
    'arm_strength': {
        'positive': ['cannon', 'rocket', 'velocity', 'zip'],
        'negative': ['weak arm', 'lacks velocity', 'floats']
    },
    'athleticism': {
        'positive': ['explosive', 'dynamic', 'burst', 'agile'],
        'negative': ['stiff', 'plodding', 'slow', 'tight hips']
    }
}

Topic Modeling

LDA Configuration:

from sklearn.decomposition import LatentDirichletAllocation

lda = LatentDirichletAllocation(
    n_components=10,      # Number of topics
    max_iter=20,          # Training iterations
    random_state=42
)

Common Football Topics: 1. Game Analysis (yards, touchdowns, defense) 2. Recruiting (commits, prospects, visits) 3. NFL Draft (round, pick, projection) 4. Transfers (portal, destination, NIL) 5. Injuries (out, questionable, surgery)

Feature Engineering

Model Building Pipeline

Raw Text → Clean → Tokenize → Vectorize → Model → Predict
    ↓         ↓        ↓         ↓         ↓
Scouting → Remove  → Split   → TF-IDF → GBM → Grade
Report    noise     words     features        prediction

Performance Benchmarks

Common Pitfalls

Evaluation Metrics

Classification: - Precision: True positives / Predicted positives - Recall: True positives / Actual positives - F1 Score: Harmonic mean of precision and recall

Regression: - MAE: Average absolute error - RMSE: Root mean squared error - R²: Variance explained

Applications Summary

Quick Implementation Checklist

For Scouting Report Analysis: - [ ] Clean and normalize text - [ ] Extract entities (players, teams, positions) - [ ] Calculate sentiment scores - [ ] Extract attribute mentions - [ ] Generate structured output

For Media Monitoring: - [ ] Process incoming articles - [ ] Extract entity mentions - [ ] Calculate sentiment per entity - [ ] Track over time - [ ] Generate alerts

Industry Tools