Part III: Supervised Learning

You have data. You have features. You have a pipeline. Now you need a model.

Part III is the algorithmic core of this book — nine chapters covering the supervised learning methods that power the majority of production ML systems. But this is not an algorithm encyclopedia. Every algorithm is presented through the same lens: When does it work? When does it fail? What are the tradeoffs? How do you evaluate it honestly? And how do you explain it to someone who signs your paycheck?

The progression is deliberate.

Chapter 11: Linear Models Revisited starts with the workhorse. Logistic regression with regularization is the model that half the companies claiming to use "AI" actually have in production. It is fast, interpretable, and surprisingly hard to beat on well-engineered features. This is your baseline — every other model must justify its complexity by outperforming it.

Chapter 12: Support Vector Machines explores the elegance of maximum-margin classifiers and the kernel trick. SVMs are less dominant than they were a decade ago, but the ideas are foundational, and on certain problems they remain the right tool.

Chapter 13: Tree-Based Methods introduces the family that dominates tabular data: decision trees, random forests, and the bagging principle. Trees are intuitive, which is both their strength and their danger.

Chapter 14: Gradient Boosting covers XGBoost, LightGBM, and CatBoost — the algorithms that win competitions and run in production. If you learn one algorithm family deeply, make it this one.

Chapter 15: Naive Bayes and Nearest Neighbors makes the case for simplicity. Sometimes the best model is the one that trains in 0.1 seconds and explains itself without SHAP.

Chapter 16: Model Evaluation Deep Dive is arguably the most important chapter in the book. How you evaluate your model matters more than which model you choose. Cross-validation, leakage detection, metric selection, and the statistical comparison of models — this is where amateurs and professionals diverge.

Chapter 17: Class Imbalance addresses the reality that every interesting classification problem has imbalanced classes. If your classes are balanced, you probably made up the dataset.

Chapter 18: Hyperparameter Tuning covers grid search, random search, and Bayesian optimization — and makes the case that tuning is important but wildly over-invested compared to feature engineering.

Chapter 19: Model Interpretation closes the part with SHAP, partial dependence, and the art of explaining your model to humans. A model you cannot explain is a model you cannot trust.

Progressive Project Milestones

Part III is where the StreamFlow churn model comes alive:

  • M4 (Chapter 11): Logistic regression baseline — the benchmark every model must beat
  • M5 (Chapters 13–14): Model comparison — Random Forest, XGBoost, LightGBM with cross-validation
  • M6 (Chapter 16): Proper evaluation — stratified CV, leakage audit, learning curves
  • M7 (Chapter 17): Address the 8.2% churn rate imbalance
  • M8 (Chapter 18): Bayesian hyperparameter optimization on the best model
  • M9 (Chapter 19): SHAP explanations — global importance and individual predictions

By the end of Part III, you will have a tuned, evaluated, interpretable churn model. It will not yet be deployed — that comes in Part VI. But it will be the kind of model you would stake your reputation on.

What You Need

  • All Part I and Part II chapters completed
  • The StreamFlow preprocessing Pipeline from Chapter 10
  • scikit-learn, XGBoost, LightGBM, SHAP installed (see requirements.txt)

Chapters in This Part