Chapter 6 Key Takeaways: The Business of Machine Learning

The Business Comes First

  1. The business of machine learning is the business first, and the machine learning second. The most common cause of ML project failure is not technical — it is solving the wrong problem, misaligning with stakeholders, or deploying a model that the organization cannot act on. Tom Kowalski's $2 million pricing engine, which achieved 94% accuracy but was never used, illustrates this principle in its purest form.

  2. The ML part of an ML project accounts for only 10-20% of total effort. Data preparation consumes 60-80% of project time. Deployment and maintenance consume the rest. Organizations that invest disproportionately in modeling talent while neglecting data engineering, deployment infrastructure, and business alignment consistently underperform.

Framing and Scoping

  1. Every ML project begins with a business decision, not with data or algorithms. The first question is always: "What action will someone take based on this model's output?" If you cannot answer this, you do not have an ML problem — you have a curiosity. Use the Translation Framework (identify the decision, define the prediction target, determine the prediction type, specify the window, define the action space) to convert vague business needs into precise ML problem statements.

  2. Professor Okonkwo's Five Questions separate viable ML projects from premature ones. (1) Is the data available? (2) Is the problem well-defined? (3) Is the value measurable? (4) Can the organization tolerate errors? (5) Is the organization ready? If the answer to any question is "no," address that prerequisite before proceeding with model development.

  3. The ML Canvas forces clarity before coding begins. A one-page strategic tool covering value proposition, prediction target, data sources, features, training data, model output, decision integration, evaluation metrics, failure modes, and monitoring plan. If you cannot complete the canvas, you are not ready to start building.

Metrics and Value

  1. Model accuracy does not equal business value. A fraud detection model with 99% accuracy that catches zero fraud is useless. Always maintain two parallel sets of metrics: model metrics (precision, recall, AUC) and business metrics (revenue retained, cost savings, customer satisfaction). The business metrics are the ones that matter.

  2. The precision-recall tradeoff is a business decision, not a technical one. Choosing between precision and recall requires knowing the relative cost of false positives versus false negatives. This can only be answered by business stakeholders with domain knowledge. If the business cannot answer it, the model's threshold is being set arbitrarily.

Failure Modes and Risk

  1. Seven failure modes kill ML projects, and most are preventable. Wrong problem framing, data leakage, overfitting to the wrong objective, scope creep, stakeholder misalignment, the notebook-to-production gap, and insufficient feedback loops. Prevention requires upfront investment in problem definition, clear scope documentation, cross-functional alignment, and deployment planning from day one.

  2. The POC trap is real and common. The gap between a successful proof of concept and a production system is enormous — different data pipelines, different scale, different reliability requirements, different security posture. Set expectations before the POC begins: "If this works, we will need X additional weeks and Y additional resources to go to production."

Strategy and Organization

  1. The build-vs-buy decision should be evaluated across five dimensions. Strategic differentiation, data uniqueness, talent availability, time pressure, and total cost of ownership. Build for core competitive capabilities; buy for commodity functions; consider "build on buy" (custom models on vendor infrastructure) for the middle ground.

  2. ML is a team sport — and data science is not always the bottleneck. A minimum viable ML team includes a product manager, data engineer, data scientist, ML engineer, and domain expert. The scarcest role is often not the data scientist but the person who can translate between business needs and technical capabilities. That translation role — the AI product manager — is a natural fit for MBA graduates.

  3. ML timelines are inherently uncertain. The feasibility sprint (2-4 weeks) is the highest-leverage planning tool. It converts "we think this might work" into "we have evidence this can work" for a bounded investment, and it provides the data needed for realistic project estimation. Always propose a feasibility sprint before committing to a full ML project.

Governance and Economics

  1. ML project governance is risk management, not bureaucracy. Stage gates (problem approval, data readiness, model validation, deployment approval, post-deployment review), model review boards, model cards, and data sheets are essential infrastructure for ML at enterprise scale. Governance is especially critical for models that affect individuals, handle sensitive data, or operate in regulated industries.

  2. Maintenance costs are the most consistently underestimated cost category in ML. Models degrade over time as the world changes (concept drift). The ongoing cost of monitoring, retraining, and maintaining ML systems in production far exceeds the initial cost of developing them. Budget explicitly for maintenance, and assign clear ownership for every deployed model.

  3. Connect every ML project to a dollar amount. If you cannot draw a line from the model to a measurable business outcome — revenue retained, costs reduced, efficiency gained — you cannot prioritize the project and you cannot declare success. Ravi Mehta's presentation to Athena's executive team exemplifies this discipline: every pilot on his shortlist has an estimated annual value, a three-year TCO, and an honest assessment of uncertainty.

These takeaways span the strategic, organizational, and economic dimensions of ML in business. They apply before, during, and after every technical chapter in Part 2. Return to them when you find yourself deep in algorithm selection or hyperparameter tuning, and ask: "Am I solving the right problem, for the right stakeholders, with the right team, measured by the right metrics?"