Chapter 7 Key Takeaways: AI Decision-Making

One-Page Summary

The Three Decision Modes

Mode Core Question What It Does Key Risk
Recommendation "What might you want?" Suggests items, content, or actions based on preferences Filter bubbles, engagement optimization over wellbeing
Classification "What category is this?" Assigns inputs to predefined categories Misclassification, categories that don't fit reality
Prediction "What will happen?" Estimates future outcomes from historical patterns False certainty, self-fulfilling prophecies

Five Key Concepts

  1. AI decisions are probability estimates, not truths. When an AI system "decides" something, it calculates the probability that an input matches a pattern. A risk score of 0.82 means "inputs like this were associated with this outcome 82% of the time in the training data" — it does not mean this specific outcome will occur.

  2. False positives and false negatives trade off against each other. Making a system more sensitive (fewer false negatives) increases false positives, and vice versa. Where you set this threshold is a values decision, not a technical one.

  3. Prediction is not explanation. A system can predict what will happen without explaining why. Effective intervention requires understanding causes, not just correlations.

  4. The accuracy-interpretability trade-off means the most accurate models are often the hardest to understand. Context determines which matters more: a cancer screening tool needs high accuracy; a credit denial needs high interpretability.

  5. Feedback loops occur when AI decisions influence the data the system learns from, potentially amplifying small biases into large systemic patterns. They can be disrupted through randomization, external data, and human oversight.

Critical Vocabulary

  • Collaborative filtering: Recommending based on what similar users liked
  • Content-based filtering: Recommending based on item attributes
  • Proxy variable: A neutral-seeming variable that correlates with a protected characteristic
  • Feedback loop: When a system's output becomes its input, reinforcing existing patterns
  • Interpretability: How well a human can understand why a system made a decision
  • False positive: System says "yes" when the truth is "no"
  • False negative: System says "no" when the truth is "yes"

The Evaluation Framework

When you encounter an AI decision system, ask:

  1. What mode? Is it recommending, classifying, predicting, or a combination?
  2. What's the threshold? Where does it sit on the false positive / false negative trade-off? Who chose that threshold, and why?
  3. Can it explain itself? Is the system interpretable? Does it need to be?
  4. Who's affected? Do errors fall equally on all groups, or do some groups bear more of the burden?
  5. Is there a loop? Does the system's output feed back into its input data?

Project Checkpoint Reminder

For your AI Audit Report: Identify your system's decision type(s), trace a specific decision from input to output, assess the accuracy-interpretability trade-off, and check for feedback loops and proxy variables.