Key Takeaways: Chapter 15 — Fairness: Definitions, Tensions, and Trade-offs
Core Takeaways
-
Fairness is not a single concept — it is a family of competing definitions. Demographic parity, equalized odds, calibration, and individual fairness each capture a different moral intuition about what equality requires. They sound complementary. They are not. The same system can be "fair" by one definition and "unfair" by another.
-
Demographic parity demands equal selection rates across groups. It captures the intuition that a fair system should produce equal outcomes regardless of group membership. Its strength is simplicity and alignment with addressing structural inequality. Its weakness is that it ignores accuracy — a coin flip satisfies demographic parity. And it may harm the group it appears to protect if, for example, approving unqualified loan applicants leads to defaults.
-
Equalized odds demands equal error rates across groups. It captures the intuition that a fair system should make the same types of mistakes regardless of group membership. Its strength is protecting individuals from group-specific errors — ensuring that innocent Black defendants are no more likely to be falsely flagged than innocent white defendants. Its weakness is that it requires reliable ground truth, which is often contaminated by the same biases the system is meant to correct.
-
Calibration demands equal predictive accuracy across groups. It captures the intuition that the system's scores should mean the same thing for everyone — a "7" is a "7" regardless of race. Its strength is informational: decision-makers can trust the score. Its weakness is that a calibrated system can produce dramatically disparate outcomes, because calibration reflects base rate differences rather than correcting them.
-
The impossibility theorem proves these definitions cannot all be satisfied simultaneously. When base rates differ across groups — which they almost always do in the domains where fairness matters most — satisfying calibration necessarily violates equalized odds, and vice versa. This is not a limitation of current technology; it is a mathematical law.
-
Choosing a fairness metric is a political and ethical decision. The impossibility theorem means that building a "fair" algorithm requires choosing which kind of fairness to prioritize. Prioritizing demographic parity says: equal outcomes matter most. Prioritizing equalized odds says: equal error rates matter most. Prioritizing calibration says: equal predictive accuracy matters most. These are different value commitments with different consequences for different people. No algorithm can make this choice; people must.
-
Individual fairness does not escape the politics of group fairness. Individual fairness requires that "similar" individuals receive "similar" treatment. But defining similarity is itself a value-laden act. If similarity is defined by credit scores, educational background, or zip code, structural inequality is encoded as neutral fact. The politics are not eliminated; they are hidden in the definition of similarity.
-
The fairness-accuracy trade-off is real but not absolute. Imposing fairness constraints on a system optimized for accuracy generally reduces overall accuracy. But the trade-off is not always severe, and in some cases addressing bias can improve both fairness and performance (by correcting errors that the biased system was making). The severity of the trade-off depends on the magnitude of base rate differences and the specific fairness definition chosen.
-
Base rate differences are products of structural inequality, not neutral facts. The impossibility theorem applies because base rates differ. Those differences — in recidivism, health outcomes, loan defaults, educational achievement — are consequences of centuries of discriminatory policy, not of inherent group characteristics. Accepting base rates as given and then optimizing a "fair" algorithm within those constraints is solving the wrong problem at the wrong level.
-
Algorithmic fairness and social justice are complementary, not substitutable. An algorithm cannot correct at the point of prediction what society has failed to correct at the point of causation. Fairness constraints on algorithms are necessary but insufficient. Addressing the root causes of differential base rates — poverty, discrimination, unequal access — is the work that makes algorithmic fairness constraints less binding and less consequential. You need both.
Key Concepts
| Term | Definition |
|---|---|
| Demographic parity | Fairness criterion requiring equal selection rates across groups. Also called statistical parity. |
| Equalized odds | Fairness criterion requiring equal true positive rates and equal false positive rates across groups. |
| Equal opportunity | Relaxed version of equalized odds requiring only equal true positive rates across groups. |
| Calibration | Fairness criterion requiring that among individuals predicted positive, the actual positive rate be equal across groups. Also called predictive parity. |
| Individual fairness | Fairness criterion requiring that similar individuals (by a task-relevant metric) receive similar predictions. |
| Group fairness | Any fairness criterion that evaluates equity at the level of demographic groups rather than individuals. |
| Impossibility theorem | Mathematical result proving that calibration and equalized odds cannot be simultaneously achieved when base rates differ across groups. |
| Base rate | The actual prevalence of the positive outcome in a group — the ground truth rate. |
| False positive rate (FPR) | Proportion of actual negatives incorrectly predicted positive. |
| True positive rate (TPR) | Proportion of actual positives correctly predicted positive. Also called sensitivity or recall. |
| Positive predictive value (PPV) | Proportion of positive predictions that are actually positive. Also called precision. |
| Fairness-accuracy trade-off | The tension between maximizing overall accuracy and satisfying fairness constraints, which generally require adjusting predictions in ways that reduce aggregate performance. |
Key Debates
-
Which definition of fairness should be prioritized? There is no objective answer. The choice depends on context, values, and stakeholder priorities. Criminal justice may prioritize equalized odds (protecting the innocent from disparate errors). Healthcare may prioritize calibration (ensuring accurate risk communication). College admissions may prioritize demographic parity (ensuring diverse representation). Each choice has trade-offs.
-
Should we accept base rate differences as given? If base rates reflect structural inequality, treating them as fixed constraints means building "fair" systems on top of unfair foundations. But if we reject base rates and demand equal outcomes regardless, we risk making predictions that are less accurate for everyone. The pragmatic answer: work on both levels simultaneously — constrain algorithms in the short term and address structural causes in the long term.
-
Can regulation mandate "fairness" without specifying which definition? Several jurisdictions have proposed or enacted regulations requiring "fair" algorithms. But without specifying which fairness definition is required, these regulations are incomplete — or will be defined by courts through litigation, placing the fairness choice in the hands of judges rather than democratic processes.
-
Is the impossibility theorem a reason for despair or clarity? Some see the theorem as proving that algorithmic fairness is hopeless. Others see it as liberating: it clarifies that fairness is a choice, not a puzzle with a hidden solution. Once you accept that you must choose, you can focus on making the choice transparently, inclusively, and accountably.
Looking Ahead
Chapter 15 revealed that fairness has multiple competing definitions and that they are often mutually incompatible. But there is another layer to the problem: even if we choose a fairness definition and build a system that satisfies it, can we explain how the system makes its decisions? Chapter 16, "Transparency, Explainability, and the Black Box Problem," examines what happens when algorithmic systems produce consequential decisions that no one — not the developer, not the operator, not the person affected — can explain.
Use this summary as a study reference and a quick-access card for the four fairness definitions and the impossibility theorem. These concepts will recur throughout the remainder of Part 3 and in the governance discussions of Parts 4 and 5.