Quiz: Chapter 33

Fairness, Bias, and Responsible ML

Instructions: Answer all questions. Multiple-choice questions have one correct answer unless otherwise stated. Short-answer questions should be answered in 2-4 sentences.

Question 1 (Multiple Choice)

A hiring model was trained on 10 years of historical hiring decisions from a company where 78% of successful candidates were male. The model has learned to associate male-gendered language in resumes with positive hiring outcomes. What type of bias does this represent?

  • A) Representation bias
  • B) Measurement bias
  • C) Historical bias
  • D) Aggregation bias

Answer: C) Historical bias. The training data faithfully records past hiring decisions, but those decisions reflect a decade of gender imbalance in hiring. The model is not measuring the wrong thing (measurement bias) or underrepresenting a group (representation bias). It is learning to reproduce a pattern that reflects systemic inequality. Removing gender from the feature set does not fix this --- gendered language patterns, extracurricular activities, and career trajectories serve as proxy variables.

Question 2 (Multiple Choice)

A hospital readmission model achieves an AUC of 0.85 for white patients and 0.71 for Asian patients. The training data contains 6,390 white patients and 710 Asian patients. What is the most likely primary cause of the performance gap?

  • A) Historical bias --- the model learned that Asian patients are inherently different
  • B) Representation bias --- the model has insufficient data to learn patterns for Asian patients
  • C) Measurement bias --- clinical features are recorded differently for Asian patients
  • D) Aggregation bias --- Asian patients have a different feature-outcome relationship

Answer: B) Representation bias. With 6,390 white patients and only 710 Asian patients (a 9:1 ratio), the model has far more information about patterns that predict readmission for white patients. The model's lower AUC for Asian patients is not because Asian patients are harder to predict --- it is because the model has not seen enough Asian patients to learn their patterns. Oversampling, targeted data collection, or separate models for underrepresented groups are potential mitigations.

Question 3 (Short Answer)

Explain why removing the race variable from a model's feature set is insufficient to prevent racial bias in predictions. Use the concept of proxy variables in your answer.

Answer: Removing the protected attribute (race) only eliminates direct use of that variable. Many other features in the model are correlated with race --- zip code, insurance type, primary language, admission source, and comorbidity patterns all carry racial information because they reflect the same systemic structures that produce racial disparities. These correlated features act as proxy variables, allowing the model to reconstruct much of the racial signal indirectly. This approach is called "fairness through unawareness" and is widely recognized as the least effective fairness strategy. The model may still produce disparate outcomes across racial groups even without direct access to the race variable.

Question 4 (Multiple Choice)

A model satisfies demographic parity if:

  • A) The model's precision is the same for all groups
  • B) The probability of a positive prediction is the same for all groups, regardless of the true label
  • C) The true positive rate and false positive rate are the same for all groups
  • D) The model's predicted probabilities are calibrated within each group

Answer: B) The probability of a positive prediction is the same for all groups, regardless of the true label. Demographic parity requires $P(\hat{Y} = 1 \mid A = a) = P(\hat{Y} = 1 \mid A = b)$ for all groups $a$ and $b$. It does not condition on the true label. Option A describes predictive parity. Option C describes equalized odds. Option D describes calibration by group.

Question 5 (Multiple Choice)

A readmission model has the following true positive rates by racial group: White 0.74, Black 0.61, Hispanic 0.68, Asian 0.65. Which fairness criterion is violated?

  • A) Demographic parity only
  • B) Equal opportunity (and therefore equalized odds)
  • C) Predictive parity only
  • D) Calibration by group only

Answer: B) Equal opportunity (and therefore equalized odds). Equal opportunity requires equal TPR across groups, and these TPRs range from 0.61 to 0.74 --- a 13 percentage point gap. Since equalized odds requires both equal TPR and equal FPR, and equal TPR is already violated, equalized odds is also violated. The TPR tells us nothing directly about demographic parity (which concerns overall positive prediction rates) or predictive parity (which concerns precision) or calibration (which concerns probability accuracy).

Question 6 (Short Answer)

State the impossibility theorem in your own words. Then explain why it matters for the Metro General readmission model, given that Black patients have a higher base rate of readmission than white patients.

Answer: The impossibility theorem states that when base rates (the actual positive rates) differ between groups, no model can simultaneously achieve equal false positive rates across groups, equal false negative rates across groups, and calibrated probabilities within each group. For Metro General, Black patients have a higher readmission base rate than white patients. This means the hospital must choose: they can equalize how often the model catches readmissions across groups (equal TPR), or they can equalize how often the model falsely flags non-readmissions across groups (equal FPR), or they can ensure the model's probability estimates are accurate within each group (calibration) --- but they cannot have all three. This is not a technical limitation that better algorithms will solve; it is a mathematical constraint that requires a values-based decision about which type of error equality matters most.

Question 7 (Multiple Choice)

Which of the following is a post-processing bias mitigation technique?

  • A) Reweighting the training data to equalize group representation
  • B) Adding a fairness regularization term to the loss function
  • C) Applying group-specific classification thresholds to equalize error rates
  • D) Oversampling underrepresented groups using SMOTE

Answer: C) Applying group-specific classification thresholds to equalize error rates. Post-processing techniques modify the model's outputs after training, without changing the model itself. Group-specific thresholds are the most common post-processing technique --- each group gets a different threshold chosen to equalize a fairness metric (TPR, FPR, or precision). Options A and D are pre-processing (modify the data before training). Option B is in-processing (modify the training algorithm).

Question 8 (Multiple Choice)

You apply group-specific thresholds to the Metro General model to equalize TPR across racial groups at 0.70. Overall accuracy drops from 0.76 to 0.74. What is the most accurate interpretation?

  • A) The model became worse and should not be deployed
  • B) The model became fairer at the cost of a 2 percentage point accuracy reduction, and the organization must decide whether that tradeoff is acceptable
  • C) The accuracy drop proves that fairness and accuracy are fundamentally incompatible
  • D) Group-specific thresholds are a flawed technique and should be avoided

Answer: B) The model became fairer at the cost of a 2 percentage point accuracy reduction, and the organization must decide whether that tradeoff is acceptable. This is the fairness-accuracy tradeoff in practice. A 2-point accuracy reduction is typical and often acceptable. The decision is not technical --- it is a values judgment about whether equitable error rates across racial groups are worth a small reduction in overall performance. That decision belongs to clinical leadership, ethics committees, and the communities served by the hospital, not solely to the data science team.

Question 9 (Short Answer)

What is a model card, and why is it important for responsible ML deployment? Name at least four sections a model card should include.

Answer: A model card is a standardized document that accompanies a deployed model, describing its purpose, performance, limitations, and fairness properties. It serves as the primary documentation artifact for responsible deployment, enabling auditors, users, and governance teams to understand what the model does, who it was evaluated on, where it fails, and how fairly it performs across demographic groups. Key sections include: (1) Model Details (name, version, description), (2) Intended Use and out-of-scope applications, (3) Training Data description, (4) Performance metrics (overall and per-group), (5) Fairness Metrics, (6) Limitations and known failure modes, (7) Ethical Considerations, and (8) Evaluation Data description.

Question 10 (Multiple Choice)

The disparate impact ratio for a particular group is 0.72 relative to the reference group. Under the 80% rule (four-fifths rule), this means:

  • A) The model is within acceptable bounds
  • B) There is evidence of disparate impact for that group
  • C) The model satisfies equalized odds
  • D) The model is perfectly calibrated for that group

Answer: B) There is evidence of disparate impact for that group. The disparate impact ratio compares the positive prediction rate of a group to that of a reference group. A ratio below 0.80 (the four-fifths threshold) is generally considered evidence of disparate impact. At 0.72, the group is receiving positive predictions at 72% of the rate of the reference group, which falls below the 0.80 threshold. This does not automatically mean the model is illegal or must be changed, but it flags a disparity that warrants investigation.

Question 11 (Multiple Choice)

A model satisfies equalized odds but violates demographic parity. What does this tell you about the data?

  • A) The model is biased and needs mitigation
  • B) The base rates (actual positive rates) likely differ between groups
  • C) The model uses the protected attribute as a feature
  • D) The training data has representation bias

Answer: B) The base rates (actual positive rates) likely differ between groups. Equalized odds requires equal TPR and FPR across groups. If equalized odds is satisfied but demographic parity is not, the model predicts positive at different rates for different groups --- but this is justified by different base rates. Groups with higher actual positive rates will have higher positive prediction rates. This is exactly the situation the impossibility theorem describes: when base rates differ, equalized odds and demographic parity are in tension.

Question 12 (Short Answer)

You are presenting the results of a fairness audit to Metro General's Chief Medical Officer. She asks: "So should we use group-specific thresholds or not?" Draft a 3-sentence response that honestly presents the tradeoff and explains what the decision hinges on.

Answer: Group-specific thresholds allow us to equalize how effectively the model identifies at-risk patients across racial groups --- right now, the model catches 74% of white readmissions but only 61% of Black readmissions, and threshold adjustment can bring both to approximately 70%. The cost is a 2-point drop in overall accuracy, meaning a few more patients in some groups will receive unnecessary follow-up calls. The decision hinges on whether equitable detection of at-risk patients across racial groups is worth that small accuracy cost --- and given that each missed readmission represents a patient who does not receive the care coordination call that could prevent a $14,400 hospital stay, I would recommend the equalized thresholds, but that is ultimately a clinical and institutional values decision that should involve your ethics committee and patient advisory board.

Question 13 (Multiple Choice)

Which of the following statements about the fairness-accuracy tradeoff is most accurate?

  • A) Fairness always requires a large sacrifice in accuracy
  • B) With enough data and the right algorithm, you can always eliminate the tradeoff
  • C) The accuracy cost of fairness is usually small (1-3 percentage points), and the tradeoff is a values decision about whether to accept it
  • D) Fairness and accuracy are completely independent --- improving one never affects the other

Answer: C) The accuracy cost of fairness is usually small (1-3 percentage points), and the tradeoff is a values decision about whether to accept it. Empirical evidence across many domains shows that equalizing error rates across groups typically reduces overall accuracy by a modest amount. The impossibility theorem guarantees that some tradeoff exists when base rates differ, so option B is wrong. Option A overstates the cost. Option D ignores the mathematical reality of the impossibility theorem.

Question 14 (Multiple Choice)

You are building a loan approval model. Which fairness metric is most appropriate if your primary concern is that applicants of equal creditworthiness should be approved at equal rates, regardless of race?

  • A) Demographic parity
  • B) Equalized odds
  • C) Predictive parity
  • D) Calibration by group

Answer: B) Equalized odds. The question asks about applicants of "equal creditworthiness" --- meaning we are conditioning on the true label (will this person actually default or not?). Equalized odds requires that, among people who will actually default, the model flags them at the same rate across groups (equal TPR), and among people who will not default, the model incorrectly flags them at the same rate across groups (equal FPR). This directly captures the idea that equally qualified applicants should receive equal treatment. Demographic parity does not condition on the true label. Predictive parity concerns what happens after a positive prediction, not whether equally qualified people are treated equally.

Question 15 (Short Answer)

Explain why fairness monitoring in production is necessary even if a model passes a fairness audit at deployment time. Give a specific example of how fairness can degrade after deployment.

Answer: Fairness metrics can drift in production just as accuracy metrics can, because the demographic composition of incoming data and the relationships between features and outcomes may change over time. For example, Metro General might experience a shift in its patient population --- perhaps a new insurance program enrolls more patients from underserved communities, changing the racial composition of incoming patients. If the new patients have different feature distributions (more comorbidities, different insurance types) than the patients in the original training data, the model's error rates for that demographic may worsen even though the model has not changed. Without ongoing fairness monitoring that tracks per-group TPR, FPR, and calibration alongside the overall performance metrics from Chapter 32, the hospital would not detect that its equalized thresholds are no longer producing equitable outcomes.

This quiz accompanies Chapter 33: Fairness, Bias, and Responsible ML. Return to the chapter for full context.