Chapter 8: Exercises

Difficulty Scale: - ⭐ Foundational — tests basic comprehension and vocabulary - ⭐⭐ Developing — applies concepts to new scenarios - ⭐⭐⭐ Advanced — requires synthesis and critical analysis - ⭐⭐⭐⭐ Expert — open-ended research, organizational design, or original analysis

† Recommended for team/classroom discussion or assignment submission

Part A: Vocabulary and Concept Checks

Exercise 1 ⭐ Match each bias type to its stage in the ML lifecycle:

Write a one-sentence explanation of why each bias type is difficult to detect using standard model performance metrics.

Exercise 2 ⭐ Define each of the following terms in your own words without consulting the chapter text. Then compare your definitions to the Vocabulary Builder section and identify any gaps or mischaracterizations.

a) Proxy variable b) Benchmark c) Occult hypoxemia d) Disaggregated metrics e) Alignment tax

Exercise 3 ⭐ True or False — for each statement, write True or False and explain your reasoning in two to three sentences:

a) Removing race from a training dataset eliminates the risk of racial discrimination in model outputs.

b) A model with 97% overall accuracy is always preferable to a model with 93% overall accuracy for high-stakes decisions.

c) Historical bias is a type of data collection error that can be corrected by improving sampling methodology.

d) RLHF completely eliminates cultural bias from large language models that have been fine-tuned using this technique.

e) Aggregation bias occurs when a model performs differently on subgroups, even though the same algorithm was applied uniformly to all of them.

Part B: Application Exercises

Exercise 4 ⭐⭐ A financial technology company is building a small business loan scoring model. The following features are under consideration for inclusion:

• Business revenue (last 12 months)
• Business age (years in operation)
• Owner's personal credit score
• Business zip code
• Owner's educational institution
• Industry category (SIC code)
• Number of prior business bankruptcies
• Owner's first name

For each feature: (a) identify whether it may function as a proxy for a protected characteristic, (b) identify which protected characteristic it may proxy for, and (c) explain the mechanism by which the proxy relationship arises. Conclude with a recommendation about whether the feature should be included, excluded, or included with additional safeguards.

Exercise 5 ⭐⭐ † A large urban hospital system is evaluating a clinical sepsis prediction AI. The vendor's evaluation report shows:

• Overall AUROC: 0.89 (excellent)
• Overall sensitivity at 85% threshold: 81%
• Overall specificity at 85% threshold: 79%
• No disaggregated performance data by demographic group

Write a memo (600–800 words) from the perspective of the hospital's Chief Medical Officer to the procurement committee. The memo should: (1) explain what information is missing and why it matters, (2) identify the specific risks of deploying the system without disaggregated data, (3) specify what additional information the committee should request from the vendor before making a deployment decision, and (4) recommend a conditional approval or rejection, with justification.

Exercise 6 ⭐⭐ The following historical datasets are being considered for use in training AI systems. For each, identify: (a) the type of historical bias present, (b) the specific historical conditions that introduced the bias, and (c) what risk the bias creates if the data is used for training without intervention.

a) Resume data from a Fortune 500 company's historic hiring records (1975–2005) to train a resume screening model.

b) Police arrest data from three major US cities (2000–2015) to train a predictive policing risk model.

c) Home mortgage application approval data from a regional bank (1980–2000) to train a loan underwriting model.

d) Emergency room treatment records from an academic medical center (1990–2010) to train a triage prioritization model.

Exercise 7 ⭐⭐ Examine the following scenario and identify which of the six bias types from the Suresh and Guttag taxonomy is (or are) present. Provide specific evidence for each identified bias type.

Scenario: A retail company trains a customer churn prediction model using data from its loyalty program members. The loyalty program was launched in 2018 and disproportionately attracts customers who shop online (70% of members shop primarily online; 30% primarily in-store). The company serves a roughly equal mix of online and in-store customers in its target market. The model is trained to predict 60-day churn and achieves 88% accuracy on a held-out test set drawn from loyalty program members. The company then deploys the model to predict churn for all customers, including the 50% who are not loyalty program members.

Exercise 8 ⭐⭐ † Conduct a brief feature audit for the following hypothetical model. For each feature listed, calculate a qualitative "proxy risk" (Low / Medium / High) for the specified protected characteristic, and justify your rating with reference to documented social patterns.

Model: Predicts whether a job applicant will receive an interview invitation.

Features: 1. College GPA — proxy risk for socioeconomic status? 2. College name — proxy risk for race? 3. Internship employer name — proxy risk for socioeconomic status? 4. LinkedIn connection count — proxy risk for age? 5. Years of prior work experience — proxy risk for age? 6. Extracurricular activity keywords — proxy risk for religion? 7. Resume language formality score (NLP-derived) — proxy risk for national origin?

Conclude with a recommendation for which features should undergo additional review before the model is deployed.

Part C: Analysis and Synthesis

Exercise 9 ⭐⭐⭐ The COMPAS recidivism risk assessment tool was found by ProPublica to assign higher risk scores to Black defendants at significantly higher rates than to white defendants with comparable criminal histories. The tool's developers, Northpointe, countered that the tool was "fair" because it had equal predictive accuracy (calibration) across racial groups — the probability that a person with a given score would reoffend was similar for Black and white defendants.

a) Explain the two different fairness metrics at play (predictive accuracy / calibration vs. error rate parity) and why they conflict.

b) How does historical bias in arrest data — specifically, differential policing intensity — contribute to the observed disparities, regardless of which fairness metric is used to evaluate the tool?

c) If you were advising a state criminal justice reform commission, what would you recommend regarding the use of COMPAS or similar tools in bail and sentencing decisions? Your recommendation should engage with: (1) the feasibility of alternatives to algorithmic risk tools, (2) the question of who bears the risk of false positive and false negative errors, and (3) the governance structures required for accountable use of algorithmic tools in criminal justice.

Exercise 10 ⭐⭐⭐ A hospital system has adopted an AI triage tool that prioritizes patients for care based on predicted health need. An internal audit has found that Black patients with equivalent predicted health needs are being prioritized lower than white patients. The root cause analysis reveals that the model uses healthcare cost as a proxy for health need — and Black patients historically incur lower healthcare costs than white patients with equivalent underlying illness burden, because they have faced greater barriers to accessing care.

a) Identify the specific bias type(s) responsible for the observed disparity.

b) Explain why the disparity exists even if the model has no explicit race variable.

c) Propose three remediation options for the hospital system, and evaluate the tradeoffs (technical feasibility, clinical validity, legal risk, resource requirements) of each option.

d) Semmel et al. (2019, "Dissecting racial bias in an algorithm used to manage the health of populations") documented exactly this problem with a widely used commercial health management algorithm. What regulatory or accreditation changes would prevent similar problems from occurring in future AI-assisted clinical tools?

Exercise 11 ⭐⭐⭐ † You are the AI Ethics Lead at a mid-size bank preparing to respond to an RFP for an AI-powered commercial credit underwriting platform. Three vendors have submitted proposals. Using the following criteria, develop a vendor evaluation rubric for bias risk:

1. Training data documentation (datasheet or equivalent)
2. Representation of underserved communities in training data
3. Evaluation methodology — disaggregated performance metrics
4. Fairness testing approach and results
5. Proxy variable audit documentation
6. Post-deployment monitoring capabilities
7. Model card availability and quality
8. Contractual responsibility for discriminatory outcomes

For each criterion: (a) describe what adequate vendor documentation would look like, (b) describe what would constitute a red flag, and (c) assign a weight (1–5) reflecting the criterion's relative importance for regulatory compliance and ethical risk management. Justify your weights.

Exercise 12 ⭐⭐⭐ Language model bias is sometimes defended on the grounds that the model is "just reflecting the data" — and the data is an accurate representation of text that humans have written. Evaluate this defense.

Your response should address: (a) whether accuracy of representation is sufficient ethical justification for reproducing harmful associations, (b) whether the argument proves too much (i.e., would justify historical discrimination on the grounds that it reflected historical attitudes), (c) what obligations model developers have beyond accurate representation of their training data, and (d) whether the "just reflecting the data" defense is more or less plausible for different types of bias (historical bias, representation bias, stereotype propagation).

Write your response as a structured ethical argument of 600–800 words.

Part D: Organizational Design and Policy

Exercise 13 ⭐⭐⭐ † Design a "Bias Prevention Protocol" for an organization beginning to develop AI systems for use in employee performance evaluation. The protocol should specify:

a) Data requirements: What data is permissible to use? What historical data requires additional scrutiny? What data is prohibited?

b) Development practices: Who must be involved in model development? What documentation is required at each stage?

c) Evaluation requirements: What performance metrics must be computed? What demographic breakdowns are required? What fairness thresholds must be met before deployment?

d) Governance: Who approves deployment? Who has authority to halt deployment or discontinue use? How are identified bias problems escalated?

e) Post-deployment monitoring: What is monitored, how frequently, and by whom? What triggers mandatory re-evaluation?

Your protocol should be 800–1,000 words and should be written in the form of an organizational policy document, not an academic essay.

Exercise 14 ⭐⭐⭐ The "Datasheets for Datasets" framework (Gebru et al., 2018) recommends that dataset creators document: motivation, composition, collection process, preprocessing, uses, distribution, and maintenance.

Select one of the following widely used public datasets and prepare a mock datasheet, answering each of the seven categories to the best of your knowledge, and explicitly flagging areas where you do not have sufficient information:

a) ImageNet (large-scale visual dataset) b) Common Crawl (web text corpus) c) Wikipedia English dump (as used in LLM training)

After completing the datasheet, write a 300-word assessment: what does your datasheet reveal about the bias risks of using this dataset to train a model intended to serve a global, demographically diverse user base?

Exercise 15 ⭐⭐⭐⭐ Conduct original research to identify a current commercial AI system that has been reported to exhibit one or more of the six bias types from the Suresh and Guttag taxonomy. (Sources may include peer-reviewed research, investigative journalism, regulatory findings, or civil society organization reports.)

Write a structured case study of 1,200–1,500 words that: 1. Describes the system and its intended function 2. Identifies the specific bias type(s) present and the evidence for each 3. Analyzes the organizational and technical failures that allowed the bias to persist 4. Evaluates any responses by the developer or deployer 5. Proposes specific, concrete remediation measures 6. Assesses the broader regulatory and accountability questions raised

Include at least five cited sources. At least two should be peer-reviewed research; at least one should be a primary source (regulatory document, company disclosure, or original data).

Part E: Scenario-Based Judgment

Exercise 16 ⭐⭐ † You are a product manager at a software-as-a-service company that provides AI-powered hiring tools to corporate clients. A client — a major manufacturing company — reports that their HR director has observed that the tool appears to score candidates with "Southern European" surnames lower than candidates with "Northern European" surnames on a composite "cultural fit" metric. The client is asking whether this is a bias problem.

a) What questions would you ask to begin diagnosing the source of the disparity?

b) What immediate interim measures would you recommend while the investigation is ongoing?

c) If the investigation confirms that the tool encodes national origin bias in its cultural fit scoring, what are your company's obligations to: (i) the current client, (ii) other current clients using the same product, (iii) candidates who have been harmed by previous biased scoring?

d) What changes to your development process would prevent similar issues in future products?

Exercise 17 ⭐⭐⭐ An investment in a new clinical AI tool for a hospital system requires the approval of both the clinical leadership and the hospital's AI Ethics Committee. The tool has the following evaluation characteristics:

• Overall diagnostic accuracy: 91%
• Accuracy for white patients: 94%
• Accuracy for Black patients: 82%
• Accuracy for Hispanic patients: 85%
• Accuracy for Asian patients: 89%

The vendor argues that the tool should be approved because its 91% aggregate accuracy surpasses the 87% accuracy of the current manual diagnostic process, and the manual process itself has documented racial disparities.

Write a structured argument (400–500 words) for the position that this tool should NOT be deployed as-is, followed by a structured argument (400–500 words) for the position that it SHOULD be deployed, with conditions. Then write a 200-word synthesis identifying which position you find more persuasive and why.

Exercise 18 ⭐⭐⭐⭐ † You have been appointed to a state government task force on AI in the criminal justice system. The task force's mandate is to recommend standards for the use of algorithmic risk assessment tools in bail, sentencing, and parole decisions. The task force must balance: (1) evidence that well-validated risk assessment tools can reduce human inconsistency and implicit bias in judicial decisions, (2) evidence that algorithmic tools trained on historically biased data reproduce and amplify that bias, and (3) constitutional and civil rights concerns about algorithmic opacity in liberty-restricting decisions.

Prepare a 1,000–1,200 word policy recommendation memo that addresses: a) The conditions under which algorithmic risk assessment tools may be used b) The data and methodological standards tools must meet c) The transparency and explainability requirements for defendants and their counsel d) The governance and oversight structures required e) A sunset review requirement — when and how the standards will be re-evaluated

Your memo should acknowledge the strongest arguments on all sides of the policy debate.

Exercise 19 ⭐⭐ The pulse oximeter case illustrates measurement bias in a physical device that generates data subsequently used in AI clinical decision systems. Identify two other physical measurement devices or sensors commonly used in clinical AI systems and investigate whether measurement bias by demographic group has been documented for those devices.

For each device: (a) describe the physical measurement principle, (b) identify any documented demographic variation in measurement accuracy, (c) assess the consequence for AI systems that use the device's output as training data or as input features, and (d) identify what regulatory or clinical practice changes would mitigate the risk.

Exercise 20 ⭐⭐ Review the documented Gemini historical image generation controversy (early 2024, in which Google's Gemini model generated racially diverse images for historical figures who were factually white Europeans, such as Nazi German soldiers depicted as people of color).

a) Identify what types of bias (from the Suresh and Guttag taxonomy) this incident illustrates — or does it illustrate a different type of problem entirely?

b) The controversy was used by some commentators to argue that AI safety and bias mitigation efforts had "gone too far" — prioritizing demographic balance over factual accuracy. Evaluate this argument.

c) What does the incident reveal about the challenges of RLHF as a bias mitigation tool?

d) What design and governance changes would have prevented this specific failure while still pursuing the legitimate goal of reducing demographic bias in AI-generated images?

Part F: Reflection and Integration

Exercise 21 ⭐⭐ † Reflect on an AI system you use regularly in your personal or professional life — a recommendation system, a search engine, a voice assistant, a navigation tool, or any other AI-powered product.

a) Identify which of the six bias types you think is most likely to affect this system, based on what you know or can infer about its development.

b) Describe a specific experience that might be evidence of bias in the system's outputs.

c) What information would you need to confirm or rule out your hypothesis?

d) What would you want the system's developer to disclose about the system's training data and evaluation methodology?

Exercise 22 ⭐⭐⭐ Chapter 8 opens with the observation that Google addressed the "gorilla" labeling incident by blocking words rather than fixing the underlying model. Nine years later, the underlying bias had not been corrected.

a) What does this response reveal about organizational incentives in AI bias remediation?

b) Apply the concept of "ethics washing" to this case. Does Google's response constitute ethics washing? What markers are present, and what markers of genuine ethics are absent?

c) What would a genuine remediation effort have required, in terms of technical work, organizational resources, and public accountability?

d) What should a regulator or civil society organization have demanded from Google after the initial incident, and what accountability mechanism should have enforced those demands?

Exercise 23 ⭐⭐ The chapter argues that "the communities most affected by AI bias are frequently excluded from the processes that produce and evaluate AI systems." Identify three specific mechanisms by which community participation can be incorporated into AI development processes, at three different stages: (1) data collection, (2) evaluation design, and (3) post-deployment monitoring.

For each mechanism: describe what meaningful participation looks like (as distinct from tokenistic consultation), identify the organizational conditions required to make it genuine, and describe a realistic obstacle to its implementation.

Exercise 24 ⭐⭐⭐⭐ † Design a semester-long "Bias Audit Practicum" for a data science team at a mid-size financial services company. The practicum should be practical and directly applicable to the team's actual work. It should:

a) Identify the 5–7 most important skills a data scientist needs to conduct bias audits effectively b) For each skill, specify a learning activity, a practice exercise, and an assessment criterion c) Include a team project in which participants conduct a bias audit of a real model the company is using or developing d) Specify what documentation artifacts the team will produce e) Identify the subject-matter experts (internal and external) the practicum should draw on

Write the practicum design as a structured curriculum document of 800–1,000 words.

Exercise 25 ⭐⭐⭐ The chapter introduces a tension between technical solvability and social roots: many AI bias problems have technical components but social causes. Using two specific examples from Chapter 8 (one from any of the six bias types and one from the LLM section), argue for or against the following proposition:

"Technical interventions to reduce AI bias are valuable and necessary, but organizations that focus primarily on technical fixes while neglecting the social conditions that produce bias are engaging in a form of ethics washing — substituting tractable technical action for harder, more consequential social and organizational change."

Your response should: (a) accurately characterize the technical interventions available for each example, (b) accurately characterize the social conditions producing the bias, (c) assess the relative effectiveness of technical vs. social/organizational interventions, and (d) take a clear position on the proposition with evidence-based justification. Length: 700–900 words.