Exercises: Bias and Fairness
Comprehension Exercises
Exercise 9.1: Mapping the Pipeline
For each of the following AI systems, identify which stage of the pipeline is the most likely primary source of bias. Explain your reasoning in 2–3 sentences.
a) A language translation tool that performs well for European languages but poorly for many African and Indigenous languages.
b) A credit scoring algorithm trained on loan repayment data from the past 30 years, a period during which lending discrimination was documented and widespread.
c) A hiring tool that uses "cultural fit" ratings from interviewers as a label for identifying good candidates.
d) A medical imaging AI that classifies lung nodules as benign or malignant, trained on a dataset where 94% of patients were white.
e) A content recommendation system on a news site that optimizes for click-through rate, which ends up promoting sensationalized content about certain communities.
Exercise 9.2: Bias Type Identification
Classify each of the following scenarios as primarily involving historical bias, representation bias, measurement bias, or aggregation bias. Some scenarios may involve more than one type — identify all that apply and explain which is primary.
a) An AI recruiting tool is trained on a company's hiring data from 2005–2015. During that period, the company's engineering team was 85% male. The tool learns to favor resumes with characteristics associated with male applicants.
b) A speech recognition system works well for American English speakers but frequently misunderstands speakers with Indian, Nigerian, or Scottish accents.
c) A predictive policing tool uses arrest records as its measure of "crime." Because drug enforcement has historically targeted certain neighborhoods, the arrest data overrepresents drug crime in those areas relative to actual drug use rates.
d) A clinical decision support tool uses a single risk model for liver disease across all patients. However, the relationship between alcohol consumption and liver damage varies significantly based on genetic factors that differ across ethnic groups.
e) A university uses an AI system to predict which admitted students are "at risk" of dropping out. The system uses first-semester GPA as its primary outcome variable, but first-generation college students often have lower first-semester GPAs due to adjustment challenges and then recover — while the AI flags them as high-risk.
Exercise 9.3: The Fairness Trade-Off
A hospital is deploying an AI system that flags patients who may need additional follow-up care after surgery. The system produces a risk score from 0 to 100.
The hospital's data shows the following: - Among older patients (65+), 40% actually experience complications requiring follow-up. - Among younger patients (18–40), 10% experience complications requiring follow-up.
The hospital wants the system to be "fair" across age groups.
a) If the hospital applies demographic parity (flags the same percentage of patients in each age group), what problems might arise? Who could be harmed?
b) If the hospital applies calibration (ensures that a risk score of, say, 70 means a 70% chance of complications regardless of age group), why will it necessarily flag a higher proportion of older patients?
c) If the hospital applies equalized odds (ensures the false positive and false negative rates are the same across age groups), what trade-off is it making?
d) In this specific context — surgical follow-up care — which definition of fairness do you think is most appropriate, and why? Is there a "right answer"?
Application Exercises
Exercise 9.4: Proxy Variable Hunt
For each protected characteristic listed below, identify at least two variables that could serve as proxy variables in a typical dataset. Then explain why simply removing the protected characteristic from the model would be insufficient to prevent discrimination.
a) Race/ethnicity b) Gender c) Socioeconomic status d) Disability status
Exercise 9.5: ContentGuard Bias Audit
You are conducting a bias audit of ContentGuard, the content moderation system. Using the bias audit framework from the chapter, work through the following:
a) Problem formulation: ContentGuard is designed to identify "harmful content." What assumptions are embedded in this framing? Who defines "harmful"?
b) Data collection: The training data consists of content flagged by human moderators over the past three years. What groups might be overrepresented or underrepresented in this data? Why?
c) Measurement: ContentGuard uses "content flagged by moderators" as its ground truth for harmfulness. Why might this be a problematic measure? What alternative measures could you propose?
d) Deployment: ContentGuard is deployed globally on a platform used in 190+ countries. What types of bias might emerge in deployment that were not present in training?
e) Recommendations: Propose two specific changes (one technical, one organizational) that could reduce bias in ContentGuard.
Exercise 9.6: The Résumé Experiment
Consider an experiment (based on real research) in which identical résumés are submitted to an AI screening tool, but the names at the top are changed. One set of résumés has names commonly associated with white applicants ("Emily Walsh," "Greg Baker"). Another set has names commonly associated with Black applicants ("Lakisha Washington," "Jamal Jones").
a) If the AI screening tool produces different scores for identical résumés with different names, what type of bias is at work?
b) Some companies argue that their AI tools don't use names in scoring — names are stripped out before analysis. Does this solve the problem? Why or why not? (Hint: think about proxy variables.)
c) How would you design a study to test whether an AI hiring tool discriminates, even when names are removed?
d) If you found evidence of discrimination, what mitigation strategy would you recommend — pre-processing, in-processing, or post-processing? Justify your choice.
Analysis Exercises
Exercise 9.7: Evaluating a Fairness Claim
A technology company publishes a blog post announcing their new AI hiring tool with the following claim: "Our system is completely unbiased. We have removed all protected characteristics from the model, and our accuracy rate is 94% across all applicants."
Write a 300–400 word critical analysis of this claim. Your analysis should address: - Why removing protected characteristics is insufficient - Why a single overall accuracy rate can be misleading - What additional information you would need to evaluate fairness - At least two specific questions you would ask the company
Exercise 9.8: The Impossibility in Practice
Read the following scenario, then answer the questions below.
A state government is evaluating an AI tool that predicts which individuals on parole are likely to commit new offenses. The tool will be used to determine the level of supervision (high, medium, low) each parolee receives. An independent audit reveals the following:
- The tool correctly identifies 80% of white parolees who reoffend (sensitivity/true positive rate).
- The tool correctly identifies 60% of Black parolees who reoffend.
- Among parolees who do NOT reoffend, 15% of white parolees are incorrectly flagged as high-risk.
- Among parolees who do NOT reoffend, 30% of Black parolees are incorrectly flagged as high-risk.
a) Which fairness criterion is being violated here? Explain in your own words what the numbers reveal about the system's behavior.
b) If the vendor adjusts the tool to equalize the false positive rates across groups, what other fairness criterion might be affected?
c) A state legislator argues: "The tool is just reflecting reality — the reoffense rates are different across groups." How would you respond, drawing on the concepts from this chapter?
d) Propose a policy recommendation for how the state should proceed. Should it modify the tool? Restrict its use? Abandon it? Justify your recommendation with reference to the frameworks in this chapter.
Reflection Exercises
Exercise 9.9: Your Own Bias Encounters
Describe a time when you personally experienced or observed algorithmic bias — a search result that seemed off, a recommendation that reflected assumptions about you, a filter that flagged something inappropriately. Analyze the experience using the vocabulary from this chapter: - What type of bias do you think was involved? - At what pipeline stage did the bias likely enter? - Who was harmed, and was the harm visible or invisible?
Exercise 9.10: The Fairness Values Question
The impossibility theorem tells us that we must choose between definitions of fairness. But how should we choose?
Write a 200–300 word reflection on who should make fairness trade-off decisions for AI systems, and how. Consider: - Should it be engineers? Executives? Government regulators? Affected communities? Some combination? - What democratic processes or institutional structures might be needed? - How do you balance efficiency (someone needs to make a decision) with legitimacy (the affected people should have a voice)?