Quiz: Bias and Fairness

Multiple Choice

1. Which of the following best describes "historical bias" in AI systems?

a) The AI system uses outdated software that has known bugs b) The training data accurately reflects real-world patterns of inequality, and the model learns those patterns c) The system was designed using methods from early AI research that are no longer considered best practice d) The developers intentionally programmed the system to discriminate

2. A medical AI system is trained on data from hospitals in major metropolitan areas. It performs well for patients in those hospitals but poorly for rural patients with different healthcare access patterns. What type of bias is most likely at work?

a) Aggregation bias b) Historical bias c) Representation bias d) Measurement bias

3. What is a "proxy variable"?

a) A variable that is intentionally added to increase model accuracy b) A feature so closely correlated with a protected characteristic that it effectively encodes that characteristic c) A placeholder variable used during model testing but removed before deployment d) A variable that represents another variable's value at a previous time step

4. The "fairness through unawareness" approach (removing protected characteristics from a model) is generally considered insufficient because:

a) Legal regulations require including demographic data in all models b) Other features in the model can serve as proxies for the removed characteristics c) Removing features always reduces model accuracy below acceptable levels d) Protected characteristics are required for the model to make any predictions

5. According to the impossibility theorem of fairness, when base rates differ between groups:

a) It is impossible to build an accurate model b) AI systems should not be used for decision-making c) Demographic parity, equalized odds, and calibration cannot all be satisfied simultaneously d) Only calibration can be achieved, because the other definitions are mathematically flawed

6. In the COMPAS debate, ProPublica found that Black defendants who did NOT reoffend were more likely to be incorrectly flagged as high-risk, while Northpointe showed that risk scores were calibrated (meant the same thing across groups). Why were both sides technically correct?

a) They used different datasets for their analyses b) ProPublica made a statistical error that happened to produce a true conclusion c) They measured fairness using different, mathematically incompatible definitions d) The time periods of their analyses did not overlap

7. What is a "runaway feedback loop" in the context of AI bias?

a) A system error that causes the model to retrain itself indefinitely b) A cycle in which the AI's predictions generate data that reinforces and amplifies the original predictions c) A debugging process where engineers repeatedly test and adjust the model d) A communication loop between users and developers during system design

8. Which of the following is an example of measurement bias?

a) A facial recognition system trained primarily on light-skinned faces b) A hiring algorithm that uses "years of experience" rather than "competence" because experience is easier to quantify c) A recommendation system that shows fewer results because the product catalog is small d) A translation tool that doesn't support enough languages

9. A company conducts a bias audit before deploying an AI system and finds no significant disparities. Six months later, disparities have emerged. The most likely explanation is:

a) The original audit was fraudulent b) Feedback loops, distributional shifts, or changing user populations introduced new bias over time c) The AI system randomly changed its own parameters d) The company secretly modified the system after the audit

10. Which of the following statements best captures the chapter's argument about the relationship between AI bias and structural inequality?

a) AI bias is entirely caused by structural inequality, so technical solutions are useless b) AI bias is purely a technical problem that can be solved independently of social context c) Technical bias mitigation is necessary but not sufficient; it must be accompanied by institutional and structural change d) Structural inequality is irrelevant to AI bias because algorithms are objective

Short Answer

11. Explain the difference between representation bias and aggregation bias, using an example not discussed in the chapter.

12. A content moderation system has the following performance across two language groups:

What type(s) of bias does this performance gap suggest? What are the likely causes, and what mitigation strategies would you recommend?

13. The chapter describes three layers of the bias problem: technical, institutional, and structural. Pick one real-world AI system (it can be one of the anchor examples or a different system) and give one concrete example of bias at each layer.

14. Explain why the choice between different definitions of fairness (demographic parity, equalized odds, calibration) is a moral question rather than a technical question. Use a specific example to illustrate your answer.

15. A technology company claims: "Our facial recognition system has been tested and achieves 99% accuracy." Using concepts from this chapter, explain why this claim is insufficient to evaluate whether the system is fair. What additional information would you need?

Answer Key (Selected)

1. b) Historical bias occurs when training data faithfully reflects real-world patterns of inequality.

3. b) A proxy variable is a feature so closely correlated with a protected characteristic that it effectively encodes it. Zip code as a proxy for race is a classic example.

5. c) The impossibility theorem proves that demographic parity, equalized odds, and calibration cannot all be satisfied simultaneously when base rates differ between groups.

7. b) A runaway feedback loop occurs when AI predictions generate data that reinforces the original predictions, creating a self-amplifying cycle.

10. c) The chapter argues that technical fixes are necessary but not sufficient — addressing AI bias also requires institutional reform and engagement with structural inequality.

15. A single overall accuracy figure can mask dramatic disparities across demographic groups. The Gender Shades study showed systems with high overall accuracy but error rates 35 times higher for darker-skinned women than lighter-skinned men. To evaluate fairness, you would need: accuracy broken down by gender, race/ethnicity, age, and skin tone; false positive and false negative rates per group; the demographic composition of the test set; and the intended deployment context.