Chapter 4 Quiz
Test your understanding of the key concepts from this chapter. Try to answer each question before revealing the answer.
Q1. What does the phrase "garbage in, garbage out" mean in the context of AI?
Show Answer
If the training data is flawed — biased, incomplete, inaccurate, or unrepresentative — the AI system built on it will produce flawed outputs. The quality of an AI system's predictions can never exceed the quality of its training data.Q2. Which of the following is an example of structured data? - (a) A collection of X-ray images - (b) A spreadsheet of patient records with columns for age, diagnosis, and blood pressure - (c) A folder of audio recordings from doctor-patient consultations - (d) A set of handwritten clinical notes
Show Answer
**(b)** A spreadsheet of patient records with clearly defined columns and data types is structured data. The other options are all forms of unstructured data.Q3. What is the key difference between labeled and unlabeled data?
Show Answer
Labeled data has been tagged with the correct answer or category that the AI is supposed to learn (e.g., an email marked "spam" or "not spam"). Unlabeled data has no such tags — it is raw information without pre-assigned categories. Supervised learning requires labeled data; unsupervised learning works with unlabeled data.Q4. A hiring AI is trained on a company's past decisions about which candidates to invite for interviews. The company has historically hired very few women for engineering roles. What type of bias does this represent?
Show Answer
This is primarily **historical bias** — the training data accurately reflects a history of gender-imbalanced hiring, and the AI learns to replicate that pattern. It may also involve **selection bias** if the company's applicant pool was itself not representative of the broader population.Q5. Explain why removing a sensitive variable (like race) from training data does not necessarily prevent an AI system from discriminating based on that variable.
Show Answer
Other variables in the dataset can serve as **proxy variables** that are correlated with the removed attribute. For example, zip code can be a proxy for race due to residential segregation; certain school names may correlate with gender; first names can indicate ethnicity. The AI can learn to reconstruct the removed variable's influence from these proxies.Q6. What is "ghost data"?
Show Answer
Ghost data refers to information about people, situations, or phenomena that was never collected or recorded in the first place. It represents the bias of *absence* — groups or experiences that are missing from the dataset entirely. Ghost data is particularly dangerous because the AI system has no way of knowing what it does not know; missing populations simply do not appear as errors.Q7. CityScope Predict is trained on historical arrest data to predict where crimes will occur. Describe the feedback loop this creates and explain why it is problematic.
Show Answer
The feedback loop works as follows: (1) Historical arrest data shows more arrests in certain neighborhoods. (2) CityScope Predict identifies these as "high crime" areas. (3) More police are sent to these neighborhoods. (4) More police presence leads to more arrests, regardless of actual crime rates. (5) New arrest data reinforces the original pattern. (6) The cycle repeats and amplifies. This is problematic because the AI's predictions become self-fulfilling prophecies that perpetuate and deepen existing disparities in policing.Q8. What is "ground truth" in machine learning, and why is the term potentially misleading?
Show Answer
Ground truth refers to the correct answer or label that a model is trying to learn. The term is potentially misleading because it implies objective fact, when in many cases the "correct" answer is actually a human judgment, an institutional decision, or a social construction. For example, a "ground truth" label of "high risk patient" may reflect a doctor's assessment that was itself influenced by biases about race, gender, or socioeconomic status.Q9. Name three common sources of AI training data and identify one limitation or ethical concern for each.
Show Answer
Examples include: 1. **User-generated content** (social media, reviews) — users typically did not consent to AI training use; content reflects the demographics of platform users, not the general population. 2. **Institutional records** (medical, legal, financial) — reflect historical biases and discriminatory practices of the institutions that created them. 3. **Sensor/IoT data** (cameras, GPS, fitness trackers) — raises privacy and surveillance concerns; availability depends on access to technology, creating socioeconomic bias. 4. **Synthetic data** (generated by other AI) — can propagate or amplify biases present in the generating model. 5. **Deliberately collected data** (surveys, studies) — expensive and therefore rare; may still have sampling biases.Q10. What is data provenance, and what is a "datasheet for datasets"?
Show Answer
**Data provenance** is the documented history of where data came from, how it was collected, who handled it, and what processing it underwent. A **datasheet for datasets** is a standardized documentation template (proposed by Timnit Gebru and colleagues) that accompanies training data, similar to a nutrition label on food. It documents the dataset's composition, collection process, recommended uses, limitations, and other relevant information to help users understand what the data represents and how it should (and should not) be used.Q11. A dermatology AI performs well on light skin tones but poorly on dark skin tones. Which type of data bias best explains this performance gap?
Show Answer
This is primarily **ghost data** (the bias of absence) combined with **selection bias**. Dark-skinned patients are underrepresented in dermatology training datasets, meaning the model simply never learned what skin conditions look like on darker skin. The data was not collected from a representative sample of the population.Q12. Why does the chapter describe data labeling as involving "human judgment hidden in 'objective' data"?
Show Answer
Because every label requires human decisions: defining what categories to use, determining what counts as an edge case, resolving disagreements between labelers, and deciding which ambiguous examples to keep or discard. These judgments embed human values, cultural assumptions, and potential biases into data that is then treated as objective ground truth. The label "hate speech" on a social media post, for instance, reflects a specific human judgment that may vary significantly across cultures, contexts, and individuals.Q13. What ethical concerns surround the labor of data labeling, particularly for content moderation?
Show Answer
Key concerns include: (1) Low pay — many data labelers, especially in the Global South, earn well below living wages in their countries. (2) Psychological harm — workers labeling content moderation data must view graphic violence, hate speech, and abuse, often causing lasting trauma. (3) Invisibility — the labor is essential to AI systems but rarely acknowledged. (4) Precarious employment — much labeling work is gig-based with no benefits or job security. (5) Power imbalance — workers have little bargaining power relative to the major tech companies that ultimately profit from their labor.Q14. Apply the four-question data ethics framework (consent, benefit, control, risk) to the following scenario: A city uses traffic camera footage to train an AI system that optimizes traffic light timing.
Show Answer
**Consent:** Drivers and pedestrians captured by traffic cameras did not specifically consent to having their movements used for AI training. Most are unaware this use is occurring. **Benefit:** The AI could benefit everyone who uses the roads through reduced congestion. But benefits may not be evenly distributed (e.g., if optimization prioritizes high-traffic routes in wealthier areas). **Control:** The city government controls how the data is used. Individual citizens have little ability to opt out of being recorded on public roads. **Risk:** If the data is breached, it could reveal people's movement patterns. If the system is biased, it could systematically disadvantage certain routes or neighborhoods.Q15. True or False: If an AI system uses a very large dataset (millions of examples), bias is no longer a significant concern because the data is big enough to be representative.
Show Answer
**False.** Size does not guarantee representativeness. A dataset of 10 million examples that are all drawn from the same biased source is just as biased as a dataset of 1,000 examples from that source — it is just biased at scale. Large datasets can even amplify biases if the overrepresented patterns receive more statistical weight. Ghost data (missing populations) is not fixed by adding more data from the same distribution.Q16. (Spaced Review — Chapter 1) In Chapter 1, we distinguished between narrow AI and general AI. How does the data dependency discussed in this chapter help explain why general AI remains so challenging?
Show Answer
Narrow AI systems are trained on data specific to one task — medical images, chess games, text. Their performance is bounded by that data. General AI would need to understand and perform across all domains, which would require data that somehow represents all of human knowledge and experience — an impossibly vast and diverse dataset. The data challenges described in this chapter (bias, gaps, labeling subjectivity) would be multiplied enormously. The fact that even narrow AI struggles with data representativeness suggests that the data problem alone makes general AI extraordinarily difficult.Q17. (Spaced Review — Chapter 3) Chapter 3 described the difference between training, validation, and test data. How does the concept of data bias affect the reliability of test set performance metrics?
Show Answer
If the test set shares the same biases as the training set (which is common, since both are typically drawn from the same source), then strong test performance does not guarantee real-world performance across diverse populations. A model that scores 95% accuracy on a biased test set may perform much worse on underrepresented groups. This is why disaggregated evaluation — breaking down performance by demographic group — is important.Q18. A research team publishes a new AI model and reports that it was "trained on publicly available data." Based on what you learned in this chapter, what follow-up questions should you ask before accepting their results?