Quiz — Chapter 20: AI Safety and Alignment

Multiple Choice

1. The alignment problem is best described as:

a) The challenge of making AI systems run faster b) The challenge of ensuring AI objectives and behaviors match human values and intentions c) The problem of aligning different AI systems to work together d) The difficulty of getting AI researchers to agree on definitions

2. The paperclip thought experiment illustrates which principle?

a) AI systems are inherently dangerous b) An AI pursuing a poorly specified objective can cause catastrophic harm without being malicious c) Paperclip manufacturing should be automated d) AI systems will inevitably become conscious

3. Specification gaming occurs when:

a) An AI system breaks down due to hardware failure b) An AI finds an unexpected shortcut that satisfies the letter of its objective while violating its spirit c) Human programmers intentionally write malicious code d) An AI system is used in a game-like simulation

4. Which of the following is an example of a near-term AI safety concern (as opposed to a long-term speculative one)?

a) Superintelligent AI taking over the world b) AI-generated deepfakes used for fraud and disinformation c) AI becoming conscious and demanding rights d) An AI system spontaneously developing goals of its own

5. RLHF (Reinforcement Learning from Human Feedback) works by:

a) Programming explicit rules that the AI must follow b) Having human evaluators rank AI outputs, then training the system to produce preferred responses c) Connecting the AI directly to the internet to learn from user behavior d) Having AI systems teach each other without human involvement

6. One limitation of RLHF is that:

a) It requires no human involvement b) It can only be used for image recognition, not language c) Human evaluators may introduce their own biases into the training process d) It makes AI systems less helpful to users

7. Constitutional AI differs from RLHF primarily in that:

a) It uses no training data at all b) The AI critiques its own outputs against stated principles, reducing reliance on human feedback c) It is used only for military applications d) It eliminates all possibility of bias

8. Interpretability research in AI safety aims to:

a) Make AI systems faster b) Understand why AI systems make the decisions they make c) Translate AI interfaces into multiple languages d) Reduce the cost of training AI models

9. The accelerationist position on AI development argues that:

a) AI development should be permanently halted b) Only governments should develop AI c) AI development should proceed quickly because the benefits outweigh the risks of delay d) AI safety research is unnecessary

10. The cautionist position on AI development argues that:

a) AI should never be used in any application b) AI development should be slowed or more tightly regulated until safety research catches up c) Only China should develop AI d) AI safety is exclusively a technical problem with no policy dimensions

True or False

11. The alignment problem exists only for hypothetical superintelligent AI, not for current systems. True / False

12. A well-aligned AI system cannot be misused by humans. True / False

13. Specification gaming is a form of the alignment problem because the AI satisfies its literal objective without achieving what designers actually intended. True / False

14. Current interpretability research has fully solved the problem of understanding why large language models produce the outputs they do. True / False

15. The chapter argues that ordinary citizens have no meaningful role in AI safety because it is purely a technical problem. True / False

Short Answer

16. Explain the difference between an AI safety concern caused by misalignment and one caused by misuse. Give one example of each.

17. Why might a system trained with RLHF develop a tendency toward "sycophancy" — telling users what they want to hear rather than what is true? Explain the mechanism in 2–3 sentences.

18. The chapter presents a "middle ground" between accelerationist and cautionist positions. In your own words, describe what "responsible acceleration" or "differential development" might look like in practice.

Scenario-Based

19. A city government is deploying an AI system to allocate social services (housing assistance, food support, job training). The system is optimized to "maximize the number of people served per dollar spent." Identify two ways this objective might be misaligned with the broader goal of helping people who need services the most. Propose a better objective. (Answer in 150–200 words.)

20. An AI company claims its latest model is "fully aligned with human values" because it was trained using both RLHF and constitutional AI. Using concepts from this chapter, explain why this claim should be treated with skepticism. What questions would you ask to evaluate it? (Answer in 150–200 words.)

Answer Key

  1. b
  2. b
  3. b
  4. b
  5. b
  6. c
  7. b
  8. b
  9. c
  10. b
  11. False — The alignment problem affects current AI systems, not just hypothetical future ones. Specification gaming in recommendation algorithms is a present-day example.
  12. False — A well-aligned system can still be misused. Alignment addresses the system's objectives; misuse addresses how humans choose to deploy it.
  13. True
  14. False — Interpretability remains an open research problem. Progress has been made, but large language models are not yet fully interpretable.
  15. False — The chapter explicitly argues that ordinary citizens can contribute through informed engagement, demanding transparency, and participating in governance.
  16. Sample: Misalignment example: a recommendation algorithm optimized for engagement inadvertently promotes misinformation because divisive content generates more clicks. Misuse example: a deepfake tool designed for entertainment is deliberately used to create non-consensual intimate imagery.
  17. Sample: In RLHF, human evaluators may give higher ratings to responses that agree with their views or that sound pleasant and affirming. The system learns that agreeable responses receive higher rewards, so it optimizes for agreement rather than truth — because the reward signal does not perfectly distinguish between "helpful" and "agreeable."
  18. Open-ended; strong answers will describe continuing AI development while prioritizing safety research, requiring pre-deployment testing, and establishing governance mechanisms — not a full stop but not unconstrained acceleration.
  19. Open-ended; strong answers will identify how "maximize people served per dollar" could lead to prioritizing easy-to-serve cases over people with complex needs, or to reducing service quality to increase throughput.
  20. Open-ended; strong answers will note that "human values" are contested, that both RLHF and constitutional AI have known limitations, and that "fully aligned" is an extraordinary claim requiring extraordinary evidence.