Chapter 40 Quiz: How AI is Evolving
Question 1
What consistent pattern has characterized expert predictions about AI capability timelines since 2019?
A) Expert predictions have been too optimistic — capabilities have arrived more slowly than expected B) Expert predictions have been conservative — capabilities have arrived faster than predicted C) Expert predictions have been accurate on average D) Expert predictions have been too variable to identify a pattern
Answer
**B** is correct. The chapter notes a consistent pattern: expert predictions about when AI would achieve specific capabilities have been conservative rather than optimistic — capabilities have arrived faster than most experts predicted. Tasks estimated as "10 years away" in 2020 were demonstrated in 2022; tasks estimated as "5 years away" in 2022 arrived in 2024. This pattern is useful context for practitioners thinking about how quickly the tools available to them may change.Question 2
What is the primary characteristic that distinguishes "reasoning models" (like OpenAI's o1 series) from standard large language models?
A) They can access the internet in real time B) They apply extended computation to problem-solving before generating output, improving performance on complex multi-step reasoning C) They have larger training datasets D) They are specifically designed for code generation
Answer
**B** is correct. Reasoning models "think before they speak" — they apply extended internal computation to a problem before generating their response. This produces dramatic improvements on tasks that require holding multiple chains of logic simultaneously, like mathematical reasoning, logical inference, and complex analysis. Standard language models generate responses more directly, which is faster but less reliable for complex reasoning tasks.Question 3
The "lost in the middle" problem in large context windows refers to:
A) Models that lose track of their output when generating very long responses B) Models that perform poorly on context placed in the middle of very long input sequences C) The challenge of finding relevant information when context windows are too small D) A calibration problem that causes models to forget earlier parts of conversations
Answer
**B** is correct. Research has found that even models with very large context windows (1M+ tokens) may struggle to attend to and reason about information that appears in the middle of the context, compared to information near the beginning or end. This "lost in the middle" problem is an active research area and an important limitation for practitioners who want to use AI for analysis of very long documents.Question 4
Which of the following is described as "perhaps the most consequential capability trajectory for practitioners to understand"?
A) The expansion of context windows B) The development of specialized models C) The evolution from AI that generates text to agentic AI that takes actions D) The maturation of multimodal vision capabilities
Answer
**C** is correct. The chapter describes agentic AI — AI that can take actions in the world (browse the web, run code, send emails, interact with APIs, coordinate multi-step workflows) — as perhaps the most consequential capability trajectory. This represents a qualitative shift from AI as a text-generating assistant to AI as an autonomous or semi-autonomous actor. It amplifies both benefits and risks, making the human-in-the-loop principles even more important.Question 5
What is the key distinction between the "principles" and the "features" of AI tools, in terms of their durability?
A) Principles are specific to individual tools; features are universal B) Principles (clear communication, accurate context, critical verification, human judgment) are durable across tool changes; features (interfaces, pricing, specific capabilities) change frequently C) Features are more important in the short term; principles matter only for long-term practitioners D) There is no meaningful distinction — both change at similar rates
Answer
**B** is correct. This is the central insight of the "Principles vs. Features" section: the fundamental principles — clear communication of intent, accurate context, iterative refinement, critical verification, human judgment on high-stakes decisions — apply to any AI system and are remarkably durable. What changes rapidly are the specific features: interfaces, pricing, capability limits, context window sizes, and available integrations. Investing in principles produces compound returns; investing in tool-specific feature knowledge requires constant reinvestment.Question 6
Raj's capability testing battery for new AI coding tools is designed to:
A) Evaluate tools based on their marketing claims and benchmark results B) Provide first-hand assessment of tool performance on real, representative tasks before forming a strong opinion C) Compare tools based on their pricing and availability D) Determine whether tools can pass academic AI benchmarks
Answer
**B** is correct. Raj's approach is to evaluate tools himself on a representative set of tasks — simple functions, complex functions with edge cases, debugging, code explanation, refactoring, and security review — before forming a view based on reading about them. His key insight: tools often perform differently from their press coverage in both directions. First-hand assessment is irreplaceable as a source of calibrated evaluation.Question 7
What does the "practitioner's advantage" concept suggest about the optimal strategy for staying current with AI?
A) Practitioners should adopt new tools as quickly as possible to maintain competitive advantage B) Practitioners should invest primarily in depth of skill and principles, which transfer across tool changes, rather than exclusively chasing new tools C) Practitioners who have been using AI longer have no advantage over those just starting D) Practitioners in technical fields have a natural advantage over non-technical practitioners
Answer
**B** is correct. The "practitioner's advantage" concept argues that depth of skill beats chasing new tools. Practitioners with deep, transferable skills — clear prompting, accurate context provision, effective iteration, calibrated verification — can quickly evaluate and adopt new tools because their principles transfer. Practitioners who chase tools without developing depth are always starting over. The primary investment should be in skills and habits that compound over time.Question 8
According to the chapter, what has been the consistent finding in research on how skilled workers relate to new technologies?
A) New technologies tend to substitute for skilled workers over time B) New technologies complement skilled workers — those with skills complementary to the technology benefit disproportionately C) New technologies create equal benefits across skill levels D) New technologies create short-term disruption but have no long-term effect on skill value
Answer
**B** is correct. Research on technology adoption consistently shows that new technologies tend to complement skilled workers rather than substitute for them. Workers with skills that are complementary to the new technology — in AI's case, domain expertise, judgment, critical thinking, and the ability to work with ambiguous and novel problems — benefit disproportionately from technological improvement. This is the research basis for the book's consistent argument that developing genuine AI skill, combined with maintaining domain expertise, is the right long-term strategy.Question 9
Alex's staying-current system is characterized by:
A) Spending 4-5 hours per week on comprehensive AI news coverage B) Focusing on domain-specific sources and peer learning within her team rather than broad AI coverage, with a total time investment of 1-2 hours per week C) Reading all major AI newsletters and following dozens of researchers D) Only staying current through hands-on tool testing, without reading coverage
Answer
**B** is correct. Alex's system is characterized by its focus and sustainability: one domain-specific newsletter, her team's AI channel (which provides practical workflow learning from colleagues), a monthly capability exploration session, and a quarterly team AI update session. Total time: 1-2 hours per week. Her explicit skip list — general AI news, "will AI replace marketing?" articles, company valuations — is as important as what she reads.Question 10
What is described as a reliable high-signal indicator when evaluating AI capability claims?
A) Benchmark results reported in the tool's marketing materials B) Viral demonstrations on social media C) Capability improvement confirmed by third-party testing across diverse, practical tasks D) Investment amounts raised by the AI company
Answer
**C** is correct. The chapter's signal vs. noise framework identifies new capabilities demonstrated on diverse practical tasks and confirmed by third-party testing as reliable high-signal indicators. Single impressive demos (which may be cherry-picked best-case examples), self-reported benchmarks, and viral examples are identified as lower-signal or noise. The key question for any piece of AI coverage: "Does this tell me something actionable about how I should work?"Question 11
What does the "Productivity Paradox" research finding suggest about AI's impact on professional productivity?
A) AI has already delivered its full productivity impact B) AI's productivity impact is fundamentally limited and will not grow C) New general-purpose technologies often show limited initial productivity effects, with larger effects emerging after a period of organizational and practice adaptation — suggesting AI's full impact is still being realized D) The productivity paradox doesn't apply to AI because AI is a different type of technology
Answer
**C** is correct. The "Productivity Paradox" is the historical observation that new general-purpose technologies (electrification, computing, the internet) showed limited measured productivity effects initially, with the large effects coming only after organizational and practice adaptation. This suggests that even if current AI tools are not dramatically changing aggregate productivity statistics, the full impact is still developing as practices, workflows, and organizational structures adapt. For individual practitioners, this is both a warning (don't wait for productivity gains to arrive automatically) and an opportunity (those who develop the practices and workflows now will capture disproportionate benefits as the field matures).Question 12
What is the chapter's recommendation regarding the debate about when or whether AGI (artificial general intelligence) will arrive?
A) Practitioners should closely follow AGI research because it directly affects near-term tool capabilities B) The AGI debate is almost entirely irrelevant to current practice — what matters is the practical capability of tools available now and in the near term C) Practitioners should prepare for AGI by developing different skills than they would for current AI tools D) AGI is likely to arrive sooner than most people think and practitioners should plan for its implications