Chapter 13 Quiz: Diagnosing and Fixing Bad Outputs
Question 1
What is the core reframe that Chapter 13 establishes as the foundation of effective troubleshooting?
A) Bad output should be reported to the AI provider as a bug B) Bad output is diagnostic information — a signal about what the model understood from your prompt — not a failure requiring frustration or giving up C) Bad output can always be fixed with one more iteration D) Bad output indicates you need to use a more powerful AI model
Show Answer
**B** — The foundational reframe is: bad output is data, not failure. Every bad output answers the question "what did the model understand from my prompt?" which gives you information about what was missing or unclear. This shifts the experience from frustrating to systematic: instead of hoping the model does better next time, you diagnose what caused the problem and engineer the conditions for success.Question 2
Which root cause is responsible for an AI output that is accurate and well-structured but uses general, industry-standard recommendations that don't account for your specific company's size, industry, or existing constraints?
A) Hallucination B) Training data bias C) Insufficient context D) Vague instruction
Show Answer
**C** — Insufficient context is the root cause when output is plausible but generic — it could apply to any company because the model didn't know what was specific about your situation. The diagnostic signal is "this advice is reasonable for the average company but doesn't fit us." The fix is providing the specific context that differentiates your situation: company size, industry nuances, existing constraints, what you've already tried, and so on.Question 3
An AI output confidently states: "According to a 2023 Stanford study, remote workers are 23% more productive than in-office counterparts." You cannot find this study. What root cause is this, and what is the specific danger?
A) Training data bias — the model has biased data favoring remote work B) Hallucination — the model generated a plausible-sounding citation that does not exist, which is dangerous because it is presented with false confidence and attribution that makes it seem credible C) Wrong capability — the model cannot access academic research databases D) Insufficient context — the model didn't know which studies were relevant to your question
Show Answer
**B** — This is hallucination: a specific, attributed factual claim that is invented. The danger is precisely that it sounds credible. A made-up citation with a specific author, institution, year, and percentage is far more convincing than a vague claim — which is why hallucinated citations are among the most professionally dangerous AI outputs. Using them in a client deliverable or published work can result in professional embarrassment or worse when the citation cannot be verified.Question 4
The Triage Matrix is a 2x2 decision tool based on what two dimensions?
A) Output quality and user satisfaction B) Distance from goal and effort required to repair C) Root cause type and model capability D) Prompt length and response length
Show Answer
**B** — The Triage Matrix evaluates: (1) how far the current output is from what you need (distance from goal, rated 1-5), and (2) how much effort it would take to repair the current output vs. restart from scratch. Low distance + low repair effort = quick repair. High distance + low restart effort = restart. The matrix prevents you from wasting time repairing an output that can never be salvaged, or restarting when a simple fix would work.Question 5
In a long conversation, you specified "always maintain a skeptical, critical tone" at the start. By message 18, the AI is writing enthusiastically and positively. What root cause explains this?
A) Training data bias — the model defaults to positive tone because positive content is more common in training data B) Hallucination — the model invented a new tone requirement C) Context window overflow — the model's attention to early instructions degrades in very long conversations D) Vague instruction — "skeptical and critical" was not specific enough
Show Answer
**C** — Context window overflow causes the model to effectively lose track of instructions specified far earlier in the conversation. As the context grows very long, attention quality for early content (including initial tone instructions) degrades. The fix is to periodically re-state key constraints, or to start a fresh conversation and include the essential context in the new session's first message.Question 6
You asked for "talking points for a 5-minute presentation" and received a 5-page narrative essay. The content is relevant and mostly accurate. What is the most efficient repair approach?
A) Start over with a completely new prompt — the output is too different to salvage B) Use the Format Fix template: keep all the content, restructure it into the correct format (bulleted talking points, appropriate for 5 minutes) C) Use the Targeted Correction template: fix only the length issue D) Use the Context Reload template: add more context about the presentation purpose
Show Answer
**B** — The Format Fix template is ideal here: the content is good (no need to throw it away), but the format is wrong. "Good structure but wrong format. Keep all the content and restructure it as: [bulleted talking points, 5-7 bullets, each completable in 30-45 seconds, suitable for a 5-minute slot]." This is a Triage Matrix Quadrant 1 case: low distance from goal, low repair effort. Restarting would be wasteful when a simple reformatting instruction solves the problem.Question 7
Which root cause explains an output that consistently recommends solutions appropriate for large enterprise companies when your company is a 10-person startup with a limited budget?
A) Insufficient context (the model didn't know you were a startup) B) Training data bias (the model's training data is dominated by large-company contexts) C) Either A or B — the distinction matters for the fix D) Vague instruction — you should have specified "startup context"
Show Answer
**C** — This is one of the most important nuances in the diagnostic framework. If you never mentioned your company size, this is Root Cause 1 (Insufficient Context) — the fix is adding context. If you specified "10-person startup" and the AI still recommended enterprise solutions, this is Root Cause 6 (Training Data Bias) — the fix is an explicit counter-instruction ("recommendations should be appropriate for a resource-constrained early-stage company, not for established enterprises"). The diagnosis determines the repair.Question 8
What is the "retry loop" anti-pattern, and why doesn't it work for systematic failures?
A) Using the same prompt multiple times in the same session, which confuses the model B) Submitting the same prompt repeatedly, hoping for better results — this works occasionally for random variance but not for failures caused by missing context, vague instruction, or capability limits, which will produce the same type of failure every time C) Switching between multiple AI tools until one produces a satisfactory result D) Running a repair prompt more than three times on the same output
Show Answer
**B** — The retry loop is submitting the same prompt again when the first attempt fails. It occasionally helps if the failure was due to random generation variance (the model can produce different outputs on identical prompts). But for systematic failures — insufficient context, vague instruction, format mismatch, wrong capability — the conditions producing the failure are unchanged, so the same type of failure will recur. The diagnostic framework replaces the retry loop with analysis before re-prompting.Question 9
You are working with a code debugging session and suspect the model's answer is based on a misunderstanding of your code's architecture. Which repair template is most appropriate?
A) Template 1 (Targeted Correction) — correct the specific wrong answer B) Template 2 (Task Clarification) — clarify what the code does and how it differs from what the model assumed C) Template 5 (Factual Correction) — provide the correct information about how the code works D) Template 7 (Full Restart) — restart with the CoT debugging structure from Chapter 10
Show Answer
**B** — When the root cause is "wrong approach" due to a misunderstood architecture or codebase structure, the Task Clarification template is the right choice: "You misunderstood [specific aspect of the code/architecture]. What I actually need is [correct understanding]. The correct architecture is [brief description]. Please revise your approach with this understanding." This directly addresses the misunderstanding that produced the wrong approach.Question 10
Elena's investor update scenario involved AI generating an authoritative-sounding passage with fabricated statistics about data privacy. What makes this type of hallucination particularly dangerous compared to obvious factual errors?
A) It involves technical topics that are harder to verify B) The professional, confident tone and specific attribution (EDPB audit, precise percentages) make the fabricated content appear credible, reducing the chance that it will be questioned before use C) Investor updates are harder to revise once sent D) Data privacy is a topic where hallucination is especially common
Show Answer
**B** — The danger is in the presentation: specific percentages ("less than 15% of user data"), named institutions ("2023 European Data Protection Board audit"), and precise comparisons ("industry average of 67%") signal credibility and specificity. These signals reduce the reviewer's likelihood of questioning the content. The same claim stated vaguely ("our data retention is better than industry average") would prompt more skepticism than one stated with false precision. Elena's principle applies: the more authoritative and specific the tone, the more carefully the underlying claims should be verified.Question 11
A research assistant asked AI to summarize five research papers and received a summary that accurately describes four of them but completely invents the content of the fifth. The model had been given only the paper titles and asked to summarize based on its training data. What root cause explains this?
A) Insufficient context — the model didn't have enough information about the papers B) Wrong capability — the model cannot reliably summarize papers it doesn't have in its training data; for papers outside training or recent publications, it will confabulate plausible-sounding content C) Hallucination — this is Root Cause 5 but it's important to understand why D) Both B and C — this is a capability limitation that manifests as hallucination
Show Answer
**D** — This is both a wrong capability issue and a hallucination. The wrong capability element: asking the model to summarize specific research papers without providing the text assumes the model has memorized those papers, which is unreliable especially for recent or obscure publications. The hallucination element: when the model lacks reliable training data on a specific paper, it generates a plausible-sounding summary — confidently, without flagging uncertainty. The fix: provide the actual paper text, not just the title.Question 12
What is a "personal failure taxonomy" and what long-term value does it provide?
A) A list of AI tasks that never work, to avoid attempting them B) A documented record of failure patterns — task type, root cause, repair approach, and prevention — that reveals consistent failure patterns in your specific work and builds a personalized improvement roadmap C) A report to share with your AI provider to help them improve the model D) A daily log of how many AI failures you experience, for productivity tracking
Show Answer
**B** — A personal failure taxonomy is a running record of significant AI failures, logged with root cause, repair, and prevention for each entry. After accumulating 20-30 entries, grouping by task type and root cause reveals your personal failure patterns — the specific combinations of task + root cause that recur in your work. These patterns point directly to the highest-value prompt improvements you can make. Without documentation, patterns remain invisible and the same failures recur indefinitely.Question 13
When is an output genuinely unfixable and you should stop trying to repair it?
A) When it fails on the third repair attempt B) When the model has a genuine capability gap that no prompting will bridge, when the task requires specialized accuracy the model cannot reliably provide, or when the task requires knowledge the model doesn't have (unpublished data, post-training-cutoff events, proprietary internal information) C) When the output is more than 50% wrong D) When you have spent more than 5 minutes trying to repair it
Show Answer
**B** — Genuinely unfixable outputs share the characteristic that the failure is structural, not prompt-related. No amount of prompting gives a model access to your internal unpublished data, compensates for post-cutoff knowledge gaps, or makes a language model perform reliable arithmetic. Recognizing the unfixable saves significant time — the signal is when two or three well-targeted repair attempts fail to address the fundamental problem despite good diagnosis and specific repair prompts.Question 14
The chapter's research breakdown suggests that approximately 60% of AI output failures are attributable to which two root causes?
A) Hallucination and wrong capability B) Insufficient context and vague instruction C) Format mismatch and training data bias D) Context window overflow and vague instruction
Show Answer
**B** — The approximate distribution is: insufficient context (~35%) and vague instruction (~25%) account for roughly 60% of failures. This distribution has significant implications: the majority of AI failures are preventable with the fundamental prompting skills from Chapters 7-9 — providing adequate context and writing specific instructions. The more sophisticated techniques (advanced prompting, hallucination prevention) address a smaller portion of the failure space. This is the empirical case for mastering fundamentals before advanced techniques.Question 15
How does the Chapter 13 diagnostic framework connect back to the earlier chapters of Part 2?
A) It replaces earlier chapters — once you learn to fix failures, you don't need to build prompting skills B) Each root cause in the diagnostic framework maps to a preventive skill from an earlier chapter — the framework makes the earlier skills more meaningful by showing exactly what failures those skills prevent C) The framework is independent — it applies after all other chapters' techniques have been exhausted D) The framework only applies to chapters 7-9; advanced techniques from chapters 10-12 don't produce diagnosable failures