Chapter 30 Quiz: Verifying AI Output — Fact-Checking Workflows
15 questions covering verification frameworks, methods, and workflow integration.
Question 1
The "verify then trust" standard differs from "trust but verify" because:
A) "Verify then trust" is stricter and requires checking all AI output B) "Verify then trust" makes verification a workflow step that precedes action on high-stakes claims, rather than an exception triggered by doubt C) "Trust but verify" is only appropriate for AI tools, while "verify then trust" applies to all sources D) "Verify then trust" means trusting AI output unconditionally once verified
Answer
**B — "Verify then trust" makes verification a workflow step that precedes action on high-stakes claims, rather than an exception triggered by doubt.** The key difference is the default. "Trust but verify" defaults to trust with verification as occasional intervention. "Verify then trust" makes verification a planned workflow step for the claims that require it, shifting the default from trust to a structured check. Option A is incorrect — "verify then trust" doesn't require checking everything, just categorizing and verifying what the stakes require.Question 2
Why is workflow structure more reliable than vigilance for ensuring consistent verification?
A) Workflows are faster than manual checking B) Vigilance is a limited resource that depletes under time pressure; structure works even when vigilance is low C) AI tools help enforce workflows automatically D) Workflows only need to be applied to high-risk content
Answer
**B — Vigilance is a limited resource that depletes under time pressure; structure works even when vigilance is low.** This is the core insight behind workflow-based verification: individual vigilance fluctuates, especially under deadline pressure. When verification is built into the workflow as a distinct step with scheduled time, it happens regardless of how vigilant the practitioner feels in the moment. Structure outlasts willpower.Question 3
In the verification spectrum, Tier 1 (verify thoroughly) applies to:
A) All AI output, regardless of content type B) Only content produced by less sophisticated AI models C) Specific factual claims in high-accountability contexts: citations in professional documents, attributed statistics, regulatory claims, clinical/safety-critical information D) Any output that will be reviewed by a manager
Answer
**C — Specific factual claims in high-accountability contexts: citations in professional documents, attributed statistics, regulatory claims, clinical/safety-critical information.** Tier 1 is content-specific and stakes-specific. The determining factors are the type of claim (specific factual: citations, statistics, regulatory) and the stakes (professional deliverable, accountability-bearing context). The tier structure is designed to apply maximum rigor where it matters and minimum friction where it doesn't.Question 4
The three phases of the Triage-Verify-Document (TVD) framework are, in order:
A) Draft, check, publish B) Identify claims requiring verification; verify them against appropriate sources; document what was checked and what was found C) Prompt, review, archive D) Assess, question, confirm
Answer
**B — Identify claims requiring verification; verify them against appropriate sources; document what was checked and what was found.** Triage = identify and categorize the claims that need verification. Verify = work through each claim using the appropriate method for its type. Document = create a record of what was checked, what source was used, and what was found (confirmed, modified, or removed). These are three distinct phases with different activities and different attention requirements.Question 5
For verifying an academic citation, which check is the SINGLE MOST DIAGNOSTIC?
A) Confirming the author's name is spelled correctly B) Resolving the DOI at doi.org to confirm it goes to the claimed paper C) Confirming the journal name is a real journal D) Confirming the page count is plausible
Answer
**B — Resolving the DOI at doi.org to confirm it goes to the claimed paper.** DOI resolution is the most direct and decisive check: a DOI that resolves to a different paper, or to nothing, immediately identifies a fabrication problem. A real journal name, a plausible author, and correct page format can all appear in a hallucinated citation. The DOI check cuts through those surface features to the underlying question: does this paper exist and is it what was claimed?Question 6
For verifying a specific statistic with an attributed source (e.g., "42% of workers reported X, according to Pew Research"), the correct verification method is:
A) Ask a different AI model whether the statistic is accurate B) Search for the general topic on Wikipedia C) Find the original report from Pew Research and confirm the number appears there with the characterization used D) Check whether the number is in a plausible range for the domain
Answer
**C — Find the original report from Pew Research and confirm the number appears there with the characterization used.** Statistical verification requires tracing back to the original study or data source and confirming: (1) the number is actually there, (2) it characterizes what the AI said it characterizes (numbers are often accurate but misrepresented in context). Option A introduces the circularity problem. Option B may find derivative uses of the statistic but not the original. Option D is not verification — it's just plausibility assessment.Question 7
What is the "circularity problem" in AI-assisted verification?
A) When AI tools generate the same error repeatedly in a loop B) Asking an AI to verify AI-generated claims — the source is verifying itself, with no external ground truth C) When verification workflows become so complicated they circle back to the original error D) When two AI models agree on a wrong answer
Answer
**B — Asking an AI to verify AI-generated claims — the source is verifying itself, with no external ground truth.** If you ask an AI model whether its own claim is accurate, the model may re-assert the hallucination with equal confidence. Even asking a different AI model doesn't fully escape circularity — both models may have learned the same patterns from overlapping training data and make the same error. Verification must go outside the AI ecosystem to primary sources.Question 8
The most important structural change for ensuring consistent verification is:
A) Using multiple AI tools to cross-check outputs B) Scheduling a verification pass as a distinct step separate from drafting C) Never using AI for content that contains factual claims D) Always asking AI to rate its own confidence level
Answer
**B — Scheduling a verification pass as a distinct step separate from drafting.** When verification is folded into drafting, attention is divided and errors slip through. A distinct verification step — with dedicated time, focused on identifying and checking claims rather than on producing output — is substantially more reliable. This is the workflow change with the highest impact on verification consistency.Question 9
For verifying technical claims about software or APIs, what is the correct approach?
A) Check with a senior developer whether the approach sounds right B) Ask another AI model whether the code is correct C) Check current official documentation for the tool in question, and test the configuration in a non-production environment D) Verify the code compiles without errors
Answer
**C — Check current official documentation for the tool in question, and test the configuration in a non-production environment.** Technical verification has two components: documentation check (to catch outdated or incorrect specifications the model may have learned from an older version) and empirical testing (to catch errors that look correct in isolation but fail in context). Code that compiles is not necessarily code that is correct, and "sounds right" assessment is vulnerable to the same confidence-accuracy gap that affects all AI output.Question 10
A verification log is primarily useful because:
A) It satisfies client requirements for documentation B) It creates professional protection, enables pattern recognition, and reinforces the verification habit C) It allows AI models to learn from their errors in future sessions D) It replaces the need for primary source checking
Answer
**B — It creates professional protection, enables pattern recognition, and reinforces the verification habit.** The verification log serves three functions: (1) professional protection — "I verified this against X source on Y date" is defensible; (2) pattern recognition — over time you learn where your AI errors cluster; (3) habit reinforcement — the documentation practice itself makes verification more consistent. It does not replace primary source checking; it records the results of it.Question 11
The rough time budget for verification in high-stakes content (published articles, client deliverables) is approximately:
A) 5% of total project time B) 50% of total project time C) 20-30% of total project time D) No specific budget needed; verification should happen continuously during drafting
Answer
**C — 20-30% of total project time.** The heuristics in Section 6: high-stakes content 20-30%, medium-stakes 10-15%, low-stakes 5%. For a four-hour project, this means building in 48-72 minutes of verification time. This is significant relative to zero — but far less than the cost of catching and managing errors after publication or client delivery.Question 12
What is the primary risk of treating spot-check procedures as sufficient for high-stakes content?
A) It takes too long compared to full verification B) Spot-checks may miss the specific claims in high-stakes positions (citations, attributed statistics) that are exactly the ones most likely to be hallucinated C) AI models detect when they're being spot-checked and produce more accurate output D) Spot-checks are only valid for content under 500 words
Answer
**B — Spot-checks may miss the specific claims in high-stakes positions (citations, attributed statistics) that are exactly the ones most likely to be hallucinated.** High-stakes claims — the ones that will appear in footnotes, be presented to clients, be published in your name — are precisely the claims in the highest-risk hallucination domains (citations, statistics with sources). A spot-check approach may happen to miss these while catching lower-risk content. For high-stakes content, the specific claims that carry professional accountability must be specifically checked, not sampled.Question 13
Where can AI legitimately assist in the verification process (without the circularity problem)?
A) Confirming whether specific statistics are accurate B) Verifying whether a citation it generated exists C) Summarizing a primary source document you've found independently, or helping generate search queries to find primary sources D) Providing a second opinion on factual claims it has already generated
Answer
**C — Summarizing a primary source document you've found independently, or helping generate search queries to find primary sources.** AI can usefully assist verification at the edges: summarizing a lengthy source you've found (the ground truth is the document, not AI memory), helping you formulate effective search queries, or translating foreign-language sources. It cannot verify its own claims without circularity, and cross-model agreement is not reliable because models share training data and may share errors.Question 14
The reason most professionals skip verification despite knowing hallucinations exist is:
A) They believe AI tools are reliable enough that verification is unnecessary B) Verification that isn't built into the workflow competes with everything else and loses under time pressure C) Verification tools are too expensive for most professionals D) Most professionals don't know how to verify AI output effectively
Answer
**B — Verification that isn't built into the workflow competes with everything else and loses under time pressure.** The gap between knowing that verification is needed and actually doing it consistently is a workflow design problem, not a knowledge problem. Most professionals who skip verification know they shouldn't — but verification wasn't planned for, there's deadline pressure, the output looks fine, and the mental note to check gets overridden by competing demands. Structural solutions (scheduled steps, time budgets) are the answer.Question 15
The ultimate purpose of systematic verification in professional AI use is:
A) To demonstrate skepticism about AI tools to supervisors and clients B) To slow down AI-assisted workflows to a more considered pace C) To earn calibrated trust in AI output that passes through it — replacing ambient uncertainty with professional confidence D) To document that any errors were AI's fault, not the practitioner's