Chapter 29 Quiz: Hallucinations, Errors, and How to Catch Them
Test your understanding of AI hallucinations, error types, and detection strategies. 15 questions.
Question 1
What is the most precise definition of an AI hallucination?
A) When AI deliberately fabricates information to deceive the user B) When AI generates plausible-sounding but factually incorrect content with full confidence C) When AI misunderstands the user's question and gives an irrelevant answer D) When AI refuses to answer a question it doesn't know the answer to
Answer
**B — When AI generates plausible-sounding but factually incorrect content with full confidence.** Hallucinations are not intentional deception (AI has no intent), not misunderstanding (the model may answer the question perfectly while being wrong), and not refusal (which is the opposite behavior). The defining features are: plausibility, false content, and confident delivery.Question 2
Why do language models hallucinate? Choose the best answer.
A) Because they have insufficient memory to retain all the facts they were trained on B) Because they are designed to be creative and creativity sometimes overrides accuracy C) Because they generate text through probabilistic prediction rather than fact retrieval, with no mechanism to verify claims against reality D) Because their training data contains inaccurate information that they reproduce
Answer
**C — Because they generate text through probabilistic prediction rather than fact retrieval, with no mechanism to verify claims against reality.** LLMs generate the next most-likely token based on training patterns. They do not retrieve facts from a database, and they have no internal verification mechanism. When training data is sparse or absent for a specific claim, the model still produces a plausible-sounding output. Option D is partially true (training data quality matters) but does not describe the primary mechanism.Question 3
Which of the following is a PURE hallucination rather than a different error type?
A) An AI gives the correct name of a medication but states an incorrect dosage B) An AI describes a real event but gives the wrong year C) An AI generates a citation for an academic paper that was never written D) An AI provides correct information from 2021 that has since been updated
Answer
**C — An AI generates a citation for an academic paper that was never written.** This is pure hallucination: the subject matter does not exist in reality. Option A is a confident error (real topic, wrong detail). Option B is also a confident error. Option D is outdated information, a distinct error type.Question 4
The "confidence is not accuracy" principle means:
A) AI tools should express more uncertainty in their outputs B) The confidence of AI language is stylistic and does not predict whether the content is factually accurate C) You should only trust AI outputs when the model explicitly says it is uncertain D) More confident AI responses are less reliable than hedged responses
Answer
**B — The confidence of AI language is stylistic and does not predict whether the content is factually accurate.** AI models produce authoritative-sounding language in contexts where authoritative language appears in training data, regardless of whether the specific content is accurate. The confidence is a feature of the language pattern, not an epistemic state reflecting the model's certainty about truth. Option D is not reliably true — hedged responses are not necessarily more accurate.Question 5
Which domain carries the HIGHEST hallucination risk?
A) Asking AI to brainstorm ten names for a new product B) Asking AI to summarize a document you have pasted into the prompt C) Asking AI for a specific academic citation on a topic D) Asking AI to create an outline for a presentation
Answer
**C — Asking AI for a specific academic citation on a topic.** Citations are the highest-risk hallucination category documented in research and practice. The model has learned the patterns of citation formatting at high fidelity and can produce completely fabricated citations that look identical to real ones. Options A, B, and D are all lower-risk tasks — they involve ideation, structure, or summarization of provided content rather than specific factual retrieval.Question 6
The "too specific" signal refers to:
A) AI responses that are too detailed to be useful B) The observation that unusual specificity in AI output (exact percentages, specific named sources, precise dates) often correlates with fabrication C) The problem of AI being overconfident in technical domains D) AI generating too many examples when asked for a few
Answer
**B — The observation that unusual specificity in AI output (exact percentages, specific named sources, precise dates) often correlates with fabrication.** Paradoxically, greater specificity in AI output warrants greater scrutiny, not less. When a model fills in specific details — "34.5% of workers," "according to the 2023 McKinsey report," "published in January 2022" — it may be generating plausible specifics rather than reporting verified facts. Specificity feels authoritative but is a hallucination pattern, not a reliability signal.Question 7
When verifying an AI-provided citation, which of the following checks is MOST important?
A) Checking that the author name sounds like a real person B) Verifying the DOI resolves to the claimed paper and that the content matches how AI characterized it C) Checking that the journal name is a real journal D) Verifying the publication year is plausible for the field
Answer
**B — Verifying the DOI resolves to the claimed paper and that the content matches how AI characterized it.** All options involve real verification steps, but B is the most complete. A DOI that resolves to a different paper, or to nothing, immediately identifies a hallucinated citation. A real journal name and plausible author can both appear in a fabricated citation. The content check is also important — a real paper may exist but have been mischaracterized.Question 8
Outdated information differs from hallucination primarily because:
A) Outdated information is always less harmful than pure hallucination B) Outdated information was accurate at some point in time, while pure hallucination was never accurate C) Outdated information only affects technical domains D) AI models always flag when their information might be outdated
Answer
**B — Outdated information was accurate at some point in time, while pure hallucination was never accurate.** Outdated information is a distinct error type: regulations change, APIs deprecate, clinical protocols update, market conditions shift. The model's information reflected reality at training time but no longer does. This is not pure hallucination (nothing was fabricated) but is still a significant error type in fast-moving domains. Option D is incorrect — models do not reliably flag outdated information.Question 9
Which AI model characteristic is described as most relevant to citation hallucination risk?
A) The size of the model (number of parameters) B) The model's training data quality C) That the model has learned citation formatting patterns and can reproduce them for fabricated papers D) The temperature setting at which the model operates
Answer
**C — That the model has learned citation formatting patterns and can reproduce them for fabricated papers.** The danger of citation hallucination is not random error — it is that models have learned the form of citations so well that fabricated ones look exactly like real ones. Author names, journal names, volume numbers, page ranges, and DOI formats are all pattern-learned, making fabricated citations visually indistinguishable from real ones without verification.Question 10
The challenge technique involves:
A) Asking a more difficult question to test the model's knowledge limits B) Asking the AI directly whether it is certain about a claim and what its source is C) Having multiple AI tools compete to provide the most accurate answer D) Asking the AI to generate arguments against its own output
Answer
**B — Asking the AI directly whether it is certain about a claim and what its source is.** The challenge technique is to ask the model directly: "How confident are you about that?" or "What is your source for that statistic?" This can surface uncertainty the model wasn't volunteering, but it does not replace verification — a model may still confidently re-assert a hallucination when challenged. The technique is useful as a first filter, not a final one.Question 11
Which scenario represents "subtle distortion"?
A) An AI generates a citation that doesn't exist B) An AI gives an incorrect medication dosage C) An AI correctly summarizes all individual facts in a document but frames the overall conclusion in a way that is misleading D) An AI fails to mention that a regulation applies only in certain jurisdictions
Answer
**C — An AI correctly summarizes all individual facts in a document but frames the overall conclusion in a way that is misleading.** Subtle distortion is the most insidious error type: the individual factual claims may be verifiable and accurate, but the framing, emphasis, or omission creates a false overall impression. Option D could also represent subtle distortion (important omission) but C is the clearest example of distortion through framing rather than false facts.Question 12
A professional should go DIRECTLY to primary sources rather than using AI when:
A) The AI seems uncertain about the topic B) They are working with safety-critical information, very recent events, or high-accountability professional contexts where errors have serious consequences C) The topic is too complex for AI to handle D) The AI tool being used is not a premium subscription
Answer
**B — They are working with safety-critical information, very recent events, or high-accountability professional contexts where errors have serious consequences.** The decision to bypass AI for information retrieval is based on stakes and domain, not on AI's expressed uncertainty (which is unreliable as a signal) or complexity. Safety-critical facts, current regulations, specific clinical information, and verified citations for professional work are all situations where going to primary sources directly is the correct approach.Question 13
What is the primary purpose of a "verification log" in professional AI use?
A) To show clients that you checked AI output B) To document what was checked, against what source, and what the result was — providing professional protection and a quality control record C) To keep track of how many AI errors you've found D) To report AI errors to the model developers
Answer
**B — To document what was checked, against what source, and what the result was — providing professional protection and a quality control record.** A verification log is professional documentation: it records what verification was performed, what sources were used, and what was confirmed or corrected. This matters for professional accountability, for quality control in ongoing work, and as a defense if questions arise later about how a fact was verified. The habit of documentation also reinforces the verification practice itself.Question 14
Which of the following represents context collapse in an AI interaction?
A) An AI stops responding partway through a long answer B) An AI confuses two similar topics from earlier in a conversation and generates a summary that misattributes information between them C) An AI gives a shorter answer than expected D) An AI correctly summarizes one section of a document but misunderstands another
Answer
**B — An AI confuses two similar topics from earlier in a conversation and generates a summary that misattributes information between them.** Context collapse occurs when the model fails to maintain accurate context across a long conversation or document — "remembering" something slightly differently, conflating adjacent topics, or generating content that mixes elements from separate parts of the conversation. This is distinct from ordinary misunderstanding and is particularly risky in long sessions where the model's context window is being stretched.Question 15
The most important takeaway from the documented legal hallucination cases (attorneys sanctioned for fabricated citations) is:
A) Attorneys should not use AI tools for legal research B) AI tools used for legal research have a higher hallucination rate than other tools C) Plausible-looking AI output cannot be distinguished from accurate output by reading alone — verification against authoritative sources is required D) Law firms should ban AI tools until hallucination rates reach zero