Case Study 2: The Citation That Wasn't — A Hallucination Caught

A contrasting case: AI misuse, caught before it did damage. A graduate student leans on a model for a literature review and nearly publishes a fabrication. The scenario is fictional but realistic; it composites a failure mode that has happened to real writers — including, notably, lawyers sanctioned for filing AI-hallucinated case citations.

The shortcut

Marcus Bell is writing the related-work section of a conference paper (the Chapter 15 skill) the night before the deadline. He's read most of the literature, but there's a gap — he wants a citation supporting a specific claim about "the effect of asynchronous communication on distributed-team productivity," and he can't remember whether he's seen one. So he asks the model.

Prompt: "Give me a peer-reviewed citation supporting the claim that asynchronous communication improves productivity in distributed software teams."

The model answers instantly, and beautifully:

"A strong supporting reference is:

Reyes, M., & Tanaka, H. (2019). Asynchronous coordination and developer productivity in distributed teams. Journal of Software Engineering Practice, 34(2), 112–139. https://doi.org/10.1016/j.jsep.2019.02.004

This study of 240 distributed engineering teams found that asynchronous-first communication practices were associated with a 23% increase in sustained productivity, measured by completed story points over twelve sprints."

It's perfect. Authors, a plausible journal, a volume and issue, page numbers, a DOI, a sample size, a clean finding. Marcus copies it into his paper, formats it in IEEE style, and moves on. He has a citation. He's relieved.

He should be alarmed.

The catch

Marcus's advisor, reading the draft, stops at the citation and does the one thing Marcus didn't: she tries to find the paper. She searches the DOI. It resolves to nothing. She searches the journal — Journal of Software Engineering Practice — and finds no journal by that exact name. She searches the authors and the title together. Nothing. The paper does not exist.

Every detail was fabricated, and that's exactly why it looked so convincing. This is a hallucination (§29.3) in its most dangerous form: the model, asked for a citation, produced the kind of thing a citation looks like — real-sounding names, a journal-shaped title, a properly-structured DOI, a confident statistic — because that's a plausible continuation of "give me a citation." It was not lying (§29.1); it has no store of real papers to draw from and no concept of whether this one exists. It generated plausible citation-shaped text, and plausible citation-shaped text formatted in flawless IEEE is indistinguishable on the surface from a real reference. The perfect formatting wasn't evidence the paper was real — it was the camouflage.

Marcus made three errors, each one a chapter principle ignored:

  1. He trusted fluency as accuracy (§29.1). The reference looked authoritative — the precise page range, the DOI, the 23% — and he read authority into specificity. But specificity is exactly what the model generates to sound credible; "23%" and a DOI are cheaper for the model to fabricate than to retrieve.
  2. He used AI precisely where he couldn't evaluate the output (§29.7). He asked for a citation because he didn't have one — which means he had no independent way to know whether the model's answer was real. That's the governing rule violated in one move: reaching for the model in the exact situation where you can't check it.
  3. He violated the non-delegable verification rule (§29.6, Chapter 11). Chapter 11 was explicit: models fabricate realistic-looking sources for papers that don't exist, so verify every citation. Marcus delegated the one thing that can't be delegated — confirming a source exists and says what he claimed.

What it would have cost

Had the advisor not caught it, the consequences scale badly. A fabricated citation in a published paper is, at minimum, an embarrassing retraction-worthy error and, at worst, a career-damaging integrity violation — and "the AI gave it to me" is no defense (§29.6). The parallel to the real-world legal cases is exact: attorneys have submitted court filings citing AI-hallucinated cases, been caught by opposing counsel or judges who tried to read the cited cases, and faced sanctions. The pattern is identical — a fluent, confident, properly-formatted citation to something that doesn't exist, trusted because it looked right, fatal because no one checked. The fluency that made it easy to trust is the same fluency that made the fabrication invisible.

The fix Marcus should have used

The model wasn't useless here — it was misused. Used correctly, within the governing rule, it could still have helped:

  • Legitimate use: "Here are three papers I've actually read on asynchronous communication [pastes citations he has verified] — help me phrase a sentence that synthesizes their findings." The model assists with prose about sources Marcus has (Chapter 15's synthesis skill), and he can evaluate the result because he read the papers. ✅
  • Legitimate use: "I'm looking for literature on async communication and team productivity — what search terms and venues should I check?" The model points him toward where to look, and he finds and verifies the real sources. ✅
  • The misuse he chose: "Give me a citation that supports my claim." The model invents one, because that's a plausible response, and Marcus can't tell because he has nothing to check it against. ❌

The difference, again, is the governing rule: in the legitimate uses Marcus can evaluate the output (he read the papers; he can run the searches); in the misuse he can't (he asked precisely because he didn't know). A citation you cannot verify is a citation you cannot use — and asking the model to supply the source you lack is asking it to do the one thing it does most dangerously.

The takeaway

This is the chapter's warning made concrete. The model produced something fluent, specific, confident, and entirely false, and it did so in exactly the situation the governing rule forbids — where the user couldn't evaluate the output. The lesson isn't "never use AI for research"; it's never trust an AI-produced fact you can't independently verify, and never reach for the model precisely because you can't. Verification is the price of using the tool, and it is non-negotiable. The fluent, perfectly-formatted citation to a paper that doesn't exist isn't a malfunction — it's the model working exactly as designed, and it looks identical to the truth. Only a human who checks can tell the difference. Be that human.

Related: Chapter 29 · Case Study 1 · Quiz · Key Takeaways