Case Study 1 — The Tutorial That Broke at Step 4

A worked scenario. Raj Patel writes a getting-started tutorial for chronoparse, tests it the only way he knows how—by re-reading it—and ships it. It works flawlessly for him and fails for almost everyone else, at exactly the same place. This is the anchor case the chapter's §26.5 reduces to its essence; here is the full forty minutes.

The setup

After Chapter 25, Raj's chronoparse README had a working quick-start, and the "how do I install this?" issues stopped. But a new request kept coming: people could install the library and run the one-line quick-start, yet had no idea how to use it on a real problem. So Raj decided to write a proper getting-started tutorial—"Analyze a Log File with chronoparse"—that would carry a beginner from nothing to a small, satisfying win: a script that reads a file of human-written event lines and prints a real date for each.

He wrote it in an afternoon. He's a careful person, so before publishing he did what he believed was thorough testing: he read the tutorial through three times, slowly, checking each step. Each time, he mentally walked the steps and confirmed they were right. They were right—for him. Here is the heart of what he shipped:

"Step 1 — Make a project folder. Create a new folder and open it in your editor. Step 2 — Install chronoparse. Run pip install chronoparse. Step 3 — Write the script. Create analyze.py and type: import chronoparse from chronoparse import parse_all for dt in parse_all(open('events.txt')): print(dt) Step 4 — Run it. Run python analyze.py. You should see a date printed for each line of your file."

Raj ran it one last time himself before publishing. A clean list of dates printed. Ship it.

The failure

Priya, a junior developer at another company, found the tutorial through a search and was delighted—exactly the gentle on-ramp she'd wanted. She made a folder. She installed the library. She created analyze.py, typed the four lines exactly as shown, saved, and ran it. Step 4 promised "a date printed for each line of your file." Instead she got:

Traceback (most recent call last):
  File "analyze.py", line 3, in <module>
    for dt in parse_all(open('events.txt')):
FileNotFoundError: [Errno 2] No such file or directory: 'events.txt'

She read step 4 again. It said run the script and see dates. She'd run the script. She got a crash. Nothing in the tutorial mentioned events.txt as something she had to make—it just appeared in the code at step 3, and she'd assumed (reasonably) that the library created it, or that it was a built-in sample, or that something she'd done already produced it. She re-read all four steps twice. No mention. She checked whether pip install was supposed to include sample files. It wasn't. She wondered if she'd mistyped, and compared her code to the tutorial character by character. Identical. She spent ten minutes searching the error message, found generic answers about file paths that didn't explain why this tutorial's step 4 failed, and started to feel the specific demoralization a broken tutorial produces: I followed every step exactly and it still doesn't work, so either this tool is broken or I'm missing something obvious that everyone else apparently knows. Forty minutes after she started, she closed the tab and went looking for a different library—the same exit Raj's eleven README visitors had taken, one layer deeper.

She wasn't alone. The tutorial worked for no one who didn't already have an events.txt file lying around. It worked for Raj every single time, which is exactly why he never saw it.

The diagnosis

The bug is an assumed step, and it's a textbook one. Raj had an events.txt file in his project folder—he'd created it while building the library, months earlier, and it had simply been there ever since. When he wrote step 3, he typed open('events.txt') without a flicker of thought, because on his machine events.txt always existed. Creating it was never a conscious action for him, so it never became a step on the page. The tutorial assumed a state of the world—"a file named events.txt, containing event lines, exists in this folder"—that no step had established for the reader.

And critically: re-reading could never have caught this. Each time Raj read "for dt in parse_all(open('events.txt'))," his mind supplied the file's existence automatically, because for him it was a fact, not a question. His three careful read-throughs filled the gap three times. The gap is invisible from inside his own knowledge. This is the curse of knowledge (Chapter 2) in its purest operational form, and it's why the chapter insists you cannot test a tutorial by reading it.

What would have caught it: a fresh-environment run (a brand-new empty folder would have crashed on step 4 immediately, because the empty folder genuinely lacks events.txt), or—best—watching one beginner like Priya try it and stall at exactly line 3.

The fix

The repair is not better prose. Raj's sentences were fine. It's a missing step, inserted where the assumption lived:

"Step 3 — Create a sample file to analyze. In the same folder, create a file called events.txt and put a few event lines in it, like: text* *standup tomorrow at 9am* *launch in 2 weeks* *retro last Friday* * Save it. (This is the file your script will read in the next step.)

Step 4 — Write the script. Create analyze.py and type: …

Step 5 — Run it. Run python analyze.py. You should see three dates printed—one for each line of events.txt."

Notice everything the fix does. The missing action becomes a numbered step. The file's contents are shown literally, so there's nothing for the reader to invent. There's an explicit "save it." A parenthetical ties the new step to the next one ("the file your script will read"), so the reader understands why they're making it. And the expected result in the (renumbered) run step is now specific—"three dates, one for each line"—so the reader can confirm exact success instead of vaguely hoping. Raj also did the thing he should have done first: he ran the whole tutorial in a new empty folder, and when that surfaced one more tiny gap (he'd never said to make sure you're in the folder when you run the command), he fixed that too.

The lesson

Raj's tutorial wasn't badly written. It was untested in the only way that matters. Three careful re-readings found nothing because re-reading can't find an assumed step—the author's expertise fills the gap every time. One beginner, doing nothing but following the steps with an empty folder, found it in ninety seconds. The forty minutes Priya lost were a defect in the tutorial, not in Priya; her stall was data Raj never collected because he tested against the worst possible sample: himself.

The test that would have saved it: Has someone who has never done this—or you, in a brand-new empty environment—followed this tutorial to the promised success, with no help and nothing you already had lying around? If no, you don't know whether it works. You only know it works for you, and you are precisely the wrong person to ask.