Instructor Notes — Chapter 9
Teaching notes
What this chapter is actually for
§9.4. Everything else is context.
Estimands are the single most useful statistical concept in Part II, they are almost never taught to non-statisticians, and this chapter is where students acquire them. A student who leaves able to ask "which estimand?" reflexively has gained something they will use on every trial result they meet for the rest of their life.
The secondary deliverable is phase discipline — the reflex that a bigger number from an earlier phase is not a better result.
Common misconceptions
"One of the two numbers must be the real one." The most persistent. Both are real; they answer different questions. The framing that works: two questions, not two analyses. Ask students which question a patient is asking and which a pharmacologist is asking.
"Reporting the bigger number is dishonest." Not necessarily — it is often a defensible editorial compression of genuine jargon. What makes it a problem is that the compression is systematic: every step selects the same direction. Case Study 1 is built to make that visible without accusing anyone.
"Retatrutide is better than tirzepatide — 24% versus 21%." This will be said in the room. It is a category error rather than a debatable ranking, and saying so plainly is appropriate.
"Tirzepatide is better than semaglutide." Better for what? §9.5's decomposition. The cardiovascular outcomes asymmetry is the row that usually settles it, and students consistently miss it because a larger weight number feels more decisive than a missing trial.
"Dual agonist = two drugs in one." §9.1's diagram. One pharmacokinetic profile, one fixed ratio.
"GIP must be good, since adding it helps." Chapter 7 §7.5's paradox is unresolved and Chapter 9 keeps it unresolved. Students want a resolution; withholding it is the lesson.
The hardest point to teach
Regression to the mean via selection — mechanism ② in Case Study 2.
Students grasp small-sample variance easily. They do not grasp that selecting compounds for having produced good Phase 2 results is what creates the bias. The set of compounds entering Phase 3 is enriched for overestimates, so Phase 3 results are expected to be smaller on average even for compounds that work exactly as well as they truly do.
What works: a concrete demonstration. Imagine twenty compounds, all with a true effect of −15%. Phase 2 estimates scatter from −8% to −22% purely by chance. Which advance? The ones at the top. What does Phase 3 find for those? Something near −15%. The drop was not a failure; it was the noise subsiding.
Students find this genuinely surprising, and once they have it they read every pipeline announcement differently.
Demonstrations that work
The transmission chain, built live. Put SURMOUNT-1's full result on the board with all six qualifiers. Then ask the class to compress it for a press release. Then a headline. Then a social post. Have them do the compressing. They arrive at stage 5 themselves in about six minutes, and the experience of having produced the exaggeration honestly is far more instructive than being shown it.
Which estimand would you want? Give four roles — patient, pharmacologist, health system, marketer — and have students assign an estimand to each with a reason. Exercise 9.11. Fast, and it converts a definition into a judgment.
The comparison worksheet. Fill it in as a class for semaglutide versus tirzepatide, leaving the final row for last. The "what differs that isn't the molecule" row usually produces at least one observation nobody had noticed.
Timing
For a 75-minute session:
| Minutes | Content |
|---|---|
| 0–10 | §9.1–9.2 — dual agonism, briskly. The one-molecule-vs-two-drugs diagram. |
| 10–20 | §9.3 — SURMOUNT-1, with both figures on the board from the start. |
| 20–40 | §9.4 — estimands. The four-roles exercise. The centerpiece. |
| 40–50 | §9.5 — head-to-head and the "which is better" decomposition. |
| 50–62 | §9.6 and Case Study 2 — phase discipline. The twenty-compounds demonstration. |
| 62–70 | §9.7–9.9 — pipeline, briskly. |
| 70–75 | §9.10 protocol; assign the comparison worksheet. |
If time is short, cut §9.7 and §9.8 to reading. Do not cut §9.4 or the Phase 2 material.
Assessment notes
Discriminating items: 9.11, 9.14, 9.17, 9.29, 9.31.
9.29 (does semaglutide's extra ✅ make it better?) is the best summative item — it requires students to resist treating ratings as a scoreboard while still acknowledging that the cardiovascular asymmetry is real and clinically relevant. Both halves are needed for full credit.
9.31 (the comparison worksheet) is the major written assignment. Grade the final row most carefully; it is where the analysis either happens or does not.