Chapter 5 — Exercises

This is the longest exercise set in the book, deliberately. Chapter 5 is a skill rather than a body of information, and skills are built by repetition. Items marked † have worked solutions in Appendix M.

A. Recall

5.1 State the question that comes before all others when meeting a health claim, and the three follow-ups.

5.2 † List the rungs of the evidence ladder from strongest to weakest.

5.3 Give the five reasons animal results fail to transfer to humans.

5.4 What is each trial phase designed to detect? What can each not detect?

5.5 † Name the six structural features that determine a trial's quality.

5.6 Define surrogate endpoint and hard endpoint, and give two examples of each from peptide science.

5.7 In plain language, what does a p-value of 0.03 mean? What does it not mean?

5.8 What does a hazard ratio of 0.80 mean? Is it relative or absolute?

5.9 † Define estimand. Name the two estimands discussed and the question each answers.

5.10 State the six rules for using the four-tier rating system.

B. The ladder

5.11 Place each on the ladder: (a) a testimonial video; (b) a randomized trial of 40 people over 6 weeks; (c) a meta-analysis of 12 trials; (d) a study in cultured fibroblasts; (e) a rat model of tendon injury; (f) a cohort study following 20,000 people for 10 years; (g) a case report of an unusual adverse event.

5.12 † For item (g) above, explain why a case report is nearly worthless as evidence of efficacy and genuinely valuable as evidence of harm. What asymmetry is doing the work?

5.13 A meta-analysis pools ten small, poorly conducted trials and reports a significant effect. Where does this sit on the ladder, and why is the answer not simply "top rung"?

5.14 Rank these by how much they would change your mind about a compound, and justify the order: a new rat study; a Phase I safety study; a completed but unpublished Phase II; a published Phase III that failed.

C. Animal evidence

5.15 † A compound shows a strong effect in a rat model of surgically transected tendon. List every distinct reason this might not translate to a human with a chronic overuse tendinopathy.

5.16 "Over a hundred animal studies support this compound." Explain, using reason ⑤ from §5.3, why this sentence conveys less than it appears to. What would you need to know to interpret it?

5.17 Construct the strongest argument you can that a large body of consistent animal evidence should substantially raise confidence about humans. Then state the strongest response.

D. Reading trials

5.18 † A trial reports: "Treatment produced a statistically significant improvement in the primary endpoint (p = 0.04) in 38 participants over 8 weeks." Write six questions you would need answered before forming any view, in order of importance.

5.19 A trial is open-label with a self-reported symptom endpoint, and the drug has noticeable side effects. Explain the specific mechanism by which this combination inflates the apparent effect.

5.20 A trial's registered primary endpoint differs from the one reported in the published paper. Explain why this matters and how you would check.

5.21 † Nearly every obesity pharmacotherapy trial provides both arms with structured lifestyle support. Explain the two opposite ways this affects interpretation, and state what the trial result therefore describes.

5.22 A composite endpoint combines cardiovascular death, myocardial infarction, stroke, and hospitalization for unstable angina. The result is significant. What would you want to check before accepting it?

E. Surrogates

5.23 † For each, identify the surrogate and the hard endpoint it is standing in for, and state whether the connecting arrow is well established: (a) raises growth hormone; (b) lowers LDL cholesterol; (c) increases lean body mass; (d) reduces an inflammatory marker; (e) shrinks a tumor; (f) improves skin elasticity on an instrument.

5.24 Describe the antiarrhythmic example from §5.6 in your own words, and state the general lesson in one sentence.

5.25 Design a study that would establish whether a surrogate legitimately predicts an outcome. What would it have to show?

F. Relative and absolute

5.26 † A trial reports a 30% relative risk reduction. The control group event rate is 4% over two years. Calculate the treated group's event rate, the absolute risk reduction, and the approximate NNT.

5.27 The same 30% relative reduction, with a control event rate of 40%. Recalculate. Write one sentence on what the comparison demonstrates.

5.28 † SELECT reported a 20% relative reduction and about 1.5 percentage points absolute over about three years. Write two headlines: one technically accurate and maximally impressive, one technically accurate and maximally underwhelming. Then write the honest one.

5.29 A drug is reported to "double the risk" of a serious adverse event. What single additional piece of information do you need before this sentence means anything?

5.30 §5.8 notes that "marketing quotes relative benefits and absolute harms; critics quote absolute benefits and relative harms." Find a real example of each in any health coverage you can access, and describe what each framing accomplishes.

G. Estimands

5.31 † Explain the difference between the treatment-regimen and efficacy estimands, using SURMOUNT-1's two figures. Which is systematically larger, and why?

5.32 A source quotes "up to 22.5% weight loss." Identify what is missing and explain why its absence is systematically misleading rather than randomly so.

5.33 Why does the estimand problem arise in essentially every long trial of a drug with meaningful side effects? Name one class of trial where it would matter little.

H. Red flags — practice

5.34 † A product page states: "In a clinical study, 87% of users reported improvement within 30 days." Identify every red flag. Then state what study design would have to have been used for this sentence to mean something.

5.35 A claim cites three papers. On checking, one is a review article that cites a fourth paper, one is in a journal with no identifiable editorial board, and one is a conference abstract from 2019 that never published. Write your assessment.

5.36 A supplement company funds a trial of its own product, which shows benefit. Is the result worthless? Argue a position, and state what would raise or lower your confidence.

I. "Explain this to a friend"

5.37 † Explain to someone with no scientific background why "there are over a hundred studies on this" can be true while the honest verdict is still ❌.

5.38 Your friend says: "The trial showed a 20% reduction in heart attacks — that's huge." Respond accurately, without being condescending, and without leaving them thinking the drug is worthless.

5.39 Explain to a friend why a compound having no human trials is different from a compound whose human trials failed — and why both are commonly called "unproven."

J. Applying the ratings

5.40 † Write a full four-line evidence rating (claim, rating, reason, falsifier) for each: (a) semaglutide for weight loss in adults with obesity; (b) BPC-157 for tendon healing in humans; (c) a topical peptide serum for wrinkle reduction; (d) a compound in Phase 2 trials for a serious disease. Then check (a) and (b) against §5.12.

5.41 Someone writes "BPC-157: ❌." State exactly what is wrong with this formulation and rewrite it correctly.

5.42 A compound has strong evidence and a bad reputation among people you respect. Rule 4 says do not downgrade with distaste. Describe how you would notice if you were violating this rule.

5.43 † Take a compound you personally hope works. Write its rating honestly. Then write down what you notice about the experience.

K. Evidence Dossier extension

5.44 Complete Field 5 (Evidence) for every peptide in your dossier, including the "what does NOT exist" line.

5.45 † Complete Field 6 (Rating) for every peptide. Every entry needs a population, an endpoint, a one-sentence reason, and a specific falsifier. Several peptides will need more than one rating.

5.46 Review your ratings. Count how many are ✅, ⚠️, ❌, 🔬. Then answer honestly: are the compounds you personally use or want to use distributed differently from the ones you don't? Write one paragraph. There is no answer key and the exercise is the noticing.