Chapter 40 — Quiz

Twenty-two items on assembly, the internal audit, drift, and transfer. Answer them before opening the key. Several questions have a tempting wrong answer that is the exact failure mode the chapter warns about.

1. What distinguishes a dossier from a set of good notes on the same compounds?

  • A. It is longer and better sourced
  • B. It is organized by question rather than by compound, so it keeps working on compounds it does not contain
  • C. It contains the book's ratings rather than your own
  • D. It is stored in a single file

2. According to §40.2, which three fields do most readers find empty when they reach Chapter 40?

  • A. 1, 2, 3
  • B. 4, 5, 6
  • C. 9, 10, 12
  • D. 7, 8, 11

3. Why are those three the ones most often left empty?

4. The internal audit asks a different question from Chapter 37's comparison. State both questions in one sentence each.

5. You find two entries with comparable evidence bases and different ratings. Which of the following is not one of the three situations §40.3 says you could be in?

  • A. The difference is justified by design, population, or endpoint
  • B. The difference is justified by something outside Field 5 that you did not write down
  • C. The difference is justified by one compound having a more plausible mechanism
  • D. The difference cannot be justified, and one of the two ratings is wrong

6. A Field 6 row reads: "Muscle recovery — ⚠️." Name the two things missing and the rating rule being broken.

7. True or false, with one sentence of justification: an undated rating is acceptable as long as you remember roughly when you made it.

8. What is the §40.3 test for whether a Field 12 entry is specific enough?

  • A. Whether it names a real trial that exists
  • B. Whether a reasonable person could disagree about whether it had been satisfied
  • C. Whether it is under fifty words
  • D. Whether it would move the rating by at least one tier

9. Complete the sentence from §40.3: an entry with an empty Field 12 is not a conclusion, it is ______. Explain why this applies as much to a confident ❌ as to a hopeful ⚠️.

10. §40.4 names two reader biases. Which of the following is the accurate characterization?

  • A. The generous reader's bias is the dangerous one; the harsh reader is merely cautious
  • B. Both are the same mistake — a rating moved by something that is not evidence — pointing in opposite directions
  • C. The harsh reader's bias is worse, because skepticism is overrated
  • D. Neither is a real bias if the final ratings happen to match the book's

11. Which rating rule does the harsh reader break, and what is its mirror image?

12. Why is the harsh reader's bias harder to detect from the inside than the generous reader's?

13. According to §40.5, which of these should trigger a review of an entry?

  • A. A calendar reminder every three months
  • B. Any news about the compound
  • C. A trial reading out, an approval or withdrawal, a shortage changing, a safety signal, or a question you could not answer
  • D. Whenever your confidence changes

14. What is the single revision type that §40.5 says is more valuable than a change, and why?

15. State the §40.5 upgrade discipline in one sentence, then list four things that do not satisfy it.

16. Explain, in three sentences, the structural reason ratings drift upward rather than downward.

17. A friend says: "There's way more evidence for this now — dozens of papers and a company running a trial." Using §40.6, give the two-part reply.

18. Distinguish the three sentences in §40.7. For each, say whether it is a fact about you or a fact about the world.

  • "I don't know."
  • "Nobody knows."
  • "It's been tested and it doesn't work."

19. Which is the stronger state of knowledge — a compound with a large, well-conducted null outcome trial, or a compound with no human trials at all? Why do readers consistently get this backwards?

20. In §40.8, the twelve fields are applied to time-restricted eating. Field 7 (Approved use) comes out empty. What is the correct interpretation, and how does it differ from a drug's Field 7 being empty?

21. §40.9 says the dossier cannot tell you what to do. If two people hold identical dossiers and reach opposite decisions, what has the dossier nonetheless succeeded at?

22. §40.10 says Chapters 41–44 are harder than everything preceding them. Name the three structural absences that make them harder.

Answer key **1.** **B.** Length, sourcing, and storage are incidental. The structural fact is that twelve standing questions transfer to compounds the file does not contain — which is why §40.8 can point the same structure at a non-peptide claim. Option C is the anti-answer: a dossier copied out of Appendix A teaches nothing and goes stale. **2.** **C** — Fields 9 (Risks), 10 (Status), and 12 (What would change my mind). **3.** Because they are the least satisfying to write. Field 9 often has to conclude "unknown, and no reports is not the same as no harms." Field 10 requires five separate answers where people want to give one. Field 12 requires specifying what would defeat a conclusion you have just reached. All three feel like admissions rather than findings — and all three are the fields that make the document usable later. **4.** Chapter 37: *are my ratings correct*, measured against the book's. Chapter 40's audit: *did I apply one standard*, measured against my own other entries. The second is detectable from inside the document with no external reference, which is what makes it a genuine audit. **5.** **C.** Mechanism plausibility is exactly what rating rule 3 forbids using to move a rating. If mechanism is what separated your two entries, you are in situation three, not a justified difference. **6.** Missing: the population and the endpoint. Breaking rating rule 1 — a rating attaches to a claim, and a claim has both. "Muscle recovery" is a topic, not an endpoint; a defensible row would name who and measured how. **7.** **False.** An undated rating is a claim about the eternal, and it cannot be evaluated against what has happened since. Remembering "roughly" is exactly the memory that fails first, and the date is what converts the dossier into the time series §40.4 depends on. **8.** **B.** Specificity is operationally defined by whether the satisfaction condition is adjudicable. "More research is needed" is satisfied by whatever encouraging thing you read next, which means it excludes nothing. **9.** *A belief.* It applies equally to a confident ❌ because if you cannot name the finding that would move it, you did not reach that ❌ by evaluating evidence — you reached it some other way and the evidence review was paperwork. This catches the failure mode that is otherwise invisible: being right for reasons that do not generalize. **10.** **B.** **11.** Rule 4 — *never downgrade with distaste.* Its mirror is rule 3 — *never upgrade with mechanism.* Both forbid something other than evidence from moving a rating; one blocks a good story pushing up, the other blocks a bad salesperson pushing down. **12.** Because skepticism is socially and intellectually rewarded, so the harsh reader receives no corrective signal. Nobody is embarrassed for having been too critical of a supplement, so the errors compound quietly — and they are transmitted to others as evidence assessments, since a ❌ looks identical whether it came from Field 5 or from distaste. **13.** **C.** Calendar-driven review makes you go looking for news at intervals, which manufactures upgrades, because there is always something. **14.** A revision that records *news arrived and the rating did not move.* It is evidence that the standard held under pressure. A file in which every revision is an upgrade has been tracking enthusiasm rather than evidence. **15.** *Nothing upgrades a rating except the readout named in Field 12.* Four of: a press release, a conference abstract, a preprint, a new mechanism paper, a funding round, a clinician's enthusiasm, a friend's result, a review by an interested author, a podcast. **16.** News about a compound reaches you only if somebody thought it worth publicizing, and positive results are preferentially published, promoted, and covered while null results are published quietly, late, or not at all. Your own attention applies the same filter a second time, because you search for compounds you care about. The result is that accumulating apparent support is what you would experience whether or not the evidence changed. **17.** First: volume is not evidence — that is Field 5's failure mode, counting papers instead of reading designs. Second: a trial being *underway* is a fact about the future and moves nothing, because a running trial has by definition not reported. Then the mechanical step — open Field 12 and ask whether the named readout arrived. It usually has not, and the question resolves in a minute. **18.** "I don't know" — a fact about **you**; the evidence may be excellent and you have not looked. "Nobody knows" — a fact about **the world**; ❌, evidence absent. "It's been tested and it doesn't work" — a fact about **the world**, and a much stronger one; ❌, evidence present and negative. **19.** The null trial. It is a far better-understood situation, because a properly designed study in an adequate population with a real comparator returned an answer. Readers get it backwards because "nothing has disproved it" sounds like a point in a compound's favor — but it is true of every claim nobody has ever examined, which means it carries no information at all. **20.** No regulator approves eating patterns, so there is no approval pathway and the absence of approval carries **no information**. For a drug, an empty Field 7 is informative and requires you to say which of the five meanings of "not approved" applies — never submitted, rejected, approved elsewhere only, off-label, or not a drug. Recognizing a *structurally* empty field versus a *damningly* empty one is the skill being tested. **21.** It has made the disagreement be about values rather than about facts. The two people weight things the dossier does not contain — tolerance for uncertainty, the cost of the problem, what happens if it goes wrong — and locating the disagreement there rather than in the evidence is a real service. **22.** No trial arms (you cannot randomize a culture), no endpoints (nobody has defined the outcome measure, and the candidates embed value judgments), and no placebo (there is no version of the society that did not get the drug, running alongside).