Chapter 5 — Self-Check Quiz

Twenty-five questions: multiple choice and short answer. Answer before opening the key at the bottom. The short-answer items reward the honest verbs of the book — "consistent with," "supports," "excludes" — and the four Daubert factors.

Multiple choice

1. The Frye standard asks a single question about a scientific method. Which one? - A) Has the method been published in a peer-reviewed journal? - B) Is the method generally accepted in its relevant field? - C) What is the method's measured error rate? - D) Can the method's central claim be falsified?

2. Under the Daubert standard, the trial judge's primary role is best described as: - A) a neutral scorekeeper who admits whatever both sides agree on - B) a scientist who personally re-runs the disputed experiments - C) a gatekeeper who independently assesses the relevance and reliability of expert testimony - D) an advocate who decides which side's expert is more persuasive

3. Daubert v. Merrell Dow Pharmaceuticals (1993) originally arose from: - A) a criminal arson prosecution - B) a civil suit alleging a drug caused birth defects - C) a fingerprint misidentification - D) a challenge to DNA evidence

4. Which of the following is not one of the four classic Daubert factors? - A) testability / falsifiability - B) known or potential error rate - C) the expert's years of courtroom experience - D) peer review and publication

5. Kumho Tire Co. v. Carmichael (1999) is important because it held that the judge's gatekeeping duty: - A) applies only to DNA evidence - B) was abolished for experience-based testimony - C) applies to all expert testimony, not only "scientific" testimony - D) belongs to the jury, not the judge

6. A method is "generally accepted" by the practitioners who perform it but has never been tested against ground truth and has no published error rate. Under an honest Daubert analysis, it should be: - A) admitted, because acceptance is sufficient - B) excluded, because it fails testability and error-rate scrutiny - C) admitted automatically because it is not novel - D) sent to the jury without any ruling

7. The "grandfather problem" the chapter describes is: - A) older judges being more lenient than younger ones - B) long-admitted methods being kept in court because of their long history, not their validity - C) a rule that grandparents cannot serve on juries - D) the requirement that DNA be re-validated every generation

8. Which statement best captures the chapter's "bitter irony" about how Daubert is applied? - A) It is applied identically in all cases. - B) It is applied more strictly to criminal defendants than to civil plaintiffs. - C) It is applied more strictly to civil plaintiffs' novel science than to old forensic methods used against criminal defendants. - D) It is never applied at all.

9. "We have never had a conviction overturned because of our method" is not a valid statement of an error rate because: - A) error rates are illegal to disclose - B) it reflects undetected errors and the absence of ground-truth testing, not a measured frequency - C) only juries may calculate error rates - D) it is too high to be believable

10. The four Daubert factors are best understood as: - A) a rigid checklist, all of which must be satisfied in every case - B) the philosophical opposite of the scientific method - C) a flexible, non-exclusive translation of the scientific method into a legal reliability test - D) a replacement for the Federal Rules of Evidence

11. Which pairing correctly matches a forensic method to its place under an honest Daubert analysis? - A) Bite-mark matching — clears all four factors - B) Single-source nuclear DNA typing — clears all four factors - C) "Pour pattern" arson indicators — strong, measured error rate - D) Microscopic hair comparison — quantified and broadly validated

12. FRE 702, as amended after 2000, requires expert testimony to be: - A) based on sufficient facts, the product of reliable methods, and a reliable application of those methods to the case - B) generally accepted, and nothing more - C) offered only by government employees - D) supported by at least one eyewitness

13. Some U.S. states still use the Frye standard rather than Daubert. The practical consequence is: - A) no consequence; the standards are identical - B) a method may be freely admitted in one state and fought over in another - C) DNA is inadmissible in Frye states - D) only federal courts hear forensic evidence

14. In the Cameron Todd Willingham case, the fire-investigation "indicators" used to conclude arson (crazed glass, low burn/"pour" patterns) were later shown to be: - A) confirmed by modern fire science as reliable - B) folklore — routinely produced by ordinary fires reaching flashover, not proof of an accelerant - C) the result of deliberate fraud by the investigators - D) irrelevant to the conviction

15. Justice Blackmun's line that "vigorous cross-examination... and careful instruction on the burden of proof" are the means of attacking "shaky but admissible evidence" assumes that: - A) juries can reliably tell strong forensic evidence from weak - B) all forensic evidence is excluded - C) judges never make mistakes - D) cross-examination is forbidden

Short answer

16. State, in one sentence each, two ways a courtroom and a laboratory differ as truth-finding systems.

17. Explain the circularity built into the Frye standard, using a forensic example.

18. A defense attorney must cross-examine an examiner on a method with no published error rate. Write the single most important Daubert-factor question she should ask, and say what a weak answer reveals.

19. Distinguish a method's foundational validity from its reliable application in one case, and name which is more often neglected in court.

20. Why does Kumho Tire matter specifically for the pattern-comparison disciplines (fingerprints, toolmarks, bite marks)?

21. Give the honest, defensible version of this overstatement: "My examination proves the fire was deliberately set." (Assume no instrumental confirmation.)

22. In one or two sentences, explain why "the judge admitted the evidence" is not the same as "the evidence is scientifically valid."

23. Name the two Cold Case evidence threads this chapter says would clear an honest gate, and the one type of fire reasoning that should not.

24. Why does the chapter stress that the Willingham investigators "were not frauds," and what does that teach about how junk science usually enters court?

25. Validity-spectrum question: a method sits near the "discredited" end of Chapter 1's validity table. Predict how it fares against each of the four Daubert factors, and state what its admission anyway would reveal about the gate.

Answer key (try every question first) **1.** B — general acceptance in the relevant field. **2.** C — independent gatekeeper assessing relevance and reliability. **3.** B — a civil suit (the Bendectin / birth-defects litigation). **4.** C — years of courtroom experience is *not* a Daubert factor (and *Kumho Tire* specifically refuses to let experience substitute for reliability). **5.** C — gatekeeping applies to *all* expert testimony, scientific or not. **6.** B — it fails testability and error-rate scrutiny; an honest gatekeeper excludes it. **7.** B — long-admitted methods kept in on the strength of precedent, not validity. **8.** C — strict against civil plaintiffs' novel science, lenient toward old forensic methods against criminal defendants. **9.** B — undetected errors and no ground-truth testing; not a measured frequency. **10.** C — a flexible, non-exclusive translation of the scientific method. **11.** B — single-source nuclear DNA clears all four factors. **12.** A — sufficient facts, reliable methods, reliable application to the case. **13.** B — the same method can be admitted in one state and contested in another. **14.** B — folklore; flashover in ordinary fires produces the same marks; no accelerant required. **15.** A — it assumes juries can reliably weigh shaky forensic evidence (which the CSI effect questions). **16.** Any two of: different clocks (science is iterative/open-ended, the law is terminal/deadline-bound); different goals (generalizable truth vs. case-specific resolution); different standards of proof (quantified uncertainty vs. verbal standards like "beyond a reasonable doubt"); opposite postures toward authority (science is "take no one's word for it," the law admits expert opinion *because of who* gives it). **17.** A method becomes accepted within its own community of practitioners → on that basis it is admitted in court → its courtroom success reinforces the field's acceptance → and the circle closes with no one ever running a validation study. Example: bite-mark matching was "generally accepted" by forensic odontologists for decades on no validated basis. Frye admits a unanimously believed error precisely *because* it is unanimously believed. **18.** "What is the measured error rate of this method, established by blind studies against known ground truth?" A weak answer — "we don't have one" or "essentially zero, I've never been shown wrong" — reveals the method has never been validated against ground truth; the absence of an error rate *is* the finding. **19.** Foundational validity = whether the method, in general, reliably does what it claims (the question the four factors mostly target). Reliable application = whether the method was performed correctly in *this* case (contamination, labeling, interpretation). The *application* prong is the more neglected; even a valid method (DNA) can be ruined by a bad sample or a misread mixture. **20.** Because those disciplines are the ones most likely to recharacterize a conclusion as "experience" or "skill" rather than "science" to dodge reliability scrutiny. *Kumho Tire* closes that escape hatch: gatekeeping applies regardless of the label, so an examiner cannot avoid the error-rate question by calling the work seasoned intuition. **21.** Something like: "The fire shows burn patterns *consistent with* an accelerant-fueled fire, but those patterns can also be produced by an ordinary fire reaching flashover; a deliberate-fire conclusion is not supported unless ignitable-liquid residue is confirmed instrumentally." (It states limits and reserves the strong claim for confirmation — Chapter 22/23.) **22.** Admissibility is a *legal* determination by a generalist judge under time pressure and a forgiving appellate standard; validity is a *scientific* property established by testing. Old forensic methods are frequently admitted on precedent despite weak validity, so "admitted" can mean "the gate failed to stop it." **23.** Would clear an honest gate: the DNA work (Chapters 7–9) and the GC-MS instrumental confirmation of any accelerant (Chapter 23). Should *not* survive: "arson folklore" reasoning of the Willingham type — "pour patterns" / "crazed glass" read as proof of an accelerant without instrumental confirmation. **24.** To show that junk science usually arrives not as deliberate deception but as the sincere, good-faith application of an unvalidated method by practitioners who have never been required to measure how often they are wrong. That is *more* dangerous, because sincerity is persuasive and the error is invisible to the people committing it. **25.** It would likely fail testability (its core claim resists falsification or has failed the tests run), have no valid error rate, lack genuine peer-reviewed validation (prestige of the parent field is not the same thing), and survive only on an eroding "general acceptance." If a court admits it anyway, that reveals the gate is tracking precedent and reputation rather than validity — the gatekeeper has not done the job the factors describe.