Case Study 2: The Demarcation Problem in Forensic Science

When "Science" Isn't Scientific

In 2009, the U.S. National Academy of Sciences published a landmark report, Strengthening Forensic Science in the United States: A Path Forward. The report's central finding was devastating: with the exception of DNA analysis, no forensic discipline had been rigorously validated through scientific testing. Bite mark analysis, hair microscopy, toolmark comparison, blood spatter interpretation, and even fingerprint analysis lacked the empirical foundation that would make them genuinely scientific methods.

This case study examines forensic science through the lens of unfalsifiability — asking how methods that were never scientifically validated came to be treated as scientific evidence in courtrooms for decades.

The Falsifiability Deficit

Consider bite mark analysis, which was used in criminal trials from the 1970s onward. The core claim: a trained forensic odontologist can match a bite mark on a victim's skin to the dental pattern of a specific individual, to the exclusion of all other individuals.

Question 1: What would disprove this claim? A systematic study showing that forensic odontologists cannot reliably match bite marks to individuals. Such studies were eventually conducted — and found error rates that would be unacceptable in any truly scientific discipline.

Question 2: Was the claim ever rigorously tested before being used in court? No. The method was adopted based on practitioner experience and case-specific anecdotal evidence, not controlled scientific testing.

Question 3: When evidence challenged the claim, what happened? Forensic odontologists defended the method, arguing that failures reflected inadequate training rather than fundamental limitations of the technique — the classic epicycle response.

Question 4: Complexity increasing without predictive power? Yes — increasingly elaborate criteria for bite mark "matching" were developed without the fundamental reliability of the technique being established.

Question 5: Could proponents and skeptics agree on a test? Eventually yes — but only after decades of resistance. When rigorous testing was finally conducted, the results were damning.

The Human Cost

The Innocence Project has identified numerous cases where individuals were convicted — some sentenced to death — based on forensic evidence that was later shown to be unreliable:

  • Keith Allen Harward spent 33 years in prison, convicted partly on bite mark evidence that was later shown to be scientifically baseless. DNA evidence exonerated him in 2016.
  • The FBI acknowledged in 2015 that its hair microscopy unit had given flawed testimony in the vast majority of cases reviewed — spanning decades of trials.

These are not abstract epistemological failures. They are concrete cases where unfalsifiable claims — methods presented as scientific but never rigorously tested — directly contributed to wrongful imprisonment.

Why the Demarcation Problem Matters

The forensic science case demonstrates why Popper's demarcation question — "What distinguishes science from non-science?" — has practical consequences beyond philosophy seminar rooms.

When a forensic examiner testifies that a bite mark "matches" a defendant, they are presenting an unfalsifiable claim as scientific evidence. The claim appears scientific (it involves technical expertise, specialized knowledge, and confident language), but it lacks the core features of genuine science: systematic testing, known error rates, independent validation, and vulnerability to falsification.

The courtroom, unfortunately, is not well-equipped to distinguish between genuine science and unfalsifiable expertise. Judges apply the Daubert standard (in federal courts) or the Frye standard (in some state courts) to evaluate expert testimony, but these legal standards are crude epistemological tools that have consistently admitted forensic methods that lack scientific validation.

Structural Lessons

  1. Unfalsifiable claims can masquerade as science through the trappings of expertise. Technical language, institutional backing, and practitioner confidence can create the appearance of scientific validity without the substance.

  2. The absence of testing is not the same as the presence of validation. Many forensic methods were never falsified — but they were also never rigorously tested. The absence of disconfirmation was mistaken for confirmation.

  3. Institutional incentives reinforced the unfalsifiability. Prosecutors needed forensic evidence to secure convictions. Forensic laboratories needed cases to justify their budgets. Defense attorneys lacked the technical expertise to challenge methods. The entire system was incentivized to accept unfalsifiable expertise as scientific.

  4. The correction required external intervention. Forensic science did not self-correct. It took the NAS report (an external review), the Innocence Project (an external advocacy organization), and DNA evidence (an external technological development) to expose the problems.

Discussion Questions

  1. Apply the Five-Question Diagnostic to fingerprint analysis. Is it more or less falsifiable than bite mark analysis? Why?
  2. How could the legal system be reformed to better distinguish between genuinely scientific expert testimony and unfalsifiable expert opinion?
  3. What parallels exist between the forensic science case and other fields where practitioner experience is treated as equivalent to scientific evidence?
  4. The chapter argues that unfalsifiability "feels powerful from the inside." How does this apply to a forensic odontologist who has spent 30 years matching bite marks?

References

  • National Research Council (2009). Strengthening Forensic Science in the United States: A Path Forward. National Academies Press. (Tier 1)
  • The Innocence Project's case database documents specific exonerations involving forensic evidence. (Tier 1)
  • The FBI's 2015 review of microscopic hair comparison testimony acknowledged systematic flaws spanning decades. (Tier 2)
  • The President's Council of Advisors on Science and Technology (PCAST) issued a 2016 report further documenting the scientific limitations of pattern-matching forensic disciplines. (Tier 1)