Key Takeaways: The Streetlight Effect

The Big Idea

Fields systematically study, measure, and optimize what is quantifiable rather than what is significant. This reshapes entire domains around the wrong questions — not by introducing specific wrong answers but by making the right questions invisible.

Core Concepts

Goodhart's Law

"When a measure becomes a target, it ceases to be a good measure." People optimize for the metric, and the metric decouples from the underlying construct.

Campbell's Law

"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures."

The McNamara Fallacy (Four Steps)

  1. Measure what's easily measured (OK)
  2. Disregard what can't be measured (misleading)
  3. Presume what can't be measured isn't important (blindness)
  4. Say what can't be measured doesn't exist (suicide)

The Five-Step Deeper Pattern

  1. The construct is complex and hard to quantify
  2. A proxy is adopted because it's measurable
  3. The system attaches rewards/punishments to the proxy
  4. Decoupling occurs as actors optimize for the proxy
  5. Blindness sets in as proxy improvement is mistaken for construct improvement

Cross-Domain Examples

Field Construct Proxy Decoupling
Military Strategic progress Body counts Counts up, war lost
Education Learning Test scores Scores up, curriculum narrowed
Healthcare Care quality Mortality rates Rates down, sick patients turned away
Criminal justice Public safety Crime statistics Stats down, actual safety unknown
Business Corporate health Quarterly EPS EPS up, long-term value destroyed
Economics Wellbeing GDP GDP up, health/equality/environment declining
Science Research quality Citations/h-index Metrics up, replication crisis

Practical Strategies

  1. Multiple metrics, no single target
  2. Rotate which metrics are targeted
  3. Measure the gap between proxy and construct
  4. Include qualitative assessment
  5. Make the streetlight visible (ask "what are we NOT measuring?")
  6. Separate measurement from targeting

Epistemic Audit — Chapter 4 Addition

After this chapter, your audit should include: what your field measures, what it doesn't measure, whether Goodhart's Law has decoupled key metrics from their constructs, and who benefits from the current measurement system.

What's Coming Next

Chapter 5: Survivorship Bias at Scale — how fields build on the evidence that survived while ignoring what didn't.

Quick Reference: Goodhart's Law Diagnostic

Is this metric a target?     NO → Metric probably still informative
                              YES → Ask: Has the metric decoupled from the construct?
                                     NO → Healthy (but monitor)
                                     YES → Streetlight effect active