Key Takeaways: Adversarial Collaboration and Other Tools
The Big Idea
Individual dissent (Chapter 33) is necessary but insufficient — systemic error requires systemic correction. This chapter evaluates nine institutional tools for reducing error, each targeting specific failure modes. The most effective tools change incentive structures rather than relying on individual behavior change.
The Nine Tools (Quick Reference)
| Tool | What It Does | Top Verdict |
|---|---|---|
| Adversarial collaboration | Disagreeing researchers jointly design & publish | Excellent for specific disputes |
| Pre-registration | Commit to hypothesis & analysis before data | High effectiveness; widely adoptable |
| Registered reports | Journal accepts before data exist | Most effective single innovation |
| Prediction markets | Bet on whether claims replicate | Promising; underdeveloped |
| Red teams | Designated challengers find flaws | Variable; culture-dependent |
| Replication funding | Dedicated money for checking findings | Essential; underfunded |
| Prize-based science | Reward outcomes, not activity | Excellent for defined problems |
| Open data mandates | Make raw data publicly available | Essential infrastructure |
| Post-publication review | Critique published work after release | Essential; underinstitutionalized |
Key Principle
The most effective tools change structures (registered reports change what journals reward) rather than behaviors (asking researchers to be more honest). Structure beats culture.
The Overcorrection Warning
Every tool carries overcorrection risk (Theme 9). Pre-registration could suppress exploration. Replication demands could paralyze research. The goal is calibrated correction, not the opposite wrong answer.
Epistemic Audit — Chapter 34 Addition
Inventory which tools exist in your field, identify which would address your lowest Checklist scores, and design one new correction mechanism.