Case Study 1: Law and Code Refactoring -- The Invisible Purpose

Case Study 1: Law and Code Refactoring -- The Invisible Purpose

"The most dangerous phrase in the language is, 'We've always done it this way.'" -- Often attributed to Grace Hopper

The second most dangerous phrase may be, "I don't see why we do it this way."

Two Systems, One Failure Mode

This case study examines Chesterton's fence operating in two domains that shape modern life in profoundly different ways: legal regulation and software engineering. One domain governs the financial system that holds the world's wealth. The other governs the information systems that run the world's infrastructure. Despite their apparent differences, the two domains share a structural feature that makes them uniquely vulnerable to Chesterton's fence failures: in both domains, the systems are so complex that no single person understands all of their components, and the purpose of any given component may be invisible to people who were not present when it was created.

The result, in both domains, is the same: components that appear unnecessary are removed by confident reformers who do not understand what the components are doing -- and the consequences reveal, too late, what was lost.

Part I: The Legal Fence -- Financial Deregulation and the Road to Crisis

The Fence That Worked Too Well

The story of Glass-Steagall is, at its core, a story about the relationship between memory and policy. When the law was enacted in 1933, the memory of the Great Depression was raw and immediate. Every legislator who voted for the bill could recall, or had constituents who could recall, the bank failures that had wiped out the savings of ordinary Americans. The purpose of the fence was vivid, urgent, and universally understood.

But memory fades. And in the fading of memory, the purpose of the fence became invisible.

The process was gradual. In the 1960s and 1970s, the banking system was stable, and Glass-Steagall's restrictions were accepted as part of the regulatory landscape. In the 1980s, as financial innovation accelerated and global competition intensified, the restrictions began to chafe. Banks argued that the separation between commercial and investment banking prevented them from competing with European and Japanese financial institutions that faced no such restrictions. Economists argued that the separation was based on an outdated understanding of financial risk and that modern risk management tools could handle the dangers that Glass-Steagall had been designed to prevent.

The arguments were not frivolous. Financial markets genuinely had changed since 1933. Risk management tools genuinely had improved. The separation between commercial and investment banking genuinely did impose costs on the industry and, arguably, on consumers. The reformers who advocated for repeal were not reckless or ignorant. They were intelligent, educated, and well-intentioned. They simply could not see the use of the fence.

What they could see was the cost. The cost of compliance was measurable: billions of dollars in foregone revenue, reduced competitiveness, and constrained innovation. The benefit of the fence -- the prevention of a systemic banking crisis that had not occurred in sixty years -- was not measurable. It was, in a precise sense, invisible. How do you quantify the value of a catastrophe that did not happen? How do you put a dollar figure on the systemic stability that the fence was providing? You cannot. And so, in any cost-benefit analysis that considered only the measurable, the fence was a pure cost.

The Erosion Before the Repeal

Glass-Steagall was not repealed in a single act. It was eroded over decades. Beginning in the 1980s, regulators granted increasingly broad exemptions to the law's restrictions. The Federal Reserve allowed bank holding companies to engage in limited securities activities. The Office of the Comptroller of the Currency approved bank subsidiaries that operated in ways that blurred the line between commercial and investment banking. By the time the Gramm-Leach-Bliley Act formally repealed Glass-Steagall in 1999, the fence was already full of holes.

This erosion pattern is characteristic of Chesterton's fence failures in law. The fence is rarely torn down all at once. It is weakened incrementally -- an exemption here, a reinterpretation there, a waiver under special circumstances. Each incremental weakening produces no immediate catastrophe, which reinforces the narrative that the fence is unnecessary. The absence of consequences after each weakening is taken as evidence that the fence has been proven pointless, when in fact the system's remaining redundancies may still be providing protection that masks the effect of the weakening.

The erosion of Glass-Steagall followed this pattern precisely. Each exemption was followed by a period of stability, which was used as evidence that further exemptions were safe. The cumulative effect of the exemptions was to gradually reconnect the commercial and investment banking systems in ways that Glass-Steagall had been designed to prevent. By the time the formal repeal occurred, the substantive protections had already been largely dismantled.

The Consequences

The 2008 financial crisis is too complex to attribute to any single cause. But the removal of the Glass-Steagall fence was a significant contributing factor, and the mechanism was exactly what the fence's original designers had feared: commercial banks used federally insured deposits to fund speculative positions, and when the speculation failed, the losses threatened the deposit-taking system and required massive government intervention.

Citigroup, which had been created by the merger of Citicorp (a commercial bank) and Travelers Group (which included the investment bank Salomon Smith Barney) -- a merger that was technically illegal under Glass-Steagall but was permitted under the expectation that the law would be repealed -- required $45 billion in direct government investment and over $300 billion in government guarantees. The merger that Glass-Steagall would have prevented produced a company whose failure would have been catastrophic to the entire financial system.

The fence had been protecting against exactly the scenario that materialized. The people who removed it had not understood this -- not because they were foolish, but because the fence had been so effective for so long that the danger it was preventing had become theoretical rather than lived. The fence's own success was its undoing.

The Lesson for Legal Reform

The Glass-Steagall story offers several specific lessons about Chesterton's fence in the legal domain.

First, the measurability asymmetry. The costs of regulation are always more measurable than the benefits. Compliance costs show up in financial statements. Reduced competition shows up in market analyses. Forgone innovation shows up in industry reports. The prevention of a crisis that does not happen shows up nowhere. Any cost-benefit framework that considers only the measurable will systematically overestimate the costs of regulation and underestimate its benefits.

Second, the generational memory problem. The people who created the regulation experienced the crisis that motivated it. The people who removed the regulation did not. Knowledge of why the fence was built resided in the memories of people who were no longer active in the policy debate. When they retired, died, or were simply outnumbered by younger voices who had never experienced a banking crisis, the institutional memory that justified the fence was lost. The dark knowledge became truly dark.

Third, the incremental erosion pattern. Fences in law are rarely torn down at once. They are weakened gradually, with each weakening appearing safe in isolation. The cumulative effect is invisible until the final weakening pushes the system past its threshold -- a connection to phase transitions (Ch. 5) that the reformers rarely recognize.

Part II: The Code Fence -- When Cleaning Up Creates Catastrophe

The Legacy System Problem

In 2014, a major financial services company -- we will call it FinCo, to protect the specifics while preserving the structural lesson -- undertook a large-scale modernization of its core transaction processing system. The system, originally written in the early 1990s, had been maintained and extended over two decades by successive teams of developers. It processed millions of transactions daily and was, in the words of one engineer, "the beating heart of the company."

The modernization team was talented and well-resourced. They had explicit support from senior leadership, a generous timeline, and a clear mandate: simplify the codebase, improve performance, reduce technical debt, and make the system maintainable by the current team. The existing code was a labyrinth. Functions called functions that called functions in chains that were difficult to trace. Configuration files contained settings that no one could explain. Error-handling routines caught exceptions that no one believed could occur. Database queries included filters that no one could justify.

The team applied standard software engineering best practices. They wrote comprehensive test suites. They refactored aggressively. They removed code that appeared to serve no purpose. They simplified error handling. They streamlined database queries. They documented everything they changed.

The modernized system passed all tests. It was deployed to production on a Friday evening -- a decision that itself violated an old, apparently pointless rule about never deploying on Fridays.

The Monday Morning Cascade

On Monday morning, the system began behaving erratically. Transactions were failing at a rate that had never been seen before. Investigation revealed the cause: a function that the modernization team had identified as "dead code" -- a function that appeared to do nothing more than introduce a 200-millisecond delay in a specific transaction pathway -- had been removed.

The function was not dead. It had been written in 1997 by a developer (long since departed) who had discovered, through painful production experience, that a specific sequence of transactions -- which occurred only under high load conditions that happened only during the Monday morning trading rush -- could produce a race condition in the database layer. The 200-millisecond delay was a deliberate throttle that prevented the race condition from materializing. It was ugly. It was undocumented. It was architecturally questionable. But it worked, and it had been working for seventeen years.

The modernization team had removed the delay because it appeared to serve no purpose. Their test suite had not caught the problem because the test suite did not simulate the specific load conditions of a Monday morning trading rush. The function's purpose was invisible because the problem it prevented had not occurred in seventeen years -- because the function had been preventing it.

The Pattern at Scale

FinCo's experience is not unusual. Software engineering lore is rich with stories of "dead code" that turned out to be alive. The specifics vary -- a timing delay that prevents a race condition, a null check that handles a hardware-specific edge case, a configuration setting that works around a bug in a long-obsolete version of a dependency that still runs in one production environment -- but the structure is always the same.

Several patterns recur in software Chesterton's fence failures:

The workaround that outlived its documentation. A developer discovers a bug in a library, an operating system, or a piece of hardware. The developer writes a workaround. The workaround solves the problem. The developer may or may not document the workaround. Over time, the library is updated, the operating system is patched, or the hardware is replaced -- but not always, and not everywhere. The workaround remains in the code. A future developer sees the workaround, cannot reproduce the bug it addresses (because the bug may only manifest on certain hardware configurations or under certain load conditions), and removes the workaround. In the environments where the original bug has been fixed, nothing happens. In the environments where it has not been fixed, the system fails.

The defensive code that catches the impossible exception. A senior developer, burned by experience, writes code that handles exceptions that, according to the specification, cannot occur. A junior developer, reading the specification, sees the exception handling as unnecessary and removes it. Eventually, the "impossible" exception occurs -- because specifications are incomplete, hardware is imperfect, and race conditions exist in the real world -- and the system crashes instead of handling the situation gracefully.

The deployment rule that prevents invisible hazards. Many organizations have rules about when deployments can occur: not on Fridays (because if something goes wrong, the team will not be available over the weekend to fix it), not during peak hours (because the deployment itself consumes resources), not before a holiday (because reduced staffing makes incident response slower). These rules appear to be mere superstitions -- until a team that ignores them discovers the specific failure mode the rule was designed to prevent.

Dark Knowledge in Code

What makes software particularly vulnerable to Chesterton's fence failures is the relationship between code and knowledge. Code is, in a precise sense, crystallized knowledge -- it encodes decisions, solutions, and workarounds. But code encodes the what without the why. A function that introduces a 200-millisecond delay tells you what it does (delays execution by 200 milliseconds) but not why it does it (prevents a race condition that occurs only under specific load conditions that the original developer discovered in production seventeen years ago).

Comments and documentation can bridge this gap, but in practice they rarely do so completely. The urgency of production incidents does not leave time for thorough documentation. The developer who writes the workaround at 2 AM during an incident is focused on stopping the bleeding, not on explaining to future generations why the bandage is necessary. The knowledge of why exists in the developer's memory, in the incident response Slack channel that was archived and deleted, in the post-mortem document that was written but never linked to the code.

Over time, the knowledge decays. The developer leaves the company. The incident response records are archived. The post-mortem is buried in a document management system that no one searches. The code remains -- a Chesterton's fence standing in the middle of the codebase, protecting against a danger that no one currently working on the system has ever encountered.

Countermeasures in Software

The software industry has developed several practices that function as formalized applications of Chesterton's principle:

Code archaeology. Before removing a piece of code, investigate its history: read the commit messages, find the pull requests that introduced it, search for related bug reports or incident records. This is the software equivalent of seeking the fence's author.

Feature flags. Instead of removing code, disable it through a configuration flag that can be re-enabled quickly if problems appear. This is the software equivalent of lowering the fence instead of removing it.

Canary deployments. Deploy changes to a small subset of production traffic first and monitor for problems before rolling out widely. This is the software equivalent of testing incrementally.

The "Chesterton comment." A growing practice of annotating code that might appear unnecessary with a comment explaining why it exists -- "This delay prevents the race condition described in INCIDENT-4721. Do not remove without understanding the concurrent access pattern in the Monday morning batch process." This is the direct encoding of dark knowledge into the codebase.

Synthesis: The Shared Structure

The legal and software cases share a structural anatomy that constitutes the essence of Chesterton's fence:

A protection is created in response to a specific, experienced harm (the Great Depression, a production incident).
The protection works. The harm does not recur.
The protection's success makes the harm invisible. Because the harm is not occurring, it appears that the protection is unnecessary.
The knowledge of why the protection was created decays. The people who experienced the original harm retire, die, or leave. The documentation is lost or ignored. The dark knowledge becomes truly dark.
The protection's visible costs remain. Compliance costs, code complexity, performance overhead, regulatory burden -- these are always visible.
The cost-benefit analysis tips against the protection. Visible costs, invisible benefits. The analysis is biased by the very success of the protection.
The protection is removed. The harm returns.

This is the universal anatomy of the Chesterton's fence failure. It operates in law and code, in ecosystems and institutions, in traditions and regulations. The specific domain changes. The structure does not.

Discussion Questions

Why is the "measurability asymmetry" -- the fact that the costs of a Chesterton's fence are always more visible than its benefits -- a structural problem rather than a problem that better analysis could solve?
Compare the "incremental erosion" pattern in legal regulation (Glass-Steagall exemptions) to the "dead code removal" pattern in software refactoring. What structural features make both domains vulnerable to gradual fence destruction?
The chapter argues that the "Chesterton comment" in code is a direct solution to the dark knowledge problem. Design an equivalent mechanism for legal regulation. What would a "Chesterton comment" for a law or regulation look like? How would it be maintained over time?
FinCo's modernization team was talented, well-resourced, and followed software engineering best practices -- and they still committed a Chesterton's fence failure. What does this tell us about the limits of best practices as protection against this pattern?
Both Glass-Steagall's repeal and FinCo's code refactoring involved people who were genuinely competent and well-intentioned. How does this complicate the narrative that Chesterton's fence failures are caused by ignorance or recklessness?