Case Study 2: When the Data Was Wrong — How a Pre-Fill Failure Industrialized a Mispricing
A note on sourcing. Unlike Case Study 1, this case is presented as an explicit, clearly-labeled composite — a teaching reconstruction built from real, recurring industry patterns rather than a single named event. Data-quality failures of exactly this shape (wrong-match pre-fill, silent defaults, stale peril codes feeding automated pricing) are well known inside carriers and well documented in industry commentary; named, public, figure-by-figure accounts are scarce because carriers rarely publish their data failures. The composite lets us teach the failure honestly. Every figure and entity is illustrative; nothing here reports a real carrier's results.
Background: a fast, modern, automated book
Picture a regional carrier — call it Meridian Mutual (a composite; not the Harbor Steel broker) — that did everything this chapter celebrates. It modernized its small-commercial underwriting: a producer entered a business name and address, the system pre-filled the building characteristics, the occupancy class, the peril codes, and a prior-loss indicator from third-party feeds, a model scored the risk in real time, and for clean, simple risks the policy bound straight through with no human in the loop. The pipeline was fast, the producers loved it, new-business volume climbed, and the expense ratio fell — fewer underwriter hours per policy. By every metric the carrier watched in the first year, the initiative was a success.
The metric it was not watching closely enough was the quality of the pre-filled data — specifically, on one class of risk, in one set of territories, where a quiet flaw in the data pipeline was mispricing risk at machine speed, and would not show up in the numbers the carrier watched until the losses arrived.
The underwriting issue: three small data failures, compounded
No single dramatic error caused the problem. As is almost always the case with data quality, it was several small, individually plausible failures, compounding — exactly the dimensions §31.7 names.
A wrong-match problem. For a subset of addresses, the geocoding step resolved the property to the wrong parcel — often a neighboring or similarly-named one. The pre-fill then populated the file with confident, internally consistent facts about the wrong building: its construction, its square footage, its distance to hazards. Nothing looked wrong; every field was filled.
A stale-peril problem. The flood- and wind-hazard codes came from a peril database on an update cycle that lagged reality. In a handful of territories where the hazard maps had recently been revised upward, the system was still pricing on the old, lower hazard codes. Again, nothing flagged — the field was populated, the value was a real value, it was simply out of date.
A silent-default problem. When a price-driving field could not be resolved — a missing construction type, an unmatched occupancy — the pipeline did not stop and refer. It filled the blank with a favorable class default (a benign construction, a low-hazard occupancy) and let the risk bind. The submission looked complete; the model was confident; the policy bound. The carrier had priced a meaningful number of risks on fields nobody actually knew.
HOW THREE SMALL DATA FAILURES BECAME ONE BIG MISPRICING [constructed illustration]
wrong-match geocode ─┐
stale peril codes ──┼──► pre-fill produces a CONFIDENT, COMPLETE-LOOKING file
silent defaults ────┘ that is quietly WRONG on price-driving fields
│
▼
real-time model PRICES the wrong picture
│ (no human friction in the loop)
▼
STP BINDS it — fast, cheap, and mispriced
│
▼
the SAME error repeats across a whole class, at scale,
for two years — until the LOSSES finally arrive
Each failure alone might have produced a handful of mispriced policies. Wired together into an automated pipeline with no human checking the price-driving fields, they mispriced an entire slice of the book — quietly, confidently, and at a speed no manual process could ever have matched. This is the chapter's darkest warning made concrete: automation did not create the bad data; it removed the friction that used to catch it, and then acted on it faster and at scale. In the old fax-and-folder world, a human underwriter who knew the territory would have caught the stale wind code or the wrong-parcel construction. In the automated pipeline, there was no such human, and the error became the carrier's price.
What it shows
Garbage in, garbage out is the binding constraint on data-driven underwriting. Every benefit Meridian captured — speed, completeness, lower expense ratio — was real, and every one of them was built on the assumption that the data was good. When the assumption failed on one class, the whole apparatus did not warn or slow; it efficiently turned bad data into bad prices and bound them.
The expense ratio lied, and the combined ratio told the truth — late. This is §31.6's warning in practice. The metric that improved first (expenses) was the one with no lag; the metric that mattered (the loss ratio, and so the combined ratio) had a two-to-three-year tail. The carrier "succeeded" in year one and discovered the mispricing only when the losses on the underpriced, mis-territoried risks finally developed. "We automated and cut costs" was true and beside the point; the combined ratio rendered the real verdict on a delay.
The silent default was the most dangerous failure, exactly as the chapter says. A visibly missing field would have flagged; the pipeline could have referred it to a human. The favorable default was worse precisely because it was invisible — it produced a complete-looking, confident submission that hid its own ignorance. A blank that looks blank is a manageable problem. A blank that looks like a fact is a trap.
Outcome
In the composite, the failure surfaced the way such failures usually do: an adverse loss-ratio trend on a specific class in specific territories, traced — only after a real investigation — back to the data pipeline rather than to the underwriting rules or the market. The remediation is the chapter's §31.7 control list, imposed after the fact: make missingness visible and never silently default a price-driving field; require corroboration (a second independent source) on the fields that move the price; set confidence thresholds that refer thin or conflicting data to a human; and audit the data feed itself — the match rate, the currency of the peril codes, the default behavior — not merely the decisions it produced. The book of business was re-priced and partly re-underwritten; the expense savings of year one were, on that class, substantially given back in the losses of years two and three.
The deeper governance lesson outlived the fix. Meridian had outsourced the data but kept the risk: the geocoder's match rate, the peril vendor's update cycle, and the pipeline's default logic were now, in effect, underwriting decisions — made by software and vendors, but owned, in losses and in front of the regulator, by the carrier. A data-driven carrier does not get to treat its data feed as someone else's problem.
Lesson
Data-driven underwriting fails in a characteristic way: not with one dramatic wrong number a human would catch, but with several small, plausible, invisible errors — a wrong match, a stale code, a silent default — compounded and then industrialized by an automated pipeline that has removed the human friction that used to catch them. The damage shows up first as an expense-ratio win and only later as a combined-ratio reckoning, on the two-to-three-year delay that has fooled many a quarterly-minded executive. The discipline that prevents it is the same discipline a careful underwriter has always had — verify what matters, corroborate from independent sources, distrust a confident number, and make ignorance visible rather than letting it hide as a fact — scaled up from the desk to the data pipeline. Automation raises the stakes of that discipline; it does not relieve you of it.
Discussion questions
- The case identifies three failures — wrong match, stale peril codes, silent default — and calls the third the most dangerous. Defend or challenge that ranking using §31.7's data-quality dimensions. Why is an invisible error worse than a visible one?
- Meridian's expense ratio improved in year one; its combined ratio worsened later. Explain the time lag, and design the leading indicators a carrier should watch so it does not have to wait two years for the loss-ratio verdict (recall §31.6 and Chapter 3).
- "Automation did not create the bad data; it removed the friction that used to catch it." Describe exactly what the "human friction" was in the old workflow, and propose where to reinsert a targeted version of it into the automated pipeline without giving up automation's benefits.
- The carrier "outsourced the data but kept the risk." What does this imply for vendor management — the contracts, the audits, the monitoring of a feed's match rate and currency — that a data-driven carrier must run? Why can't this be the vendor's responsibility alone?
- Contrast this case with Case Study 1. Imagery (Case 1) was a data success; this pipeline was a data failure. What single discipline most separates the two outcomes — and where, in the Harbor Steel file, do you see that discipline being applied (the corroboration of the satellite roof read against the loss runs and the broker's note)?