Further Reading: Data Quality

Sources are tagged Tier 1 (confident it exists, recommended without reservation) or Tier 2 (real and worth seeking, but confirm the current edition, version, or URL yourself).

A warning about this literature specifically. Data quality is the topic in this book with the widest gap between its writing and its practice, because a large fraction of it is produced by companies selling observability platforms. That does not make it wrong — some of the best practitioner writing on this comes from vendors — but the shape of the argument is reliably the same: your problem is coverage, and coverage is what we sell. §23.8's assessment of that claim is the honest one.

Books

• Barr Moses, Lior Gavish, and Molly Vorwerck, Data Quality Fundamentals (O'Reilly, 2022). The standard reference, and the source of "data downtime" as a metric (§23.11). It is written by the founders of an observability company and is upfront about that. Read it for the vocabulary and the organizational material — the chapters on ownership and on incident response transfer directly and are better than anything else available. Discount the sections arguing that automated anomaly detection is the primary control, for §23.8's reasons. Tier 1.

• Laura Sebastian-Coleman, Measuring Data Quality for Ongoing Improvement (Morgan Kaufmann, 2013). Predates the modern tooling entirely and is better on the part the tooling does not touch: what a quality dimension actually means, and how to measure it in a way that survives being reported to someone. The treatment of the six dimensions is the careful version of §23.2, and it is honest about accuracy in the same way. Tier 1.

• Ralph Kimball and Margy Ross, The Data Warehouse Toolkit (3rd ed., Wiley, 2013), on error event schemas. Kimball's proposal — model your quality failures as a fact table, with a dimension for the rule and one for the source — is thirty years old and still the best answer to "where do quarantined rows go." It is Case Study 1's four obligations, in a schema. Tier 1.

• Google's Site Reliability Engineering (O'Reilly, 2016), on monitoring and alerting. The chapter "Monitoring Distributed Systems" contains the symptom-versus-cause distinction that §23.6's placement argument rests on, and the discussion of alert fatigue is where §23.7's material comes from. Free online, and the single best thing on this reading list about thresholds. Tier 1.

Tools

• The Great Expectations documentation. The concepts pages first — Expectations, Suites, Checkpoints, Data Docs — because the vocabulary is specific and the tool is hard to evaluate without it. Read the profiler documentation and then read §23.5's warning again; the docs present profiling as the on-ramp and it is also the fastest route to four hundred tests that cannot fail. Tier 2 — the API has changed substantially between major versions; check which you are reading.

• The dbt_expectations package. The bridge between the two worlds: Great Expectations' vocabulary as dbt tests. For most teams this is the right answer, and knowing it exists prevents a premature adoption of a second tool. Tier 2 — a community package; check maintenance status.

• The dbt documentation on tests, severity, error_if/warn_if, and --store-failures. The last one is what makes a failing test investigable rather than merely loud, and almost nobody turns it on. Tier 1.

• Soda, Monte Carlo, Elementary, and the observability category generally. Worth an evaluation once you have the six assertions in place and your problem has genuinely become breadth. Not before — §23.8's positioning is that these are a net for the unknown, and a team that buys one instead of writing the six has bought coverage of everything except the failures this book documents. Tier 2 — an active commercial category; the names and capabilities move.

On the specific arguments

• Anything careful on alert fatigue, from operations rather than from data. The literature is mature and it transfers unchanged: the mechanism by which a suite of correct alerts stops being read does not care whether the alerts are about disks or about dimensions. Case Study 2's 511-day mute is a documented pattern in that literature under other names. Tier 2 — scattered across SRE and clinical-alarm research, and the clinical work is genuinely worth reading.

• The "error budget" material in SRE. §23.1's argument that a reconciliation tolerance is a budget for permanent error is the error-budget idea applied to correctness rather than availability, and the original framing is more careful about the trade-off than this chapter has room to be. Tier 1.

• Postmortem practice — Allspaw, Dekker, and the resilience-engineering literature. Both case studies here turn on the difference between what a person did and what the system permitted: a correct mute at 04:12, a correct quarantine after a correct postmortem. The framing that treats both as system properties is what makes the fixes structural — a capped mute duration rather than a reminder, four obligations rather than good intentions. Tier 1 for Dekker's Field Guide; Tier 2 for the blog and conference material.

Practice

• code/coverage.py in this chapter. Scores a dbt manifest against the six assertions and lists the twenty-two-item register, marking which are detectable from a manifest and which need a person — seven of twenty-two, which is itself the finding. Run it against a project you own and count the empty boxes.

• Audit your mutes today. Exercise 23.16. If your alerting tool cannot tell you how many suppressions exist and how old they are, that is the answer to the exercise, and Case Study 2 is what it costs.

• Walk the register against one mart. Twenty-two items, an hour, and the output is a list rather than a feeling. Do it for the mart the CEO looks at, which is the one where §23.1's argument applies most directly.

A note on what to be skeptical of

Any claim that a tool will "automatically detect data quality issues." It will detect changes, which is a different and narrower thing, and §23.8's two limits are structural rather than implementation defects: a defect present before the baseline is invisible, and the sensitivity that survives Black Friday will not catch 5%.

And the framing that data quality is a coverage problem. Sometimes it is — a warehouse of eight hundred tables nobody has inventoried genuinely has a coverage problem. More often it is a specificity problem: the six assertions in §23.4 are missing from the four tables that matter, and no amount of breadth substitutes for them.