Exercises: Data Quality

Several of these ask you to audit something you own rather than something in this book. That is deliberate — the chapter's central claim is that auditing against a fixed list of what should exist beats counting what does, and the only way to feel that is to run the fixed list against your own work.

code/coverage.py scores a dbt manifest against the six assertions and the register, with no warehouse required.

Difficulty: ⭐ warm-up · ⭐⭐ standard · ⭐⭐⭐ deeper. Solutions: daggered (†) and odd-numbered problems are in appendices/answers-to-selected.md.

Part A — Warm-ups ⭐

23.1 † Make the "bad data is worse than no data" argument in terms of who bears the cost and when they discover it. Then state it in currency for a system you know.

23.2 Name the six quality dimensions. Which one cannot be tested, and what are the three substitutes?

23.3 † State the difference between testing the pipeline and testing the data. Give the one question that tells you which a given check is.

23.4 List the six assertions that earn their keep, and the incident each one would have caught.

23.5 † Write the grain test for a table you own, in nine lines of dbt YAML.

23.6 Why does a relationships test pass when unmatched keys resolve to the unknown member? What assertion is needed alongside it?

23.7 † Name three things a dbt test cannot do. For each, say what covers it instead.

23.8 Why is a profiler-generated expectation suite dangerous? Give the question to apply to each generated expectation.

23.9 † Give the four obligations a quarantine table creates. Which one turns a recoverable problem into a permanent one?

23.10 Why does a muted check not count as coverage?

Part B — Standard ⭐⭐

23.11 Run coverage.py against a dbt project you own. Report the number of empty boxes. Then fill the cheapest one and report what it took.

23.12 † Take the five incidents in §23.1's table. For each, write the single assertion that would have caught it, and estimate how long it would have taken to write before the incident, when nobody knew it was needed.

23.13 Write all six assertions for one mart. For each threshold, state the normal value, the threshold, and the broken value, and defend the gap between them.

23.14 † Find a check in your systems that measures the pipeline while appearing to measure the data. Apply §23.3's test: can it fail while the pipeline is completely healthy? Report what you found.

23.15 Set up Great Expectations against a CSV before it lands. Run its profiler, then delete every expectation that cannot fail and report how many of the generated ones survived.

23.16 † Audit your mutes. Count them, get their ages, and find any without an owner or a reason. Report the oldest. If your alerting tool cannot answer this, that is the finding.

23.17 Find every place in your systems where rows go to be forgotten — quarantine tables, dead-letter queues, _rejects/ prefixes, held statuses. For each, answer §23.9's four questions.

23.18 † Take one reconciliation you run and determine what it does not cover. Case Study 1's payment-processor check could not see invoiced wholesale revenue. What is yours blind to?

Part C — Deeper ⭐⭐⭐

23.19 §23.8 argues anomaly detection cannot see a defect that predates its baseline. Design a detection strategy for exactly that class — a persistent, unchanging error — and be honest about what it costs and what it still misses.

23.20 † Case Study 2 lists five failures across five chapters that all pass an artifact-counting audit: a test that cannot fail, a threshold that cannot fire, an alert to a dead channel, a check never invoked, a quarantine never drained. Design one audit that catches all five. State what it costs and what still gets past it.

23.21 §23.6 argues for testing early and then argues for weighting toward the mart. Resolve it for a specific system: given a fixed budget of twelve tests, where do yours go, and what is your reasoning?

23.22 † §23.10 claims a test's compute cost is negligible and its real cost is attention. Measure the first for a project you own — actual credits or dollars for the test portion of a build — and then estimate the second honestly. Does the claim hold?

Part D — The Kestrel Platform ⭐⭐⭐

23.23 — Increment 23: the quality layer.

(a) Run coverage.py against the project as it stands after Chapter 22 and report the number of empty boxes. It will not be zero, and that gap is the honest starting point for everything below.

(b) Fill every box for all four marts. Each threshold carries a comment stating the normal value, the threshold, and the broken value.

(c) Write platform/quality/mute_audit.py, run it weekly, and make it post a message rather than render a dashboard.

(d) Write platform/quality/reconcile_daily.py publishing the variance as a time series, not a pass/fail — and document what it does not cover.

(e) Add a Great Expectations suite validating the supplier CSV before it lands, which is the one place dbt cannot reach and where Chapter 22's Case Study 2 happened.

(f) For every quarantine, dead-letter queue, and reject prefix: a monitored count, a named owner, an idempotent replay path with --dry-run, and a retention decision. Or delete it and say so.

23.24 † Extend coverage.py so a muted check does not count toward coverage, and so the six-box matrix fails CI on any mart below six. Then run it and report how many marts fail on the first run.

The muted-check part is the exercise. It requires your coverage tool to know about your alerting tool's state, which most platforms cannot do — and finding out that they cannot is the finding.

Reflection

A. §23.13's register has twenty-two assertions, every one from a failure that happened, and none of them is difficult. Why do you think none of them existed before its incident? Is "we didn't know" the whole answer?

B. Both case studies in this chapter are about a control that existed and did not operate — a quarantine nobody drained, a check nobody heard. Chapters 19 through 22 each contain at least one more. Count them. What does the total suggest about where to spend the next hour: writing a new control, or verifying an existing one?