Exercises: CI/CD for Data Pipelines

Several of these ask you to break your own pipeline deliberately. That is the point — both of this chapter's case studies are about a control that was correct and did not operate, and the only way to know which yours are is to make them fail on purpose while you are watching.

code/pr_report.py reads dbt manifests and needs no dbt, no warehouse, and no network.

Difficulty: ⭐ warm-up · ⭐⭐ standard · ⭐⭐⭐ deeper. Solutions: daggered (†) and odd-numbered problems are in appendices/answers-to-selected.md.

Part A — Warm-ups ⭐

27.1 † Give the four ways data CI/CD is harder than application CI/CD, and the compromise each forces. How many of the four resolve to "accept something"?

27.2 Order the CI stages by yield. Which four need no warehouse?

27.3 † Name the three sources of test data and what each cannot exercise.

27.4 A 1% random sample of Kestrel's orders — what is the probability it contains at least one of the twelve wholesale distributors? Show the arithmetic and say why it matters.

27.5 † What three things does slim CI require operationally? Which is a decision rather than a task?

27.6 Name the four things state:modified+ cannot see.

27.7 † Give the three deploy shapes and the data operation each implies. What is the failure of treating a definitional change as additive?

27.8 What can and cannot be rolled back? Which category has no rollback at all, and what follows for DAG structure?

27.9 † Why is a shadow deploy's gate not "zero rows"?

27.10 What should a reviewer look at, and what should be delegated to a linter? State the rule.

Part B — Standard ⭐⭐

27.11 Time your own CI pipeline stage by stage. Report the table from §27.2's 🔎 callout for your own repository: what each stage catches, and what it costs per catch.

27.12 † Build §27.3's deliberate sample for a table you own. Start with clause 3 — every row that has ever failed a test — and report how many rows that is.

27.13 Run pr_report.py --demo, then point it at two real manifests from your own project. Report the deploy shape it computes and whether you agree.

27.14 † Reproduce Case Study 2. Point a state:modified+ build at a missing manifest and confirm it exits 0 having built nothing. Then add the four-line assertion and confirm it fails.

27.15 Take a definitional change you have merged in the past year. Determine whether history was rebuilt. If not, find the boundary date — and time how long that takes you.

27.16 † Add _built_by and _built_at to one model. Measure the storage cost after a week, both uncompressed and in Parquet, and report both.

27.17 Write the deploy-shape section into your pull-request template. Apply it retroactively to the last ten merged changes and report the distribution.

27.18 † Audit your CI for controls that fail open. For each check, ask §"The Analysis"'s question: would its failure be distinguishable from having nothing to check? Report how many fail the test.

Part C — Deeper ⭐⭐⭐

27.19 §27.5 says the industry replaced staging with a permission grant. Price both for a system you know: what a representative staging environment would cost, against the governance cost of CI reading production. Which would you choose, and what would change your mind?

27.20 † Case Study 2 identifies "do not X; instead Y" as two requirements of which only the negative gets implemented. Find three requirements of that shape in your own systems — in code, in process, or in documentation — and check which half exists.

27.21 §27.10 claims a reviewer whose attention is spent on mechanics has none left for meaning. Test it: take your last twenty review comments and classify each as mechanical or substantive. Report the ratio, and estimate what moving the mechanical ones into CI would change.

27.22 † Design the shadow-deploy gate for a definitional change: it must pass when the differences are the intended ones and fail when they are not. Be specific about how "intended" is expressed, because that is the whole difficulty.

Part D — The Kestrel Platform ⭐⭐⭐

27.23 — Increment 27: the pipeline.

(a) Six stages ordered by yield, with the first four needing no warehouse. Under twenty seconds to a first failure.

(b) The nightly job uploads manifest.json and verifies it produced one; CI falls back to a full build if it is missing, and asserts it built more than zero models.

(c) --defer configured, with the production read grant documented — masking policy, audit trail, and the reasoning. It is a governance decision.

(d) Build fixtures/pathological.sql from clause 3 first. Take a defect from any case study in this book, add the row that caused it, and confirm the fixture fails against the pre-fix code and passes after. This is the exercise that carries the chapter — a fixture set built that way is the only regression suite that grows in the direction your system actually breaks.

(e) The pull-request template requires the deploy shape, and pr_report.py pre-computes it.

(f) The export task is last in the DAG, after the assertions. OIDC rather than long-lived keys, and --store-failures so CI never prints production rows.

27.24 † Add _built_by and _built_at to every gold model, and write the query that answers "which code produced this row?" Then use it to check whether any of your models currently has more than one definition in it.

Expect at least one. Kestrel's audit found three, one of which had been created while fixing the boundary left by an earlier one.

Reflection

A. Case Study 1 shipped code without data; Case Study 2 ran a check without checking. Both are half of a two-part thing. Enumerate the two-part things in your delivery pipeline — deploy and migrate, alert and route, quarantine and drain, test and assert-it-ran — and mark which halves you have verified. The unverified halves are your next incident.

B. §27.1 argues data CI/CD is a set of trades rather than a set of best practices, and three of the four resolve to "accept something." Write down what your pipeline has accepted. If the list is empty, you have accepted the same things without recording them.