Exercises: Schema Evolution and Data Contracts

Part D asks you to write a real contract for a real source. The status: observed variant in 17.23 is the one most readers can actually deploy, because it requires nobody's agreement.

Difficulty: ⭐ warm-up · ⭐⭐ standard · ⭐⭐⭐ deeper. Solutions: daggered (†) and odd-numbered problems are in appendices/answers-to-selected.md.

Part A — Warm-ups ⭐

17.1 † State the chapter's central argument in one sentence, in terms of where a failure lands.

17.2 Name three things a data contract is not.

17.3 † Define backward, forward, and full compatibility. For each, say who can be upgraded first.

17.4 Classify under backward compatibility: add an optional field · remove an optional field · widen int to long · rename a field · reorder fields.

17.5 † Why is adding an enum value "compatible on paper and frequently breaking in practice"?

17.6 What is a registry's single most valuable behavior, and at what moment does it happen?

17.7 † Name the seven things a contract contains beyond the schema. Which two do most of the work, and why each?

17.8 Name the four enforcement points. Which is the only one available for a source you do not own? Which is the only one that catches a compliant producer that has stopped?

17.9 † Describe expand-contract. Which of its three steps takes the time, and what makes step 3 possible?

17.10 Give the four social failure modes and the fix for each.

Part B — Standard ⭐⭐

17.11 For each Part III incident, say which enforcement point (§17.5) would have caught it, or state that none would: Chapter 2's discount_cents sign inversion · Chapter 13's postal_code type change · Chapter 14's REPLICA IDENTITY misconfiguration · Chapter 15's carrier field becoming an object · Chapter 16's API rate limit. At least one has no enforcement point — say which and why.

17.12 † Take §17.4's contract and write the equivalent for Kestrel's clickstream topic. The semantics section must resolve at least four things a schema cannot express, and the guarantees section must use real numbers from Chapter 1 §1.5.

17.13 Design the CI check that fails when a producer changes a schema without updating its contract (§17.7, fix 1). Specify what it compares, where it runs, and how it handles the case where the contract legitimately needs no change.

17.14 † §17.9's second residual was an added enum value. Write the assertion for every categorical column in Kestrel's silver.orders whose values a contract enumerates. Then generalize: write the generator that produces these assertions from a contract file, so a new enumerated field gets a check automatically.

17.15 A producer needs to rename total_amount to total_cents — a rename and a unit change, which is the worst combination. Write the full expand-contract plan with dates, including what the contract says at each stage and how you know step 3 is safe.

17.16 † §17.7's 🏭 callout describes an eight-month violation log persuading a reluctant producer. Design that log: what a violation record contains, how it is captured automatically, and what the summary you would bring to the conversation looks like.

17.17 Choose a compatibility mode for each and justify: an internal event topic with three consumers you control · a topic consumed by a mobile app whose users upgrade on their own schedule · a CDC topic replayed from the beginning for backfills · an internal metrics topic with one consumer.

Part C — Deeper ⭐⭐⭐

17.18 §17.9's first residual — a semantic change with no schema change — is stated as a gap this book has no good automated answer for. Try. Design a detection mechanism for "a field's meaning changed while its type did not," and be honest about its false-positive rate.

17.19 † §17.10 lists four cases where a contract is not worth it, and explicitly excludes "the producer will not agree." Construct the strongest case that a contract nobody has agreed to is worse than none — then say what in §17.8 answers it.

17.20 The registry is "a runtime dependency of every consumer" and a cold-cache restart during a registry outage is a real failure. Design the mitigation. What do you cache, where, for how long, and what does the consumer do when it meets an id it cannot resolve?

17.21 † Compare consumer-driven contract testing (Pact and similar) to the producer-enforced registry model in this chapter. What does each assume about who owns the relationship, and which suits a data platform better? Cite the sources.

Part D — The Kestrel Platform ⭐⭐⭐

17.22 — Increment 17: the orders contract.

(a) Write platform/contracts/orders.v2.yml in full, following §17.4's seven sections. Every field in the semantics section must say something a schema cannot.

(b) Add a schema registry to docker-compose.yml. Register the orders schema with BACKWARD_TRANSITIVE compatibility.

(c) Prove the registry rejects. Attempt to register an incompatible schema — remove a required field — and record the exact error message. Then attempt a compatible one and confirm it is accepted.

(d) Add the CI check from §17.5 point 1 to your repository.

(e) Add the consumer-side assertions from §17.5 point 3 and §17.9: required fields present, types unchanged, and the enum assertion, generated from the contract file rather than hand-written.

(f) Add the scheduled guarantee checks from §17.5 point 4: freshness and volume against the contract's declared numbers.

17.23 † Write platform/contracts/carrier-b.tracking.yml with status: observed, for a source you do not control — the Kestrel carrier API, or a real public API you have access to.

Fill the observed_guarantees section with measured numbers, not guesses. Then wire the consumer-side validation and let it run for a fortnight.

This is the exercise most readers can actually deploy at work, because it requires nobody's agreement.

Reflection

A. §17.7 says a contract framed as "you may not change your system without asking us" will be resisted and should be. Have you been on the receiving end of a control framed that way? What would have made it land differently?

B. The chapter recommends writing an observed contract without agreement, and gathering evidence. Is that collaborative or adversarial? Does the answer change if the producer is internal?