Exercises: Data Architecture Principles

Architecture exercises are mostly arithmetic and argument. Several of these ask you to defend a position you may not hold, which is the point — the fastest way to find a weak argument is to build the strongest version of its opposite.

Difficulty: ⭐ warm-up · ⭐⭐ standard · ⭐⭐⭐ deeper. Solutions: daggered (†) and odd-numbered problems are in appendices/answers-to-selected.md.

Part A — Warm-ups ⭐

3.1 † State the chapter's definition of an architectural decision in one sentence. Then rank these five by reversal cost, highest first: compression codec, dimensional grain, cloud provider, orchestrator, partitioning scheme on the largest table.

3.2 What is the question that actually decides batch versus streaming? Give the three clusters of answer and what each implies you should build.

3.3 † Why did ELT displace ETL? Answer in one sentence that mentions no tool names.

3.4 Name the three cases where ETL is still correct. For each, give a one-line example.

3.5 † Fill in the guarantee for each: does a plain data lake provide (a) schema enforcement, (b) ACID transactions, (c) row-level deletes, (d) time travel? Now answer the same four for a lakehouse.

3.6 State Little's Law and define each term. Then compute $L$ for an arrival rate of 500/sec and a service time of 25 ms.

3.7 † What are the three sections of an ADR that people most often omit, and which of the three does the chapter call the most valuable?

Part B — Standard ⭐⭐

3.8 Redo the §3.5 sizing arithmetic with different assumptions. The nightly job needs 12 nodes at average and 60 at peak, runs 2.0 hours, and there are 14 peak days a year. At the frozen $2.400/node-hour, compute the annual cost of fixed-at-peak, fixed-at-average, and elastic. Express elastic as a percentage of fixed-at-peak, and compare that percentage to the chapter's 19.0%. Why did it move?

3.9 † A stakeholder asks for "real-time inventory levels on the ops dashboard so we can react to stockouts." Apply the §3.2 test. Write the three follow-up questions you would ask, predict the most likely answers, and state what you would build. Then write the two-sentence response you would send if the answer turns out to be group three.

3.10 Kestrel's clickstream consumer has p50 handling time of 8 ms and p99 of 310 ms. At the 2,900/sec peak: (a) Compute the required concurrency using p50 and using p99. (b) The topic has 12 partitions. How many handlers per partition does each answer imply? (c) Which would you provision for, and what would you monitor to know you got it wrong?

3.11 † Write ADR-003 for Kestrel: whether to retain bronze clickstream for 2 years or 90 days. Use the template in §3.7. Your "what would reverse this" section must contain at least two conditions you could write a monitoring query for. Use the frozen S3 rates.

3.12 The 🏭 callout in §3.6 describes a reporting job reading an intermediate table. Design two independent controls that would have prevented or quickly detected it. For each, state what it costs to implement and what it costs to maintain.

3.13 † The chapter says a hybrid architecture — lakehouse for bronze/silver, warehouse for gold — "is not a compromise." Argue that it is a compromise, as strongly as you can. Then say which specific Kestrel property makes your argument fail, and what would have to change for it to succeed.

Part C — Deeper ⭐⭐⭐

3.14 The chapter claims elasticity is "the entire economic argument for cloud infrastructure in a business with a peak." Construct the case for Kestrel buying its own hardware instead. Include: the peak-sizing cost, an estimate of what the same capacity costs to own over three years, and the non-cost factors. Then state the peak-to-average ratio below which owning wins.

3.15 † §3.4's ⚠️ callout describes two Spark jobs writing the same S3 prefix concurrently. Without using a table format, design a protocol that makes this safe. Then explain why every such protocol converges on reimplementing part of a transaction log, and identify which part is hardest.

3.16 ELT's premise fails at very large volume (📏 in §3.3). Find the crossover for Kestrel: at what daily event volume does the annual S3 cost of two-year bronze retention exceed $50,000? Show the arithmetic. Then argue why that dollar threshold is the wrong way to make the decision.

3.17 † Take an architecture you have seen — at work, in a blog post, in a conference talk. Write its implicit ADR: what was decided, what the alternatives probably were, and what would reverse it. Then identify the one decision in it that was most likely copied from a company with a different scale, and say what it costs.

Part D — The Kestrel Platform ⭐⭐⭐

3.18 — Increment 3: the decision records.

Create platform/docs/adr/ and write two ADRs using the §3.7 template. Number them adr-001-*.md and adr-002-*.md.

ADR-001 — the storage architecture. Work it through yourself from the Chapter 1 §1.5 numbers. Consider at least three alternatives. If you conclude warehouse-only, defend it — that is a legitimate answer at Kestrel's size and the chapter's answer is not the only correct one.

ADR-002 — batch versus streaming for the clickstream. Apply the §3.2 test explicitly. Name what acts on the data. Compute the cost of both options using the frozen basis. State the latency you are committing to.

For both: "What would reverse this" must contain at least two conditions you could write a monitoring query for. "If requirements change" is not one of them. "If bronze exceeds 8 TB" is.

3.19 † Add platform/docs/adr/README.md explaining the ADR process for a new team member: when to write one, when not to, who reviews, and what happens at the review date. Keep it under 300 words — a process document nobody reads is not a process.

Reflection

A. The chapter argues you should optimize for reversibility rather than correctness under uncertainty. Where in your own work have you done the opposite — committed hard to a decision you were not confident in? What did it cost, and what would the reversible version have looked like?

B. Of the six principles in §3.6, which is weakest in a system you know well? What is the visible symptom, and would anyone currently attribute it to that principle?