Case Study 1: Two Candidates, One System Design Question

"They gave nearly identical designs. One got an offer at senior and one was declined, and the transcripts are four hundred words apart."

Executive Summary

Two candidates answered the same question — "design a system to compute daily revenue for an e-commerce company" — in the same 45-minute slot, with the same interviewer, a week apart.

Their final architectures were nearly identical: CDC into an object store, a layered warehouse, a nightly batch, a dashboard.

                          candidate A    candidate B
requirements clarified          25/25          6/25
trade-offs stated               18/20          8/20
failure modes                   17/20          4/20
correctness                     14/15          3/15
cost arithmetic                  8/10          0/10
the diagram                      7/10         10/10
                                ─────          ─────
                                   89             31

Candidate B drew the better diagram and scored 31.

This case study is the transcript of the four moments where they diverged, and none of them is about knowledge. Both candidates knew the same things.

Skills applied: the design rubric (§39.3); the six requirements questions; naming the downside of your own choice; and the sentences that could only come from experience (§39.4).

The Question

"Kestrel Supply Co. sells outdoor equipment online. They want daily revenue reported every morning. Design it."

Forty-five minutes. A whiteboard. No further information offered.

That last part is the test. The question is under-specified on purpose, and §39.3's first 25% is entirely about noticing.

Divergence 1: The First Five Minutes

Candidate B, at 0:30:

"Sure. So I'd start with the source — I'm assuming Postgres or MySQL — and get it into an object store..."

Candidate A, at 0:30:

"Before I design it, can I ask a few things? Six, roughly, and they'll change the answer.

How much data — orders per day, and what's the peak? How fresh does 'every morning' need to be, as a time? Who reads it, and what do they do with it? What happens if it's late, and what happens if it's wrong? Does history change — can yesterday's number move? And what already exists?"

The interviewer answered all six, which took four minutes.

🎓 Interview Angle — the question that changed the outcome

Five of the six questions produced useful answers. The fifth changed the design.

"Does history change — can yesterday's number move?"

"Yes. Refunds settle for up to 90 days after the sale."

Candidate B never asked, and therefore designed an append-only pipeline — perfectly reasonable, straightforward, and unable to restate a closed month. The design was wrong from minute five and the error was invisible until the interviewer raised it at minute thirty-eight, by which point it could not be fixed.

Candidate A's design changed in response: "Given the 90-day refund window, fct_order_line has to be incremental with a lookback rather than append-only, and the monthly figure isn't final until 1 March. That's a real property — the dashboard should show an as-of date."

Three things that sentence does at once, which is why the four minutes paid for themselves:

• It names a design consequence, not a fact. The requirement changed something visible.
• It names a downstream consequence the interviewer had not asked about — the as-of date on the dashboard — which is the difference between answering the question and understanding it.
• And it is unfakeable. You do not think about as-of dates on dashboards unless you have watched somebody screenshot a number and compare it to a later screenshot.

The general lesson: the six questions are not politeness and they are not stalling. They are where a quarter of the score lives, and the fifth one is where the design actually gets decided.

Divergence 2: The Justification

Both candidates chose CDC over a nightly full extract.

Candidate B:

"I'd use CDC — Debezium probably — to get changes into the lake."

Candidate A:

"CDC rather than a nightly full extract, because the source is 340 GB and a full extract won't finish inside the window — and because I want update history, not just the current state.

The cost is a replication slot that can fill the source's disk if a consumer stalls, so I'd monitor slot lag and alert on it, and I'd want to know who gets paged when the source database's disk fills, because it isn't going to be my team."

🔎 Read the Plan — naming your own choice's downside is the cheapest high-value move available

Both candidates made the same decision. Only one of them made it a decision.

The scoring difference is in the second paragraph, and it does three things in about fifteen seconds:

It states a trade-off, which is literally the rubric line (20%). "I'd use CDC" is a preference; "CDC because X, at the cost of Y" is an engineering judgment.

The downside is specific and operational. "Replication slots can fill the source's disk" is not a thing you read in a comparison table; it is a thing that happens to you, and interviewers know which is which.

And the last clause is the one that reads as senior: "who gets paged when the source database's disk fills, because it isn't going to be my team." That is an organizational consequence of a technical choice, and noticing those is roughly what "senior" means.

The transferable form is a sentence pattern:

text "<choice>, because <requirement it satisfies>, at the cost of <specific operational downside>, which means <who is affected and what they need to do>."

Use it three or four times in a design round. It costs fifteen seconds each and it is the single highest-yield habit in this chapter.

Divergence 3: Failure Modes

At minute 25 the interviewer asked both candidates the same follow-up: "what can go wrong?"

Candidate B:

"The pipeline could fail — I'd add retries and alerting. The source could be down. I'd monitor it."

Candidate A:

"Four things, and they need different answers.

The source is late. The DAG waits rather than producing a partial answer — there's about an hour of slack before the 6am deadline, and I'd alert on remaining slack rather than on the breach, because a deadline met by five minutes and one met by an hour look the same otherwise.

Duplicate CDC rows.* Dedup on the natural key ordered by updated_at — with a deterministic tie-break*, because updated_at isn't unique and without a tie-break you get a different row on every run and it's invisible.

Late refunds. Covered by the lookback window from earlier.

A partial failure mid-run. Every task idempotent, keyed on the run's logical date, so a rerun produces the same result rather than double-counting."

⚠️ Failure Mode — "I'd add monitoring" is scored as nothing

Candidate B's answer is not wrong. Retries, alerting, and monitoring are all correct things to have.

It scores 4 of 20 because it names categories rather than making decisions, and the interviewer cannot distinguish it from an answer given by somebody who has read a blog post.

The specific tell is that every noun is generic. "Alerting" — on what, at what threshold, to whom? "Retries" — how many, with what backoff, and what happens when they are exhausted? "Monitor it" — which metric?

Candidate A's answer contains four sentences that could not be produced by someone who had not run a pipeline:

text "alert on remaining slack rather than on the breach" Ch 33 CS2 "with a deterministic tie-break" Ch 20, Ch 38 CS2 "keyed on the run's logical date" Ch 24 "because it's invisible" the tell

The last one is the strongest and the shortest. Saying why a failure is hard to detect — rather than just naming it — is what separates having read about a problem from having chased one.

And note the structure: "four things, and they need different answers." Announcing the shape of your answer before giving it is worth doing every time; it tells the interviewer you have a taxonomy rather than a list, and it keeps you from trailing off after two.

Divergence 4: Cost

At minute 38, unprompted, candidate A spent ninety seconds on arithmetic:

"Roughly: 6,600 orders a day at about 2.7 lines each is 18,000 lines. The nightly Spark job is maybe 24 nodes for an hour and a bit, so call it $75; the warehouse transform maybe $18; ingestion around $80. So about $175 a night, $5,250 a month, which is about 2.6 cents per order — and the revenue it's reporting is roughly $15 million a month, so the pipeline costs about 0.035% of what it measures.

That tells me not to optimize it. If someone asks me to cut this cost I'd want to know why."

Candidate B was not asked about cost and did not raise it. Scored 0 of 10.

💸 Cost Check — the arithmetic is worth more than its accuracy

Candidate A's numbers are approximate and one of them is wrong — the interviewer's own ingestion figure was about half of A's $80, which would put the night nearer $135 than $175. It scored 8 of 10 anyway.

Because the rubric line is "reasoned about with numbers," not "computed correctly." What is being tested is whether cost is a dimension you think in at all, and an order-of-magnitude estimate with stated assumptions demonstrates that completely.

The last two sentences are what took it from 5 to 8:

"That tells me not to optimize it. If someone asks me to cut this cost I'd want to know why."

A number with no conclusion attached is a calculation. A number that changes what you would do is an engineering judgment — and "this is 0.035% of the revenue it reports, so it is not where the money is" is exactly Chapter 33's discipline, arriving unprompted.

The practical version: in any design round, spend ninety seconds on arithmetic, out loud, with your assumptions stated, and end with what it implies. Almost nobody does, it is impossible to fake, and it is 10% of the score for a minute and a half.

What Happened

Candidate B drew the better diagram and lost on every other line.

The debrief note the interviewer wrote about B is the useful artifact:

"Knows the components. Could not tell whether they have operated any of them, because nothing in the answer was specific enough to be evidence either way. Would ask a colleague to re-interview at mid level with more probing — my read is that they may well have the experience and did not show it."

"May well have the experience and did not show it" is the most common failure in this round, and it is the one this chapter is trying to prevent. The knowledge gap between A and B was small. The evidence gap was total.

Lessons

1. Nearly identical architectures, 89 against 31. The difference is not knowledge.

2. 🎓 The fifth requirements question — does history change? — decided the design. B never asked and designed an append-only pipeline that could not restate a closed month, invisibly wrong from minute five.

3. A requirement must visibly change something. A's answer named the design consequence, the downstream consequence (an as-of date on the dashboard), and neither is fakeable.

4. 🔎 Name the downside of your own choice. "CDC, because X, at the cost of a replication slot that can fill the source's disk, which means someone outside my team gets paged." Fifteen seconds, and it is the highest-yield habit in the chapter.

5. The organizational consequence of a technical choice is roughly what "senior" means.

6. ⚠️ "I'd add monitoring" scores as nothing, because every noun is generic and it is indistinguishable from having read a blog post. Monitoring what, alerting whom, at what threshold?

7. Four short phrases carried A's failure-mode answer: alert on slack not breach · a deterministic tie-break · keyed on the logical date · "because it's invisible." The last is the strongest — saying why a failure is hard to detect separates reading from chasing.

8. Announce the shape before the list. "Four things, and they need different answers" signals a taxonomy rather than a list, and stops you trailing off after two.

9. 💸 Ninety seconds of arithmetic, out loud, with assumptions stated. A's numbers were approximate and one was wrong; it scored 8 of 10, because the rubric asks whether you reason in cost, not whether you compute correctly.

10. End the arithmetic with what it implies. "So I would not optimize this" turns a calculation into a judgment.

11. "May well have the experience and did not show it" is the most common failure in this round. The knowledge gap was small; the evidence gap was total.

Questions for Discussion

1. Candidate B's answers were all defensible. Is the rubric fair to someone who is competent and terse? What would you change?

2. The six requirements questions took four minutes of forty-five. What would you cut if you had thirty?

3. A's cost arithmetic contained an error and scored 8/10. Would you have corrected yourself mid-answer if you noticed? What does that cost or buy?

4. "Because it's invisible" is called the strongest phrase in the failure-mode answer. Find three more phrases of that kind from Parts IV and V.

5. The interviewer suggested re-interviewing B at mid level. Is that a kindness or a levelling decision? How would you want it handled if you were B?

6. Write your own version of the CDC trade-off sentence for a technology you have actually run — choice, requirement, specific operational downside, who is affected.

7. Candidate A raised cost unprompted. What else is worth raising unprompted, and what is the risk of raising too much?