Self-Assessment Quiz: On-Call for Data

Twenty questions. Aim for 16 or more. Questions 4, 9, and 15 are the three that decide whether a rotation is survivable.

Question 1

The most important difference between data on-call and service on-call is:

• A. Data teams are smaller
• B. The failure is usually not urgent, but the decision is
• C. Data pipelines fail more often
• D. There are no users

Question 2

"How many users were affected" has no data analogue. The right question is:

• A. How many rows
• B. Which decisions were made on the wrong number
• C. How many tables
• D. How long it took

Question 3

Kestrel pages overnight for exactly two things:

• A. Sev-1 and Sev-2
• B. A source whose retention is shorter than the wait, and a privacy event
• C. Any failed DAG
• D. Nothing

Question 4

The reasoning for that is:

• A. The team is lazy
• B. With a 06:00 deadline, a failure at 01:00 and one at 04:45 have identical consequences
• C. Alerts are unreliable overnight
• D. The scheduler is paused at night

Question 5

An SLO is:

• A. An external commitment with consequences
• B. Your internal target
• C. The measurement itself
• D. A contract with a vendor

Question 6

Correctness is scored:

• A. At the time of the incident's discovery
• B. Retrospectively — a day is correct until something proves otherwise
• C. Only for Sev-1s
• D. Monthly

Question 7

An SLO of 99.9% over 90 days gives:

• A. 9 days of budget
• B. 0.09 days — a single miss blows the quarter
• C. 0.9 days
• D. Unlimited budget

Question 8

Why is that worse than useless?

• A. It is too easy
• B. Once blown, one failure and nine are identical, so the rest of the window carries no signal
• C. It cannot be computed
• D. It requires more monitoring

Question 9

Burn rate must be computed over a short recent window because:

• A. It is faster
• B. Over the full window it collapses to spent/budget and cannot exceed 1 without the budget already being blown
• C. Short windows are more accurate
• D. The SLO changes weekly

Question 10

A job that crashed and produced nothing is:

• A. Sev-1
• B. Sev-3 — the SLA is not yet at risk
• C. Sev-2
• D. Not an incident

Question 11

A job that succeeded and published wrong numbers into a dashboard someone is reading is:

• A. Sev-3
• B. Sev-1
• C. Sev-4
• D. A data quality ticket

Question 12

The two things a runbook most often lacks are:

• A. Commands and links
• B. How to tell whether this is actually the problem, and what to do if it is not
• C. An owner and a date
• D. Diagrams

Question 13

Before any recovery that re-reads a source you must check:

• A. Disk space
• B. That the source still has the data — and time travel, and backups
• C. The scheduler is up
• D. Who is on call

Question 14

The general rule about retention is that it must cover:

• A. The time it takes you to notice
• B. The age of the data you might need to reconstruct
• C. One quarter
• D. The SLA window

Question 15

Communication comes before recovery because:

• A. It is quicker
• B. The recovery may take longer than the deadline, and "we do not know yet" is a complete message
• C. It is required by policy
• D. It reduces the incident's severity

Question 16

Declaring a Sev-1 is unilateral and resolving one is not, because:

• A. Resolution requires a manager
• B. The pressure at 05:50 is one-directional — toward "it is probably fine"
• C. Declarations are cheap
• D. Two people must sign off on all changes

Question 17

"Stale over wrong" should be:

• A. Judged by the on-call engineer each time
• B. Pre-decided, by the business partner, with their name on it
• C. Decided by the incident commander
• D. Never applied

Question 18

The postmortem's required data-impact section must accept which answer?

• A. Only a repair command
• B. "None, because ..." — written, not omitted
• C. Only a row count
• D. It cannot be left blank under any circumstances

Question 19

In Case Study 2's drill, the proportion of blocking steps that were writing problems was:

• A. All of them
• B. 19 of 35 — the rest were drift and missing access
• C. None
• D. 5 of 35

Question 20

The drill's author must be present and silent because:

• A. It is more efficient
• B. An author who answers converts a blocking step into a two-second conversation that does not feel like a defect
• C. It saves the author's time
• D. Silence is a management technique

Answer Key

1. B — §26.1. The incident has a deadline rather than a bleeding rate.

2. B — §26.1. Usually unanswerable at 03:00 and sometimes unanswerable ever.

3. B — §26.1's 📐 callout.

4. B — §26.1. Alerting between 22:00 and 05:00 buys nothing and costs a night. And they wrote down what it costs: a 23:00 catastrophe is found at 05:00.

5. B — §26.2. The SLI is the measurement; the SLA is external.

6. B — §26.2. Which makes it the only SLI that measures the failures in this book, and Case Study 1 is the consequence.

7. B — §26.2's ⚠️ callout.

8. B — §26.2. The remaining eighty-nine days carry no signal and the number stops being consulted.

9. B — slo.py's comment, and the bug its self-check found. Burn rate is the recent failure rate divided by the rate the SLO permits.

10. B — §26.4. It produced nothing, so nothing wrong was published.

11. B — §26.4. Sev-1 is about publication, not failure — Chapter 23 §23.1's argument encoded in a severity scale.

12. B — §26.5. A runbook that assumes a correct diagnosis will be applied to the wrong incident.

13. B — §26.5's ⚠️ callout. Three windows expiring at different speeds.

14. B — §26.5. Those are different numbers and the second is usually much smaller, which is why the mistake is easy. Chapter 24 Case Study 1 needed 141 days.

15. B — §26.6's 🏭 callout. The stakeholder's need is knowing whether to trust the number in front of them.

16. B — §26.8.

17. B — §26.8's 📐 callout. The engineer is then executing a decision the business already made, and can say so in the channel.

18. B — §26.9. "If the answer is none, say why" is the part that does the work — it converts an omission into a claim.

19. B — Case Study 2's 🔎 callout. Eleven were drift and five were missing access, and no amount of careful authorship prevents either.

20. B — Case Study 2's 📐 callout. Run both ways, the same two runbooks produced four questions and "mostly fine" with the author answering, and nine blocking steps in silence.

Topic map