Self-Assessment Quiz: What Is Data Engineering?

Twenty questions on the shape of the field. Aim for 16 or more before moving to Chapter 2. Work the whole quiz before reading the answer key; a wrong answer is more useful than a right one because it tells you which section to reread.

Question 1

The chapter defines data engineering as moving data from where it is produced to where it is used — with one clause carrying the actual engineering problem. Which clause?

• A. "at scale"
• B. "at a scale and reliability that lets someone downstream treat it as true"
• C. "using pipelines and orchestration"
• D. "from source systems to a warehouse"

Question 2

Which is the most important structural difference between a failing software service and a failing data pipeline?

• A. Data pipelines process more data
• B. Data pipelines run on a schedule instead of on request
• C. A failing data pipeline usually still succeeds — green DAG, populated table, wrong numbers
• D. Data pipelines are written in Python rather than a compiled language

Question 3

Which of these is NOT one of the six artifacts a data engineer produces?

• A. Landed raw data
• B. Tests of the data
• C. Statistical models and experiment designs
• D. Contracts with producers

Question 4

"Landed" data should be:

• A. Cleaned and deduplicated before storage, to save space
• B. Preserved in a form that reflects what the source actually said
• C. Aggregated to the grain the business needs
• D. Stored only long enough to transform it

Question 5

Kestrel's average order value is $75.83. Where does that number come from?

• A. An industry benchmark for outdoor retail
• B. $182.0M ÷ 2,400,000, computed on the page
• C. The median of the order table
• D. The payments processor's settlement report

Question 6

Why did ETL (transform before load) dominate the 1990s and 2000s?

• A. SQL was not expressive enough to transform in the warehouse
• B. Storage and compute were coupled and expensive, so cleaning before loading was economically forced
• C. Regulators required data to be cleaned before storage
• D. Extraction tools could not read raw formats

Question 7

What killed the GUI-based ETL tools, according to §1.3?

• A. They were too slow at scale
• B. They could not connect to cloud data sources
• C. Pipelines built by dragging boxes cannot be diffed, reviewed, tested, or rolled back
• D. They were proprietary and expensive

Question 8

The chapter describes dbt's main contribution as:

• A. Technical — a faster SQL execution engine
• B. Social — putting transformations in version control with tests, review, and CI
• C. Economic — reducing warehouse compute costs
• D. Architectural — introducing the medallion layers

Question 9

Which statement best captures the book's position on distributed processing?

• A. Always build distributed; you will need it eventually
• B. Never use Spark; single-node engines are always sufficient
• C. Start single-node; move to distributed when you have measured that you must
• D. Use distributed processing only for streaming workloads

Question 10

When a dashboard number is wrong because a promotions join fans out rows and changes the effective definition of revenue, who owns it?

• A. Data engineering
• B. Analytics engineering
• C. Data science
• D. Site reliability engineering

Question 11

Kestrel's peak-to-average order ratio is 6.28×. Why does the chapter call this more architecturally important than the annual totals?

• A. Because peak load determines storage cost
• B. Because a system sized for the average fails on the most expensive day, and one sized for peak is idle 364 days a year
• C. Because it determines the number of Kafka partitions
• D. Because regulators require peak capacity planning

Question 12

The duplicate-rows incident went undetected for thirty-one days primarily because:

• A. The team had no monitoring at all
• B. The DAG failed silently without logging
• C. Every job succeeded, row growth looked normal, and revenue rising in spring looked like a good quarter
• D. The source database was unavailable for comparison

Question 13

Which SQL pattern would have prevented the duplicate-rows incident?

• A. Adding a LIMIT clause to the insert
• B. DELETE the target window and INSERT it again, in one transaction
• C. Running the backfill less frequently
• D. Adding an index on order_item_id

Question 14

The four-thousand-dollar Spark job actually cost:

• A. $4,000 a night
• B. $3,840.00 a night
• C. $74.88 a night
• D. $1,401,600 a night

Question 15

What caused it?

• A. Too few executors for the data volume
• B. CAST(event_ts AS DATE) in the WHERE clause defeated partition pruning
• C. A shuffle-heavy join with no broadcast hint
• D. Reading JSON instead of Parquet

Question 16

In a Spark physical plan, the line that reveals this failure is:

• A. ReadSchema: struct<...>
• B. Exchange hashpartitioning(...)
• C. PartitionFilters: []
• D. HashAggregate(keys=[...])

Question 17

Every quantitative claim in this book must be one of four things. Which is NOT one of them?

• A. A frozen anchor figure about Kestrel
• B. Arithmetic performed on the page from those figures
• C. A figure the author remembers from production experience
• D. Output from code in this repository

Question 18

Why does the book use a single frozen cost basis throughout?

• A. To make the arithmetic easier for readers
• B. Because mixing cost bases produces contradictory conclusions from the same architecture
• C. Because cloud providers require consistent quoting
• D. Because the basis is the only accurate one

Question 19

The acceptance criterion for the Kestrel Data Platform is:

• A. The pipeline runs nightly without failures for thirty days
• B. The dashboard loads in under two seconds
• C. Gold-layer net revenue equals source net revenue to the cent, with every difference explained by a documented rule
• D. All Airflow DAGs are green

Question 20

The 6am SLA at Kestrel means:

• A. The pipeline must start at 06:00 America/New_York
• B. daily_revenue must be fresh for the previous calendar day by 06:00 America/New_York
• C. The warehouse must be available 6 hours a day
• D. Alerts must be acknowledged within 6 minutes

Answer Key

1. B — §1 Overview. "At scale" alone is not the hard part; the reliability that permits downstream trust is.

2. C — §1.2, ⚠️ Failure Mode. Software fails loudly, data fails silently. This asymmetry drives Chapters 23, 25, and 26.

3. C — §1.2. Statistical models are the data scientist's output. The six are landed data, conformed tables, tests, schedules, contracts, and documentation/lineage.

4. B — §1.2. Preserved, not cleaned. If you clean before landing, you can never answer "what did the source actually say?" — the first question in every incident.

5. B — §1.5. Derived on the page, so you can check it. This is the numbers rule in §1.8.

6. B — §1.3. Storage and compute were coupled in a fixed appliance budget. When S3 and elastic compute broke the coupling, ELT became the default.

7. C — §1.3. They were software that had escaped software engineering. The modeling from the era survives completely; the tools did not.

8. B — §1.3. Compiling templated SQL is not a hard problem. Making transformations reviewable, testable git artifacts changed the field.

9. C — §1.3, 📐 Design Decision. The named cost of this position: if you genuinely do have a petabyte, you will rewrite.

10. B — §1.4. Definition problems belong to analytics engineering. Compare with the duplicate-rows incident, which presented as a definition problem for two weeks and was decisively data engineering.

11. B — §1.5. Elasticity — paying for capacity only while using it — is the strongest argument for cloud infrastructure in a retail business.

12. C — §1.6. Note what is not on the list: no job failed. That is the whole point.

13. B — §1.6, 🔁 Idempotency Check. Delete-insert (also called insert-overwrite). Three other strategies appear in Chapter 20 §20.3.

14. B — §1.6. $3,840.00 = 160 × 10.0 × $2.400. "$4,000" was the nickname, and the gap between the nickname and the meter is itself the lesson.

15. B — §1.6. A function of a column cannot be matched against the partition key, so nothing is pruned and the scan reads 4.2 TB instead of 34 GB.

16. C — §1.6, 🔎 Read the Plan. PartitionFilters: [] — an empty bracket where the pruning predicate should be. Check this before anything else in a plan.

17. C — §1.8. Recalled figures are exactly the category the rule excludes, because that is where the errors come from.

18. B — §1.8. This rule exists because an earlier book in this series paired a ratio computed on one cost base with a profit computed on another, and readers reproduced the same wrong answer.

19. C — §1.7. Not "it runs." The number is right and you can prove it.

20. B — §1.5. Fresh for the prior calendar day, by 06:00, in America/New_York — and the executive opens it at 06:15, which is the entire perceived error budget.

Topic map

Below 12: reread the chapter before Chapter 2 — the lifecycle chapter assumes all of this. 12–15: reread the sections your misses point to. 16+: go to Chapter 2.