Self-Assessment Quiz: Batch Ingestion

Twenty questions. Aim for 16 or more. This is the chapter whose material you will use most often.

Question 1

Which three steps of a batch extract do people most often omit?

• A. Decide the range, read, land
• B. Acquire a lease, verify, record a manifest
• C. Read, land, commit
• D. Verify, commit, release

Question 2

If the watermark commits and the write then fails:

• A. The next run retries the range
• B. The range is permanently skipped, and nothing errors
• C. The extract raises an exception
• D. The data is duplicated

Question 3

Which is NOT something a full load gets right for free?

• A. Idempotency
• B. Handling deletes
• C. Self-healing past bugs
• D. Low load on the source

Question 4

Which condition usually forces incremental extraction first?

• A. The table is large
• B. The full load puts unacceptable load on the source
• C. You need history the source does not keep
• D. Storage cost

Question 5

Ranging over an auto-increment primary key:

• A. Catches inserts and updates
• B. Catches inserts only — it silently misses every update
• C. Catches deletes
• D. Requires a watermark

Question 6

Which change-detection strategy catches deletes without CDC?

• A. Timestamp column
• B. Monotonic id
• C. Full compare — key plus row hash on both sides
• D. Version counter

Question 7

updated_at set at transaction start rather than at commit causes:

• A. Duplicate rows
• B. A row committing behind an already-advanced watermark, invisible forever
• C. A cast failure
• D. Slow extracts

Question 8

Which lie is the only one that produces wrong values rather than missing or extra data?

• A. Set at transaction start
• B. Not maintained on every write path
• C. Granularity too coarse
• D. A type change during schema drift

Question 9

The upper bound of a safe watermark should come from:

• A. max(updated_at) of the rows read
• B. An authority — now() read from the primary
• C. The replica's clock
• D. The scheduler's logical date

Question 10

Why must a watermark advance even when no rows were found?

• A. To keep the manifest complete
• B. Otherwise the extract rescans an empty range forever, which is what motivated the else now() bug that lost six weeks of orders
• C. To trigger downstream jobs
• D. It should not advance

Question 11

The overlap window in a safe watermark converts:

• A. Duplicates into loss
• B. Loss into duplicates, which an idempotent write absorbs
• C. Deletes into updates
• D. Slow queries into fast ones

Question 12

The right answer to hard deletes, if you can get it, is:

• A. Periodic full key reconciliation
• B. Soft deletes in the source — a source-system change worth asking for
• C. Full loads
• D. Ignoring them

Question 13

An overlap window combined with tombstone reconciliation can:

• A. Lose rows
• B. Resurrect a deleted row
• C. Corrupt the watermark
• D. Nothing — they are independent

Question 14

When ingesting files, why is filename alone insufficient to know what you have processed?

• A. Filenames may contain invalid characters
• B. Partners overwrite the same filename with different content, and re-send after failures
• C. Filenames are not indexed
• D. It is sufficient

Question 15

A file that never arrives:

• A. Raises an error in the pipeline
• B. Produces no error anywhere — you must alert on absence
• C. Is retried automatically
• D. Fails the manifest check

Question 16

SELECT * in a bronze extract is:

• A. Always wrong
• B. Correct — you want whatever arrives — with a schema check beside it
• C. Correct only for small tables
• D. Required by Parquet

Question 17

Chunking orders by placed_at rather than by order_id produces:

• A. Faster extracts
• B. Wildly uneven chunks, because Black Friday is 6.28× the median
• C. Fewer chunks
• D. Better compression

Question 18

Throttling a backfill is described as:

• A. Expensive but necessary
• B. Free in money — node-hours are node-hours — and valuable in risk
• C. Only worthwhile for large backfills
• D. A last resort

Question 19

A backfill is a different program from an incremental load because:

• A. It uses different libraries
• B. It processes a bounded historical range, may run at higher concurrency, and will be run more than once
• C. It writes to a different table
• D. It runs less often

Question 20

Restartable and idempotent differ in that:

• A. They are the same property
• B. Restartability decides what to redo; idempotency makes redoing harmless
• C. Idempotency implies restartability
• D. Restartability implies idempotency

Answer Key

1. B — §13.1. And each omission has a signature failure: interleaved runs, an empty result you cannot interpret, and an unanswerable question later.

2. B — §13.1. The single most damaging ordering mistake in ingestion, because the next run starts from the advanced watermark.

3. D — §13.2. Load is exactly what a full load costs. The other three are free.

4. B — §13.2. Chapter 7 §7.3's problem, and it usually fires before size does.

5. B — §13.3. Correct only for genuinely append-only tables, which is a claim to verify rather than assume.

6. C — §13.3. Needs nothing but a stable key, and is frequently the right answer for a mid-size table with no reliable updated_at.

7. B — §13.4, lie 1. updated_at is a value the application writes; visibility is governed by commit order. Two different clocks.

8. D — §13.7. Additions and removals are structural and visible; a type change is semantic — and a permissive cast setting turns it into silent nulls.

9. B — §13.4, property 1. Never max() of what you happened to read.

10. B — §13.4, property 3. The bounded range makes this automatic.

11. B — §13.4, property 2. Chapter 4 §4.5's central trade.

12. B — §13.5. Product teams frequently agree, because soft deletes are useful to them too.

13. B — §13.5, 🔁 callout. Two independently correct mechanisms, jointly wrong, invisible in each one's own tests.

14. B — §13.6. Hash the content as well.

15. B — §13.6. Alert on absence, not only on failure.

16. B — §13.7. Bronze must accept whatever arrives; the check announces the change without blocking it.

17. B — §13.8. Chapter 4 §4.2's skew, appearing in the extract rather than the processing, with an identical symptom.

18. B — §13.10, 💸 callout. What it buys is on the other meter — not competing with the nightly load or the source.

19. B — §13.10. Chapter 1's incident was a one-off backfill promoted to a schedule without being re-reviewed as one.

20. B — §13.11. The chunk manifest gives you both, and you need both.

Topic map