Self-Assessment Quiz: Data Lakes

Twenty questions. Aim for 16 or more.

Question 1

In S3, events/date=2025-11-28/part-0.parquet is:

• A. A path with two directories and a file
• B. A single opaque key in a flat key-value store
• C. A symbolic link
• D. A prefix with an implicit directory object

Question 2

"Renaming a directory" of 34,560 objects requires:

• A. One atomic metadata operation
• B. 34,560 copies and 34,560 deletes, non-atomically
• C. A single MOVE API call
• D. Rewriting the bucket index

Question 3

Object immutability is the reason:

• A. S3 is cheap
• B. Lakehouse table formats exist
• C. Parquet uses row groups
• D. Listing is paginated

Question 4

S3's consistency model:

• A. Is still eventually consistent for overwrites
• B. Became strongly read-after-write consistent for all operations in December 2020
• C. Depends on the storage class
• D. Requires a consistency layer for correctness

Question 5

A Parquet reader's first action is to:

• A. Read the first row group
• B. List all files in the prefix
• C. Read the last 8 bytes to find the footer
• D. Decompress the first page

Question 6

The Parquet footer is at the end of the file because:

• A. It compresses better there
• B. The writer can stream data without knowing statistics in advance
• C. It makes the file smaller
• D. Readers process files backwards

Question 7

A truncated Parquet file is:

• A. Partially readable, up to the truncation point
• B. Entirely unreadable — it fails cleanly
• C. Readable with a repair tool only
• D. Readable but silently missing rows

Question 8

The decisive reason for separate buckets per layer is:

• A. Lifecycle policies
• B. Access control is per bucket
• C. Cost attribution
• D. Blast radius

Question 9

Putting the schema version in the object path:

• A. Is required by Parquet
• B. Costs nothing to add and is impossible to retrofit
• C. Improves compression
• D. Enables partition pruning

Question 10

Bronze is partitioned by ingest date rather than event date because:

• A. Ingest date compresses better
• B. Partitioning bronze by event date means a late arrival must modify an old partition — a non-atomic rewrite while people are reading
• C. Event date is not always available
• D. Silver requires it

Question 11

"Landing raw" means storing:

• A. Parsed JSON with types applied
• B. Exactly what the source sent, plus receipt metadata, payload unparsed
• C. Deduplicated records with a schema
• D. Only the fields you currently use

Question 12

The payload is kept as a string rather than parsed because:

• A. Strings compress better
• B. Parsing at landing forces a decision about unparseable records, and every answer loses information
• C. Parquet cannot store nested data
• D. It makes the write faster

Question 13

The strongest argument against keeping raw data forever is:

• A. Storage cost
• B. Erasure obligations — raw data contains personal data you have committed to deleting
• C. Query performance
• D. Catalog size

Question 14

Partitioning Kestrel's clickstream by customer_id would produce roughly:

• A. 365 partitions
• B. 8,760 partitions
• C. 1.9 million partitions
• D. 47,000 partitions

Question 15

For a lake dataset, the correct treatment of a high-cardinality column you filter on is:

• A. Partition by it
• B. Sort by it within a low-cardinality partition
• C. Create an index on it
• D. Store it in a separate dataset

Question 16

A consumer committing every 30 seconds across 12 partitions produces roughly how many files a year?

• A. 34,560
• B. 420,000
• C. 12.6 million
• D. 150 million

Question 17

The largest cost of the small-files problem is:

• A. Request charges
• B. Latency — 63 minutes of pure request latency at 100-way parallelism for a full-year scan
• C. Storage
• D. Catalog entries

Question 18

Which is NOT one of the three properties of a good compaction job?

• A. It sorts
• B. It writes then swaps
• C. It runs on a lag
• D. It deletes the small files before writing

Question 19

Which practice is NOT one of the seven that separate a lake from a swamp?

• A. A catalog of what exists and who owns it
• B. Compaction and layout maintenance
• C. Storing everything in a single bucket for simplicity
• D. A retention policy, enforced automatically

Question 20

The minimum viable interface for a lake other people consume is:

• A. Direct file access with a documented path convention
• B. A catalog over the files
• C. A table format such as Delta or Iceberg
• D. A REST API

Answer Key

1. B — §9.1. The slashes are just characters. Everything else in §9.1 follows from this.

2. B — §9.1, ⚠️ callout. Slow, billed, and — the real problem — a reader listing mid-move gets a partial result and does not error.

3. B — §9.1. Updating one row means rewriting a whole file, and doing that safely while others read requires a protocol object storage does not provide.

4. B — §9.1, and Chapter 4 §4.4. Pre-2021 workaround advice is obsolete and still ranks well in search.

5. C — §9.2. Then the footer, which gives schema, offsets, and statistics with no data read.

6. B — §9.2. And a reader can fetch it with one small range request before deciding what else to fetch.

7. B — §9.2. A real operational property: an interrupted write fails cleanly rather than returning partial data silently.

8. B — §9.3. A bucket policy versus a prefix-condition policy — expressible either way, and much easier to get subtly wrong the second way.

9. B — §9.3, rule 2. When a breaking change arrives, v2/ sits beside v1/.

10. B — §9.3, 📐 callout. The write path gets the simple option because the alternative's failure is worse.

11. B — §9.4.

12. B — §9.4. Drop it and it is gone; null the fields and the original is lost; fail the batch and you lose availability for one bad record. Storing bytes defers the decision.

13. B — §9.4, 🔐 callout. Not cost — at Kestrel it is $7.84 a month.

14. C — §9.5. 1,900 LIST calls before reading a byte, $0.76 per query in requests, and a catalog that degrades.

15. B — §9.5. Most of the skipping benefit, none of the metadata explosion.

16. C — §9.6. 34,560/day × 365, averaging 27 KB.

17. B — §9.6. Against roughly 30 seconds for the same bytes in 1,332 well-sized files. Requests are only $5.05.

18. D — §9.6. Never delete first — that leaves a window in which the partition is empty or partial, and readers do not error.

19. C — §9.7. Separate buckets per layer is the recommendation; a single bucket is what makes access control and lifecycle policy harder.

20. B — §9.8. Direct file access is fine for exploration and a liability as an interface, because every consumer must know the path convention and nothing tells them when it changes.

Topic map