Self-Assessment Quiz: Python Transformations

Twenty questions. Aim for 16 or more.

Question 1

pandas was designed for:

• A. Distributed batch processing
• B. Interactive analysis at a REPL, on data that fits in memory
• C. Streaming
• D. SQL compatibility

Question 2

Polars deliberately has no:

• A. Window functions
• B. Index
• C. Joins
• D. Lazy mode

Question 3

DuckDB's most distinctive property, relative to the other two, is that it:

• A. Is written in Rust
• B. Streams rather than materializing
• C. Uses Arrow
• D. Is faster

Question 4

pandas' four structural problems are: everything in memory at a multiple, no query optimizer, single-threaded, and:

• A. Poor documentation
• B. The index and its copy-versus-view ambiguity
• C. No Parquet support
• D. Slow I/O

Question 5

Copy-on-Write, the pandas 3.0 default, means chained assignment:

• A. Works reliably
• B. Consistently does nothing, rather than sometimes working
• C. Raises an error
• D. Is faster

Question 6

scan_parquet plus a lazy chain lets Polars:

• A. Use less CPU
• B. Push filters and column projections into the file read
• C. Skip the write
• D. Parallelize

Question 7

Measured, that laziness was worth:

• A. Nothing
• B. 1.5× in time and 2.5× in memory
• C. 10× in time
• D. Memory only

Question 8

Which column of a benchmark like §22.6's matters more on one machine?

• A. Seconds
• B. Peak memory — it decides whether the job runs at all
• C. Rows out
• D. They are equivalent

Question 9

In the 20M-row measurement, pandas' time was dominated by:

• A. Reading Parquet
• B. Writing
• C. groupby().agg() over a million groups — 77.5%
• D. The join

Question 10

The measured memory multipliers over Parquet size were roughly:

• A. All about the same
• B. DuckDB 2.9×, Polars-lazy 10.9×, pandas 28.2×
• C. DuckDB highest
• D. pandas lowest

Question 11

A memory multiplier is a property of:

• A. The engine
• B. The query, as much as the engine
• C. The file format
• D. The hardware

Question 12

The first version of the benchmark measured nothing because:

• A. The data was too small
• B. All four engines ran in one process, and RSS does not fall when a DataFrame is freed
• C. It used the wrong timer
• D. Compression was off

Question 13

The rule of thumb for one machine is roughly:

• A. RAM ÷ 2 for any engine
• B. RAM ÷ 10 streaming, RAM ÷ 30 materializing
• C. RAM × 2
• D. Row count, not size

Question 14

Which of the three spills to disk automatically when it runs out of memory?

• A. pandas
• B. Polars
• C. DuckDB
• D. All three

Question 15

DuckDB's SUM over a BIGINT column returns:

• A. BIGINT
• B. HUGEINT, which Parquet stores as DOUBLE
• C. DOUBLE directly
• D. DECIMAL

Question 16

Why does that matter even when no precision is lost?

• A. It is slower
• B. The money column is now a float in every downstream consumer, and all further arithmetic is float arithmetic
• C. It uses more space
• D. It breaks the join

Question 17

Type inference at the read makes your schema:

• A. Explicit
• B. A property of whatever arrived this morning
• C. Immutable
• D. Faster to load

Question 18

On a CSV column of pure digits with leading zeros, and no type declared:

• A. All three readers keep the zeros
• B. All three lose them
• C. DuckDB keeps them; pandas and Polars infer an integer
• D. pandas keeps them; the others do not

Question 19

pl.read_csv(path, schema_overrides={"sku": pl.String}) versus pl.read_csv(path).with_columns(pl.col("sku").cast(str)):

• A. Equivalent
• B. The first keeps the leading zeros; the second preserves the damage the inference already did
• C. The second is faster
• D. Neither works

Question 20

Exit code 137 in a data job almost always means:

• A. A syntax error
• B. A timeout
• C. SIGKILL from the OOM killer, which leaves no traceback because it cannot be caught
• D. A network failure

Answer Key

1. B — §22.1. Most complaints about pandas in a pipeline are complaints about using an interactive tool non-interactively.

2. B — §22.1. Which removes the class of bug where an operation silently aligned on labels you had forgotten about.

3. B — §22.1. And it is what produced the 320 MB against 3,138 MB in §22.6.

4. B — §22.2.

5. B — §22.2. An improvement, and a migration hazard for code that relied on the sometimes-working case.

6. B — §22.3. Columns the query never mentions are never decompressed.

7. B — §22.6. Same answer, same library, one keyword — the cheapest improvement in the chapter.

8. B — §22.6. Every engine finished in under eleven seconds; the memory spread was 9.8×.

9. C — §22.6. "pandas is 25× slower" is much less useful than naming the operation.

10. B — §22.7.

11. B — Case Study 1. Polars-lazy used more than pandas-with-projection on that workload and far less on §22.6's.

12. B — §22.6's ⚠️ callout. pandas appeared cheapest on memory, which is the opposite of the truth, because it ran last.

13. B — §22.7. And ask how much one run touches: Kestrel's clickstream is 341 GB a year and 934 MB a day.

14. C — §22.8. It changes the failure mode from "the process dies" to "the query is slower."

15. B — §22.9. Returning HUGEINT is a good decision — a sum of 64-bit values cannot overflow. The Parquet mapping is the problem.

16. B — §22.9. Assert the output schema, not only the output values — a value test passes on a float column holding exactly the right numbers.

17. B — §22.11.

18. C — Case Study 2, measured. All three are defensible; nothing in CSV says which reading is intended. Do not rely on the good sniffer either — it is still inference.

19. B — Case Study 2's 📐 callout. A type declaration is only a control at the boundary where the information still exists.

20. C — Case Study 1. 128 + 9. The absence of a traceback is information: it narrows the cause to something outside your process's control.

Topic map