Affiliate disclosure

Book titles on this page link to Amazon. As an Amazon Associate, DataField.Dev earns from qualifying purchases — at no additional cost to you.

Further Reading: Data Warehouses

Sources are tagged Tier 1 (confident it exists, recommended without reservation) or Tier 2 (real and worth seeking, but confirm the current edition, version, or URL yourself).

Warehouse documentation is vendor documentation, which means it is excellent at explaining what a product does and structurally incapable of telling you when not to use it. The papers below are the antidote: they explain the mechanisms, and mechanisms do not have a sales target.

The papers that explain the mechanisms

  • Daniel Abadi, Samuel Madden, and Nabil Hachem, "Column-Stores vs. Row-Stores: How Different Are They Really?" (2008), SIGMOD. The paper that settled why columnar storage wins, by taking a row store and adding columnar features one at a time to see which mattered. The answer — that the benefit comes from compression, late materialization, and block iteration together rather than from layout alone — is more interesting than the headline. This is §8.2 and §8.3, from the source. Tier 1.

  • Mike Stonebraker et al., "C-Store: A Column-oriented DBMS" (2005), VLDB. The ancestor of Vertica and, indirectly, of most modern columnar engines. Read for the design reasoning around compression and projections. Tier 1.

  • Benoit Dageville et al., "The Snowflake Elastic Data Warehouse" (2016), SIGMOD. How separated storage and compute is actually implemented: micro-partitions, the metadata layer, virtual warehouses, and the pruning that makes it work. §8.4 and §8.5's Snowflake material compressed from here. Written by the people who built it and unusually clear about the trade-offs. Tier 1.

  • Sergey Melnik et al., "Dremel: Interactive Analysis of Web-Scale Datasets" (2010), VLDB, and the 2020 retrospective "Dremel: A Decade of Interactive SQL Analysis at Web Scale." Dremel is BigQuery's engine. The original paper introduced the nested columnar format that became Parquet's repetition/definition-level encoding; the retrospective is the better read, because it says which of the original design decisions survived a decade and which did not. Tier 1.

  • Anurag Gupta et al., "Amazon Redshift and the Case for Simpler Data Warehouses" (2015), SIGMOD. Redshift's design paper. Note the date — it describes classic coupled Redshift, and RA3 and Serverless changed a great deal. Read it for the distribution-key reasoning, which has not changed and which is §8.5's characteristic Redshift mistake. Tier 1, with the date caveat.

The vendor documentation, used correctly

Read these for mechanism, not for recommendation:

  • Snowflake: "Understanding Snowflake Table Structures" (micro-partitions and clustering), and the "Query Profile" documentation. The clustering depth material is what §8.8 compresses. The Query Profile page is required reading before you first look at one — the "Bytes spilled to remote storage" statistic in particular. Tier 2 — versioned and rewritten regularly.

  • BigQuery: "Introduction to partitioned tables," "Introduction to clustered tables," and "Estimate and control costs." The last one covers maximum_bytes_billed and custom quotas — §8.7's controls 1 and 2, from the source. Tier 2.

  • Redshift: "Distribution styles" and "Sort keys." The two settings that determine whether Redshift is fast or slow, explained by the people who chose to make them your problem. Tier 2.

  • The DuckDB blog. Unusually honest performance write-ups, including ones where DuckDB is not the right tool. Useful precisely because it is a vendor blog that says so. Tier 2.

On benchmarks, since §8.9's case study depends on it

  • The TPC-H and TPC-DS specifications (tpc.org). The standard analytical benchmarks. Worth knowing what they measure, because every vendor benchmark you read is derived from one of them, usually with modifications the vendor will describe in a footnote. Reading the specification once makes vendor benchmark claims much easier to evaluate. Tier 1.

  • Mark Raasveldt et al., "Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing" (2018), DBTest. Eight pitfalls in database benchmarking, from the DuckDB authors. It is the academic version of this chapter's Case Study 2 — cold versus warm cache, incomparable configurations, cherry-picked queries, and ignoring load time. Read before you run a benchmark, and again before you believe one. Tier 1.

On cost

  • The FinOps Foundation's materials (finops.org). Vendor-neutral practice for cloud cost management: allocation, tagging, showback, and unit economics. §8.7's four controls are FinOps fundamentals applied to a warehouse, and the tagging discipline in this chapter's Case Study 1 is the single most emphasized practice in the FinOps literature. Tier 2 — an evolving body of material.

  • Each vendor's usage and metering views: Snowflake's SNOWFLAKE.ACCOUNT_USAGE schema (especially QUERY_HISTORY and WAREHOUSE_METERING_HISTORY), BigQuery's INFORMATION_SCHEMA.JOBS, and Redshift's SVL_* and SYS_* views. These are the tables the cost analysis in Case Study 1 runs against, and knowing they exist is most of the battle — a surprising number of teams do their cost analysis from the invoice instead. Tier 2 — schema names change.

On the wider argument

  • Jordan Tigani, "Big Data Is Dead" (2023). Recommended in Chapter 5 and relevant again: the claim that most organizations' data fits on one machine bears directly on §8.9's thresholds. The chapter's position — that concurrency and governance, not volume, are the usual honest reasons to adopt a warehouse — is a partial agreement with Tigani and worth reading alongside. Tier 2.

  • Chapter 3's further reading on the lakehouse papers. Where warehouses are going: the Armbrust lakehouse paper and the Delta Lake paper both argue that warehouse guarantees can be delivered over open formats, which is Chapter 10's subject and which would, if it fully succeeds, make this chapter's vendor comparison much less consequential. Tier 1.

If you only read one thing

Read the Snowflake paper (Dageville et al., 2016). About twelve pages, and it is the clearest available explanation of how separated storage and compute actually works — micro-partitions, pruning metadata, and elastic virtual warehouses. Once you have the mechanism, the pricing model stops being arbitrary and starts being a consequence, which is what lets you predict cost rather than measure it afterwards.

Then, before your first benchmark, read "Fair Benchmarking Considered Difficult." Eight pages, and it will save you a week.