Further Reading: Data Governance and Cataloging

Sources are tagged Tier 1 (confident it exists, recommended without reservation) or Tier 2 (real and worth seeking, but confirm the current edition, version, or URL yourself).

A warning about this literature before anything else. Data governance has more published material than almost any topic in this book and a worse ratio of useful to useless. Much of it is written by vendors, organized around maturity models, and measures programs by the completeness of their artifacts rather than by the decisions those artifacts change — which is precisely the failure §30.1 is about. Read it with the decision test in hand.

Start here

• Chapter 23 and Chapter 26 of this book. Not a deflection. The register of assertions is the most effective governance artifact Kestrel has, and the SLO document is the second, and neither was built as governance. §30.11's argument is that this is the general case rather than an accident.

• Piethein Strengholt, Data Management at Scale (O'Reilly, 2nd ed. 2023). The most practical book on this subject. Chapters on metadata, ownership, and the interplay between governance and architecture are the ones to read; the mesh material overlaps Chapter 35. Strengholt is unusually honest about what does not work. Tier 1.

• The DAMA-DMBOK (Data Management Body of Knowledge, 2nd ed.). The reference work, and it is a reference work — do not read it front to back. Its value is as a checklist of things you may not have considered, and its cost is that it presents every practice as necessary, which is exactly the posture §30.1 argues against. Use it to find gaps, not to build a program. Tier 1 for existence; read selectively.

On catalogs specifically

• The dbt documentation on docs, meta, and exposures. §30.11's argument in software form: metadata lives in the same file as the model and is reviewed in the same pull request. The exposures feature is under-used and directly answers "if this breaks, who notices?" Tier 1 — versioned; this book pins dbt-core 1.9.1.

• The OpenLineage specification, and Marquez as a reference implementation. The vendor-neutral lineage standard. Read the spec rather than a vendor's page about it — it is short, and it makes clear how much of "automated lineage" is emitted by the tools rather than inferred. Tier 1.

• DataHub, Amundsen, and OpenMetadata — the open-source catalogs. All three are real, all three are substantially more work to run than their landing pages suggest, and all three are worth reading the data model of even if you never deploy one. Amundsen's original engineering blog post (Lyft, 2019) is the honest account of why they built it: search, not compliance. Tier 1 for the projects; Tier 2 for the specific blog URL.

• Shirshanka Das et al. on DataHub's architecture (LinkedIn engineering, and the subsequent talks). The push-based metadata model is the interesting idea — metadata is emitted by producers rather than crawled — and it is the same by-product argument as §30.11 at platform scale. Tier 2 — find the current write-up.

On the semantic layer

• The dbt Semantic Layer documentation, and the MetricFlow specification. Case Study 1's problem, addressed structurally. Read the metric specification format first — it makes concrete what "the definition exists once" means. Tier 1 — moving quickly; check the current shape.

• Cube's documentation on data modeling. A different take on the same problem, and worth reading alongside dbt's for the contrast. Tier 1.

• Anything on the history of OLAP semantic layers — Business Objects universes, Cognos frameworks, SSAS cubes. This problem was solved reasonably well in the 1990s and the solution was discarded with the tools that carried it, which is worth knowing before treating the semantic layer as new. Tier 2 — mostly product documentation and retrospectives.

On access, review, and least privilege

• The Google SRE book's chapter on access and the "Building Secure and Reliable Systems" companion (O'Reilly, 2020), particularly the material on least privilege in practice and on making the safe path the fast path. Case Study 2's pre-authorized restoration path is that idea. Tier 1.

• NIST SP 800-53, control family AC (Access Control), and AC-2(3) specifically. Dry, and the authoritative statement of what an access review is supposed to accomplish. Read AC-2 and then ask Case Study 2's question of your own review. Tier 1 — free, and findable by control number.

• Your warehouse's own metadata schema. Snowflake ACCOUNT_USAGE (GRANTS_TO_ROLES, GRANTS_TO_USERS, QUERY_HISTORY, ACCESS_HISTORY), BigQuery INFORMATION_SCHEMA.JOBS and Data Access audit logs, Postgres information_schema.role_table_grants with pg_stat_statements. This is the single highest-value reading in this section — Case Study 2's entire audit was two queries against documentation that was already there. Tier 1.

• Snowflake's ACCESS_HISTORY view in particular. It reports column-level access, which turns "who can see customer email addresses" from an inference about grants into an observation about reads. Tier 1 — edition-gated on some plans; check yours.

On classification, retention, and the boundary

• Your own legal team. §30.6's point is that classification is a legal decision and engineering's job is to make it operable. An hour with the person who owns the policy is worth more than any book here, and it is skipped remarkably often.

• The ICO's guidance on data retention (UK) and the equivalent from your regulator. Written for practitioners, free, and clearer than most vendor material on the distinction between a policy and its enforcement. Tier 2 — jurisdiction-specific; find yours.

• AWS S3 Lifecycle configuration documentation, and the equivalent for GCS and ADLS. §30.7's missing mechanism. Ten minutes of reading; it is the enforcement half of every retention policy that currently has only a period. Tier 1.

On the human part

• Anything careful on the "documentation is written by the wrong person" problem. The curse-of-knowledge literature in psychology is the underlying result, and §30.9's claim that an author cannot write down what they find obvious is a direct application. Tier 2 — a general literature; one good summary chapter will do.

• Studies of how developers and analysts actually find things. The consistent finding across decades — people ask a person first — is the one most catalog projects are designed as though they had not read. Tier 2 — scattered across software-engineering venues.

• Melanie Mitchell's and others' writing on institutional metrics and Goodhart's law, applied to governance programs measured by artifact completeness. A program that is scored on how many fields are populated will populate fields. Tier 2.

Practice

• code/catalog_audit.py in this chapter. Eleven findings in a deliberately messy fixture, twenty-two self-checks, and Exercise 30.11 adds three checks of your own.

• Run the grep. Case Study 1's is four minutes and finds something in almost every organization. If it finds nothing, your pattern is probably too narrow — try the column names rather than the metric name.

• Run the two queries. Exercise 30.7. Granted versus used, against a warehouse you have access to. The gap is almost always larger than the people who run it expect, and it is the fastest way to make this chapter concrete.

A note on what to be skeptical of

Maturity models. They rank organizations on a five-level scale and the levels are almost never tied to outcomes. Ask what decision moving from level 2 to level 3 changes, and the useful ones will survive the question.

"Automated governance" and "AI-powered cataloging." Automated lineage is real and useful. Automated description generation produces text that is grammatically fine, semantically empty, and worse than an empty field — because an empty field is honest and a generated one is not. The gotchas that people read were written by people who had just answered a question.

Any governance program whose success metric is completeness. §30.1's test is the antidote, and Exercise 30.16's honest-empties count is the practice version of it.