33 min read

> *"The proposal had four domains, four domain teams, and a self-serve platform. What it did not have

Prerequisites

  • Chapter 17
  • Chapter 30
  • Chapter 33
  • Chapter 34

Learning Objectives

  • State the problem data mesh solves, and recognize when you do not have it.
  • Measure whether your central team is actually a bottleneck.
  • Test a proposed domain partition instead of asserting it.
  • Apply the four principles as conditions rather than aspirations.
  • Price a mesh adoption in headcount before it returns anything.
  • Extract the two ideas that are worth adopting at any size.
  • Say when a mesh is right, using a threshold rather than a trend.

Chapter 35: Data Mesh

"The proposal had four domains, four domain teams, and a self-serve platform. What it did not have was any statement of what each domain would own that the other three did not need."

Overview

Data mesh answers a real problem and is adopted, at real expense, by organizations that do not have it.

The problem is genuine. As an organization grows, a central data team becomes a queue. Every new dataset, every change, every question routes through the same handful of people, who know less about each domain than the domain does and have less context than the requester. At sufficient scale this is a serious structural failure, and data mesh is a serious answer to it.

The misapplication is not subtle. It is adopting a decentralization pattern without having anything to decentralize — no domain boundaries, no domain teams, no bottleneck, and often no more than a handful of data engineers in total.

This chapter takes the idea seriously and is direct about when it is wrong, which means doing something the mesh literature almost never does: making the preconditions measurable. code/mesh_readiness.py asks four questions with numbers rather than adjectives — is the central team a bottleneck, do you have domains, are you ready, what does it cost — and applies them to Kestrel.

Kestrel's answer is no. The chapter says so, shows the arithmetic, and then extracts the two ideas from data mesh that are worth adopting at any size, because they are — and they are not the ones the proposals usually lead with.


35.1 The Problem It Solves

Picture an organization with two hundred engineers and six data engineers.

Every dataset routes through those six. They are competent, they are busy, and they know less about payments than the payments team does. So a request arrives as a specification the requester wrote, gets built by someone without the domain context to notice the specification is wrong, and comes back in three weeks needing changes.

Four symptoms, and if you do not have at least three of them you do not have this problem:

A queue with a wait time nobody defends. Not "we're busy" — an actual measured wait, in weeks, that has been stable or growing for a year.

Requests that are specifications rather than problems. The requester has already designed the solution because they gave up on a conversation.

A central team that cannot answer "is this number right?" for most of the datasets it owns, because correctness is a domain question.

Domain teams that have built shadow pipelines. The clearest signal, and the most reliable: when people route around you, the bottleneck is real.

🎓 Interview Angle — "should we adopt a data mesh?"

A question you will be asked, and the wrong answer is either yes or no.

The answer that works asks for evidence of the problem before discussing the solution:

"Before anything about mesh — is the central team actually a bottleneck? I'd want the intake queue for a year, the wait times, and specifically what share of it is work a domain team could even do. That last number is usually the surprise.

Then: do domains exist? Not in the org chart — in the data. If most of the dependency edges cross the proposed boundaries, they aren't domains, and decentralizing along them creates coordination cost without creating autonomy.

And I'd want to know whether any domain has an engineer who could own a pipeline. If the answer is no, the mesh is a plan to hire, not a plan to reorganize — which might be the right plan, but it should be argued as one."

If you want one sentence: "data mesh is an answer to a coordination problem, and coordination problems are a function of headcount — so the first number I'd want is how many people you have."


35.2 The Four Principles, Stated Fairly

Zhamak Dehghani's formulation has four principles, and each is a genuinely good idea.

1. Domain-oriented decentralized ownership. The team that produces the data owns the data product built from it. Not "consults on" — owns, including the pipeline, the schema, and the on-call.

2. Data as a product. A dataset is a product with a consumer, a contract, an SLO, documentation, and someone accountable for its quality. Not an output; a product.

3. Self-serve data platform. A central platform team provides the capabilities — ingestion, storage, orchestration, observability, governance — so a domain team can ship without a platform engineer. The platform team's product is the platform, not the data.

4. Federated computational governance. Global standards, decided by a federation of domain representatives, and enforced by automation rather than by review.

Read those four again and notice something: three of them are good advice at any size. Chapters 17, 30, and 34 have been arguing for #2 and #4 for the whole book. Only #1 requires decentralization, and only #1 requires that you have domains.

That asymmetry is the chapter's central point and §35.11 returns to it.


35.3 Is Your Central Team Actually the Bottleneck?

The premise is measurable and almost nobody measures it. --bottleneck takes a year of Kestrel's intake:

category                         count  wait(d)   delegable?
────────────────────────────────────────────────────────────
'why is this number wrong'          88      2.0       yes
access / permissions                71      1.0       no
change an existing model            62      4.0       yes
ad hoc extract                      53      3.0       yes
fix a broken pipeline               48      0.5       no
new gold model                      34      9.0       yes
infrastructure change               19     14.0       no
new source ingestion                11     21.0       no
────────────────────────────────────────────────────────────
TOTAL                              386      3.8

386 requests a year across four data engineers — 96.5 each. That is roughly two a week per person, which is real load and is not obviously a crisis.

Three findings, and the second and third both argue against a mesh.

61.4% of the queue is work a domain team could do. Which sounds like a strong case for decentralizing — until you look at the wait times.

🔎 Read the Plan — the delegable work is the work that is already fast

text mean wait, all work 3.84 days mean wait, delegable work 3.75 days

The work a domain team could take is already moving faster than average. Decentralizing it would relieve the central team of 61% of its volume and almost none of its delay.

And the two slowest categories are both undelegable:

text new source ingestion 21.0 days a domain team cannot do this infrastructure change 14.0 days a domain team cannot do this

These are the requests that people complain about, they are 8% of the volume and the overwhelming majority of the frustration, and a data mesh does not touch them. They are slow because they require platform work — which under a mesh is still done by the platform team, now with more consumers and a higher bar for self-service.

The generalizable diagnostic: split your queue by wait time, not by volume, and ask who could do the slow half. If the answer is "only the central team," decentralization addresses the wrong constraint, and the honest fix is to make source ingestion and infrastructure changes faster — which is Chapter 28's work, not Chapter 35's.

This is the single most useful hour in the chapter and it costs one query against a ticket system.

And the biggest category is not a build request at all. "Why is this number wrong" is 88 requests a year — 23% of everything — and it is a question about meaning, which is exactly what Chapter 30 §30.9's catalog and §30.8's certified definitions address. A mesh would move those questions to domain teams, which is plausibly better; a catalog answers them without anyone moving.


35.4 Do You Actually Have Domains?

Somebody proposes four domains: orders, customers, inventory, marketing. Are those domains?

The question is answerable. --coupling takes the proposed partition and measures how much of the dependency graph respects it:

models                    24
dependency edges          30
within a domain           18
CROSSING a domain         12   (40%)

40% of edges cross a boundary, which is high — but the fan-out is the number that settles it:

domain fan-out (how many other domains each depends on):
    orders       3   customers, inventory, platform
    customers    2   marketing, orders
    marketing    2   customers, orders
    inventory    1   orders
    platform     0   -

Every proposed domain depends on at least one other. There is no autonomous unit in the partition.

⚠️ Failure Mode — a partition is not a domain boundary

Any set of tables can be partitioned into four groups. The partition tells you nothing; the edges do.

The test is autonomy, and it has a precise form: could this domain build, deploy, and change its data products without coordinating with another domain? If the answer is no, you have not created autonomy — you have created a coordination requirement with a team boundary through the middle of it, which is strictly worse than the centralized version, because now the coordination is between teams instead of within one.

Kestrel's orders domain depends on three others, including platform for a shared date dimension. Marketing's attribution model reads orders' revenue table. Inventory's stock-cover model reads it too.

The failure mode this produces is well documented in software and is identical here: a change to fct_order_line now requires agreement from four teams rather than a decision by one. Kestrel's proposal would have converted a one-person schema change into a four-team negotiation, and called the result decentralization.

The number to compute before proposing domains: for each proposed domain, how many others does it depend on? Two or more, for most of them, and you are proposing coordination rather than autonomy.

And then the specific table that makes it impossible.

📏 Scale Note — the table every domain reads

text models read by MORE THAN ONE domain: gold.fct_order_line 4 customers, inventory, marketing, orders gold.dim_customer 3 customers, marketing, orders gold.fct_session 2 customers, marketing

fct_order_line is read by all four proposed domains, and it is the most-used table in the platform. Which domain owns it?

Every available answer is bad:

  • orders owns it. Then three domains depend on orders' release schedule, orders' on-call, and orders' priorities for a table that is central to their work. orders becomes the new central team, with less capacity and no mandate.
  • platform owns it. Then the platform team owns the most important business table, which is exactly the centralization the mesh was meant to remove — now with a team whose stated product is infrastructure.
  • Each domain builds its own version. Chapter 30 Case Study 1, deliberately. Four definitions of revenue, by design.

A shared table that four domains need is a signal that the boundary is in the wrong place, and the mesh literature's answer — "it is a polyseme; each domain has its own representation" — is real and applies to a concept like customer, not to a fact table 24 models deep in a dependency graph.

The honest reading: Kestrel's data has one domain in it, and it is called "the business." At 2.4 million orders a year through one commerce system, that is what a small e-commerce company's data looks like, and no partition will change it.


35.5 Data as a Product

The best idea in the mesh, and the one that requires nothing else from it.

A data product has: a consumer who is known · a contract (Chapter 17) · an SLO (Chapter 26) · documentation and a grain (Chapter 30) · an owner who can act (§30.4) · a deprecation path (§30.2).

Kestrel has all six, built over Parts V and VI, with a central team. The mesh's contribution is the framing: treat the dataset as a product with a customer, rather than as an output of a pipeline.

Two things that framing changes in practice, and both are real:

You ask who the consumer is, and sometimes there isn't one. Chapter 25 §25.12 found tables nobody read. Product framing makes that question routine rather than a periodic audit.

Breaking changes become a product decision. Chapter 17's contract with a deprecation window is a product manager's instinct applied to a table.


35.6 The Self-Serve Platform

The second-best idea, and the one that is hardest to cost correctly.

The goal: a domain team ingests a source, builds a model, deploys it, and monitors it, without a platform engineer.

Kestrel's honest status, from --readiness:

SELF-SERVE PLATFORM     4/9
    [ ] (3) a domain can ingest a source unaided
    [x] (2) a domain can deploy a model unaided
    [ ] (2) a domain can get compute unaided
    [x] (2) observability is provided, not built

Deploying a model is self-serve — dbt, a pull request, CI (Chapter 27). Ingesting a source is not, and it is the 21-day queue item.

💸 Cost Check — a self-serve platform is more work than doing the work

This is the line every mesh proposal gets wrong, and the arithmetic is not close.

Doing it yourself: an engineer writes an ingestion pipeline for a new source. Two to five days, and they can make every decision expediently because they are the only user.

Making it self-serve: the same capability, as a product — a declarative interface, validation, sensible errors, documentation, a test harness, and support for the sources you have not seen yet. Kestrel estimated eight weeks for source ingestion alone, and estimates for platform work of this kind are famously optimistic.

At 11 new sources a year, doing it yourself costs about 33 engineer-days annually. Self-serve costs 40 days up front and then some smaller ongoing number. It breaks even in a bit over a year — if the volume holds and the platform does not need rework, and it does not begin to pay until a domain team actually uses it.

Which is the real problem: the payback requires domain teams that do not exist yet. Kestrel would be building self-serve ingestion for four teams with no data engineers, who would then need to learn the platform, which is a second cost nobody prices.

The general rule: build self-serve for a capability whose users already exist and are already asking. Chapter 27's CI and Chapter 25's observability qualify — the four data engineers use them daily. Source ingestion does not, and the honest sequencing is to create the demand before building the supply.


35.7 Federated Computational Governance

The fourth principle, and the word doing the work is "computational."

Global standards, decided by a federation, enforced by automation. Kestrel scores 5 of 7 here, which is its best principle — and the reason is instructive:

FEDERATED GOVERNANCE    5/7
    [ ] (2) a body exists that can decide
    [x] (3) global standards are enforced automatically
    [x] (2) a metric can be defined once

The automation exists — Chapter 34's layer_check.py, Chapter 23's register, Chapter 30's catalog_audit.py, Chapter 31's manifest generation. What does not exist is the federation, because there are no domains to federate.

Which is exactly backwards from how most organizations arrive here, and it is good news: the computational half is the hard half, it is what this book has spent six chapters building, and it works without any of the decentralization.


35.8 Conway's Law, and Which Way the Arrow Points

"Organizations design systems that mirror their communication structure." Conway's law is the foundation the mesh is built on, and it is usually invoked backwards.

The mesh's move is the inverse Conway maneuver: change the organization to get the architecture you want. That is a legitimate and powerful technique. It is also, stated plainly, a reorganization — and it should be argued as one, by people who can authorize one.

⚠️ Failure Mode — an architecture proposal that is secretly a reorganization proposal

This is the most common way a mesh adoption fails, and it fails quietly.

A data team proposes a mesh. The proposal is technical: domains, data products, a self-serve platform, federated governance. It is reviewed by technical people and approved on technical merit.

What it actually requires is that four product teams accept a new responsibility — owning a data pipeline, carrying its on-call, and prioritizing its consumers' requests against their own roadmap. Nobody in the approval chain can compel that, and the proposal did not ask them to, because it was framed as an architecture.

What happens next is predictable and Kestrel's team had seen it at a previous employer:

text month 1-3 platform team builds self-serve tooling month 4 domain teams are "onboarded" -- given access and a workshop month 5-9 domain teams do not use it; they have roadmaps month 10 the platform team resumes building pipelines, now with self-serve tooling nobody uses and a mesh in the org chart

The tooling is not wasted — the platform team uses it. The reorganization never happened, because it was never actually proposed to anyone who could make it.

The test that separates a real proposal from this one: name the person who will approve a change to four teams' responsibilities, and confirm they have seen the proposal. If you cannot, you are proposing tooling and calling it an architecture. That is not necessarily bad — the tooling may be worth building — but it should be funded and measured as tooling.

And the arrow points both ways, which is the part worth knowing. If your organization already has strong domain boundaries — separate product lines, separate P&Ls, separate engineering orgs — Conway's law says your data architecture will drift toward a mesh whether or not you adopt one, because the communication structure is already meshed.

In that case the choice is not whether to have a mesh. It is whether to have a governed one or an accidental one, and the mesh literature is genuinely valuable there. Kestrel's organization has one product line and one engineering team, so its data architecture is centralized because its communication structure is — and forcing a mesh onto it would be fighting Conway's law rather than using it.

35.9 Kestrel's Readiness, and the Honest Answer

DOMAIN OWNERSHIP        1/9
    [x] (1) domains exist in the org chart
    [ ] (3) each domain has >=1 engineer who can build data
    [ ] (3) cross-domain edges are a minority
    [ ] (2) domains can deploy without the central team

DATA AS A PRODUCT       5/7
SELF-SERVE PLATFORM     4/9
FEDERATED GOVERNANCE    5/7
────────────────────────────
TOTAL                  15/32  = 47%

47% is not a passing or failing grade; the pattern is the finding.

🔎 Read the Plan — every failure is about a domain acting independently

Sort Kestrel's fifteen conditions into passed and failed and look at what separates them:

text PASSED (8) FAILED (7) contracts documented a domain has an engineer discoverable without asking cross-domain edges are a minority SLOs published domains deploy without the central team deploy a model unaided every dataset has an owner who can act observability provided a domain can ingest a source unaided standards enforced automatically a domain can get compute unaided a metric defined once a governance body exists domains exist in the org chart

Every passed condition is something the central team built. Every failed one is something a domain team would have to do.

That is not a coincidence and it is not a coverage gap. It says Kestrel has built the supporting infrastructure of a data mesh — the contracts, the catalog, the SLOs, the automated governance — and has none of the decentralization, because the decentralization requires people who do not exist.

The useful reframing: Kestrel is not 47% of the way to a mesh. It is 100% of the way to the parts of a mesh that a central team can build alone, and 0% of the way to the parts that require an organization. Averaging those two into 47% obscures the only decision that matters.

Which is why a readiness score should always be read by principle rather than in total — and why a proposal that leads with a total is usually leading with the wrong number.

The honest answer for Kestrel: no. Four data engineers, three analysts, two scientists, one commerce system, one domain. A data mesh at Kestrel would create four coordination boundaries where none is needed, and would need to hire the teams to put on either side of them.


35.10 What It Costs

--cost prices the four principles in headcount, before any of them returns anything:

proposed domains                     4
data engineers per domain (min)      1
...so domain engineers               4
platform team (does NOT shrink)      4
federated governance               0.5
────────────────────────────────────────
total                              8.5
today                                4
ADDED                              4.5   (2.12x)
at $180,000 loaded            $810,000/year

⚠️ Failure Mode — the platform team does not shrink

The line every proposal gets wrong, and it is the one that determines whether the arithmetic works.

The intuition: work moves to the domains, so the central team gets smaller.

What actually happens: the central team stops doing pipeline work and starts doing platform work — which is more work, for a long time (§35.6), and which has to be delivered before domain teams can do anything. The central team is the bottleneck for its own removal.

So there is a period — Kestrel estimated eighteen months — during which you are paying for both: a platform team building self-serve capabilities and domain engineers who cannot yet self-serve. This is the transition cost, it is the largest number in the proposal, and it is almost never in the proposal.

The multiple is what makes it decidable. At Kestrel's size, a mesh is 2.12× the data engineering headcount and $810,000 a year of new cost. At forty engineers and twelve domains, the same arithmetic gives a multiple under 1.5, because the platform team is already large and the domain engineers largely already exist inside product teams.

That is the real threshold, and it is about headcount rather than about architecture. A mesh gets cheaper as you get bigger, which is the same shape as the problem it solves — which is a good sign that it is a real answer to a real problem, at a size Kestrel is nowhere near.


35.11 The Two Ideas Worth Stealing

Data mesh's four principles, sorted by whether they require decentralization:

Principle Requires domains? Worth adopting at Kestrel's size?
Domain-oriented ownership yes ❌ no
Data as a product no yes
Self-serve platform no ⚠️ partially
Federated computational governance the federation, yes the computational half

Two ideas transfer completely, and neither requires a single organizational change.

Data as a product. A dataset with a known consumer, a contract, an SLO, and an owner. Kestrel has this and got it from the mesh literature, applied by a central team — which the literature does not forbid and does not emphasize.

Computational governance. Standards enforced by automation rather than by a review board. This is the best idea in the mesh and it is orthogonal to the mesh, and Chapters 23, 27, 30, 31, and 34 are all instances of it.

🧱 Kestrel Platform — what was adopted, and what it cost

From a mesh proposal that was rejected, three things were adopted:

text data product manifest every gold table: consumer, contract, SLO, owner -> 4 tables had NO consumer; deprecated (Ch 30) output ports one documented interface per product, versioned -> replaced 11 ad-hoc extracts with 3 ports computational standards layer_check + catalog_audit + the register -> already existed; the mesh gave them a name

Cost: about three weeks. No reorganization, no new headcount, no domain teams.

The output-port change is the one worth copying. Eleven scheduled extracts had accumulated, each built for one consumer, each a separate thing to maintain and to include in Chapter 31's deletion manifest. Framing them as products with ports collapsed them to three, because the framing forced the question "who is the consumer, and is this the same product?" — and eight times the answer was "the same product, shaped differently for one person's convenience."

This is data mesh delivering real value at a company that should not adopt data mesh, and it is the honest outcome of taking the idea seriously rather than either buying it or dismissing it.


35.12 When It Is Right

Four conditions. You need all four, and the first is not negotiable.

1. Domain teams with engineers who can build data pipelines. Not analysts, not "a data champion" — engineers with capacity. Without this, a mesh is a hiring plan.

2. A measured bottleneck in delegable work. §35.3. Not volume — wait time, on work a domain team could actually do.

3. Domains that are actually separable. §35.4's fan-out test. Most domains depending on at most one other, and no single table read by all of them.

4. A platform team that can build platform. Different skills from pipeline work, and a headcount that survives the transition period during which it is paying for both.

The size at which this typically holds: somewhere north of thirty data-capable engineers across five or more genuinely distinct business lines. That is not a rule, and the number is soft. What is not soft is that Kestrel, at nine data people and one commerce system, is not near it.

And one condition that is not on the list: growth. "We will be big enough in two years" is the most common argument for premature adoption and it is the weakest, because the coordination cost arrives immediately and the benefit arrives only if the growth does.


🏭 From the Pipeline — the mesh that was announced and the platform that was built

A three-hundred-person company announced a data mesh. Eight domains, each owning its data products, with a federated governance council and a self-serve platform.

Eighteen months later there were eleven domains, no platform, and forty-one pipelines.

What happened, in order:

The domains were declared and staffed immediately. Each got a data engineer, or half of one, or an analytics engineer who had volunteered.

The platform was scheduled for "phase two." It was the largest piece of work and the least visible, and it slipped.

So each domain built its own ingestion. Correctly — they had been told to own their data products — and eight teams independently wrote a watermark extractor, each with a different subset of §13.4's four lies handled.

The governance council met monthly and produced standards nobody could implement, because implementing them required the platform.

And the central data team, which had been the bottleneck, was now supporting eight bespoke stacks rather than one. Their workload went up.

The recovery took another year and it inverted the order:

text 1. build the platform FIRST -- ingestion, storage, orchestration, quality, a catalog -- as a paved road, not a mandate 2. migrate ONE domain onto it, and make that domain's life visibly better than the others' 3. let the others ask 4. THEN the governance council, which now has something to govern

Step 2 is the one that made it work. The first migrated domain shipped faster and got paged less, and the other seven asked to be next — which is a much stronger mechanism than a mandate and it required no council.

The lesson is §35.9's precondition, stated as a sequence rather than as a checklist: the self-serve platform is not one of four principles, it is the prerequisite for the other three. Decentralising ownership onto no platform does not distribute the work; it duplicates it, eight times, and the duplication is invisible for about a year.

🔁 Idempotency Check — a data product must be rebuildable by its domain

"Data as a product" has obligations, and the one that is never in the list is the one this book cares about: the domain must be able to rebuild its own product from its own inputs.

text obligation usually listed? testable? ───────────────────────────────────────────────────────────────────── discoverable yes yes addressable yes yes trustworthy (an SLO) yes yes self-describing (a schema) yes yes interoperable (standards) yes partially secure (access control) yes yes REBUILDABLE from its inputs NO yes -- and it is the one that proves the rest

A domain that cannot rebuild its product does not own it. It operates a pipeline whose history is somebody else's problem, and the first time a definition changes, the restatement is a request to the central team — which is exactly the bottleneck the mesh was adopted to remove.

Three consequences for an adoption:

The domain needs bronze, or access to it. Chapter 34's rebuild is what makes this possible, and a mesh where domains consume silver from a central team have decentralised the transformation and centralised the fidelity — which is a coherent architecture and is not a mesh.

The rebuild has to be affordable at the domain's scale. $198.96 for a whole platform is fine; the same cost per domain, eight times, on a quarterly cadence, is a number somebody should see before it is committed to.

And it has to be tested, not assumed. Exercise 35.15's readiness assessment should include it: "can each domain rebuild its products from raw, and has any of them tried?" It is the same question Chapter 34 asks of a platform, and a mesh is a platform with more owners.

📏 Scale Note — the team size at which each principle starts paying

The four principles do not arrive together, and adopting one early is usually harmless while adopting another early is expensive.

text principle starts paying at costs at Kestrel's size ───────────────────────────────────────────────────────────────────────── data as a PRODUCT immediately almost nothing -- an (owner, SLO, contract, owner, an SLO, and a discoverability) contract per dataset self-serve PLATFORM ~15-20 engineers, it IS the platform; or 2+ writing you have one already teams federated GOVERNANCE ~3+ domains with a council of one team conflicting needs is a meeting domain OWNERSHIP when a central team ACTIVELY HARMFUL: 4 (decentralised) is a bottleneck engineers cannot staff for domain KNOWLEDGE N domains

The top row is free and almost nobody does it. Every obligation of a data product — a named owner, a stated SLO, a contract, a way to find it — is available to a centralised team today, and Exercise 35.8 asks students to notice that. A four-person team that adopts data-as-a-product and nothing else has taken most of the available benefit.

The bottom row is the one that is harmful early, and it is the one people adopt first because it is the headline. Four engineers split across three domains is three part-time platform teams, and the operational load per person goes up rather than down (Chapter 5 §5.1).

The ordering that follows is §35.13's, and the scale note is why: product first, platform second, governance third, decentralisation last — because each is a prerequisite for the next and the last one is the only one that can make things worse.

The number to watch is not headcount. It is how often the central team says "we don't know enough about that domain to model it" — because that is the bottleneck the mesh addresses, and if it is not happening, decentralising addresses nothing.

🧪 Try It — score your own organisation, out loud, honestly

bash cd part-07-architecture-patterns/chapter-35-data-mesh/code python mesh_readiness.py --self-check # 102 assertions python mesh_readiness.py --assess my-org.yml

Fill in my-org.yml for your own organisation and score it. The four preconditions, honestly, with evidence rather than intention:

```text 1. IS THERE A BOTTLENECK, and is it DOMAIN KNOWLEDGE? evidence: time from request to delivered dataset, BY DOMAIN. A flat distribution means the bottleneck is capacity, not knowledge -- and a mesh addresses nothing.

  1. CAN A DOMAIN STAFF AND OPERATE A DATA PRODUCT? evidence: name the person who would carry the pager. Per domain. "We'd hire" is not evidence.

  2. DOES A SELF-SERVE PLATFORM EXIST? evidence: could a domain ingest, store, transform, test, and publish WITHOUT the central team writing code? Try it.

  3. IS THERE A GOVERNANCE FORUM WITH AUTHORITY? evidence: name a decision it has made that somebody disliked and complied with. ```

Expect to fail at least two. Chapter 35's central claim is that most organisations do, and a cohort or a team that all score highly is not scoring honestly — precondition 2's "name the person" and precondition 4's "name a decision" are the two that produce the honest answer, because both require a specific fact rather than an assessment.

Then do the more useful half: for the precondition you fail hardest, write what it would take. The answer is usually a hiring plan or a platform roadmap, not a technical design — and a submission that proposes a technical fix for an organisational precondition has misread the chapter.

And score it again in a year. The instrument is cheap to re-run and the trend is more informative than the score, because a mesh is a destination rather than a decision.

🧭 Version Note — the mesh discourse cooled, and the good parts stayed

Data mesh was announced in 2019, peaked in about 2022, and is now discussed more carefully — which is the normal life cycle and is worth knowing when you read anything about it.

text era the claim what survived ───────────────────────────────────────────────────────────────────────── 2019 four principles, as an the principles, and they are architecture good 2021 vendors named products "mesh" nothing; it was a platform with a new label 2022 every large org announced one the ones that built the platform first (§35.13) now "mesh" is used loosely to DATA AS A PRODUCT, which is the mean decentralised anything principle nearly everyone can adopt today for almost nothing

The durable contribution is the first principle, and it is the one that needs no reorganisation. An owner, an SLO, a contract, and a way to find it — available to a centralised four-person team, this quarter (§35.8, and the 📏 above).

The claim that did not survive contact is that decentralised ownership is the starting point. Every published recovery story inverts the order: platform, then one migrated domain, then the others ask (§35.13's 🏭).

And a vocabulary warning worth having. "We're doing a mesh" now means anything from a rigorous four-principle adoption to "each team has its own dbt project." When you meet the word, ask which of the four principles they mean — the answer is usually one, and it is usually the first, and that is fine as long as everyone knows.

🔐 Privacy & Governance — federated governance is the principle with legal consequences

Three of the four principles are about delivery. The fourth is about obligations, and it is the one where getting the decentralisation wrong is not merely inefficient.

text what MUST stay global what CAN be federated ───────────────────────────────────────────────────────────────────── the classification scheme applying it to a domain's own data the retention POLICY choosing a shorter retention the deletion MECHANISM -- operating it for their own products one manifest, generated the definition of a "data product" and its obligations what each domain publishes the lawful basis for processing -- nothing. This is not a domain decision.

The left column is not centralisation for its own sake. A per-domain classification scheme means a deletion request has N interpretations, and the organisation cannot say what it holds about a person — which is a regulatory obligation rather than a coordination problem.

The failure mode is specific and it is not hypothetical: each domain implements erasure for its own products, and nobody can enumerate the products. Chapter 31's argument — the manifest must be generated from classification tags — becomes strictly harder in a mesh, because the tags are applied by N teams to N schemas. Generating it requires the scheme to be global even when the tagging is not.

Two things to require of any adoption, both of which belong in the platform rather than in the council:

Classification is a platform primitive, not a per-domain convention. A domain says what its column is; the scheme says what the categories are.

And the deletion manifest is generated across all domains, by one job, from those tags. A per-domain manifest is a per-domain answer to a question that is asked of the organisation.

35.13 If You Do Adopt: The Order That Works

The four principles are usually adopted in the order they are listed, which is close to the worst order.

The listed order — domains first, then products, then platform, then governance — hands responsibility to teams before they have anything to do it with. They get ownership of a pipeline, no tooling, no standards, and a central team that is busy building the platform they were promised.

The order that works inverts it:

1. COMPUTATIONAL GOVERNANCE   automated standards, enforced in CI
                              cost: weeks. Needs no reorganization.
                              You are doing this anyway (Ch 23, 27, 30, 34).

2. DATA AS A PRODUCT          contracts, SLOs, owners, deprecation
                              cost: weeks. Central team applies it to itself.
                              This is where you discover which datasets
                              have no consumer.

3. SELF-SERVE PLATFORM        built for the capabilities people already ask
                              for, one at a time, measured by adoption
                              cost: quarters. START WITH THE 21-DAY ITEM.

4. DOMAIN OWNERSHIP           last, per domain, only when that domain has
                              an engineer and asks for it
                              cost: a reorganization.

Three reasons this order is better, and the third is the one that decides it:

Every step is independently valuable. Stop after step 1 and you have automated governance. Stop after step 2 and you have data products. Stop after step 3 and you have a better central team. Only step 4 requires the organization to change, and it is the only step whose value depends on the previous ones.

It builds the evidence for step 4. After step 3, you can measure whether domain teams use the self-serve tooling. A domain that adopts the tooling voluntarily is a domain ready to own a product; one that does not is a domain that would have been given a responsibility it did not want.

And it fails safely. The listed order fails by giving teams responsibility they cannot discharge, which damages trust in the platform team and in the idea. This order fails by leaving you with a well-governed central platform, which is where you started, only better.

📐 Design Decision — adopt domain ownership one domain at a time, and let it be voluntary

Big-bang adoption is the norm and it is wrong for a specific reason: it removes the evidence.

If all four domains onboard simultaneously and the result is poor, you cannot tell whether the mesh was wrong, the tooling was immature, or two of the four domains were not ready. Every hypothesis survives, so the retrospective produces a compromise rather than a conclusion.

One domain at a time, starting with the most autonomous one (§35.4's fan-out test picks it for you — Kestrel's would be inventory, with a fan-out of 1):

  • It is a real trial with a real outcome. Six months later you know whether it worked.
  • The failure is contained. One domain's data products, not four.
  • The second domain benefits from the first one's discoveries, which are always about the platform rather than about the domain.

And voluntary matters more than sequencing. A domain team that asks to own its data products has already made the trade against its own roadmap; one that is assigned them has not, and will prioritize accordingly — which shows up as a data product with a stale SLO and an unanswered support channel about four months in.

The uncomfortable implication: if no domain volunteers, that is your answer. It is a cheaper answer than eighteen months of transition, and it is the same evidence §35.3's queue analysis provides, gathered from the other end.

35.14 Summary

Data mesh answers a real problem — a central team becoming a queue at scale — and is adopted, at real expense, by organizations that do not have it.

🎓 Ask for evidence of the problem before discussing the solution. The queue, the wait times, the delegable share, the fan-out. Coordination problems are a function of headcount, so the first number is how many people you have.

🔎 Split the queue by wait time, not by volume. Kestrel: 386 requests a year, 61.4% delegable — and the delegable work already waits less than average (3.75 vs 3.84 days). The two slowest categories, source ingestion at 21 days and infrastructure at 14, are both undelegable: 8% of volume, most of the frustration, and a mesh does not touch them.

The biggest single category is "why is this number wrong" — 23% of all requests — which is a question about meaning that a catalog answers without anyone reorganizing.

⚠️ A partition is not a domain boundary. Any set of tables partitions into four groups; the edges decide. Kestrel: 40% of edges cross, and every proposed domain depends on at least one other — orders on three. You have not created autonomy; you have put a team boundary through the middle of a coordination requirement, which is worse than centralizing, because now it is between teams.

📏 fct_order_line is read by all four proposed domains, and every ownership answer is bad: orders becomes the new central team, platform owns the most important business table, or you get four definitions of revenue by design. A table four domains need means the boundary is in the wrong place — and the polyseme answer applies to a concept, not to a fact table 24 models deep.

💸 A self-serve platform is more work than doing the work. Ingestion: 2–5 days to build one, ~8 weeks to make it self-serve, breaking even in a bit over a year — and the payback requires domain teams that do not exist yet. Build self-serve for capabilities whose users already exist and are already asking.

⚠️ The platform team does not shrink. It changes job to something harder, must deliver before domain teams can act, and is therefore the bottleneck for its own removal. Kestrel estimated eighteen months of paying for both.

The cost, priced: 2.12× the data engineering headcount and $810,000 a year at Kestrel's size — and a multiple under 1.5 at forty engineers and twelve domains. The threshold is headcount, not architecture, and a mesh getting cheaper as you get bigger is a good sign that it is a real answer to a real problem.

🔎 Read a readiness score by principle, never in total. Kestrel's 15/32 hides the only fact that matters: every passed condition is something the central team built, and every failed one is something a domain team would have to do. Kestrel is 100% of the way to the parts a central team can build alone and 0% of the way to the parts that require an organization.

🧱 Two ideas transfer completely and require no reorganization. Data as a product — a known consumer, a contract, an SLO, an owner — and computational governance, standards enforced by automation rather than a review board, which is the best idea in the mesh and is orthogonal to it.

Kestrel adopted three things from a rejected proposal, in three weeks, and the output-port change is the one to copy: eleven ad-hoc extracts collapsed to three products, because the framing forced the question "who is the consumer, and is this the same product?"

⚠️ A mesh proposal is a reorganization proposal. The test: name the person who will approve a change to four teams' responsibilities, and confirm they have seen it. If you cannot, you are proposing tooling — which may be worth building, and should be funded and measured as tooling.

Conway's law points both ways. An organization that already has strong domain boundaries will drift toward a mesh whether or not it adopts one, and there the question is whether it is governed or accidental. Kestrel has one product line and one engineering team, so forcing a mesh would be fighting Conway's law rather than using it.

📐 Adopt the principles in reverse order: computational governance, then data as a product, then self-serve starting with the 21-day item, and domain ownership last, per domain, only when that domain has an engineer and asks for it. Every step is independently valuable, step 3 generates the evidence for step 4, and the failure mode is "a well-governed central platform" rather than "teams holding responsibilities they cannot discharge."

One domain at a time, and voluntary. Big-bang adoption removes the evidence — if it goes badly you cannot tell which hypothesis was wrong. And if no domain volunteers, that is your answer, arrived at far more cheaply than through eighteen months of transition.

Four conditions for a real mesh: domain engineers with capacity · a measured bottleneck in delegable work · domains that are separable · a platform team that survives the transition. And "we'll be big enough in two years" is the weakest argument for adoption, because the coordination cost arrives immediately and the benefit only if the growth does.

Chapter 36 is event-driven architecture — designing around events rather than tables, and the architectural end state that Chapters 14, 15, and 29 have been approaching from different sides.


Key terms: data mesh · domain-oriented ownership · data product · output port · self-serve platform · federated computational governance · bounded context · autonomy · coupling · fan-out · polyseme · Conway's law