Chapter 12 — Key Takeaways (NoSQL and Specialized Stores)

The page for an adoption proposal, and for the day you have to extract from one of these.

Why platforms accumulate stores

Nobody chooses six databases; everybody ends up with six. Each arrives as a reasonable answer to a real problem, by someone under delivery pressure. Nobody counts. Chapter 5 §5.1's two-systems-per-engineer is the count.

The one honest reason to add a store: an access pattern the existing store serves badly enough to matter — measured.

The three bad reasons: "Postgres can't do that" (frequently it can) · "it'll scale better" (measure — nine months for two seconds) · "the team knows it" (real, and a bad primary reason).

The classes

DynamoDB is not quite key-value: partition key + sort key gives range queries within a partition, so it has wide-column capability and wide-column modelling discipline.

The extraction table — consult BEFORE adopting

Three rules: 1. Extract from the source of truth, not from a derived store. 2. A store with no change feed will be full-scanned. Price that at adoption — it misses deletes. 3. When the store cannot tell you what changed, make the application tell you (outbox, Ch. 36 §36.4).

Time-series: cardinality, not volume

http_requests{method, status, endpoint}
  4 × 6 × 200                          =         4,800 series   fine
+ customer_id (1.9M)                   = 9,120,000,000 series   OOM in 11 minutes

Never put an unbounded identifier in a label. customer · session · order · request · email · URL · error message. The person who does it is always trying to improve observability.

Kestrel's clickstream is not time-series data, however many timestamps it has.

Defenses: sample_limit per target (a failed scrape is loud and harmless) · alert on series growth rate (a step change is always a deploy) · CI lint on label names with an exception list · know how to drop a metric without deploying · a tiny external monitor, because you cannot debug your monitoring with your monitoring.

Vector stores are a pipeline

1. Embeddings must be generated and kept fresh — that is an incremental-processing problem.
2. The model is a versioned dependency. Vectors from two versions are not comparable.
3. The vector is not the answer — the join to real data is.

⚠️ Absence does not raise an exception. 6,000 products silently missing from every similarity search for five weeks: no error, no count change, and nobody watching for what did not appear.

Three defenses:

-- 1. Version the INDEX, not the rows: build alongside, assert, flip.
-- 2. Store model_version and source_hash WITH the vector.
-- 3. Coverage assertion -- the nine words that catch it:
SELECT COUNT(*) FROM products p WHERE p.active
   AND NOT EXISTS (SELECT 1 FROM embeddings e
                    WHERE e.product_id = p.product_id
                      AND e.model_version = :current);   -- expect 0

pgvector is sufficient under ~10M vectors with moderate query volume, especially when you join to relational data. Kestrel has 47,000.

The three most over-adopted, and the test

Absorb before you adopt — what PostgreSQL already does

None is as good as the specialist. All are good enough for longer than people expect — and each one avoided is a system you do not operate, monitor, upgrade, secure, extract from, or staff a second operator for.

Two diagnostics worth stealing

GROUP BY the property you assume is uniform:

SELECT vector_dims(embedding), COUNT(*) FROM embeddings GROUP BY 1;
SELECT length(sku),            COUNT(*) FROM products   GROUP BY 1;

Finds the class of problem where the schema permits variation you did not intend — no type check catches it, because the type is satisfied.

A job may be partially successful; it may not be quietly partially successful. Quarantine with a reason, and exit non-zero.