Appendix I: The Kestrel Platform

The complete inventory of what the book builds, chapter by chapter, so you can check what you have and find what you are missing.

⚠️ This appendix contains the Chapter 38 reconciliation figures. It is one of the four places validate.py permits them. Do not read §I.9 before computing the capstone yourself.

I.1 The Repository

kestrel-platform/
├── docker-compose.yml            App. A
├── .env                          gitignored
├── requirements.txt
├── platform/
│   ├── ingest/                   ch 13-16
│   │   ├── load_orders.py            batch + high-water mark
│   │   ├── load_customers.py         full refresh
│   │   ├── cdc_replay.py             ch14
│   │   ├── api_client.py             ch16: retries, backoff, pagination
│   │   └── timestamp_audit.py        ch34 CS2: flags offsetless timestamps
│   ├── contracts/                ch17
│   │   ├── orders.yml
│   │   └── check_contracts.py
│   ├── quality/                  ch23
│   │   ├── register.py               22 assertions
│   │   └── quarantine.py             ch34 section 34.12
│   ├── layers/                   ch34
│   │   ├── layer_check.py            9 rules, in CI
│   │   ├── suppressions.yml          expiring, with reasons
│   │   ├── replay.py                 rebuild + cost
│   │   └── depth_report.py
│   ├── events/                   ch36
│   │   ├── outbox_relay.py
│   │   ├── consumer.py               idempotent base class
│   │   ├── upcast.py
│   │   └── projections/
│   ├── ml/                       ch32
│   │   ├── asof.py                   ONE implementation
│   │   ├── materialize.py            changed-only + weekly full
│   │   ├── skew_check.py             daily, by segment
│   │   └── feature_age.py
│   ├── privacy/                  ch31
│   │   ├── scan.py
│   │   ├── manifest.py               GENERATED from catalog tags
│   │   ├── resolver.py               5 identifier spaces
│   │   └── dags/erasure.py
│   ├── governance/               ch30
│   │   ├── catalog_audit.py
│   │   ├── access_review.py
│   │   └── metric_audit.py           the quarterly grep
│   ├── cost/                     ch33
│   │   ├── rates.yml                 ONE source of truth
│   │   ├── estimate.py
│   │   ├── pr_cost_bot.py
│   │   └── waste.py
│   ├── finance/                  ch38
│   │   ├── rules.md                  the four rules, with owners
│   │   └── reconcile.py
│   ├── migration/                ch37
│   │   ├── inventory.yml
│   │   ├── reconcile/
│   │   └── rollback/
│   └── ci/
│       └── purity.py                 ch38 CS2
├── dbt/                          ch 18-20
│   ├── dbt_project.yml
│   └── models/{staging,marts}/
├── dags/                         ch24
├── infra/                        ch28
└── scripts/
    ├── validate.py
    ├── xref_audit.py
    └── wordcount.py

I.2 Ingestion (ch 13–16)

load_orders.py — incremental by high-water mark, with the four ways updated_at lies documented inline and a reconciliation against source row counts.

cdc_replay.py — Debezium-shaped records into bronze, with the (order_id, updated_at, cdc_lsn) dedup and the deterministic tie-break that Chapter 38 Case Study 2 shows the cost of omitting.

api_client.py — exponential backoff with jitter, cursor pagination, a circuit breaker, idempotency keys, and terminal-versus-retryable error classification.

timestamp_audit.py — flags any timestamp arriving without an offset. 14 of Kestrel's 41 external timestamp columns; nine remain unresolved, which is correct.

I.3 Contracts and Quality (ch 17, 23)

contracts/orders.yml — schema, semantics, guarantees, and a deprecation window. Consumer-driven: the producer's CI fails when a consumer's assertion breaks.

quality/register.py — 22 assertions across validity, completeness, consistency, timeliness, and uniqueness, each with a severity and an owner. Plus the two added later: an input-volume band (ch38 §38.13) and a ceiling on every published count (ch30 CS1).

quality/quarantine.py — rejected rows with the assertion that rejected them, a replayable _batch_id, a paging size limit, and a 30-day retention that forces the drain.

I.4 Transformation (ch 18–22)

models/staging/          stg_orders, stg_order_lines, stg_customers, stg_events
models/marts/core/       dim_customer (SCD2), dim_product, dim_date, fct_order_line
models/marts/finance/    daily_revenue, fin_margin

fct_order_line is incremental with a 90-day lookback, because refunds settle late and a closed month must be restatable (ch38 §38.5).

dim_customer is SCD Type 2 with valid_from/valid_to/is_current, and it doubles as a point-in-time-correct feature table (ch32).

I.5 Orchestration and Delivery (ch 24, 27, 28)

dags/kestrel_daily.py — the 6am SLA path: ingest → sessionize → dbt build → quality → reconcile. Every task keyed on the logical date, max_active_runs=1, deferrable sensors, and an alert on remaining SLA slack below 45 minutes rather than on the breach.

CI — state:modified+ against a stored production manifest, layer_check, purity.py, the cost bot, and the sealed-figure check.

infra/ — Terraform with remote state, separated by blast radius, OIDC for CI, and a nightly drift plan that now includes grants (ch30 CS2).

I.6 Governance, Privacy, Cost (ch 30, 31, 33)

catalog_audit.py — 22 checks; the demo fixture yields 11 findings.

privacy/manifest.py — generated from classification tags, not written. A new table carrying customer_id joins the deletion manifest automatically and fails verification loudly if it has no mechanism. This is the highest-leverage single design in the book.

cost/rates.yml — the frozen rate card: $2.400`/node-hour, `$0.023/GB-month, $2.00/credit, $6.25`/TiB, `$0.09/GB egress.

I.7 The Verification Layer

Four things that produce no data and gate everything:

python platform/layers/layer_check.py --check       # 9 rules; must be 0 blocking
python platform/privacy/manifest.py --verify        # every customer_id table has a path
python platform/ci/purity.py                        # no network, now(), or random in a model
python platform/layers/replay.py --from bronze --into scratch_verify

The last one costs $198.96 and four hours, and has failed three times in two years.

I.8 Line Counts

Under twelve thousand lines for a platform that ingests 2.4M orders and 5.1 billion events a year, reconciles to the cent, and can be rebuilt from raw for $198.96. Most of it is assertions.

I.9 The Capstone Reconciliation

Stop here if you have not computed it yourself. Chapter 38 §38.6.

November 2026, as of 2026-12-01:

                                  lines              cents
──────────────────────────────────────────────────────────────
source, all rows                856,117     $24,102,447.18
  less R1 test orders           854,705     $24,061,239.18
  less R2 cancelled             811,798     $22,856,356.83
  less R3 gift cards            781,380     $21,945,202.00
  less R4 refunds               781,380     $20,430,983.06
──────────────────────────────────────────────────────────────

GOLD LINES                 781,380
GOLD GROSS REVENUE         $21,945,202.00
REFUNDS                    $1,514,218.94
GOLD NET REVENUE           $20,430,983.06

The four rules, with owners:

Verification, and this is the part that matters more than the closing:

781,380 lines / (6,480,000 / 12)     = 1.447x
$21,945,202 / ($182,000,000 / 12)    = 1.447x
$21,945,202 / 781,380                = $28.085 per line
Chapter 1: $75.83 AOV / 2.70 lines   = $28.084 per line

Two independent ratios agreeing to three decimals, and a per-line figure matching Chapter 1's independently-derived one to four significant figures — neither of which the reconciliation forced.

capstone.py --self-check asserts all of it: 38 checks.

I.10 Getting the Code

Every chapter's code/ directory is in the repository alongside its prose. They are standalone, dependency-free, and deterministic.

# verify everything
python scripts/validate.py
python scripts/xref_audit.py
for f in part-*/chapter-*/code/*.py; do
    echo -n "$(basename $f): "; python "$f" --self-check | head -1
done

Expected: 0 errors, 0 broken references, and every self-check passing. A failure on an unmodified file is a bug worth reporting; a failure after you have changed a fixture is the file working as designed.