Affiliate disclosure
Book titles on this page link to Amazon. As an Amazon Associate, DataField.Dev earns from qualifying purchases — at no additional cost to you.
Further Reading: Event-Driven Architecture
Sources are tagged Tier 1 (confident it exists, recommended without reservation) or Tier 2 (real and worth seeking, but confirm the current edition, version, or URL yourself).
A note on this literature. It is unusually good, unusually old, and unusually scattered. The best material predates the current vocabulary by a decade and comes from the domain-driven design and enterprise-integration communities rather than from the data community — which is why data engineers often reinvent it. The other half is Martin Fowler's site, which is free and better than most books.
Start here
-
Martin Fowler, "What do you mean by 'Event-Driven'?" (2017). Short, free, and it is §36.2 — the distinction between event notification, event-carried state transfer, event sourcing, and CQRS. If you read one thing from this chapter's list, read this, because the confusion it clears up is the source of most over-engineering in this territory. Tier 1.
-
Martin Fowler, "Event Sourcing" (2005) and "CQRS" (2011). The canonical write-ups. The event sourcing piece is honest about the costs in a way most later material is not, and it is twenty years old, which is worth noticing. Tier 1.
-
Martin Kleppmann, Designing Data-Intensive Applications (O'Reilly, 2017), Chapter 11. Streams, logs, change capture, and the relationship between a stream and a table. §36.4's "state is a fold" is Kleppmann's stream-table duality, argued more rigorously. Tier 1.
-
Jay Kreps, "The Log: What every software engineer should know about real-time data's unifying abstraction" (2013). The essay that made "the log is the source of truth" a mainstream idea, and it is still the clearest statement of it. Read it after Fowler's taxonomy, so you can place it. Tier 1 for the ideas; Tier 2 for the URL.
On the outbox and the dual write
-
Chris Richardson's microservices.io patterns, specifically Transactional Outbox, Polling Publisher, and Transaction Log Tailing. The most precise available statement of §36.5, with the trade-offs enumerated. Free, short, and the diagrams are the ones to copy into a design doc. Tier 1.
-
The Debezium documentation on the outbox event router. The CDC-on-the-outbox pattern Kestrel adopted, documented by the tool that implements it. Read the section on why the outbox table's schema should differ from your business tables — it is §36.8's argument, made by the CDC vendor, which should tell you something. Tier 1 — versioned.
-
Anything careful on idempotent consumers and effectively-once processing. The Kafka documentation's transactions section and KIP-98 are the primary sources; Chapter 29 §29.6 of this book is the honest summary of what the guarantee actually covers. Tier 2 for the KIPs.
On designing events
-
Eric Evans, Domain-Driven Design (Addison-Wesley, 2003), on aggregates and domain events. An aggregate is the thing §36.1's version number is scoped to, and if you cannot identify your aggregates, §36.13's second disqualifying condition applies. Tier 1.
-
Vaughn Vernon, Implementing Domain-Driven Design, the chapters on domain events and event sourcing. More concrete than Evans and directly about the design question §36.8 poses. Tier 1.
-
Alberto Brandolini's material on Event Storming. A workshop technique for discovering events with domain experts, and the fastest way to find out whether your events are past-tense and whether anybody agrees on what they mean. Half a day, a wall, and sticky notes. Tier 2 — mostly talks and a leanpub book.
-
Greg Young's talks and writing on event sourcing. The most experienced practitioner voice, and the most willing to say when it is wrong, which is §36.13. His "8 Lines of Code" and versioning material are the useful ones. Tier 2 — conference talks; find current recordings.
On ordering and commutativity
-
Anything on CRDTs (conflict-free replicated data types). §36.9's commutativity analysis is the entry-level version of a rigorous field. You do not need CRDTs to apply the lesson, and knowing they exist tells you the property has a name and a theory: a projection whose reducer forms a commutative monoid needs no ordering, ever. Tier 1 — Shapiro et al.'s papers; a survey is enough.
-
The Kafka documentation on partitioning and ordering guarantees. Read the exact wording: Kafka guarantees order within a partition, which is per-key ordering and not global ordering, and Case Study 2 turns on that distinction. Tier 1 — versioned.
-
Lamport, "Time, Clocks, and the Ordering of Events in a Distributed System" (1978). The foundation. Read it once — it is nine pages, and the idea that ordering is a property you construct rather than discover is what makes §36.9's version numbers obvious rather than clever. Tier 1.
On replay, schema, and retention
-
Greg Young, Versioning in an Event Sourced System (Leanpub). The definitive treatment of §36.10, including upcasting, and honest about the fact that this is the hard part. Tier 2 — leanpub, and worth confirming availability.
-
The Confluent Schema Registry documentation on compatibility modes.
BACKWARD,FORWARD, andFULL, and which one lets an old consumer read a new event. §36.10's "additive changes only" isBACKWARDcompatibility, named. Tier 1. -
The Kafka documentation on log compaction. Read it specifically to understand why compaction and event sourcing are incompatible on the same topic (§36.12) — compaction keeps the latest record per key, which for an event-sourced aggregate discards the history that is the data. Tier 1.
-
Chapter 31 §31.5 of this book on crypto-shredding, and the NIST guidance behind it. §36.12's erasure problem is Chapter 31's problem in a store that cannot be updated.
On the parts that are not the pattern
-
Chapter 23 §23.11 of this book, and Case Study 1. Reconciliation independence is the transferable lesson from this chapter and has nothing to do with events: two numbers derived from a common ancestor agree on everything the ancestor got wrong.
-
Anything on the "outbox vs. CDC vs. dual write" decision written by someone who has run all three. These are rarer than they should be. Look for write-ups that give a loss rate, which is the number §36.5 measures and which almost nobody publishes.
Practice
-
code/event_lab.pyin this chapter. The dual write measured against the outbox, four projections including two retroactive, the shuffle test, and the storage comparison. Thirty-nine self-checks. Exercise 36.11's snapshot assertion — snapshot plus tail equals full fold — is the most valuable addition you can make to it. -
Search your codebase for dual writes. Exercise 36.4. A database write followed by a publish, an HTTP call, or a queue send, outside a transaction. Estimate the loss and check whether any reconciliation would detect it.
-
Sort your projections by commutativity. Exercise 36.9. An afternoon, and it may save two quarters.
-
Replay something. Exercise 36.10. It will fail, and the failure is the point.
A note on what to be skeptical of
Any claim that event sourcing gives you an audit log for free. It gives you a technical history of state changes. An audit log needs actor, reason, and intent — which are only there if you designed the events to carry them (§36.8), and a CDC-derived stream never does.
Any comparison that leads with storage. §36.11: 10× a rounding error is a rounding error, and the argument you actually need to make is about engineering time.
"Exactly-once." Chapter 29 §29.6 and §36.5: the outbox gives at-least-once, and the consumer makes it effectively-once. A vendor claiming exactly-once is describing a scope you need to read carefully.
And any event-sourcing advocacy that does not mention replay having failed. Everyone's has. A write-up that only describes replay working has not exercised it — which is §36.10's whole warning, and it applies to the writing as much as to the system.