Distributed Transactions

Dual-write problem: No single transaction spans a database and a message broker. Crash between DB write and broker publish → inconsistency.

Outbox pattern: Write event to outbox table in the same DB transaction as the state change. A separate publisher reads the outbox and publishes to the broker, then marks entries published. Guarantees at-least-once delivery; consumers must be idempotent.

sequenceDiagram participant Co as Coordinator participant Inv as Inventory participant Pay as Payment Note over Co,Pay: 2PC — locks held across the WHOLE exchange Co->>Inv: prepare (lock) Co->>Pay: prepare (lock) Inv->>Co: vote yes Pay->>Co: vote yes Co->>Inv: commit Co->>Pay: commit Note over Co,Pay: SAGA — each step commits immediately, locally Co->>Inv: reserve stock (commits immediately, no lock held) Co->>Pay: charge card (commits immediately) Note over Co,Pay: if Pay fails: run compensating action on Inv, not a rollback

2PC: Phase 1 (Prepare): coordinator asks all participants to lock and prepare. Phase 2 (Commit/Abort): if all voted yes, commit; else abort. Problem: blocking — if coordinator crashes after Phase 1, participants hold locks indefinitely. Single point of failure. Long-held distributed locks kill throughput.

SAGA: Decompose into local transactions, each with a compensating transaction. Orchestration: central coordinator drives steps, triggers compensation on failure. Choreography: events drive reactions across services.

Trade-offs vs 2PC: no distributed locks, eventual consistency (not strong), compensation must be idempotent, write-skew window exists during saga execution.

Worked example — order placement as a SAGA: placing an order touches three services: inventory (reserve stock), payment (charge the card), order (mark confirmed). Orchestration-style: an order-saga coordinator calls inventory to reserve 1 unit of item X (local transaction, commits immediately); then calls payment to charge $50 (local transaction, commits immediately); then calls order to mark the order confirmed. If the payment call fails (card declined), the coordinator runs the compensating transaction for the already-committed inventory step — release the reservation on item X — rather than rolling back a distributed transaction, because there never was one; each step was already durably committed on its own. The system is briefly inconsistent while this unwinds (item X shows as reserved-but-order-not-confirmed for a few hundred milliseconds), which is the eventual-consistency trade-off named above. Contrast with 2PC: a 2PC coordinator would hold locks on the inventory row for the entire multi-service round trip, so if the payment service is slow, the inventory lock — and every other order trying to reserve stock on that same row — is blocked for as long as payment takes to respond.

Worked example — the outbox pattern closing the dual-write gap: an order service needs to both update its orders table to status = 'confirmed' and publish an OrderConfirmed event to a message broker. Writing directly to the broker after committing the DB transaction has a gap: if the process crashes between the commit and the publish, the order is confirmed in the database but the event never fires, and nothing downstream (shipping, notifications) ever finds out. The outbox pattern instead inserts a row into an outbox table — same transaction, same commit — containing the event payload. A separate poller reads unpublished outbox rows, publishes each to the broker, and marks it published. If the process crashes right after the DB commit but before publishing, the outbox row is still there on restart; the poller picks it up and publishes it late, but it does get published. The row acts as a durable, transactional "to-do" that survives the crash the direct dual-write couldn't.