Transactional Outbox vs Sagas: Keeping FastAPI Services Honest When Transactions Cross Boundaries

Here’s the failure mode that eventually finds every growing FastAPI service: a request handler opens a database transaction, commits an order row, and then calls publish("order.created") against a message broker. The commit succeeds. The publish call times out because the broker had a two-second hiccup. The database thinks the order exists. Nothing downstream — billing, fulfillment, notifications — ever hears about it.

That’s the dual-write problem, and it’s structural, not a bug you can fix by retrying harder. Any time you write to two different systems (a database and a message broker, or a database and an external API) and they don’t share a transaction, you will eventually get one write without the other. The transactional outbox pattern exists specifically to close this gap for the common case: a single service needs to update its own data and reliably tell the outside world about it.

This article covers what the outbox pattern actually is, how to build it in FastAPI and PostgreSQL today, how it differs from sagas, and a decision framework for choosing between them — because they solve related but distinct problems, and reaching for the wrong one is how teams end up building distributed transaction machinery they didn’t need.

The Direct Answer

Use a transactional outbox when one service needs to atomically commit a state change and guarantee that an event describing it eventually reaches the outside world. Use a saga when a business process spans multiple services, each with its own database, and you need to coordinate a sequence of steps with defined rollback behavior if a later step fails.

The outbox pattern is about reliable delivery of a single event from a single transactional boundary. A saga is about coordinating multiple services, each of which might use an outbox internally to publish its own step’s event reliably. In production systems, you often see both together — the outbox is the cell, the saga is the organism.

Why You Can’t Just “Also Publish” After Commit

To see why this needs a pattern instead of effort, it helps to be precise about what a database transaction actually guarantees. ACID — atomicity, consistency, isolation, durability — is a promise scoped entirely to your database engine. PostgreSQL’s write-ahead log ensures that once a transaction commits, its effects survive a crash, and its isolation level controls what concurrent transactions can see of each other’s uncommitted work via MVCC (multi-version concurrency control). None of that machinery extends past the database connection. A message broker, a webhook endpoint, a third-party API — these are outside the blast radius of your COMMIT, so there’s no way to make “write the row” and “send the message” succeed or fail together using transactional guarantees alone.

The naive fixes don’t hold up:

Publish before commit: you might notify the world about something that gets rolled back a moment later.
Publish after commit, synchronously, in the request path: slow or down brokers now determine your API latency and error rate, and a crash between commit and publish still loses the event.
Fire-and-forget background task: if the process dies before the task runs, the event is gone, with no record it was ever supposed to happen.

What you actually want is to make the event part of the same database transaction as the business write, since that’s the one place you already have atomicity.

The Transactional Outbox Pattern

The outbox pattern does exactly that: instead of publishing to the broker directly, you write the event into an outbox table in the same PostgreSQL transaction as your business data change. Since it’s the same transaction, either both rows exist or neither does — you’ve turned a cross-system dual write into a single-system write, which PostgreSQL’s ACID guarantees already cover. A separate process — the relay or poller — reads unpublished rows from the outbox table and pushes them to your broker, marking them published once the broker acknowledges receipt.

This is broker-agnostic by design. The relay can push to Kafka, a generic message queue, a Redis Pub/Sub channel, or an event bus — the pattern doesn’t care, because the durability guarantee comes from the outbox table, not from the broker. Some teams implement the relay using change data capture (CDC) against the PostgreSQL write-ahead log instead of polling, but a polling relay is simpler to reason about and perfectly adequate for most workloads that aren’t pushing extreme event volume. This article builds the polling version because it’s the one you can ship this sprint without new infrastructure.

Outbox Table Schema

CREATE TABLE outbox_events (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    aggregate_type  TEXT NOT NULL,       -- e.g. 'order'
    aggregate_id    TEXT NOT NULL,       -- e.g. the order's UUID
    event_type      TEXT NOT NULL,       -- e.g. 'order.created'
    payload         JSONB NOT NULL,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at    TIMESTAMPTZ,
    attempts        INT NOT NULL DEFAULT 0,
    last_error      TEXT
);

CREATE INDEX idx_outbox_unpublished
    ON outbox_events (created_at)
    WHERE published_at IS NULL;

The partial index matters more than it looks: the relay only ever scans unpublished rows, and on a busy table that index stays small regardless of how many events have already shipped.

Writing to the Outbox Inside the Business Transaction

Using FastAPI with SQLAlchemy’s async engine, the key discipline is that the outbox insert happens on the same AsyncSession as the business write, inside the same transaction block:

from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import insert
import json

async def create_order(session: AsyncSession, order_data: dict) -> str:
    async with session.begin():  # single transaction boundary
        order = Order(**order_data)
        session.add(order)
        await session.flush()  # populate order.id without committing

        session.add(OutboxEvent(
            aggregate_type="order",
            aggregate_id=str(order.id),
            event_type="order.created",
            payload=json.dumps({
                "order_id": str(order.id),
                "customer_id": order.customer_id,
                "total_cents": order.total_cents,
            }),
        ))
    # commit happens automatically at the end of the `async with` block
    return str(order.id)

session.flush() is doing real work here — it sends the INSERT to PostgreSQL so order.id is available for the outbox payload, without ending the transaction. If anything after this point raises, both inserts roll back together; if the async with session.begin() block exits cleanly, both commit together. This is the entire trick. There’s no distributed coordinator, no two-phase commit, just a single transaction doing double duty.

The Relay / Poller

The relay is a separate async loop — a standalone worker process, not part of the request/response cycle — that polls for unpublished rows, attempts delivery, and marks success or records failure:

import asyncio
from sqlalchemy import select, update

async def relay_loop(session_factory, broker_publish, poll_interval=1.0):
    while True:
        async with session_factory() as session:
            async with session.begin():
                result = await session.execute(
                    select(OutboxEvent)
                    .where(OutboxEvent.published_at.is_(None))
                    .order_by(OutboxEvent.created_at)
                    .limit(100)
                    .with_for_update(skip_locked=True)
                )
                events = result.scalars().all()

                for event in events:
                    try:
                        await broker_publish(
                            topic=event.event_type,
                            key=event.aggregate_id,
                            value=event.payload,
                            idempotency_key=str(event.id),
                        )
                        event.published_at = func.now()
                    except Exception as exc:
                        event.attempts += 1
                        event.last_error = str(exc)[:500]

        await asyncio.sleep(poll_interval)

SKIP LOCKED is what makes this safe to run as multiple relay instances for throughput or failover — competing workers simply skip rows someone else already has locked instead of blocking on them. This is the same mechanism you’d reach for building any PostgreSQL-backed job queue, and it holds up well at moderate volume.

At-Least-Once Delivery and Why Idempotency Isn’t Optional

The relay can crash, be killed mid-loop, or succeed at publishing but fail before marking the row published. In every one of those cases, the safe failure mode is to retry — which means the same event can be delivered more than once. This is why the outbox pattern, like essentially every reliable messaging system, gives you at-least-once delivery, not exactly-once. Exactly-once delivery across a network boundary is not something you can honestly promise; you can only get exactly-once effect by making delivery idempotent on the consumer side.

Concretely, that means every consumer of these events needs to deduplicate. The event’s id (the outbox row’s UUID) is a natural idempotency key — consumers check a deduplication store (a table, a Redis set with a TTL, whatever fits the stack) before acting on an event, and skip it if that key has already been processed.

async def handle_order_created(event_id: str, payload: dict, session: AsyncSession):
    inserted = await session.execute(
        insert(ProcessedEvents)
        .values(event_id=event_id)
        .on_conflict_do_nothing()
    )
    if inserted.rowcount == 0:
        return  # already handled, skip
    # ... do the actual work

This single guard is what lets you retry freely, which in turn is what lets the relay be simple. Don’t skip it to save a table.

Retries, Backoff, and Dead Letters

A relay that retries forever on a permanently broken message (bad payload, deleted downstream resource) will loop pointlessly and hide the fact that something needs human attention. The standard shape is: retry with exponential backoff and jitter to avoid thundering-herd retries against a struggling broker, cap the attempts, and after the cap route the event to a dead-letter queue — a separate table or topic — for manual inspection rather than silently dropping it.

MAX_ATTEMPTS = 8

if event.attempts >= MAX_ATTEMPTS:
    event.dead_lettered_at = func.now()
else:
    backoff = min(2 ** event.attempts + random.uniform(0, 1), 300)
    event.next_attempt_at = func.now() + timedelta(seconds=backoff)

Treat dead-lettered rows as an alerting signal, not a cleanup chore — a growing dead-letter count is usually the earliest indicator that a downstream consumer or broker has degraded.

Observability: Watching the Relay, Not Just the App

An outbox relay is invisible infrastructure until it’s broken, which is exactly why it needs explicit monitoring rather than being assumed healthy. At minimum, track: the age of the oldest unpublished row (a growing lag means the relay is falling behind or stuck), the dead-letter queue depth, and per-event-type publish latency and error rate. Wiring these into whatever you already use for error tracking and metrics — Sentry for exceptions in the relay loop, a metrics/tracing stack for the lag and throughput numbers — turns “did the outbox work” from a mystery into a dashboard. If you’re also running background job queues elsewhere in the stack, the same instrumentation discipline applies: queue depth and oldest-pending-item age are the two numbers that tell you whether a worker is keeping up.

Sagas: When the Problem Is Bigger Than One Service

The outbox solves reliable publishing at one boundary. It does not tell you what to do when a business operation needs several services to each do their part, and a later step can fail after earlier steps already committed. That’s the saga pattern’s job.

A saga breaks a cross-service business transaction into a sequence of local transactions, each owned by one service, with a compensating transaction defined for each step to undo its effect if a later step fails. There is no global lock and no shared transaction — each step commits independently, and if step four fails, you run the compensations for steps three, two, and one in reverse, rather than rolling back a transaction that never existed across service boundaries.

There are two ways to coordinate the sequence:

Choreography has each service publish an event when its step completes, and the next service in line reacts to that event to perform its own step — fulfillment listens for payment.confirmed, notifications listen for order.shipped, and so on, with no central coordinator. This is simple to start and keeps services decoupled, but as the number of steps grows, the overall workflow logic is scattered across every participating service’s event handlers, which gets hard to audit or change confidently.

Orchestration introduces a central coordinator that explicitly calls each service in sequence, tracks which step the workflow is on, and decides what compensation to trigger on failure. This centralizes the workflow logic into one place you can read top to bottom, at the cost of that orchestrator becoming a dependency every participating service now has.

	Choreography	Orchestration
Coordination	Each service reacts to events from others	Central coordinator drives each step explicitly
Workflow visibility	Scattered across services’ event handlers	Centralized in one orchestrator
Coupling	Services decoupled from each other	Services coupled to the orchestrator’s contract
Best fit	Few steps, stable workflow	Many steps, workflow changes often, needs auditability

Either way, each step in a saga is naturally modeled as a state machine: pending, completed, compensating, compensated, failed. Persisting that status per step — not just inferring it from logs — is what lets you recover a saga after a crash instead of guessing where it left off. And each step that publishes an event to trigger the next one should itself use an outbox, so step completion and event publication for that step stay atomic — this is the point where the two patterns compose rather than compete.

Two-Phase Commit: Why Sagas Won Instead

It’s worth naming the alternative sagas replaced: two-phase commit (2PC), where a distributed transaction coordinator asks every participant to “prepare” (lock resources, confirm it can commit) and only then tells everyone to “commit” together. 2PC gives you a real atomic cross-service transaction, which sounds strictly better than compensating actions — until you account for what it costs. Every participant holds locks for the full duration of the coordinator’s round trip, which means one slow or down service stalls every other participant mid-transaction, and the coordinator itself becomes a single point of failure for every transaction that touches it. Sagas trade that strict atomicity for availability and service independence: each step commits immediately and locally, and failures are handled after the fact via compensation rather than by blocking everyone upfront. In a microservices setup where services are deployed and scaled independently, that tradeoff is almost always the right one — it’s the same CAP-theorem-flavored tension between consistency and availability that shows up throughout distributed systems design, and most teams building on independently deployed services choose eventual consistency over blocking atomicity.

Durable Execution Engines: The Third Option

If you find yourself hand-rolling saga state machines with retry logic, compensation tracking, and crash recovery across several services, it’s worth knowing there’s a class of tools built exactly for this: durable execution engines such as Temporal or Inngest, which let you write a workflow as ordinary-looking code while the engine persists progress after every step. If the process crashes mid-workflow, execution resumes from the last completed step rather than from the beginning, which handles a lot of the failure-handling plumbing a hand-rolled saga orchestrator would otherwise need to reimplement. For a single FastAPI service publishing its own events, this is overkill — the outbox table is simpler and sufficient. For a saga with many steps, long-running human-in-the-loop stages, or frequent workflow changes, a durable execution engine can remove an entire category of hand-rolled bugs — it’s a build-vs-adopt decision worth making deliberately rather than defaulting into years of custom orchestrator maintenance.

Which One When: A Decision Framework

Situation	Reach for
One service needs to reliably publish an event after a DB write	Transactional outbox
A business process spans multiple services, each needing local commit + possible undo	Saga (choreography or orchestration)
Few steps, workflow is stable, teams own their own reactions	Saga with choreography
Many steps, workflow logic changes often, needs central auditability	Saga with orchestration
Saga has grown complex enough that crash recovery and retries are themselves a project	Durable execution engine
You need all-or-nothing atomicity across services and can tolerate blocking	2PC (rare in practice for this reason)

Closing Judgment

Start with the outbox. It’s a table, an index, and a loop — no new infrastructure, and it directly closes the dual-write hole that caused you to search for this in the first place. Reach for a saga only once a single business operation genuinely spans services with independent databases, and reach for a durable execution engine only once the saga’s own bookkeeping becomes the hard part. Building these in the wrong order — orchestrating sagas across services that don’t exist yet, or adopting a workflow engine before you’ve outgrown a polling relay — is how teams end up maintaining distributed systems machinery for a problem they didn’t actually have.

FAQ

Does the outbox pattern replace my message broker? No. The outbox guarantees the event gets created reliably and eventually sent; the broker still handles delivery to consumers, fan-out, and routing. The outbox and the broker solve different halves of the reliability problem.

Can I use the outbox pattern without a message broker at all? The pattern doesn’t require a broker: consumers could in principle read the outbox table directly (or a read-only replica of it) rather than subscribing to published events. The trade-off is that you lose push-based delivery and broker fan-out, so consumers must poll and you have to build any multi-consumer distribution yourself — the atomicity guarantee itself is unaffected.

How is this different from just using PostgreSQL’s LISTEN/NOTIFY? LISTEN/NOTIFY can wake up a relay faster than polling, but it isn’t durable on its own — a notification sent while no one is listening is lost. Pair it with the outbox table as the durability layer, and use NOTIFY only as a low-latency trigger to poll sooner.

Do I need Kafka or a specific broker to implement this? No. As a design property, the durability guarantee comes from the outbox table itself, not from any particular transport — so the relay’s publish step can be adapted to different transports such as a queue, a pub/sub channel, or another event bus, rather than being tied to one broker.

What breaks if the outbox insert uses a different transaction or session than the business write? If the outbox row is written in a different transaction or session than the business write, the two writes are no longer atomic. A crash or failure between them can leave one committed without the other — the business write with no corresponding outbox row, or vice versa — which is the dual-write problem again, just moved one layer down. The fix is purely mechanical: insert into the outbox table on the same session, inside the same session.begin() block as the business write, always.