The transactional outbox and the dual-write problem

Two systems never commit together: write the event in the change's own transaction, and let a relay publish it, possibly twice.

Before you read: predict

ExamHall's submit endpoint saves the submission and tells the grading service through the message broker:

Candidate a-311 submits. The pod running submit is killed just after broker.publish returns. Then the grading service reads the broker.

What do the database and the broker hold for a-311?

async function submit(attemptId) {  await db.query('BEGIN');  await db.query('INSERT INTO submissions(attempt_id) VALUES ($1)', [attemptId]);  await broker.publish('attempt.submitted', { attemptId });  await db.query('COMMIT');}

The 30-second version

A database commit and a broker publish are two writes to two systems, and nothing makes them one. Publish before COMMIT and a rollback leaves an event for nothing; publish after and a crash loses it. So insert the event as an outbox row in the same transaction, and let a separate relay publish unsent rows, at least once.

Two systems never commit together: write the event in the change's own transaction, and let a relay publish it, possibly twice.

How it works

Step 1 of 11

Two systems. The database commits; the broker accepts a message. No transaction spans both, so either write can happen without the other.

Analogy: A mail-order firm's order binder

The order clerk files each order sheet together with its sealed dispatch note in the day's binder, in one movement. The courier, waiting apart, later takes every note not yet ticked to the warehouse, and ticks each one once the warehouse has it.

At the order desk	In the outbox
phoning the warehouse before the sheet is filed, then the customer cancels	publishing before `COMMIT`, then a rollback (step 2)
filing the sheet, meaning to phone later, then being called away	publishing after `COMMIT`, then a crash (step 3)
the sheet and its note filed in one movement, or neither	the outbox row in the change's transaction (step 4)
the courier, never the clerk, carries the notes	the relay (step 5)
two couriers, each taking notes nobody else is holding	`SKIP LOCKED` claims (step 6)
a courier called away after the handover, before the tick	published, not marked: sent again (step 8)

Breaks down when: a paper note handed over twice is still one note, which the warehouse can see; a broker given an event twice holds two messages, so the consumer must recognise the repeat itself (async-idempotent-consumers).

Contrasting case: a counter sale where the customer carries the parcel out. One act in one place: no second system, so no note to send.

In real terms: the event is one more row of the change's transaction; everything after COMMIT is a relay retrying until the broker has it.

Examples, run for real

4.1 ExamHall EXH-2430: three submitted attempts that were never graded

The submit handler committed each submission, then added the grading message; on a refusal it logged the error and returned. During a broker outage at the end of an exam, three submissions committed. MR !2730 writes the event row in the submission's own transaction and leaves publishing to a relay:

4.1 before (commit, then publish; a refusal is logged): 3 submissions committed · publishes refused 3 · after the broker recovers: graded 0 of 3
4.1 after MR !2730 (event row in the submission's transaction, then a relay): 3 submissions committed · relay round refused, outbox rows unsent 3 · after the broker recovers: graded 3 of 3, unsent 0

The handler's message had nowhere to wait, so a refused publish was gone (step 3). The outbox row waited in the table, and the relay sent it once the broker answered (steps 4–5).

4.2 Payments ledger: what a relay can read

A ledger writes each transfer's outbox row first and the transfer second, in one transaction. A CHECK refuses one transfer, so its transaction rolls back. A polling claim and a logical replication slot (test_decoding, used as a row counter) then read the outbox:

4.2 4 transfers attempted · 3 committed · 1 rolled back (CHECK cents > 0) · polling claim returns tr-1, tr-2, tr-4 · logical slot holds outbox INSERTs for tr-1, tr-2, tr-4

Both see the three committed rows and nothing of the refused one: the event row lives or dies with its change (step 4), whichever relay reads it (steps 6 and 11).

Picture it

A clerk in a green jacket holds a loose sheet while lifting a green binder's cover over a sealed blue envelope; a man in a blue jacket waits apart. — An illustration to help it stick. The mechanism is in the sections above.

Try it

Answer first, then check. Nothing is saved.

Question 1 of 5

The grading service must get exactly one attempt.submitted for each submission that commits and none for any other. Four versions of submit are tried. Each handles three submissions: s-1 is killed just before its first COMMIT is sent; s-2 is killed just after that COMMIT returns; s-3 is killed just after the handler's own broker.publish returns, or runs to its end in a version whose handler never publishes. In the versions with a relay, the relay then runs until no row is unsent, and is never killed.

Which version meets the goal for all three submissions?

Question 2 of 5

A ledger service's outbox holds four unsent rows: 1 acct-17 account.opened, 2 acct-40 account.opened, 3 acct-17 deposit, 4 acct-40 deposit. Two relays, A and B, each run rounds of:

A starts its first round, and the network stalls A's first publish. While it is stalled, B runs one round. Then the stall clears, A finishes its round, and both relays keep running rounds until nothing is unsent.

In what order does the ledger receive acct-17's events?

await db.query('BEGIN');const { rows } = await db.query(  'SELECT * FROM outbox WHERE sent_at IS NULL ORDER BY id LIMIT 2 FOR UPDATE SKIP LOCKED');for (const row of rows) await broker.publish(row.type, row);await db.query('UPDATE outbox SET sent_at = now() WHERE id = ANY($1)', [rows.map((r) => r.id)]);await db.query('COMMIT');

Question 3 of 5

ExamHall's outbox holds six unsent attempt.submitted rows, ids 1 to 6. Two grading relays, A and B, each run rounds of:

A's first claim runs first, and B's first claim runs while A's round is still open. A's pod is then killed just after the broker accepts row 2. B finishes its first round and keeps running rounds until no row is unsent.

Which rows does B's second round publish?

await db.query('BEGIN');const { rows } = await db.query(  'SELECT * FROM outbox WHERE sent_at IS NULL ORDER BY id LIMIT 3 FOR UPDATE SKIP LOCKED');for (const row of rows) {  await broker.publish(row.type, row);  await db.query('UPDATE outbox SET sent_at = now() WHERE id = $1', [row.id]);}await db.query('COMMIT');

Question 4 of 5

A wallet service's outbox holds six unsent rows: 1 w-3 topup, 2 w-8 topup, 3 w-3 payment, 4 w-8 payment, 5 w-5 topup, 6 w-3 refund. Two relays run rounds of BEGIN, then

SELECT * FROM outbox o WHERE o.sent_at IS NULL ORDER BY o.id LIMIT 2 FOR UPDATE SKIP LOCKED

then publish each claimed row in id order, mark the claimed rows sent and COMMIT. Relay A starts first, and its first publish stalls; while it is stalled, relay B keeps running rounds until its claim comes back empty. The wallet ledger must receive each wallet's events in id order, and all six rows must be published.

Which one change to the claim meets that, with both relays still running?

Question 5 of 5

A ticketing service runs each sale as one transaction:

The database runs with wal_level = logical. A logical replication slot is created before the first sale, and a relay reads it, turning each row added to outbox into an event and ignoring removals. Three sales commit; a fourth runs the same statements but ends with ROLLBACK instead of COMMIT.

What does the box office receive?

await db.query('BEGIN');await db.query('INSERT INTO sales(sale_id) VALUES ($1)', [saleId]);const { rows } = await db.query(  "INSERT INTO outbox(aggregate_id, type) VALUES ($1, 'ticket.sold') RETURNING id", [saleId]);await db.query('DELETE FROM outbox WHERE id = $1', [rows[0].id]);await db.query('COMMIT');

Go deeper

Pro tips

Give each outbox row a unique event id and send it with the message, so the consumer can recognise a repeat. When: a relay restarts after publishing and before marking (step 8). Verify: Debezium's router puts the row's id in a message header; its docs say "You can use this ID, for example, to remove duplicate messages." Status: DOC (Debezium Outbox Event Router, fetched 2026-10-01).

Source

Debezium and PostgreSQL behaviour can change: the facts below are as of 2026-10-01. No item is keyed on a source; every key is executed. Valid as of 2026-10-01.

Chris Richardson, microservices.io, Pattern: Transactional outbox: https://microservices.io/patterns/data/transactional-outbox.html (microservices-outbox.txt). The curriculum row's source. Quoted: "sending a message in the middle of a transaction is not reliable. There’s no guarantee that the transaction will commit"; "if a service sends a message after committing the transaction there’s no guarantee that it won’t crash before sending the message"; "The Message relay might publish a message more than once." Supports steps 1–5 and 8–9.
microservices.io, Pattern: Polling publisher: https://microservices.io/patterns/data/polling-publisher.html (microservices-polling.txt) and Pattern: Transaction log tailing: https://microservices.io/patterns/data/transaction-log-tailing.html (microservices-tail.txt), the two relay patterns the outbox page links. Quoted: polling's drawback "Tricky to publish events in order"; log tailing's "Tail the database transaction log and publish each message/event inserted into the outbox". Supports steps 9–11.
Debezium documentation 3.7, Outbox Event Router: https://debezium.io/documentation/reference/stable/transformations/outbox-event-router.html (debezium-outbox-router.txt). Quoted, as of 2026-10-01: the aggregateid column "is important for maintaining correct order in Kafka partitions"; "The SMT automatically filters out DELETE operations on an outbox table"; the id column: "You can use this ID, for example, to remove duplicate messages." Supports step 11 and tip 1.
PostgreSQL 17 documentation, SELECT, The Locking Clause: https://www.postgresql.org/docs/17/sql-select.html (pg-select.txt). Quoted: "With SKIP LOCKED, any selected rows that cannot be immediately locked are skipped", and "can be used to avoid lock contention with multiple consumers accessing a queue-like table". Supports step 6 and every two-relay run.

Checked against the tools and docs as of 2026-10-01.

Predict first