spring-kafka-exactly-once-boot4/docs/05-database-boundary.md

5. The boundary: Kafka, your database, and the thing that cannot be done

Runnable companion: DatabaseBoundaryTest

5.1 State it plainly

A Kafka transaction is atomic across Kafka partitions and Kafka consumer offsets. That is the entire scope. It cannot include:

• a database write,
• an HTTP call to another service,
• a file,
• an email, an SMS, a push notification,
• a cache invalidation.

There is no two-phase commit between Kafka and your database, and there is not going to be. Kafka's transaction coordinator has no protocol for enlisting a foreign resource manager.

So: a listener that writes to a database is at-least-once with respect to that database, no matter what your Kafka configuration says. The redelivery that makes Kafka's guarantee work is exactly the thing that duplicates your row.

5.2 What ChainedKafkaTransactionManager was, and why it is deprecated

Spring used to ship ChainedKafkaTransactionManager for this, and it is deprecated along with its parent ChainedTransactionManager. The reason is worth understanding rather than just obeying: chaining transaction managers is best-effort ordering, not distributed coordination. It commits one, then the other. If the process dies in between, they disagree, and nothing reconciles them.

It narrows the window. It does not close it. A narrowed window is not a guarantee, and treating it as one is worse than knowing you have none — because you will skip the idempotency work.

5.3 What to do instead

Option A — idempotent writes (usually right)

Give the work a natural identity and enforce it with a UNIQUE constraint. Then a duplicate attempt is a no-op regardless of why it duplicated.

String dedupeKey = "%s-%d-%d".formatted(topic, partition, offset);
if (!repository.existsByDedupeKey(dedupeKey)) {
    repository.save(new ProcessedOrder(dedupeKey, payload));
}

topic-partition-offset is a good default key: Kafka guarantees it is unique and stable, it is on every consumer record, and it requires no coordination or payload changes. An event ID carried in the message is better when the same logical event can arrive on more than one topic or be replayed from a different offset.

Do the check-and-insert inside the database transaction and let the UNIQUE constraint be the real enforcement — the exists check is an optimisation, not the guarantee, because two concurrent attempts can both pass it.

Measured in DatabaseBoundaryTest: with a naive insert the listener ran twice and wrote 2 rows for one message; with the dedupe key it ran twice and wrote 1.

Option B — the transactional outbox (when you cannot be idempotent)

Invert the problem. The database is the source of truth:

1. In one local database transaction, write your business change and a row into an outbox table.
2. A separate relay — a poller, or change-data-capture such as Debezium — reads the outbox and publishes to Kafka.
3. The relay is at-least-once, so consumers must tolerate duplicates, which idempotent consumers already do.

This gives genuine atomicity for the part that matters (business state and the intent to publish), using only a local transaction. The cost is a table, a relay, and publish latency.

Option C — make the listener's side effect naturally idempotent

Sometimes the work is already idempotent and nobody noticed: UPDATE ... SET status='PAID', PUT /resource/123, an upsert keyed by business ID, a SET in Redis. If the operation is absorbing rather than accumulating, redelivery is harmless. Prefer designing for this over detecting duplicates.

5.4 Ordering, if you use @Transactional anyway

With a KafkaTransactionManager on the container and @Transactional on the listener method, you get:

container begins Kafka transaction
  interceptor begins DB transaction
    listener body runs
  DB transaction COMMITS        <-- first
Kafka transaction COMMITS       <-- second

The DB commits first, deliberately: if the Kafka commit then fails, the offset is not advanced, the record is redelivered, and an idempotent DB write absorbs it. Had Kafka committed first, a DB failure would lose the write entirely with the offset already advanced.

So @Transactional is still worth having — it makes the DB write roll back when your listener throws, which is the common case. It just does not make the two systems atomic, and the crash window between the two commits is real. Idempotency is what covers it.

5.5 The rule

Exactly-once processing = at-least-once delivery + idempotent side effects.

Kafka transactions give you the delivery half. You build the other half.

Anyone who tells you Kafka gives you end-to-end exactly-once across your whole system is describing Kafka-to-Kafka and has not thought about the database.

---

Next: 06-performance.md.