Exactly-once with Spring Kafka on Spring Boot 4: demonstrations, docs and captured output

Companion suite for the ankurm.com guide. Every test runs against an in-JVM KRaft broker
(spring-kafka-test), so there is no Docker requirement. Verified on Spring Boot 4.1.0,
Spring Kafka 4.1.0, kafka-clients 4.2.1, JDK 25.0.3. results/full-run.txt is unedited
output of mvn test: 15 tests, 0 failures.

Demonstrations
  _01_idempotence   verifies idempotence from the BROKER via Admin.describeProducers rather
                    than by reading config back: a default producer registers one producer
                    state, an acks=1 producer registers zero. Also covers the ConfigException
                    you get only when idempotence is requested explicitly.
  _02_transactions  transaction markers consuming offsets, aborted records surviving in the
                    log, read_committed vs read_uncommitted, and a correction: markers do NOT
                    inflate consumer lag (a drained consumer reaches zero), they break
                    counting records by offset arithmetic (measured 100% overstatement).
  _03_read_process_write
                    the Spring loop with container-managed transactions; a listener that
                    fails after sending runs twice and is committed once.
  _04_fencing       shared transactional.id fences the zombie; a random per-startup id lets
                    both producers commit and silently defeats the guarantee entirely.
  _05_database      Kafka EOS does not extend to a database. Naive listener writes 2 rows for
                    1 message; the same listener keyed on topic-partition-offset writes 1.
  _06_performance   cost per transaction size, median of 5 interleaved rounds.

Docs
  Seven chapters: Boot 4 setup (auto-configuration moved to spring-boot-starter-kafka),
  idempotence, transactions and markers, fencing and KIP-890, the database boundary and the
  outbox pattern, performance, and corner cases.

Measured highlights
  acks=1 producer          0 broker-tracked producer states (idempotence silently off)
  5 records in one tx      endOffset 6; offset span overstates record count by 100%
  1 record  per tx            711 records/s
  100 records per tx       78,770 records/s  (~111x)

docs/05-database-boundary.md

@@ -0,0 +1,112 @@
+# 5. The boundary: Kafka, your database, and the thing that cannot be done
+
+> Runnable companion: [`DatabaseBoundaryTest`](../src/test/java/com/ankurm/kafka/eos/_05_database/DatabaseBoundaryTest.java)
+
+## 5.1 State it plainly
+
+A Kafka transaction is atomic across **Kafka partitions and Kafka consumer offsets**. That is the
+entire scope. It cannot include:
+
+- a database write,
+- an HTTP call to another service,
+- a file,
+- an email, an SMS, a push notification,
+- a cache invalidation.
+
+There is no two-phase commit between Kafka and your database, and there is not going to be. Kafka's
+transaction coordinator has no protocol for enlisting a foreign resource manager.
+
+So: **a listener that writes to a database is at-least-once with respect to that database, no matter
+what your Kafka configuration says.** The redelivery that makes Kafka's guarantee work is exactly
+the thing that duplicates your row.
+
+## 5.2 What `ChainedKafkaTransactionManager` was, and why it is deprecated
+
+Spring used to ship `ChainedKafkaTransactionManager` for this, and it is deprecated along with its
+parent `ChainedTransactionManager`. The reason is worth understanding rather than just obeying:
+chaining transaction managers is **best-effort ordering, not distributed coordination**. It commits
+one, then the other. If the process dies in between, they disagree, and nothing reconciles them.
+
+It narrows the window. It does not close it. A narrowed window is not a guarantee, and treating it
+as one is worse than knowing you have none — because you will skip the idempotency work.
+
+## 5.3 What to do instead
+
+### Option A — idempotent writes (usually right)
+
+Give the work a natural identity and enforce it with a `UNIQUE` constraint. Then a duplicate
+attempt is a no-op regardless of *why* it duplicated.
+
+```java
+String dedupeKey = "%s-%d-%d".formatted(topic, partition, offset);
+if (!repository.existsByDedupeKey(dedupeKey)) {
+    repository.save(new ProcessedOrder(dedupeKey, payload));
+}
+```
+
+`topic-partition-offset` is a good default key: Kafka guarantees it is unique and stable, it is on
+every consumer record, and it requires no coordination or payload changes. An event ID carried in
+the message is better when the same logical event can arrive on more than one topic or be replayed
+from a different offset.
+
+Do the check-and-insert inside the database transaction and let the `UNIQUE` constraint be the
+real enforcement — the `exists` check is an optimisation, not the guarantee, because two concurrent
+attempts can both pass it.
+
+Measured in `DatabaseBoundaryTest`: with a naive insert the listener ran twice and wrote **2 rows**
+for one message; with the dedupe key it ran twice and wrote **1**.
+
+### Option B — the transactional outbox (when you cannot be idempotent)
+
+Invert the problem. The database is the source of truth:
+
+1. In **one local database transaction**, write your business change *and* a row into an `outbox`
+   table.
+2. A separate relay — a poller, or change-data-capture such as Debezium — reads the outbox and
+   publishes to Kafka.
+3. The relay is at-least-once, so consumers must tolerate duplicates, which idempotent consumers
+   already do.
+
+This gives genuine atomicity for the part that matters (business state and the intent to publish),
+using only a local transaction. The cost is a table, a relay, and publish latency.
+
+### Option C — make the listener's side effect naturally idempotent
+
+Sometimes the work is already idempotent and nobody noticed: `UPDATE ... SET status='PAID'`,
+`PUT /resource/123`, an upsert keyed by business ID, a `SET` in Redis. If the operation is
+absorbing rather than accumulating, redelivery is harmless. Prefer designing for this over
+detecting duplicates.
+
+## 5.4 Ordering, if you use `@Transactional` anyway
+
+With a `KafkaTransactionManager` on the container and `@Transactional` on the listener method, you
+get:
+
+```
+container begins Kafka transaction
+  interceptor begins DB transaction
+    listener body runs
+  DB transaction COMMITS        <-- first
+Kafka transaction COMMITS       <-- second
+```
+
+The DB commits first, deliberately: if the Kafka commit then fails, the offset is not advanced, the
+record is redelivered, and an idempotent DB write absorbs it. Had Kafka committed first, a DB
+failure would lose the write entirely with the offset already advanced.
+
+So `@Transactional` is still worth having — it makes the DB write roll back when your listener
+throws, which is the common case. It just does not make the two systems atomic, and the crash
+window between the two commits is real. Idempotency is what covers it.
+
+## 5.5 The rule
+
+> **Exactly-once processing = at-least-once delivery + idempotent side effects.**
+>
+> Kafka transactions give you the delivery half. You build the other half.
+
+Anyone who tells you Kafka gives you end-to-end exactly-once across your whole system is describing
+Kafka-to-Kafka and has not thought about the database.
+
+---
+
+Next: [06-performance.md](06-performance.md).