asmhatre/spring-kafka-exactly-once-boot4
1
0
Fork 0
Code Activity

Exactly-once with Spring Kafka on Spring Boot 4: demonstrations, docs and captured output

Browse Source 
Companion suite for the ankurm.com guide. Every test runs against an in-JVM KRaft broker
(spring-kafka-test), so there is no Docker requirement. Verified on Spring Boot 4.1.0,
Spring Kafka 4.1.0, kafka-clients 4.2.1, JDK 25.0.3. results/full-run.txt is unedited
output of mvn test: 15 tests, 0 failures.

Demonstrations
  _01_idempotence   verifies idempotence from the BROKER via Admin.describeProducers rather
                    than by reading config back: a default producer registers one producer
                    state, an acks=1 producer registers zero. Also covers the ConfigException
                    you get only when idempotence is requested explicitly.
  _02_transactions  transaction markers consuming offsets, aborted records surviving in the
                    log, read_committed vs read_uncommitted, and a correction: markers do NOT
                    inflate consumer lag (a drained consumer reaches zero), they break
                    counting records by offset arithmetic (measured 100% overstatement).
  _03_read_process_write
                    the Spring loop with container-managed transactions; a listener that
                    fails after sending runs twice and is committed once.
  _04_fencing       shared transactional.id fences the zombie; a random per-startup id lets
                    both producers commit and silently defeats the guarantee entirely.
  _05_database      Kafka EOS does not extend to a database. Naive listener writes 2 rows for
                    1 message; the same listener keyed on topic-partition-offset writes 1.
  _06_performance   cost per transaction size, median of 5 interleaved rounds.

Docs
  Seven chapters: Boot 4 setup (auto-configuration moved to spring-boot-starter-kafka),
  idempotence, transactions and markers, fencing and KIP-890, the database boundary and the
  outbox pattern, performance, and corner cases.

Measured highlights
  acks=1 producer          0 broker-tracked producer states (idempotence silently off)
  5 records in one tx      endOffset 6; offset span overstates record count by 100%
  1 record  per tx            711 records/s
  100 records per tx       78,770 records/s  (~111x)
This commit is contained in:
Ankur
2026-07-31 22:27:35 +05:30
commit e23ca359eb
26 changed files with 3635 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

138
docs/03-transactions-and-markers.md Normal file
View File

@@ -0,0 +1,138 @@
# 3. Transactions, isolation, and the markers nobody mentions
> Runnable companion: [`TransactionMarkerOffsetTest`](../src/test/java/com/ankurm/kafka/eos/_02_transactions/TransactionMarkerOffsetTest.java),
> [`ReadProcessWriteTest`](../src/test/java/com/ankurm/kafka/eos/_03_read_process_write/ReadProcessWriteTest.java)
## 3.1 What a transaction adds over idempotence
Idempotence protects one producer session writing to one partition. A **transaction** makes a set of
writes atomic across partitions — *and*, crucially, lets a consumer offset commit be one of those
writes.
That last part is the whole trick behind read-process-write. The offset for the input record is
written **into the same transaction** as the output record, via `sendOffsetsToTransaction`. There is
no window between "produced the output" and "committed the offset" in which a crash produces a
duplicate, because there is no between.
Setting `transactional.id` on a producer also forces `enable.idempotence=true` — transactions are
built on top of idempotence, not as an alternative to it.
## 3.2 In Spring, you write none of that
```yaml
spring.kafka.producer.transaction-id-prefix: order-processor-tx-
spring.kafka.consumer.isolation-level: read_committed
```
```java
@KafkaListener(topics = "orders.incoming", groupId = "order-processor")
public void process(String order) {
kafkaTemplate.send("orders.validated", order.toUpperCase());
}
```
The container begins the transaction before the listener runs, enlists the `KafkaTemplate` send,
sends the offsets into the transaction, and commits — or aborts everything if the method throws.
The listener body is deliberately ordinary. **The guarantee is entirely in the configuration.**
## 3.3 `isolation.level` defaults to the wrong thing
`read_uncommitted` is the default. A consumer reading a transactional topic with the default
configuration will happily deliver records from transactions that were **aborted**.
Measured, on one partition, after committing 5 records and aborting 3:
```
read_committed : 5 records [committed-0 .. committed-4]
read_uncommitted : 8 records [committed-0 .. committed-4, aborted-0, aborted-1, aborted-2]
```
`isolation.level` is a **consumer-side filter**, not a broker-side delete. The aborted records are
physically in the log with permanent offsets and they consume disk. Producing transactionally
without setting `read_committed` on every consumer gives you the cost of transactions and none of
the benefit.
## 3.4 Transaction markers consume offsets
When a transaction commits or aborts, Kafka writes a **control record** — a transaction marker —
into every partition it touched. Control records are real log entries that **consume an offset**,
but are never delivered to your consumer.
```
records sent : 5
endOffset : 6 <- records occupy 0..4; the commit marker takes 5
last delivered offset : 4
```
Three consequences:
**Offsets are not contiguous.** Send five records in a transaction and the next record is at offset
6, not 5. Any code assuming `offset + 1` is the next record is wrong. In the Spring
read-process-write test, three messages processed in three transactions landed at offsets **0, 2 and
4** — a marker between each.
**Counting records by offset arithmetic is broken.** After 5 committed records, 3 aborted records
and 2 markers:
```
beginningOffset : 0
endOffset : 10
end - beginning : 10 <- looks like the record count
records actually retrievable: 5
overstatement : 100%
```
Retention estimates, "how many events between these offsets", partition-size dashboards and tests
that assert on offsets are all wrong by however much your marker and abort rate happens to be.
**Consumer lag, however, is fine.** This is worth stating because the opposite is widely repeated.
A drained `read_committed` consumer's *position* advances over markers and aborted records exactly
as it advances over delivered ones, so `endOffset - committedOffset` reaches zero:
```
partition end offset : 10
consumer committed offset : 10
lag : 0
```
The claim that transactions inflate consumer lag does not survive contact with a broker. The claim
that they break offset arithmetic does.
## 3.5 The Last Stable Offset, and why transaction duration is a consumer-latency setting
A `read_committed` consumer can only read up to the **Last Stable Offset** — the offset before the
first still-open transaction. If a producer opens a transaction and holds it for 30 seconds, every
`read_committed` consumer on that partition stalls for 30 seconds, *including for records committed
by other producers in the meantime*.
So `transaction.timeout.ms` and how long you keep a transaction open are consumer-latency knobs, not
just producer concerns. Long-running transactions are the most common cause of "our consumers
randomly freeze for exactly N seconds".
## 3.6 Aborted records only reach the log if they were flushed
A detail that surprised this repository's own test. `KafkaTemplate.send()` is asynchronous. If the
transaction aborts while the record is still sitting in the producer's accumulator,
`abortTransaction()` **discards the un-flushed batch** and the record never reaches the broker at
all.
So aborted records occupy permanent offsets only when they were already flushed — which happens
under load, on large batches, or when you block on the send future. Both behaviours are correct and
neither is something to rely on. Rely on `read_committed`.
## 3.7 Exactly-once means committed once, not executed once
The clearest demonstration in this repository: a listener that fails *after* sending its output.
```
times the listener ran for 'delta' : 2
DELTA records visible (read_committed) : 1
```
The work happened twice. One result was committed. **That is what exactly-once means in Kafka, and
it has never meant that your code runs once.** Every side effect outside Kafka happened twice —
see [05-database-boundary.md](05-database-boundary.md).
---
Next: [04-fencing.md](04-fencing.md).