spring-kafka-exactly-once-boot4/docs/04-fencing.md

4. Zombie fencing and transactional.id

Runnable companion: ZombieFencingTest

4.1 The zombie

A processor reads, transforms and writes. It stalls — a long GC pause, a network partition, a container the orchestrator has decided is dead. A replacement starts, the group rebalances, the new instance owns the partition.

Then the old instance wakes up. It holds a partially completed transaction and does not know it was replaced. If it were allowed to commit, the same input would be processed twice and both results would be committed. That is the zombie.

4.2 The defence

transactional.id is a stable, durable identity the transaction coordinator remembers, along with an epoch. Every initTransactions() for an existing id bumps the epoch and invalidates anything holding the old one.

Measured:

Producer A: initTransactions, beginTransaction, send   (transaction OPEN)
Producer B: initTransactions with the SAME id          (epoch bumped)
Producer A: commitTransaction
  -> REJECTED with ProducerFencedException
Producer B: commits normally

The zombie's work is discarded by the broker, not by any code you wrote.

4.3 The two ways to get it wrong

Random per start-up — fencing never happens

// Catastrophic, and everything appears to work
props.put(TRANSACTIONAL_ID_CONFIG, "processor-" + UUID.randomUUID());

Measured: producer A opens a transaction, producer B starts with a fresh UUID and commits, then A commits too. Nothing is fenced. Both commit. The coordinator saw two unrelated identities and had no reason to fence either.

This is the most damaging Kafka transactions misconfiguration precisely because it is invisible: transactions commit, tests pass, and you have exactly the duplicate-processing problem you turned transactions on to prevent.

Shared across instances — everything fences everything

Every instance bumps every other instance's epoch continuously, and the group thrashes with ProducerFencedExceptions.

4.4 Getting it right in Spring

spring.kafka.producer.transaction-id-prefix: order-processor-tx-

Spring appends a suffix per producer. The prefix must be stable across restarts and unique per application instance — typically the application name plus a stable instance identifier (a StatefulSet ordinal, a pod index, a configured node id), not a random value and not a value shared by every replica.

Since Spring Kafka 3.2, TransactionIdSuffixStrategy lets you control suffix allocation, which matters when producers are cached and reused across partitions.

With EOSMode.V2 — the only mode since Spring Kafka 3.0 — the producer no longer needs a transactional.id tied to the specific input partition. V2 (KIP-447) uses consumer group metadata for fencing, so one producer per instance is sufficient. Older guidance about transactional.id per topic-partition describes the removed EOSMode.ALPHA and does not apply.

4.5 Kafka 4.x: KIP-890 changes which exception you see

KIP-890 strengthened the transaction protocol against hanging transactions. Under transaction.version=2 — the default on Kafka 4.x, and what the embedded broker in this repository reports at start-up — the producer epoch is bumped on every transaction, not only at initTransactions().

Practical effect: the exception on fencing may be InvalidProducerEpochException rather than the ProducerFencedException that older articles promise. Catch both, or catch neither and let the container restart the producer, which is what Spring does.

A hanging transaction is worth understanding because its blast radius is large: it holds the Last Stable Offset back, which stalls every read_committed consumer on the partition and prevents log compaction. KIP-890 exists to make that class of bug impossible rather than merely rare.

---

Next: 05-database-boundary.md.