spring-kafka-exactly-once-boot4/docs/06-performance.md

6. What transactions actually cost

Runnable companion: TransactionCostTest

6.1 The useful version of "transactions are slow"

The cost is per transaction, not per record.

Committing requires a round trip to the transaction coordinator plus a durable marker write into every partition the transaction touched, plus acknowledgement of those writes. That is a fixed cost. It does not care whether the transaction contained one record or ten thousand.

So the question is never "should I use transactions" in isolation. It is "how many records go in each one".

6.2 Measured

10,000 records of 256 bytes, single-node in-JVM broker, median of 5 interleaved rounds, JDK 25.0.3 / Kafka 4.2.1 on an AMD Ryzen 5 5600U laptop:

Configuration	records/s	round-to-round spread	vs previous row
no transaction (idempotent only)	801,880	26%	—
1 record / transaction	711	11%	—
10 records / transaction	8,452	12%	11.9x
100 records / transaction	78,770	55%	9.3x
1,000 records / transaction	330,969	77%	4.2x
10,000 records / transaction	878,071	32%	2.7x

One record per transaction is roughly 111x slower than one hundred, and about three orders of magnitude below the non-transactional baseline.

Each 10x increase in transaction size buys close to 10x throughput while commit cost dominates, then flattens as per-record cost takes over — the knee is somewhere around 1,000 records here.

6.3 How to read these numbers honestly

They come from a laptop and a single-node in-JVM broker, so the absolute throughput is not meaningful. Two specific caveats:

• The non-transactional baseline is noisy (26% spread, and it drifted 30% across the run). Sending 10,000 records with no commit is fast enough that the measurement is dominated by flush timing and OS scheduling. Ratios against the baseline are indicative only.
• The transactional measurements are stable because each is dominated by a fixed number of commits. The comparisons between transaction sizes are the trustworthy part, and they are the point.

On a real cluster the per-commit cost is larger — real network, real replication, real disk — so the penalty at small transaction sizes is worse than shown here and batching matters more.

The test itself went through three revisions before producing numbers worth quoting; the reasons are documented in its source, and they are the usual ones: insufficient warm-up, measurement order accruing warm-up to whichever configuration ran last, and one path paying for a metadata fetch its competitor had already done. If you adapt it, keep the interleaving and the drift report.

6.4 The setting that controls this in Spring

max.poll.records. Not anything named "transaction".

With a KafkaTransactionManager, the container's transaction boundary is the batch returned by poll(). So the number of records per transaction is exactly max.poll.records (default 500), capped by how many are actually available.

spring:
  kafka:
    consumer:
      max-poll-records: 500      # == records per transaction

Raising it is the single most effective throughput fix for a transactional listener. The price is that a failure anywhere in the batch rolls back and redoes the whole batch, so you are trading throughput against the cost of redoing work — and against max.poll.interval.ms, since the whole batch must be processed within it.

6.5 The other costs, which are not throughput

• Latency. A read_committed consumer cannot read past the Last Stable Offset, so downstream consumers see nothing until the transaction commits. Larger transactions mean higher end-to-end latency. This is a direct trade against §6.4.
• Storage. Every transaction writes a marker per partition. High-frequency small transactions on many partitions produce a surprising amount of control-record overhead.
• Redelivery amplification. One poisoned record in a batch of 500 rolls back all 500.
• Coordinator load. Every transaction is state on a broker, in __transaction_state.

6.6 Rules of thumb

• Do not use one transaction per record. It is three orders of magnitude off the achievable rate.
• Start at the default max.poll.records=500 and tune from measurements, not from feeling.
• If latency matters more than throughput, lower it deliberately and know what you are buying.
• If you do not need atomic offset+output commits, do not use transactions at all — an idempotent producer plus an idempotent consumer is cheaper and simpler, and covers most systems.

---

Next: 07-corner-cases.md.