spring-messaging-demo/kafka-error-handling/docs/05-retryable-topic.md

← The DLT · Module README

5. Non-blocking retries with @RetryableTopic

Blocking retries stall the partition. @RetryableTopic republishes the failed record to a separate topic and lets the main partition carry on.

@RetryableTopic(attempts = "4", backOff = @BackOff(delay = 500, multiplier = 2.0),
        sameIntervalTopicReuseStrategy = SameIntervalTopicReuseStrategy.SINGLE_TOPIC,
        exclude = Failures.PermanentFailure.class)
@KafkaListener(topics = "invoices", groupId = "invoices")
public void onInvoice(ConsumerRecord<String, Payment> record, ...) { ... }

Two API changes in Spring Kafka 4.x will stop older examples compiling:

• the attribute is backOff, not backoff
• the annotation is org.springframework.kafka.annotation.BackOff, not org.springframework.retry.annotation.Backoff. Spring Kafka 4 dropped the spring-retry dependency and brought its own.

The failure is package org.springframework.retry.annotation does not exist, which reads like a missing dependency and is not.

Also new in 4.1: sameIntervalTopicReuseStrategy defaults to SINGLE_TOPIC in RetryTopicConfigurationBuilder, aligning it with the annotation's default.

What it actually does

From docs/output/retry-topics.txt — a failing record and a good one published back to back on the same partition:

=== @RetryableTopic delivery trace ===
  +0      ms  invoices                     transient-1
  +531    ms  invoices-retry-500           transient-1
  +550    ms  invoices                     ok-1
  +1554   ms  invoices-retry-1000          transient-1
  +3560   ms  invoices-retry-2000          transient-1
  DLT: [transient-1 on invoices-dlt]

Read the third line. ok-1 was processed at +550 ms, while transient-1 was still two retries from giving up. With a blocking handler it would have waited for the whole schedule.

Retry topics are named by the delay, not the attempt number. invoices-retry-500, invoices-retry-1000, invoices-retry-2000 — that is TopicSuffixingStrategy.SUFFIX_WITH_DELAY_VALUE, the default. So provisioning topics ahead of time means knowing your whole back-off schedule in advance, and changing the multiplier changes the topic names, orphaning whatever is still sitting in the old ones. Deploy that change the way you would a rename.

The cost

Per-key ordering is gone for any record that fails. That is not a side effect; it is the mechanism. If invoice-7 fails and invoice-7's next event succeeds, they are processed out of order, and no configuration prevents it.

So the decision is not "blocking or non-blocking", it is:

	blocking (DefaultErrorHandler)	non-blocking (@RetryableTopic)
ordering under failure	preserved	lost for the failing key
partition throughput under failure	stalled	unaffected
topics to provision	1 + DLT	1 + one per distinct delay + DLT
long back-offs	limited by max.poll.interval.ms	unlimited

If your consumer is idempotent and order-insensitive — most notification, indexing and cache-warm consumers are — retry topics are strictly better. If it applies state transitions per key, blocking retries with a short schedule and a fast DLT are usually the safer answer.

Use exclude (or include) rather than retrying everything: a PermanentFailure here skips the retry topics entirely and goes straight to invoices-dlt.

@DltHandler

@DltHandler
public void onDlt(ConsumerRecord<String, Payment> record,
        @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) { ... }

Without one, the framework still creates and populates the DLT — it just logs and moves on, and nothing in your application has looked at the record. A @DltHandler that increments a counter and writes a structured log line is the minimum worth having, because a DLT nobody watches is a queue that grows until someone notices the disk.

Module README