90 lines
3.9 KiB
Markdown
90 lines
3.9 KiB
Markdown
[← The DLT](04-the-dlt.md) · [Module README](../README.md)
|
|
|
|
# 5. Non-blocking retries with `@RetryableTopic`
|
|
|
|
Blocking retries stall the partition. `@RetryableTopic` republishes the failed record to a
|
|
separate topic and lets the main partition carry on.
|
|
|
|
```java
|
|
@RetryableTopic(attempts = "4", backOff = @BackOff(delay = 500, multiplier = 2.0),
|
|
sameIntervalTopicReuseStrategy = SameIntervalTopicReuseStrategy.SINGLE_TOPIC,
|
|
exclude = Failures.PermanentFailure.class)
|
|
@KafkaListener(topics = "invoices", groupId = "invoices")
|
|
public void onInvoice(ConsumerRecord<String, Payment> record, ...) { ... }
|
|
```
|
|
|
|
**Two API changes in Spring Kafka 4.x will stop older examples compiling:**
|
|
|
|
- the attribute is **`backOff`**, not `backoff`
|
|
- the annotation is **`org.springframework.kafka.annotation.BackOff`**, not
|
|
`org.springframework.retry.annotation.Backoff`. Spring Kafka 4 dropped the spring-retry
|
|
dependency and brought its own.
|
|
|
|
The failure is `package org.springframework.retry.annotation does not exist`, which reads like a
|
|
missing dependency and is not.
|
|
|
|
Also new in 4.1: `sameIntervalTopicReuseStrategy` defaults to `SINGLE_TOPIC` in
|
|
`RetryTopicConfigurationBuilder`, aligning it with the annotation's default.
|
|
|
|
## What it actually does
|
|
|
|
From [`docs/output/retry-topics.txt`](output/retry-topics.txt) — a failing record and a good one
|
|
published back to back on the same partition:
|
|
|
|
```
|
|
=== @RetryableTopic delivery trace ===
|
|
+0 ms invoices transient-1
|
|
+531 ms invoices-retry-500 transient-1
|
|
+550 ms invoices ok-1
|
|
+1554 ms invoices-retry-1000 transient-1
|
|
+3560 ms invoices-retry-2000 transient-1
|
|
DLT: [transient-1 on invoices-dlt]
|
|
```
|
|
|
|
Read the third line. `ok-1` was processed at +550 ms, while `transient-1` was still two retries
|
|
from giving up. With a blocking handler it would have waited for the whole schedule.
|
|
|
|
**Retry topics are named by the delay, not the attempt number.** `invoices-retry-500`,
|
|
`invoices-retry-1000`, `invoices-retry-2000` — that is
|
|
`TopicSuffixingStrategy.SUFFIX_WITH_DELAY_VALUE`, the default. So provisioning topics ahead of
|
|
time means knowing your whole back-off schedule in advance, and **changing the multiplier changes
|
|
the topic names**, orphaning whatever is still sitting in the old ones. Deploy that change the
|
|
way you would a rename.
|
|
|
|
## The cost
|
|
|
|
**Per-key ordering is gone for any record that fails.** That is not a side effect; it is the
|
|
mechanism. If `invoice-7` fails and `invoice-7`'s next event succeeds, they are processed out of
|
|
order, and no configuration prevents it.
|
|
|
|
So the decision is not "blocking or non-blocking", it is:
|
|
|
|
| | blocking (`DefaultErrorHandler`) | non-blocking (`@RetryableTopic`) |
|
|
|---|---|---|
|
|
| ordering under failure | preserved | lost for the failing key |
|
|
| partition throughput under failure | stalled | unaffected |
|
|
| topics to provision | 1 + DLT | 1 + one per distinct delay + DLT |
|
|
| long back-offs | limited by `max.poll.interval.ms` | unlimited |
|
|
|
|
If your consumer is idempotent and order-insensitive — most notification, indexing and cache-warm
|
|
consumers are — retry topics are strictly better. If it applies state transitions per key,
|
|
blocking retries with a short schedule and a fast DLT are usually the safer answer.
|
|
|
|
Use `exclude` (or `include`) rather than retrying everything: a `PermanentFailure` here skips the
|
|
retry topics entirely and goes straight to `invoices-dlt`.
|
|
|
|
## `@DltHandler`
|
|
|
|
```java
|
|
@DltHandler
|
|
public void onDlt(ConsumerRecord<String, Payment> record,
|
|
@Header(KafkaHeaders.RECEIVED_TOPIC) String topic) { ... }
|
|
```
|
|
|
|
Without one, the framework still creates and populates the DLT — it just logs and moves on, and
|
|
nothing in your application has looked at the record. A `@DltHandler` that increments a counter
|
|
and writes a structured log line is the minimum worth having, because a DLT nobody watches is a
|
|
queue that grows until someone notices the disk.
|
|
|
|
[Module README](../README.md)
|