Add the kafka-error-handling module
This commit is contained in:
89
kafka-error-handling/docs/05-retryable-topic.md
Normal file
89
kafka-error-handling/docs/05-retryable-topic.md
Normal file
@@ -0,0 +1,89 @@
|
||||
[← The DLT](04-the-dlt.md) · [Module README](../README.md)
|
||||
|
||||
# 5. Non-blocking retries with `@RetryableTopic`
|
||||
|
||||
Blocking retries stall the partition. `@RetryableTopic` republishes the failed record to a
|
||||
separate topic and lets the main partition carry on.
|
||||
|
||||
```java
|
||||
@RetryableTopic(attempts = "4", backOff = @BackOff(delay = 500, multiplier = 2.0),
|
||||
sameIntervalTopicReuseStrategy = SameIntervalTopicReuseStrategy.SINGLE_TOPIC,
|
||||
exclude = Failures.PermanentFailure.class)
|
||||
@KafkaListener(topics = "invoices", groupId = "invoices")
|
||||
public void onInvoice(ConsumerRecord<String, Payment> record, ...) { ... }
|
||||
```
|
||||
|
||||
**Two API changes in Spring Kafka 4.x will stop older examples compiling:**
|
||||
|
||||
- the attribute is **`backOff`**, not `backoff`
|
||||
- the annotation is **`org.springframework.kafka.annotation.BackOff`**, not
|
||||
`org.springframework.retry.annotation.Backoff`. Spring Kafka 4 dropped the spring-retry
|
||||
dependency and brought its own.
|
||||
|
||||
The failure is `package org.springframework.retry.annotation does not exist`, which reads like a
|
||||
missing dependency and is not.
|
||||
|
||||
Also new in 4.1: `sameIntervalTopicReuseStrategy` defaults to `SINGLE_TOPIC` in
|
||||
`RetryTopicConfigurationBuilder`, aligning it with the annotation's default.
|
||||
|
||||
## What it actually does
|
||||
|
||||
From [`docs/output/retry-topics.txt`](output/retry-topics.txt) — a failing record and a good one
|
||||
published back to back on the same partition:
|
||||
|
||||
```
|
||||
=== @RetryableTopic delivery trace ===
|
||||
+0 ms invoices transient-1
|
||||
+531 ms invoices-retry-500 transient-1
|
||||
+550 ms invoices ok-1
|
||||
+1554 ms invoices-retry-1000 transient-1
|
||||
+3560 ms invoices-retry-2000 transient-1
|
||||
DLT: [transient-1 on invoices-dlt]
|
||||
```
|
||||
|
||||
Read the third line. `ok-1` was processed at +550 ms, while `transient-1` was still two retries
|
||||
from giving up. With a blocking handler it would have waited for the whole schedule.
|
||||
|
||||
**Retry topics are named by the delay, not the attempt number.** `invoices-retry-500`,
|
||||
`invoices-retry-1000`, `invoices-retry-2000` — that is
|
||||
`TopicSuffixingStrategy.SUFFIX_WITH_DELAY_VALUE`, the default. So provisioning topics ahead of
|
||||
time means knowing your whole back-off schedule in advance, and **changing the multiplier changes
|
||||
the topic names**, orphaning whatever is still sitting in the old ones. Deploy that change the
|
||||
way you would a rename.
|
||||
|
||||
## The cost
|
||||
|
||||
**Per-key ordering is gone for any record that fails.** That is not a side effect; it is the
|
||||
mechanism. If `invoice-7` fails and `invoice-7`'s next event succeeds, they are processed out of
|
||||
order, and no configuration prevents it.
|
||||
|
||||
So the decision is not "blocking or non-blocking", it is:
|
||||
|
||||
| | blocking (`DefaultErrorHandler`) | non-blocking (`@RetryableTopic`) |
|
||||
|---|---|---|
|
||||
| ordering under failure | preserved | lost for the failing key |
|
||||
| partition throughput under failure | stalled | unaffected |
|
||||
| topics to provision | 1 + DLT | 1 + one per distinct delay + DLT |
|
||||
| long back-offs | limited by `max.poll.interval.ms` | unlimited |
|
||||
|
||||
If your consumer is idempotent and order-insensitive — most notification, indexing and cache-warm
|
||||
consumers are — retry topics are strictly better. If it applies state transitions per key,
|
||||
blocking retries with a short schedule and a fast DLT are usually the safer answer.
|
||||
|
||||
Use `exclude` (or `include`) rather than retrying everything: a `PermanentFailure` here skips the
|
||||
retry topics entirely and goes straight to `invoices-dlt`.
|
||||
|
||||
## `@DltHandler`
|
||||
|
||||
```java
|
||||
@DltHandler
|
||||
public void onDlt(ConsumerRecord<String, Payment> record,
|
||||
@Header(KafkaHeaders.RECEIVED_TOPIC) String topic) { ... }
|
||||
```
|
||||
|
||||
Without one, the framework still creates and populates the DLT — it just logs and moves on, and
|
||||
nothing in your application has looked at the record. A `@DltHandler` that increments a counter
|
||||
and writes a structured log line is the minimum worth having, because a DLT nobody watches is a
|
||||
queue that grows until someone notices the disk.
|
||||
|
||||
[Module README](../README.md)
|
||||
Reference in New Issue
Block a user