60 lines
2.7 KiB
Markdown
60 lines
2.7 KiB
Markdown
# `kafka-error-handling` — retries, DLT and poison pills
|
|
|
|
Companion project for
|
|
[**Kafka Error Handling with Spring Kafka 4.1: DLT, Retry Topics and Poison Pills**](https://ankurm.com/spring-kafka-4-1-error-handling-dlt-retry-topics/)
|
|
on ankurm.com.
|
|
|
|
Six tests against a **real Kafka broker** started in-process in KRaft mode. No Docker, no local
|
|
install. `./scripts/run-all.sh` regenerates everything under [`docs/output/`](docs/output/).
|
|
|
|
## Versions
|
|
|
|
| | Version |
|
|
|---|---|
|
|
| JDK | 25 (Temurin 25.0.4.1+1) |
|
|
| Spring Boot | 4.1.1 |
|
|
| Spring Kafka | 4.1.1 |
|
|
| kafka-clients | 4.2.1 (Boot-managed) |
|
|
|
|
## Profiles
|
|
|
|
| Profile | What it wires |
|
|
|---|---|
|
|
| `dlt` | `DefaultErrorHandler` + `DeadLetterPublishingRecoverer`, `FixedBackOff(1000, 2)`, `PermanentFailure` classified non-retryable |
|
|
| `dltbytes` | the same, with a `byte[]`-aware template map so poison pills keep their original bytes |
|
|
| `retrytopic` | `@RetryableTopic` non-blocking retries with a `@DltHandler` |
|
|
| `defaults` | the stock `DefaultErrorHandler`, for reading its behaviour |
|
|
|
|
## Documentation
|
|
|
|
1. [Two kinds of failure, and why they need different machinery](docs/01-two-kinds-of-failure.md)
|
|
2. [What the default actually does](docs/02-default-error-handler.md)
|
|
3. [Poison pills](docs/03-poison-pills.md)
|
|
4. [The dead-letter topic](docs/04-the-dlt.md)
|
|
5. [Non-blocking retries with `@RetryableTopic`](docs/05-retryable-topic.md)
|
|
|
|
## Captured output
|
|
|
|
| File | Shows |
|
|
|---|---|
|
|
| [`default-backoff.txt`](docs/output/default-backoff.txt) | ten deliveries, zero delay |
|
|
| [`retry-and-dlt.txt`](docs/output/retry-and-dlt.txt) | measured back-off and the DLT headers |
|
|
| [`poison-pill.txt`](docs/output/poison-pill.txt) | base64 payload, and the byte-aware fix |
|
|
| [`retry-topics.txt`](docs/output/retry-topics.txt) | the non-blocking delivery trace |
|
|
| [`tests.txt`](docs/output/tests.txt) | 6 tests |
|
|
|
|
## Six things this module exists to prove
|
|
|
|
1. **The default is ten deliveries, zero milliseconds apart, then the record is dropped.** Not
|
|
"retry with backoff", and not "dead-letter".
|
|
2. **The DLT suffix is `-dlt`, not `.DLT`.** Get it wrong and the recoverer logs a WARN and the
|
|
record is lost — your safety net silently drops it.
|
|
3. **`kafka_dlt-exception-fqcn` is always `ListenerExecutionFailedException`** for listener
|
|
failures. The useful header is `-exception-cause-fqcn`.
|
|
4. **A poison pill reaches the DLT base64-encoded**, because the recoverer reuses the JSON
|
|
producer. A per-type template map fixes it; the module shows both transcripts.
|
|
5. **`@RetryableTopic` names retry topics by delay** — `-retry-500`, `-retry-1000` — so changing
|
|
the multiplier renames them.
|
|
6. **`@Backoff` from spring-retry no longer exists here.** Spring Kafka 4 ships its own
|
|
`@BackOff`, and the attribute is `backOff`.
|