54 lines
2.4 KiB
Markdown
54 lines
2.4 KiB
Markdown
[Module README](../README.md) · [DefaultErrorHandler →](02-default-error-handler.md)
|
|
|
|
# 1. Two kinds of failure, and why they need different machinery
|
|
|
|
Kafka delivery is at-least-once. The container commits offsets after your listener returns
|
|
([the basics article](https://ankurm.com/spring-boot-4-1-kafka-producer-consumer-serialisation/)
|
|
covers why), so if the listener throws, the offset does not move and the record comes back.
|
|
Everything in this module is about what happens next.
|
|
|
|
There are two failures, and conflating them is why a bad record can take a partition down for
|
|
hours.
|
|
|
|
## Failure inside the listener
|
|
|
|
Your code threw. The record deserialized fine; the work failed. The container catches it, hands
|
|
it to a `CommonErrorHandler`, and that decides whether to retry, how long to wait, and what to do
|
|
when the attempts run out.
|
|
|
|
This splits again, and the split matters more than any back-off setting:
|
|
|
|
| | example | retrying it |
|
|
|---|---|---|
|
|
| **transient** | timeout, 503, deadlock, connection reset | may succeed |
|
|
| **permanent** | validation failure, missing entity, malformed field | will fail identically |
|
|
|
|
Retrying a permanent failure ten times buys nothing and costs ten times the latency plus nine
|
|
misleading log lines. Spring Kafka lets you say so:
|
|
|
|
```java
|
|
handler.addNotRetryableExceptions(PermanentFailure.class);
|
|
```
|
|
|
|
That single line is worth more than tuning the back-off, and it is the one most people skip.
|
|
|
|
## Failure before the listener
|
|
|
|
The bytes on the topic are not what the deserializer expects. Someone changed a schema, or
|
|
published with a different serializer, or your `__TypeId__` header names a class you do not
|
|
trust.
|
|
|
|
This one is nastier, because it happens **inside `poll()`**, before any listener exists to throw
|
|
from. There is no error handler in the path. The consumer cannot advance past the offset,
|
|
retries the same record on the next poll, fails again, and does that forever — at whatever rate
|
|
the poll loop runs. The partition is stopped and the only symptom is a growing lag with a
|
|
consumer that looks healthy.
|
|
|
|
That is a **poison pill**, and the cure is a different mechanism from the one above:
|
|
`ErrorHandlingDeserializer`, covered in [chapter 3](03-poison-pills.md).
|
|
|
|
Keeping these two apart is the whole point of this module. One needs a retry policy; the other
|
|
needs a wrapper around the deserializer. Neither fixes the other.
|
|
|
|
[DefaultErrorHandler →](02-default-error-handler.md)
|