Add the kafka-error-handling module
This commit is contained in:
53
kafka-error-handling/docs/01-two-kinds-of-failure.md
Normal file
53
kafka-error-handling/docs/01-two-kinds-of-failure.md
Normal file
@@ -0,0 +1,53 @@
|
||||
[Module README](../README.md) · [DefaultErrorHandler →](02-default-error-handler.md)
|
||||
|
||||
# 1. Two kinds of failure, and why they need different machinery
|
||||
|
||||
Kafka delivery is at-least-once. The container commits offsets after your listener returns
|
||||
([the basics article](https://ankurm.com/spring-boot-4-1-kafka-producer-consumer-serialisation/)
|
||||
covers why), so if the listener throws, the offset does not move and the record comes back.
|
||||
Everything in this module is about what happens next.
|
||||
|
||||
There are two failures, and conflating them is why a bad record can take a partition down for
|
||||
hours.
|
||||
|
||||
## Failure inside the listener
|
||||
|
||||
Your code threw. The record deserialized fine; the work failed. The container catches it, hands
|
||||
it to a `CommonErrorHandler`, and that decides whether to retry, how long to wait, and what to do
|
||||
when the attempts run out.
|
||||
|
||||
This splits again, and the split matters more than any back-off setting:
|
||||
|
||||
| | example | retrying it |
|
||||
|---|---|---|
|
||||
| **transient** | timeout, 503, deadlock, connection reset | may succeed |
|
||||
| **permanent** | validation failure, missing entity, malformed field | will fail identically |
|
||||
|
||||
Retrying a permanent failure ten times buys nothing and costs ten times the latency plus nine
|
||||
misleading log lines. Spring Kafka lets you say so:
|
||||
|
||||
```java
|
||||
handler.addNotRetryableExceptions(PermanentFailure.class);
|
||||
```
|
||||
|
||||
That single line is worth more than tuning the back-off, and it is the one most people skip.
|
||||
|
||||
## Failure before the listener
|
||||
|
||||
The bytes on the topic are not what the deserializer expects. Someone changed a schema, or
|
||||
published with a different serializer, or your `__TypeId__` header names a class you do not
|
||||
trust.
|
||||
|
||||
This one is nastier, because it happens **inside `poll()`**, before any listener exists to throw
|
||||
from. There is no error handler in the path. The consumer cannot advance past the offset,
|
||||
retries the same record on the next poll, fails again, and does that forever — at whatever rate
|
||||
the poll loop runs. The partition is stopped and the only symptom is a growing lag with a
|
||||
consumer that looks healthy.
|
||||
|
||||
That is a **poison pill**, and the cure is a different mechanism from the one above:
|
||||
`ErrorHandlingDeserializer`, covered in [chapter 3](03-poison-pills.md).
|
||||
|
||||
Keeping these two apart is the whole point of this module. One needs a retry policy; the other
|
||||
needs a wrapper around the deserializer. Neither fixes the other.
|
||||
|
||||
[DefaultErrorHandler →](02-default-error-handler.md)
|
||||
Reference in New Issue
Block a user