Add the kafka-basics module
This commit is contained in:
82
kafka-basics/docs/03-serialisation.md
Normal file
82
kafka-basics/docs/03-serialisation.md
Normal file
@@ -0,0 +1,82 @@
|
||||
[← Producing](02-producing.md) · [Module README](../README.md) · [Keys and partitions →](04-keys-and-partitions.md)
|
||||
|
||||
# 3. Serialisation: two Jackson families, and the one your IDE will suggest is the wrong one
|
||||
|
||||
Kafka moves `byte[]`. Everything else is a `Serializer` and a `Deserializer`, and under Spring
|
||||
Boot 4 there is a fork in the road that no tutorial mentions yet.
|
||||
|
||||
Spring Kafka 4.1 ships **two complete JSON families**:
|
||||
|
||||
| Class | Jackson | Mapper type |
|
||||
|---|---|---|
|
||||
| `JsonSerializer` / `JsonDeserializer` / `JsonSerde` | 2.x | `com.fasterxml.jackson.databind.ObjectMapper` |
|
||||
| `JacksonJsonSerializer` / `JacksonJsonDeserializer` / `JacksonJsonSerde` | 3.x | `tools.jackson.databind.json.JsonMapper` |
|
||||
|
||||
Spring Boot 4 is a Jackson 3 application. `spring-boot-starter-jackson` brings
|
||||
`tools.jackson.core:jackson-databind` 3.1.5. But the class named `JsonSerializer` — the one
|
||||
every existing example configures, and the one autocomplete offers first — is the Jackson 2 one.
|
||||
|
||||
It does not fail at startup. It fails on the first payload containing a `java.time` value:
|
||||
|
||||
```
|
||||
org.apache.kafka.common.errors.SerializationException: Can't serialize data
|
||||
[OrderEvent[orderId=o-1, ..., placedAt=2026-08-29T10:15:30Z]] for topic [orders]
|
||||
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException:
|
||||
Java 8 date/time type `java.time.Instant` not supported by default:
|
||||
add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling
|
||||
```
|
||||
|
||||
The message tells you to add a Jackson 2 module, and doing so works — and leaves you running two
|
||||
Jackson stacks, one for your HTTP layer and one for your messaging layer, with independent
|
||||
configuration. The better fix is one word:
|
||||
|
||||
```yaml
|
||||
spring:
|
||||
kafka:
|
||||
producer:
|
||||
value-serializer: org.springframework.kafka.support.serializer.JacksonJsonSerializer
|
||||
consumer:
|
||||
value-deserializer: org.springframework.kafka.support.serializer.JacksonJsonDeserializer
|
||||
```
|
||||
|
||||
Jackson 3 handles `java.time` with no module and no configuration, and `BigDecimal` keeps its
|
||||
scale through a round trip — both asserted in
|
||||
[`SerialisationTest`](../src/test/java/com/ankurm/kafkabasics/SerialisationTest.java):
|
||||
|
||||
```
|
||||
{"orderId":"o-1","customerId":"c-1","amount":10.00,"placedAt":"2026-08-29T10:15:30Z"}
|
||||
```
|
||||
|
||||
`amount` is `10.00`, not `10.0` and not `"10.00"`. Use `BigDecimal` for money and this survives;
|
||||
use `double` and it does not.
|
||||
|
||||
## The type header
|
||||
|
||||
The three-argument `serialize(topic, headers, data)` overload writes a `__TypeId__` header
|
||||
naming the class:
|
||||
|
||||
```java
|
||||
headers.lastHeader("__TypeId__") -> "com.ankurm.kafkabasics.OrderEvent"
|
||||
```
|
||||
|
||||
The consumer reads it and builds that class. Which is convenient and is also a remote-code-
|
||||
selection primitive, so the deserializer refuses any class outside its trusted packages:
|
||||
|
||||
```
|
||||
... is not in the trusted packages: [java.util, java.lang]
|
||||
```
|
||||
|
||||
Three ways out, in descending order of preference:
|
||||
|
||||
1. `spring.json.trusted.packages: com.ankurm.kafkabasics` — an allow-list of your own packages.
|
||||
2. `spring.json.value.default.type` — ignore the header, always build this class. Best when
|
||||
producer and consumer are owned by different teams, because it makes the consumer's contract
|
||||
its own decision rather than the producer's.
|
||||
3. `spring.json.trusted.packages: "*"` — do not.
|
||||
|
||||
Option 2 has a second benefit worth stating plainly: the `__TypeId__` header couples the two
|
||||
services by **fully qualified class name**. Renaming or moving a class in the producer breaks
|
||||
every consumer that trusts the header, at runtime, with no compile-time warning. `type.mapping`
|
||||
(a logical name to class map on both sides) is the version of this that survives a refactor.
|
||||
|
||||
[Keys and partitions →](04-keys-and-partitions.md)
|
||||
Reference in New Issue
Block a user