61 lines
2.9 KiB
Markdown
61 lines
2.9 KiB
Markdown
# `kafka-basics` — producer, consumer and serialisation, from nothing
|
|
|
|
Companion project for
|
|
[**Spring Boot 4.1 and Apache Kafka: Producer, Consumer and Serialisation from Scratch**](https://ankurm.com/spring-boot-4-1-kafka-producer-consumer-serialisation/)
|
|
on ankurm.com.
|
|
|
|
Eight tests against a **real Kafka broker**, started in-process in KRaft mode. No Docker daemon,
|
|
no local Kafka install, no ZooKeeper. `./scripts/run-all.sh` regenerates everything under
|
|
[`docs/output/`](docs/output/).
|
|
|
|
## Versions
|
|
|
|
| | Version | Notes |
|
|
|---|---|---|
|
|
| JDK | 25 (Temurin 25.0.4.1+1) | current LTS |
|
|
| Spring Boot | 4.1.1 | GA of the 4.1 line was 10 June 2026 |
|
|
| Spring Kafka | 4.1.1 | Boot-managed |
|
|
| kafka-clients | **4.2.1** | Boot-managed. Central has 4.3.1; let the BOM decide |
|
|
| Jackson | 3.1.5 (`tools.jackson`) | why `JacksonJsonSerializer` and not `JsonSerializer` |
|
|
| Testcontainers | 2.0.5 | artifact is `testcontainers-kafka`, not `kafka` |
|
|
|
|
Versions were read from `repo1.maven.org/.../maven-metadata.xml` and from Boot's own
|
|
`spring-boot-dependencies` POM, not from release announcements.
|
|
|
|
## Quickstart
|
|
|
|
```bash
|
|
./scripts/run-all.sh # runs every test and regenerates docs/output/
|
|
mvn test # the same thing without the capture
|
|
```
|
|
|
|
## Documentation
|
|
|
|
1. [The on-ramp, and the dependency that is not the one you remember](docs/01-the-on-ramp.md)
|
|
2. [Producing, and the return value everybody throws away](docs/02-producing.md)
|
|
3. [Serialisation: two Jackson families](docs/03-serialisation.md)
|
|
4. [Keys and partitions: the ordering guarantee in disguise](docs/04-keys-and-partitions.md)
|
|
5. [Consuming](docs/05-consuming.md)
|
|
6. [Acknowledgement, and a property that is not where you look for it](docs/06-acknowledgement.md)
|
|
7. [Testing without installing Kafka](docs/07-testing.md)
|
|
|
|
## Captured output
|
|
|
|
| File | Produced by |
|
|
|---|---|
|
|
| [`effective-config.txt`](docs/output/effective-config.txt) | client defaults vs Boot's overrides vs effective |
|
|
| [`key-to-partition.txt`](docs/output/key-to-partition.txt) | real placement against a real broker |
|
|
| [`serialised-payload.txt`](docs/output/serialised-payload.txt) | the Jackson 3 wire format |
|
|
| [`tests.txt`](docs/output/tests.txt) | 8 tests |
|
|
|
|
## Four things this module exists to prove
|
|
|
|
1. **`spring-kafka` alone gives you no auto-configuration under Boot 4.** You need
|
|
`spring-boot-starter-kafka`; the symptom is a missing `KafkaTemplate` bean.
|
|
2. **`JsonSerializer` is the Jackson 2 one.** It cannot write a `java.time.Instant` with its
|
|
default mapper. `JacksonJsonSerializer` is the Jackson 3 one and can.
|
|
3. **The default partitioner is `murmur2 & 0x7fffffff`, not `Math.abs(murmur2)`.** They disagree
|
|
on two of the six keys in the committed transcript.
|
|
4. **`ConsumerFactory.isAutoCommit()` answers `true` while no consumer auto-commits.** The
|
|
container overrides the property per-consumer and never tells the factory.
|