Add the kafka-basics module
This commit is contained in:
59
kafka-basics/docs/05-consuming.md
Normal file
59
kafka-basics/docs/05-consuming.md
Normal file
@@ -0,0 +1,59 @@
|
||||
[← Keys and partitions](04-keys-and-partitions.md) · [Module README](../README.md) · [Acknowledgement →](06-acknowledgement.md)
|
||||
|
||||
# 5. Consuming
|
||||
|
||||
```java
|
||||
@KafkaListener(topics = "orders", groupId = "orders-basic")
|
||||
public void onOrder(@Payload OrderEvent event,
|
||||
@Header(KafkaHeaders.RECEIVED_PARTITION) int partition,
|
||||
@Header(KafkaHeaders.OFFSET) long offset) { ... }
|
||||
```
|
||||
|
||||
There is no poll loop, no offset commit and no rebalance listener, because the container owns
|
||||
all three. That is the actual value Spring Kafka adds over the plain client, and it is worth
|
||||
knowing what it is doing on your behalf — see [chapter 6](06-acknowledgement.md).
|
||||
|
||||
## The single most common "my listener never fires"
|
||||
|
||||
`auto.offset.reset` decides where a consumer group with **no committed offset** starts. The
|
||||
kafka-clients default is `latest`: skip everything already in the topic. So a brand-new group
|
||||
attached to a topic full of records reads nothing, and the listener you just wrote never runs.
|
||||
|
||||
```yaml
|
||||
spring:
|
||||
kafka:
|
||||
consumer:
|
||||
auto-offset-reset: earliest
|
||||
```
|
||||
|
||||
Boot does not change this default; the value in this module's `application.yaml` does. It only
|
||||
applies when there is no committed offset — once the group has committed once, this setting is
|
||||
irrelevant, which is why the symptom disappears the first time it works and never comes back.
|
||||
|
||||
The other reliable cause is a `groupId` mismatch between the annotation and
|
||||
`spring.kafka.consumer.group-id`. The annotation wins, and a typo there creates a brand-new group
|
||||
that then hits the paragraph above.
|
||||
|
||||
## Concurrency and what it can and cannot buy
|
||||
|
||||
`spring.kafka.listener.concurrency` creates that many consumers in the group, each on its own
|
||||
thread. The ceiling is the partition count: with 3 partitions, `concurrency: 10` gives you three
|
||||
working consumers and seven idle ones. There is no configuration that makes two threads consume
|
||||
one partition, because that would break the ordering guarantee from
|
||||
[chapter 4](04-keys-and-partitions.md).
|
||||
|
||||
## Headers worth knowing
|
||||
|
||||
| Constant | What it carries |
|
||||
|---|---|
|
||||
| `KafkaHeaders.RECEIVED_KEY` | the producer's key, through the key deserializer |
|
||||
| `KafkaHeaders.RECEIVED_PARTITION` | which partition this came from |
|
||||
| `KafkaHeaders.OFFSET` | the offset within that partition |
|
||||
| `KafkaHeaders.RECEIVED_TIMESTAMP` | broker or producer timestamp, per topic config |
|
||||
| `__TypeId__` | the class name the producer wrote — see [chapter 3](03-serialisation.md) |
|
||||
|
||||
Declare the key as `required = false` unless you are certain every record has one. A `null` key
|
||||
on a `required = true` header parameter fails the conversion, and the failure arrives as a
|
||||
deserialization-time error rather than as a null.
|
||||
|
||||
[Acknowledgement →](06-acknowledgement.md)
|
||||
Reference in New Issue
Block a user