spring-messaging-demo/kafka-basics/docs/05-consuming.md

← Keys and partitions · Module README · Acknowledgement →

5. Consuming

@KafkaListener(topics = "orders", groupId = "orders-basic")
public void onOrder(@Payload OrderEvent event,
        @Header(KafkaHeaders.RECEIVED_PARTITION) int partition,
        @Header(KafkaHeaders.OFFSET) long offset) { ... }

There is no poll loop, no offset commit and no rebalance listener, because the container owns all three. That is the actual value Spring Kafka adds over the plain client, and it is worth knowing what it is doing on your behalf — see chapter 6.

The single most common "my listener never fires"

auto.offset.reset decides where a consumer group with no committed offset starts. The kafka-clients default is latest: skip everything already in the topic. So a brand-new group attached to a topic full of records reads nothing, and the listener you just wrote never runs.

spring:
  kafka:
    consumer:
      auto-offset-reset: earliest

Boot does not change this default; the value in this module's application.yaml does. It only applies when there is no committed offset — once the group has committed once, this setting is irrelevant, which is why the symptom disappears the first time it works and never comes back.

The other reliable cause is a groupId mismatch between the annotation and spring.kafka.consumer.group-id. The annotation wins, and a typo there creates a brand-new group that then hits the paragraph above.

Concurrency and what it can and cannot buy

spring.kafka.listener.concurrency creates that many consumers in the group, each on its own thread. The ceiling is the partition count: with 3 partitions, concurrency: 10 gives you three working consumers and seven idle ones. There is no configuration that makes two threads consume one partition, because that would break the ordering guarantee from chapter 4.

Headers worth knowing

Constant	What it carries
KafkaHeaders.RECEIVED_KEY	the producer's key, through the key deserializer
KafkaHeaders.RECEIVED_PARTITION	which partition this came from
KafkaHeaders.OFFSET	the offset within that partition
KafkaHeaders.RECEIVED_TIMESTAMP	broker or producer timestamp, per topic config
__TypeId__	the class name the producer wrote — see chapter 3

Declare the key as required = false unless you are certain every record has one. A null key on a required = true header parameter fails the conversion, and the failure arrives as a deserialization-time error rather than as a null.

Acknowledgement →