Add the kafka-basics module
This commit is contained in:
67
kafka-basics/docs/02-producing.md
Normal file
67
kafka-basics/docs/02-producing.md
Normal file
@@ -0,0 +1,67 @@
|
||||
[← The on-ramp](01-the-on-ramp.md) · [Module README](../README.md) · [Serialisation →](03-serialisation.md)
|
||||
|
||||
# 2. Producing, and the return value everybody throws away
|
||||
|
||||
[`OrderProducer`](../src/main/java/com/ankurm/kafkabasics/OrderProducer.java) has three send
|
||||
methods because there are exactly three useful answers to "when do I find out this failed".
|
||||
|
||||
```java
|
||||
public void sendAndForget(OrderEvent event) {
|
||||
this.template.send(TOPIC, event.orderId(), event); // returns a CompletableFuture. Ignored.
|
||||
}
|
||||
```
|
||||
|
||||
`KafkaTemplate.send` is asynchronous and returns a `CompletableFuture<SendResult<K, V>>`.
|
||||
Discarding it discards the only notification you will get. The method returns normally, the
|
||||
record may never reach the broker, and nothing in your logs says so — the producer's own retry
|
||||
and expiry messages are at `WARN` under `org.apache.kafka`, which most applications turn down.
|
||||
|
||||
This is the most common way to lose messages in a Spring Kafka application, and it looks like
|
||||
correct code.
|
||||
|
||||
```java
|
||||
this.template.send(TOPIC, event.orderId(), event)
|
||||
.whenComplete((result, ex) -> { /* log, meter, compensate */ });
|
||||
```
|
||||
|
||||
Handle the future, or block on it when the caller genuinely must not proceed without a durable
|
||||
write:
|
||||
|
||||
```java
|
||||
SendResult<String, OrderEvent> result = this.template.send(record).get();
|
||||
result.getRecordMetadata().partition(); // where it landed
|
||||
result.getRecordMetadata().offset(); // and at what offset
|
||||
```
|
||||
|
||||
Blocking costs a network round trip plus the replication acknowledgement, so it belongs at the
|
||||
edges of a system, not inside a loop.
|
||||
|
||||
## Durability comes from defaults you did not set
|
||||
|
||||
Spring Boot sets **nothing** on the producer beyond bootstrap servers and serializers. Here is
|
||||
the effective configuration, printed by
|
||||
[`EffectiveConfigTest`](../src/test/java/com/ankurm/kafkabasics/EffectiveConfigTest.java) and
|
||||
committed at [`docs/output/effective-config.txt`](output/effective-config.txt):
|
||||
|
||||
```
|
||||
property kafka-clients default set by Spring Boot
|
||||
acks all -
|
||||
enable.idempotence true -
|
||||
retries 2147483647 -
|
||||
max.in.flight.requests.per.connection 5 -
|
||||
delivery.timeout.ms 120000 -
|
||||
```
|
||||
|
||||
Since Kafka 3.0 the client defaults are `acks=all` and `enable.idempotence=true`, so a stock
|
||||
Boot application already has a durable, deduplicating producer. Two consequences:
|
||||
|
||||
- **You do not need to set `acks=all`.** It is already on.
|
||||
- **An old runbook that sets `acks=1` or `retries=0` is now a downgrade.** Those lines were
|
||||
written when the defaults were weaker, and deleting them makes the system safer, which is the
|
||||
opposite of how configuration usually ages.
|
||||
|
||||
`delivery.timeout.ms` at two minutes is the one worth revisiting: it is the total budget for a
|
||||
send including retries, and a request that exceeds it fails permanently. If your future never
|
||||
completes, that is the clock you are waiting on.
|
||||
|
||||
[Serialisation →](03-serialisation.md)
|
||||
Reference in New Issue
Block a user