Ankur Mhatre
e7e64a047b
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
60 lines
2.5 KiB
Markdown
60 lines
2.5 KiB
Markdown
prev: [Ordering](02-ordering.md) · [README](../README.md) · next: [Consumer scaling](04-consumer-scaling.md)
|
|
|
|
# 3. Replay
|
|
|
|
"Can I read that again?" is asked after a bad deploy, and by then it is too late to change
|
|
brokers.
|
|
|
|
## Kafka
|
|
|
|
```
|
|
group replay-group-1, first read : 12 records
|
|
group replay-group-2, brand new group : 12 records
|
|
group replay-group-1, after seekToBeginning: 12 records
|
|
```
|
|
|
|
([`docs/output/kafka-replay.txt`](output/kafka-replay.txt)) Consuming moves an offset and removes
|
|
nothing. A new group, a reset offset and a `seekToBeginning` all read the same records, for as
|
|
long as retention keeps them.
|
|
|
|
## RabbitMQ
|
|
|
|
```
|
|
first drain of the queue : 12 messages
|
|
second drain of the queue : 0 messages
|
|
queue depth afterwards : 0
|
|
```
|
|
|
|
([`docs/output/rabbit-replay.txt`](output/rabbit-replay.txt)) Acknowledging deletes. There is no
|
|
offset to rewind and no second reader that can see what the first consumed. Reading a message
|
|
twice has to be arranged in advance — a second queue bound to the same exchange, or a copy written
|
|
somewhere else — and it cannot be arranged afterwards.
|
|
|
|
This is not a defect. A router that deletes what it has delivered is much cheaper to operate than
|
|
a log, and most work queues genuinely do not need history. It only becomes a defect at the moment
|
|
you need history and do not have it.
|
|
|
|
RabbitMQ streams (3.9+) are a separate, log-shaped feature that does support replay. They are a
|
|
different thing living in the same broker, with their own client protocol and their own semantics
|
|
— worth knowing about, and not what you get from `queueDeclare`.
|
|
|
|
## Pulsar
|
|
|
|
```
|
|
subscription replay-sub, first read : 12 messages
|
|
subscription replay-sub, after seek(earliest): 12 messages
|
|
subscription replay-sub-2, brand new : 12 messages
|
|
```
|
|
|
|
([`docs/output/pulsar-replay.txt`](output/pulsar-replay.txt)) `seek(MessageId)` and
|
|
`seek(timestamp)` rewind a live subscription; a new subscription starting from `Earliest` reads
|
|
everything still stored.
|
|
|
|
The difference from Kafka is the **default**, and it is the one that surprises people: Kafka keeps
|
|
a record for the retention period regardless of who read it, while Pulsar deletes a message once
|
|
every subscription has acknowledged it, unless a retention policy on the namespace says otherwise.
|
|
A Pulsar namespace with the default retention and one well-behaved subscription keeps nothing —
|
|
so the replay you are counting on requires a policy you have to set.
|
|
|
|
next: [Consumer scaling](04-consumer-scaling.md)
|