Add the broker-comparison module
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
This commit is contained in:
59
broker-comparison/docs/03-replay.md
Normal file
59
broker-comparison/docs/03-replay.md
Normal file
@@ -0,0 +1,59 @@
|
||||
prev: [Ordering](02-ordering.md) · [README](../README.md) · next: [Consumer scaling](04-consumer-scaling.md)
|
||||
|
||||
# 3. Replay
|
||||
|
||||
"Can I read that again?" is asked after a bad deploy, and by then it is too late to change
|
||||
brokers.
|
||||
|
||||
## Kafka
|
||||
|
||||
```
|
||||
group replay-group-1, first read : 12 records
|
||||
group replay-group-2, brand new group : 12 records
|
||||
group replay-group-1, after seekToBeginning: 12 records
|
||||
```
|
||||
|
||||
([`docs/output/kafka-replay.txt`](output/kafka-replay.txt)) Consuming moves an offset and removes
|
||||
nothing. A new group, a reset offset and a `seekToBeginning` all read the same records, for as
|
||||
long as retention keeps them.
|
||||
|
||||
## RabbitMQ
|
||||
|
||||
```
|
||||
first drain of the queue : 12 messages
|
||||
second drain of the queue : 0 messages
|
||||
queue depth afterwards : 0
|
||||
```
|
||||
|
||||
([`docs/output/rabbit-replay.txt`](output/rabbit-replay.txt)) Acknowledging deletes. There is no
|
||||
offset to rewind and no second reader that can see what the first consumed. Reading a message
|
||||
twice has to be arranged in advance — a second queue bound to the same exchange, or a copy written
|
||||
somewhere else — and it cannot be arranged afterwards.
|
||||
|
||||
This is not a defect. A router that deletes what it has delivered is much cheaper to operate than
|
||||
a log, and most work queues genuinely do not need history. It only becomes a defect at the moment
|
||||
you need history and do not have it.
|
||||
|
||||
RabbitMQ streams (3.9+) are a separate, log-shaped feature that does support replay. They are a
|
||||
different thing living in the same broker, with their own client protocol and their own semantics
|
||||
— worth knowing about, and not what you get from `queueDeclare`.
|
||||
|
||||
## Pulsar
|
||||
|
||||
```
|
||||
subscription replay-sub, first read : 12 messages
|
||||
subscription replay-sub, after seek(earliest): 12 messages
|
||||
subscription replay-sub-2, brand new : 12 messages
|
||||
```
|
||||
|
||||
([`docs/output/pulsar-replay.txt`](output/pulsar-replay.txt)) `seek(MessageId)` and
|
||||
`seek(timestamp)` rewind a live subscription; a new subscription starting from `Earliest` reads
|
||||
everything still stored.
|
||||
|
||||
The difference from Kafka is the **default**, and it is the one that surprises people: Kafka keeps
|
||||
a record for the retention period regardless of who read it, while Pulsar deletes a message once
|
||||
every subscription has acknowledged it, unless a retention policy on the namespace says otherwise.
|
||||
A Pulsar namespace with the default retention and one well-behaved subscription keeps nothing —
|
||||
so the replay you are counting on requires a policy you have to set.
|
||||
|
||||
next: [Consumer scaling](04-consumer-scaling.md)
|
||||
Reference in New Issue
Block a user