Ankur Mhatre
e1f8aa7402
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
2.5 KiB
prev: Ordering · README · next: Consumer scaling
3. Replay
"Can I read that again?" is asked after a bad deploy, and by then it is too late to change brokers.
Kafka
group replay-group-1, first read : 12 records
group replay-group-2, brand new group : 12 records
group replay-group-1, after seekToBeginning: 12 records
(docs/output/kafka-replay.txt) Consuming moves an offset and removes
nothing. A new group, a reset offset and a seekToBeginning all read the same records, for as
long as retention keeps them.
RabbitMQ
first drain of the queue : 12 messages
second drain of the queue : 0 messages
queue depth afterwards : 0
(docs/output/rabbit-replay.txt) Acknowledging deletes. There is no
offset to rewind and no second reader that can see what the first consumed. Reading a message
twice has to be arranged in advance — a second queue bound to the same exchange, or a copy written
somewhere else — and it cannot be arranged afterwards.
This is not a defect. A router that deletes what it has delivered is much cheaper to operate than a log, and most work queues genuinely do not need history. It only becomes a defect at the moment you need history and do not have it.
RabbitMQ streams (3.9+) are a separate, log-shaped feature that does support replay. They are a
different thing living in the same broker, with their own client protocol and their own semantics
— worth knowing about, and not what you get from queueDeclare.
Pulsar
subscription replay-sub, first read : 12 messages
subscription replay-sub, after seek(earliest): 12 messages
subscription replay-sub-2, brand new : 12 messages
(docs/output/pulsar-replay.txt) seek(MessageId) and
seek(timestamp) rewind a live subscription; a new subscription starting from Earliest reads
everything still stored.
The difference from Kafka is the default, and it is the one that surprises people: Kafka keeps a record for the retention period regardless of who read it, while Pulsar deletes a message once every subscription has acknowledged it, unless a retention policy on the namespace says otherwise. A Pulsar namespace with the default retention and one well-behaved subscription keeps nothing — so the replay you are counting on requires a policy you have to set.
next: Consumer scaling