Ankur Mhatre
e1f8aa7402
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
31 lines
1.2 KiB
Plaintext
31 lines
1.2 KiB
Plaintext
== Pulsar: no partition ceiling, but the receiver queue decides who gets the work ==
|
|
|
|
one non-partitioned topic, 40 messages, 5 consumers, Shared subscription
|
|
|
|
receiverQueueSize left at the default (1000):
|
|
consumer-1 -> 40 messages
|
|
consumer-2 -> 0 messages
|
|
consumer-3 -> 0 messages
|
|
consumer-4 -> 0 messages
|
|
consumer-5 -> 0 messages
|
|
consumers that received nothing : 4
|
|
|
|
receiverQueueSize(1):
|
|
consumer-1 -> 8 messages
|
|
consumer-2 -> 8 messages
|
|
consumer-3 -> 8 messages
|
|
consumer-4 -> 8 messages
|
|
consumer-5 -> 8 messages
|
|
consumers that received nothing : 0
|
|
|
|
The topic has no partitions and five consumers can still share the work --
|
|
in Kafka the same shape needs at least five partitions, chosen when the
|
|
topic was created. But the default receiver queue is 1000 messages, so the
|
|
first consumers to connect pull the whole 40-message backlog into their own
|
|
buffers before the rest ask for anything, and the subscription looks
|
|
broken. Which consumers win is a race and varies between runs -- one
|
|
consumer taking all forty, or two taking twenty each -- but the consumers
|
|
that lose it see nothing at all. This is the same trap as RabbitMQ's
|
|
unbounded prefetch, with a different name and a much larger default.
|
|
|