Ankur Mhatre
e7e64a047b
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
35 lines
1.3 KiB
Plaintext
35 lines
1.3 KiB
Plaintext
== RabbitMQ: consumers scale, and prefetch decides whether they actually do ==
|
|
|
|
one queue, 40 messages, 5 consumers, each taking 10 ms per message
|
|
|
|
no basicQos at all (unlimited prefetch, the AMQP default):
|
|
consumer-1 -> 40 messages
|
|
consumer-2 -> 0 messages
|
|
consumer-3 -> 0 messages
|
|
consumer-4 -> 0 messages
|
|
consumer-5 -> 0 messages
|
|
consumers that received nothing : 4
|
|
|
|
basicQos(1):
|
|
consumer-1 -> 8 messages
|
|
consumer-2 -> 9 messages
|
|
consumer-3 -> 8 messages
|
|
consumer-4 -> 8 messages
|
|
consumer-5 -> 7 messages
|
|
consumers that received nothing : 0
|
|
|
|
Every consumer on a queue competes for the same messages, so adding
|
|
consumers adds throughput and there is no structural ceiling of the kind
|
|
Kafka's partition count imposes. But with the AMQP default the broker
|
|
pushes as many messages as a consumer will take, so whichever consumer
|
|
connects first can be handed the entire backlog while the others sit idle.
|
|
basicQos is not a tuning knob you get to postpone; it is what makes the
|
|
fan-out real. Spring AMQP sets it for you --
|
|
AbstractMessageListenerContainer.DEFAULT_PREFETCH_COUNT is 250 -- which is
|
|
better than unlimited and still large enough to concentrate a small
|
|
backlog on one consumer.
|
|
|
|
The price of all this is the ordering measurement: there is no key, so
|
|
nothing constrains related messages to one consumer.
|
|
|