1
0

Add the broker-comparison module

Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and
consumer scaling by driving the three client libraries directly, plus an
operational-footprint measurement of each broker's own distribution.

Nine tests, three brokers started without Docker, and every number in the
documentation regenerated by scripts/run-all.sh.
This commit is contained in:
2026-09-01 23:27:08 +05:30
parent f4b0112e2c
commit e1f8aa7402
30 changed files with 1835 additions and 7 deletions

View File

@@ -0,0 +1,40 @@
== Operational footprint, measured on one 2-core / 3.8 GB Linux box ==
Each broker started from its shipped distribution with default configuration and a
700 MB heap cap, on Temurin JDK 21. Times are one sample on a small box: treat them as
orders of magnitude, not as a benchmark.
broker : kafka
time from launch to first
accepted connection : 10035 ms
resident memory at idle : 333 MB
server processes : 1
listening ports : 9092 (broker), 9093 (controller)
unpacked distribution size : 135 MB
settings in shipped default
configuration file : 24 (server.properties)
broker : rabbit
time from launch to first
accepted connection : 15053 ms
resident memory at idle : 115 MB
server processes : 1
listening ports : 5672 (AMQP), 4369 (epmd), 25672 (inter-node)
unpacked distribution size : 26 MB
settings in shipped default
configuration file : 0 (rabbitmq.conf.example)
epmd processes : 1
NOTE: RabbitMQ ships no configuration file at all. rabbitmq.conf.example is
entirely commented out and the boot log records "Config file(s): (none)".
Everything in that file is documentation, not a default.
broker : pulsar
time from launch to first
accepted connection : 20801 ms
resident memory at idle : 610 MB
server processes : 1
listening ports : 6650 (binary), 8080 (admin/REST), 2181 (ZooKeeper)
unpacked distribution size : 344 MB
settings in shipped default
configuration file : 357 (standalone.conf)

View File

@@ -0,0 +1,16 @@
== Kafka: partitions are the ceiling on consumer parallelism ==
topic 'orders-scaling', 3 partitions, 5 consumers in one group
consumer-1 -> partitions [0]
consumer-2 -> partitions [1]
consumer-3 -> partitions [2]
consumer-4 -> no partitions (idle)
consumer-5 -> no partitions (idle)
consumers with no partitions: 2
A partition is assigned to at most one consumer in a group, so the number
of partitions is a hard ceiling on consumer parallelism. Adding consumers
beyond it adds idle processes, not throughput.

View File

@@ -0,0 +1,30 @@
== Kafka: ordering is per partition ==
produced 12 records, keys D/A/F round-robin, values 1..12 in order
(keys chosen so that murmur2 spreads them: D->0, A->1, F->2. A, B and C
all hash to partition 1 with three partitions, which is worth knowing
before you decide your keys are well distributed.)
consumed in this order:
F=2@p2
F=5@p2
F=8@p2
F=11@p2
A=1@p1
A=4@p1
A=7@p1
A=10@p1
D=3@p0
D=6@p0
D=9@p0
D=12@p0
per key:
F -> partition [2] values [2, 5, 8, 11]
A -> partition [1] values [1, 4, 7, 10]
D -> partition [0] values [3, 6, 9, 12]
Order is preserved within each key because a key hashes to one partition.
Across keys it is not: the values above are not 1..12 in order, because a
consumer drains one partition's buffer before the next.

View File

@@ -0,0 +1,10 @@
== Kafka: the log is the storage ==
group replay-group-1, first read : 12 records
group replay-group-2, brand new group : 12 records
group replay-group-1, after seekToBeginning: 12 records
Consuming does not remove anything. A consumer group is a cursor over a log
that the broker keeps until retention expires, so a new group, a reset
offset or a seek all read the same records again.

View File

@@ -0,0 +1,30 @@
== Pulsar: no partition ceiling, but the receiver queue decides who gets the work ==
one non-partitioned topic, 40 messages, 5 consumers, Shared subscription
receiverQueueSize left at the default (1000):
consumer-1 -> 40 messages
consumer-2 -> 0 messages
consumer-3 -> 0 messages
consumer-4 -> 0 messages
consumer-5 -> 0 messages
consumers that received nothing : 4
receiverQueueSize(1):
consumer-1 -> 8 messages
consumer-2 -> 8 messages
consumer-3 -> 8 messages
consumer-4 -> 8 messages
consumer-5 -> 8 messages
consumers that received nothing : 0
The topic has no partitions and five consumers can still share the work --
in Kafka the same shape needs at least five partitions, chosen when the
topic was created. But the default receiver queue is 1000 messages, so the
first consumers to connect pull the whole 40-message backlog into their own
buffers before the rest ask for anything, and the subscription looks
broken. Which consumers win is a race and varies between runs -- one
consumer taking all forty, or two taking twenty each -- but the consumers
that lose it see nothing at all. This is the same trap as RabbitMQ's
unbounded prefetch, with a different name and a much larger default.

View File

@@ -0,0 +1,24 @@
== Pulsar: the subscription type decides ==
12 messages, keys A/B/C, values 1..12 in order, two consumers per
subscription, receiverQueueSize 1 so the first consumer cannot take the
whole backlog.
Shared subscription, which consumers saw each key:
A -> [consumer-1, consumer-2]
B -> [consumer-1, consumer-2]
C -> [consumer-1, consumer-2]
Key_Shared subscription, which consumers saw each key:
A -> [consumer-1]
B -> [consumer-1]
C -> [consumer-1]
A Shared subscription round-robins individual messages, so messages with
the same key end up on different consumers and can be processed at the
same time: there is no per-key order left to speak of. Key_Shared hashes
the key to one consumer, which is Kafka's guarantee -- except that the
assignment belongs to the subscription and is recomputed as consumers come
and go, rather than being fixed by a partition count chosen when the topic
was created.

View File

@@ -0,0 +1,14 @@
== Pulsar: acknowledged, but still there ==
subscription replay-sub, first read : 12 messages
subscription replay-sub, after seek(earliest): 12 messages
subscription replay-sub-2, brand new : 12 messages
Acknowledgement moves a cursor; the message itself lives in the managed
ledger. seek(MessageId) and seek(timestamp) rewind a live subscription,
which Kafka can also do by resetting offsets. What differs is the default:
Pulsar deletes a message once every subscription has acknowledged it,
unless a retention policy on the namespace says otherwise, whereas Kafka
keeps it for the retention period regardless of who read it. A Pulsar
topic with no retention policy and no subscriptions keeps nothing.

View File

@@ -0,0 +1,34 @@
== RabbitMQ: consumers scale, and prefetch decides whether they actually do ==
one queue, 40 messages, 5 consumers, each taking 10 ms per message
no basicQos at all (unlimited prefetch, the AMQP default):
consumer-1 -> 40 messages
consumer-2 -> 0 messages
consumer-3 -> 0 messages
consumer-4 -> 0 messages
consumer-5 -> 0 messages
consumers that received nothing : 4
basicQos(1):
consumer-1 -> 8 messages
consumer-2 -> 9 messages
consumer-3 -> 8 messages
consumer-4 -> 8 messages
consumer-5 -> 7 messages
consumers that received nothing : 0
Every consumer on a queue competes for the same messages, so adding
consumers adds throughput and there is no structural ceiling of the kind
Kafka's partition count imposes. But with the AMQP default the broker
pushes as many messages as a consumer will take, so whichever consumer
connects first can be handed the entire backlog while the others sit idle.
basicQos is not a tuning knob you get to postpone; it is what makes the
fan-out real. Spring AMQP sets it for you --
AbstractMessageListenerContainer.DEFAULT_PREFETCH_COUNT is 250 -- which is
better than unlimited and still large enough to concentrate a small
backlog on one consumer.
The price of all this is the ordering measurement: there is no key, so
nothing constrains related messages to one consumer.

View File

@@ -0,0 +1,18 @@
== RabbitMQ: FIFO per queue, per consumer ==
one queue, one consumer, 12 messages
completion order : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
one queue, two consumers, prefetch 1, consumer-1 slower than consumer-2
completion order : [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1, 12]
out-of-order steps: 1
A queue is FIFO and a single consumer sees it that way. The moment a second
consumer is added the broker hands the next message to whichever consumer
is free, so the order in which work finishes is no longer the order in
which it was published. There is no key: RabbitMQ has no notion of a
partition to which related messages could be pinned. Ordering across
related messages means one queue and one consumer, and therefore no
horizontal scaling for that queue -- or a consistent-hash exchange, which
is a plugin.

View File

@@ -0,0 +1,12 @@
== RabbitMQ: there is nothing to replay ==
first drain of the queue : 12 messages
second drain of the queue : 0 messages
queue depth afterwards : 0
Acknowledging a message deletes it. The broker is a router with buffers,
not a log: there is no offset to rewind and no second reader that can see
what the first one consumed. Reading the same message twice means
arranging it in advance -- a second queue bound to the same exchange, or a
copy written somewhere else -- and it cannot be arranged after the fact.

View File

@@ -0,0 +1,12 @@
[INFO] Running com.ankurm.brokers.KafkaComparisonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.88 s -- in com.ankurm.brokers.KafkaComparisonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS
[INFO] Running com.ankurm.brokers.RabbitComparisonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.732 s -- in com.ankurm.brokers.RabbitComparisonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS
[INFO] Running com.ankurm.brokers.PulsarComparisonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.00 s -- in com.ankurm.brokers.PulsarComparisonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS