Add the broker-comparison module
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
This commit is contained in:
40
broker-comparison/docs/output/footprint.txt
Normal file
40
broker-comparison/docs/output/footprint.txt
Normal file
@@ -0,0 +1,40 @@
|
||||
== Operational footprint, measured on one 2-core / 3.8 GB Linux box ==
|
||||
|
||||
Each broker started from its shipped distribution with default configuration and a
|
||||
700 MB heap cap, on Temurin JDK 21. Times are one sample on a small box: treat them as
|
||||
orders of magnitude, not as a benchmark.
|
||||
|
||||
broker : kafka
|
||||
time from launch to first
|
||||
accepted connection : 10035 ms
|
||||
resident memory at idle : 333 MB
|
||||
server processes : 1
|
||||
listening ports : 9092 (broker), 9093 (controller)
|
||||
unpacked distribution size : 135 MB
|
||||
settings in shipped default
|
||||
configuration file : 24 (server.properties)
|
||||
|
||||
broker : rabbit
|
||||
time from launch to first
|
||||
accepted connection : 15053 ms
|
||||
resident memory at idle : 115 MB
|
||||
server processes : 1
|
||||
listening ports : 5672 (AMQP), 4369 (epmd), 25672 (inter-node)
|
||||
unpacked distribution size : 26 MB
|
||||
settings in shipped default
|
||||
configuration file : 0 (rabbitmq.conf.example)
|
||||
epmd processes : 1
|
||||
NOTE: RabbitMQ ships no configuration file at all. rabbitmq.conf.example is
|
||||
entirely commented out and the boot log records "Config file(s): (none)".
|
||||
Everything in that file is documentation, not a default.
|
||||
|
||||
broker : pulsar
|
||||
time from launch to first
|
||||
accepted connection : 20801 ms
|
||||
resident memory at idle : 610 MB
|
||||
server processes : 1
|
||||
listening ports : 6650 (binary), 8080 (admin/REST), 2181 (ZooKeeper)
|
||||
unpacked distribution size : 344 MB
|
||||
settings in shipped default
|
||||
configuration file : 357 (standalone.conf)
|
||||
|
||||
16
broker-comparison/docs/output/kafka-consumer-scaling.txt
Normal file
16
broker-comparison/docs/output/kafka-consumer-scaling.txt
Normal file
@@ -0,0 +1,16 @@
|
||||
== Kafka: partitions are the ceiling on consumer parallelism ==
|
||||
|
||||
topic 'orders-scaling', 3 partitions, 5 consumers in one group
|
||||
|
||||
consumer-1 -> partitions [0]
|
||||
consumer-2 -> partitions [1]
|
||||
consumer-3 -> partitions [2]
|
||||
consumer-4 -> no partitions (idle)
|
||||
consumer-5 -> no partitions (idle)
|
||||
|
||||
consumers with no partitions: 2
|
||||
|
||||
A partition is assigned to at most one consumer in a group, so the number
|
||||
of partitions is a hard ceiling on consumer parallelism. Adding consumers
|
||||
beyond it adds idle processes, not throughput.
|
||||
|
||||
30
broker-comparison/docs/output/kafka-ordering.txt
Normal file
30
broker-comparison/docs/output/kafka-ordering.txt
Normal file
@@ -0,0 +1,30 @@
|
||||
== Kafka: ordering is per partition ==
|
||||
|
||||
produced 12 records, keys D/A/F round-robin, values 1..12 in order
|
||||
(keys chosen so that murmur2 spreads them: D->0, A->1, F->2. A, B and C
|
||||
all hash to partition 1 with three partitions, which is worth knowing
|
||||
before you decide your keys are well distributed.)
|
||||
|
||||
consumed in this order:
|
||||
F=2@p2
|
||||
F=5@p2
|
||||
F=8@p2
|
||||
F=11@p2
|
||||
A=1@p1
|
||||
A=4@p1
|
||||
A=7@p1
|
||||
A=10@p1
|
||||
D=3@p0
|
||||
D=6@p0
|
||||
D=9@p0
|
||||
D=12@p0
|
||||
|
||||
per key:
|
||||
F -> partition [2] values [2, 5, 8, 11]
|
||||
A -> partition [1] values [1, 4, 7, 10]
|
||||
D -> partition [0] values [3, 6, 9, 12]
|
||||
|
||||
Order is preserved within each key because a key hashes to one partition.
|
||||
Across keys it is not: the values above are not 1..12 in order, because a
|
||||
consumer drains one partition's buffer before the next.
|
||||
|
||||
10
broker-comparison/docs/output/kafka-replay.txt
Normal file
10
broker-comparison/docs/output/kafka-replay.txt
Normal file
@@ -0,0 +1,10 @@
|
||||
== Kafka: the log is the storage ==
|
||||
|
||||
group replay-group-1, first read : 12 records
|
||||
group replay-group-2, brand new group : 12 records
|
||||
group replay-group-1, after seekToBeginning: 12 records
|
||||
|
||||
Consuming does not remove anything. A consumer group is a cursor over a log
|
||||
that the broker keeps until retention expires, so a new group, a reset
|
||||
offset or a seek all read the same records again.
|
||||
|
||||
30
broker-comparison/docs/output/pulsar-consumer-scaling.txt
Normal file
30
broker-comparison/docs/output/pulsar-consumer-scaling.txt
Normal file
@@ -0,0 +1,30 @@
|
||||
== Pulsar: no partition ceiling, but the receiver queue decides who gets the work ==
|
||||
|
||||
one non-partitioned topic, 40 messages, 5 consumers, Shared subscription
|
||||
|
||||
receiverQueueSize left at the default (1000):
|
||||
consumer-1 -> 40 messages
|
||||
consumer-2 -> 0 messages
|
||||
consumer-3 -> 0 messages
|
||||
consumer-4 -> 0 messages
|
||||
consumer-5 -> 0 messages
|
||||
consumers that received nothing : 4
|
||||
|
||||
receiverQueueSize(1):
|
||||
consumer-1 -> 8 messages
|
||||
consumer-2 -> 8 messages
|
||||
consumer-3 -> 8 messages
|
||||
consumer-4 -> 8 messages
|
||||
consumer-5 -> 8 messages
|
||||
consumers that received nothing : 0
|
||||
|
||||
The topic has no partitions and five consumers can still share the work --
|
||||
in Kafka the same shape needs at least five partitions, chosen when the
|
||||
topic was created. But the default receiver queue is 1000 messages, so the
|
||||
first consumers to connect pull the whole 40-message backlog into their own
|
||||
buffers before the rest ask for anything, and the subscription looks
|
||||
broken. Which consumers win is a race and varies between runs -- one
|
||||
consumer taking all forty, or two taking twenty each -- but the consumers
|
||||
that lose it see nothing at all. This is the same trap as RabbitMQ's
|
||||
unbounded prefetch, with a different name and a much larger default.
|
||||
|
||||
24
broker-comparison/docs/output/pulsar-ordering.txt
Normal file
24
broker-comparison/docs/output/pulsar-ordering.txt
Normal file
@@ -0,0 +1,24 @@
|
||||
== Pulsar: the subscription type decides ==
|
||||
|
||||
12 messages, keys A/B/C, values 1..12 in order, two consumers per
|
||||
subscription, receiverQueueSize 1 so the first consumer cannot take the
|
||||
whole backlog.
|
||||
|
||||
Shared subscription, which consumers saw each key:
|
||||
A -> [consumer-1, consumer-2]
|
||||
B -> [consumer-1, consumer-2]
|
||||
C -> [consumer-1, consumer-2]
|
||||
|
||||
Key_Shared subscription, which consumers saw each key:
|
||||
A -> [consumer-1]
|
||||
B -> [consumer-1]
|
||||
C -> [consumer-1]
|
||||
|
||||
A Shared subscription round-robins individual messages, so messages with
|
||||
the same key end up on different consumers and can be processed at the
|
||||
same time: there is no per-key order left to speak of. Key_Shared hashes
|
||||
the key to one consumer, which is Kafka's guarantee -- except that the
|
||||
assignment belongs to the subscription and is recomputed as consumers come
|
||||
and go, rather than being fixed by a partition count chosen when the topic
|
||||
was created.
|
||||
|
||||
14
broker-comparison/docs/output/pulsar-replay.txt
Normal file
14
broker-comparison/docs/output/pulsar-replay.txt
Normal file
@@ -0,0 +1,14 @@
|
||||
== Pulsar: acknowledged, but still there ==
|
||||
|
||||
subscription replay-sub, first read : 12 messages
|
||||
subscription replay-sub, after seek(earliest): 12 messages
|
||||
subscription replay-sub-2, brand new : 12 messages
|
||||
|
||||
Acknowledgement moves a cursor; the message itself lives in the managed
|
||||
ledger. seek(MessageId) and seek(timestamp) rewind a live subscription,
|
||||
which Kafka can also do by resetting offsets. What differs is the default:
|
||||
Pulsar deletes a message once every subscription has acknowledged it,
|
||||
unless a retention policy on the namespace says otherwise, whereas Kafka
|
||||
keeps it for the retention period regardless of who read it. A Pulsar
|
||||
topic with no retention policy and no subscriptions keeps nothing.
|
||||
|
||||
34
broker-comparison/docs/output/rabbit-consumer-scaling.txt
Normal file
34
broker-comparison/docs/output/rabbit-consumer-scaling.txt
Normal file
@@ -0,0 +1,34 @@
|
||||
== RabbitMQ: consumers scale, and prefetch decides whether they actually do ==
|
||||
|
||||
one queue, 40 messages, 5 consumers, each taking 10 ms per message
|
||||
|
||||
no basicQos at all (unlimited prefetch, the AMQP default):
|
||||
consumer-1 -> 40 messages
|
||||
consumer-2 -> 0 messages
|
||||
consumer-3 -> 0 messages
|
||||
consumer-4 -> 0 messages
|
||||
consumer-5 -> 0 messages
|
||||
consumers that received nothing : 4
|
||||
|
||||
basicQos(1):
|
||||
consumer-1 -> 8 messages
|
||||
consumer-2 -> 9 messages
|
||||
consumer-3 -> 8 messages
|
||||
consumer-4 -> 8 messages
|
||||
consumer-5 -> 7 messages
|
||||
consumers that received nothing : 0
|
||||
|
||||
Every consumer on a queue competes for the same messages, so adding
|
||||
consumers adds throughput and there is no structural ceiling of the kind
|
||||
Kafka's partition count imposes. But with the AMQP default the broker
|
||||
pushes as many messages as a consumer will take, so whichever consumer
|
||||
connects first can be handed the entire backlog while the others sit idle.
|
||||
basicQos is not a tuning knob you get to postpone; it is what makes the
|
||||
fan-out real. Spring AMQP sets it for you --
|
||||
AbstractMessageListenerContainer.DEFAULT_PREFETCH_COUNT is 250 -- which is
|
||||
better than unlimited and still large enough to concentrate a small
|
||||
backlog on one consumer.
|
||||
|
||||
The price of all this is the ordering measurement: there is no key, so
|
||||
nothing constrains related messages to one consumer.
|
||||
|
||||
18
broker-comparison/docs/output/rabbit-ordering.txt
Normal file
18
broker-comparison/docs/output/rabbit-ordering.txt
Normal file
@@ -0,0 +1,18 @@
|
||||
== RabbitMQ: FIFO per queue, per consumer ==
|
||||
|
||||
one queue, one consumer, 12 messages
|
||||
completion order : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
|
||||
|
||||
one queue, two consumers, prefetch 1, consumer-1 slower than consumer-2
|
||||
completion order : [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1, 12]
|
||||
out-of-order steps: 1
|
||||
|
||||
A queue is FIFO and a single consumer sees it that way. The moment a second
|
||||
consumer is added the broker hands the next message to whichever consumer
|
||||
is free, so the order in which work finishes is no longer the order in
|
||||
which it was published. There is no key: RabbitMQ has no notion of a
|
||||
partition to which related messages could be pinned. Ordering across
|
||||
related messages means one queue and one consumer, and therefore no
|
||||
horizontal scaling for that queue -- or a consistent-hash exchange, which
|
||||
is a plugin.
|
||||
|
||||
12
broker-comparison/docs/output/rabbit-replay.txt
Normal file
12
broker-comparison/docs/output/rabbit-replay.txt
Normal file
@@ -0,0 +1,12 @@
|
||||
== RabbitMQ: there is nothing to replay ==
|
||||
|
||||
first drain of the queue : 12 messages
|
||||
second drain of the queue : 0 messages
|
||||
queue depth afterwards : 0
|
||||
|
||||
Acknowledging a message deletes it. The broker is a router with buffers,
|
||||
not a log: there is no offset to rewind and no second reader that can see
|
||||
what the first one consumed. Reading the same message twice means
|
||||
arranging it in advance -- a second queue bound to the same exchange, or a
|
||||
copy written somewhere else -- and it cannot be arranged after the fact.
|
||||
|
||||
12
broker-comparison/docs/output/tests.txt
Normal file
12
broker-comparison/docs/output/tests.txt
Normal file
@@ -0,0 +1,12 @@
|
||||
[INFO] Running com.ankurm.brokers.KafkaComparisonTest
|
||||
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.88 s -- in com.ankurm.brokers.KafkaComparisonTest
|
||||
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
|
||||
[INFO] BUILD SUCCESS
|
||||
[INFO] Running com.ankurm.brokers.RabbitComparisonTest
|
||||
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.732 s -- in com.ankurm.brokers.RabbitComparisonTest
|
||||
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
|
||||
[INFO] BUILD SUCCESS
|
||||
[INFO] Running com.ankurm.brokers.PulsarComparisonTest
|
||||
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.00 s -- in com.ankurm.brokers.PulsarComparisonTest
|
||||
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
|
||||
[INFO] BUILD SUCCESS
|
||||
Reference in New Issue
Block a user