Ankur Mhatre
e7e64a047b
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
17 lines
526 B
Plaintext
17 lines
526 B
Plaintext
== Kafka: partitions are the ceiling on consumer parallelism ==
|
|
|
|
topic 'orders-scaling', 3 partitions, 5 consumers in one group
|
|
|
|
consumer-1 -> partitions [0]
|
|
consumer-2 -> partitions [1]
|
|
consumer-3 -> partitions [2]
|
|
consumer-4 -> no partitions (idle)
|
|
consumer-5 -> no partitions (idle)
|
|
|
|
consumers with no partitions: 2
|
|
|
|
A partition is assigned to at most one consumer in a group, so the number
|
|
of partitions is a hard ceiling on consumer parallelism. Adding consumers
|
|
beyond it adds idle processes, not throughput.
|
|
|