Ankur Mhatre
e1f8aa7402
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
56 lines
2.9 KiB
Markdown
56 lines
2.9 KiB
Markdown
prev: [Consumer scaling](04-consumer-scaling.md) · [README](../README.md) · next: [What Spring adds](06-what-spring-adds.md)
|
|
|
|
# 5. Operational footprint
|
|
|
|
"Ops burden" cannot be benchmarked. The number of moving parts, the memory floor and the size of
|
|
the configuration surface can be, and they are what people are actually asking about.
|
|
|
|
Each broker started from its shipped distribution with default configuration and a 700 MB heap
|
|
cap, on one 2-core / 3.8 GB Linux box, Temurin JDK 21
|
|
([`docs/output/footprint.txt`](output/footprint.txt)):
|
|
|
|
| | Kafka 4.2.1 | RabbitMQ 3.10.25 | Pulsar 4.2.4 standalone |
|
|
|---|---|---|---|
|
|
| launch to first accepted connection | 10.0 s | 15.1 s | 20.8 s |
|
|
| resident memory at idle | 333 MB | 115 MB | 610 MB |
|
|
| server processes | 1 | 1 (+ `epmd`) | 1 |
|
|
| listening ports | 9092, 9093 | 5672, 4369, 25672 | 6650, 8080, 2181 |
|
|
| unpacked distribution | 135 MB | 26 MB | 344 MB |
|
|
| settings in the shipped default config | 24 | **0** | 357 |
|
|
|
|
Read those numbers as orders of magnitude, not as a benchmark: one sample, one small machine.
|
|
|
|
Three things they say clearly.
|
|
|
|
**RabbitMQ ships no configuration file at all.** The boot log records `Config file(s): (none)`;
|
|
`rabbitmq.conf.example` is entirely commented out. Everything works out of the box, and the
|
|
smallest resident footprint here is the one that is not a JVM. Against that, `epmd` on 4369 and
|
|
the inter-node port on 25672 are the Erlang distribution, which is also how clustering works and
|
|
how clustering goes wrong.
|
|
|
|
**Pulsar standalone is three systems in one process.** Port 2181 is ZooKeeper and the on-disk
|
|
data is BookKeeper's; the standalone distribution hides that behind one command. A real
|
|
deployment does not: you operate ZooKeeper (or, from Pulsar 3.x, an alternative metadata store),
|
|
BookKeeper bookies and Pulsar brokers as three tiers with three scaling stories. The 357-setting
|
|
`standalone.conf` is the honest signal here — that is the configuration surface, and running it
|
|
seriously means learning most of it.
|
|
|
|
**Kafka since KRaft is genuinely simpler than it was.** One process, two ports, 24 settings, no
|
|
ZooKeeper. The old "Kafka means also running ZooKeeper" objection is a version behind; anyone
|
|
comparing on that basis is comparing to 2022.
|
|
|
|
## What the numbers do not include
|
|
|
|
The footprint above is one node at idle. The thing that actually determines ops burden is what
|
|
happens at three in the morning:
|
|
|
|
- **Kafka**: rebalances, consumer lag as the primary signal, partition-count decisions you cannot
|
|
cleanly reverse, and a broad ecosystem of tools that assume Kafka.
|
|
- **RabbitMQ**: the management plugin is genuinely good, queue depth is a direct and obvious
|
|
signal, and the hard problem is network partitions in a cluster — which the Erlang distribution
|
|
makes fast to detect and awkward to resolve.
|
|
- **Pulsar**: the fewest people on your team will have run it. That is not a technical property
|
|
and it is usually the deciding one.
|
|
|
|
next: [What Spring adds](06-what-spring-adds.md)
|