Ankur Mhatre
e1f8aa7402
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
60 lines
3.6 KiB
Markdown
60 lines
3.6 KiB
Markdown
prev: [What Spring adds](06-what-spring-adds.md) · [README](../README.md)
|
|
|
|
# 7. The decision table
|
|
|
|
Every row below is either a measurement in `docs/output/` or a statement about operating the
|
|
thing. Nothing here is a vendor claim.
|
|
|
|
| | Kafka 4.2.1 | RabbitMQ 3.10.25 | Pulsar 4.2.4 |
|
|
|---|---|---|---|
|
|
| ordering with one consumer | FIFO per partition | FIFO per queue | FIFO |
|
|
| ordering with many consumers | per key, always | **none** | per key with `Key_Shared`, none with `Shared` |
|
|
| who chooses that | topic design | plugin, or one consumer | each subscription, independently |
|
|
| replay after the fact | yes, within retention | **no** | yes, if retention is configured |
|
|
| second independent reader | new consumer group, any time | must be arranged in advance | new subscription, any time |
|
|
| consumer parallelism ceiling | partition count | none | none |
|
|
| changing that ceiling | repartition; breaks key→partition | nothing to change | nothing to change |
|
|
| the client-side trap | none by default | unlimited prefetch (Spring: 250) | `receiverQueueSize` 1000 |
|
|
| memory at idle | 333 MB | **115 MB** | 610 MB |
|
|
| processes to operate | 1 (KRaft) | 1 + `epmd` | broker + BookKeeper + metadata store |
|
|
| shipped default settings | 24 | **0** | 357 |
|
|
| Spring project maturity | very high | very high | good, much smaller community |
|
|
|
|
## Choosing
|
|
|
|
**Choose RabbitMQ** when the work is a queue of tasks: each message is an instruction, order
|
|
between instructions does not matter, and once it is done it is done. It is the smallest thing
|
|
that works, it has the best out-of-the-box operability of the three, and queue depth is a metric
|
|
anyone can interpret. Choose it in the knowledge that you are giving up replay permanently —
|
|
adding it later is a broker migration, not a configuration change.
|
|
|
|
**Choose Kafka** when the messages are events other people will want to read: when more than one
|
|
consumer will exist, when someone will need to reprocess history after a bug, or when ordering per
|
|
entity is part of the contract. Pay for it with a partition count decided too early, consumers
|
|
that scale only as far as that number, and an operational model where lag is the thing you watch.
|
|
KRaft has removed the old ZooKeeper objection.
|
|
|
|
**Choose Pulsar** when you genuinely need what neither of the others gives you: per-subscription
|
|
choice of ordering versus fan-out, consumer counts not bounded by a number chosen at topic
|
|
creation, or multi-tenancy with real isolation. It is the most capable design here. It is also
|
|
three systems, 357 settings, five times RabbitMQ's memory floor, and the one your team has least
|
|
experience with — and that last point decides more incidents than the first three prevent.
|
|
|
|
**If the honest answer is "we do not know yet"**, that argues for Kafka, on grounds that have
|
|
nothing to do with the technology: you can hire for it, your monitoring vendor supports it, and
|
|
the failure modes are documented by thousands of people who hit them first. That is a real
|
|
engineering argument and it is fine to make it out loud.
|
|
|
|
## The question that dissolves the choice
|
|
|
|
A surprising number of these decisions are made for a system that has one producer, one consumer
|
|
and fewer than a hundred messages a second. At that volume all three brokers work, none of the
|
|
measurements above will ever be reached, and the decision is entirely about what your team can
|
|
operate at three in the morning.
|
|
|
|
The measurements matter when you can name which row you are relying on. If you cannot, you are
|
|
picking an operational burden, not a broker — and the cheapest one to operate is the one in the
|
|
`115 MB` cell.
|
|
|
|
[README](../README.md)
|