Ankur Mhatre
e1f8aa7402
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
50 lines
2.5 KiB
Markdown
50 lines
2.5 KiB
Markdown
[README](../README.md) · next: [Ordering](02-ordering.md)
|
|
|
|
# 1. Three storage models, and everything that follows from them
|
|
|
|
Almost every difference between these brokers follows from one sentence about where a message
|
|
lives.
|
|
|
|
- **Kafka** is an append-only log, partitioned. The broker keeps every record for the retention
|
|
period and remembers nothing about individual consumers except an offset. Reading does not
|
|
remove anything.
|
|
- **RabbitMQ** is a router with queues. The broker owns each message until a consumer
|
|
acknowledges it, and then deletes it. It can route, expire, and dead-letter on its own.
|
|
- **Pulsar** is a log too, but the cursor and the storage are separate services: brokers are
|
|
stateless and BookKeeper holds the data. Acknowledgement moves a cursor, and a message is
|
|
deleted once every subscription has passed it — unless a retention policy says otherwise.
|
|
|
|
Read those three sentences again before reading any comparison table, including the one in
|
|
[chapter 7](07-the-decision-table.md). They predict most of it.
|
|
|
|
| | Kafka | RabbitMQ | Pulsar |
|
|
|---|---|---|---|
|
|
| unit of parallelism | partition | queue | subscription |
|
|
| set when | topic is created | queue is declared | consumer subscribes |
|
|
| consuming | moves an offset | deletes the message | moves a cursor |
|
|
| a second reader | new consumer group | another queue, bound in advance | new subscription |
|
|
|
|
The last row is the one people underestimate. In Kafka and Pulsar you can add a reader that sees
|
|
history you have already processed. In RabbitMQ you cannot add one after the fact at all — the
|
|
messages are gone — so the decision to have a second consumer has to be made *before* the
|
|
messages arrive.
|
|
|
|
## What was measured, and how
|
|
|
|
Three questions, asked identically of all three brokers by driving the client libraries directly
|
|
rather than through three different Spring abstractions:
|
|
|
|
| Question | Chapter |
|
|
|---|---|
|
|
| What ordering survives when you add a second consumer? | [02](02-ordering.md) |
|
|
| Can you read the same message twice? | [03](03-replay.md) |
|
|
| How far do consumers scale, and what stops them? | [04](04-consumer-scaling.md) |
|
|
| What does each one cost to run? | [05](05-operational-footprint.md) |
|
|
|
|
Versions are the ones a Spring Boot 4.1.1 application gets: kafka-clients 4.2.1, amqp-client
|
|
5.30.0, pulsar-client 4.2.4, all read from `spring-boot-dependencies-4.1.1.pom`. The brokers are
|
|
Kafka 4.2.1, RabbitMQ 3.10.25 and Pulsar 4.2.4, each started from its own distribution by a script
|
|
in `scripts/`.
|
|
|
|
next: [Ordering](02-ordering.md)
|