Add the broker-comparison module
Kafka, RabbitMQ and Pulsar measured side by side on ordering, replay and consumer scaling by driving the three client libraries directly, plus an operational-footprint measurement of each broker's own distribution. Nine tests, three brokers started without Docker, and every number in the documentation regenerated by scripts/run-all.sh.
This commit is contained in:
49
broker-comparison/docs/01-three-models.md
Normal file
49
broker-comparison/docs/01-three-models.md
Normal file
@@ -0,0 +1,49 @@
|
||||
[README](../README.md) · next: [Ordering](02-ordering.md)
|
||||
|
||||
# 1. Three storage models, and everything that follows from them
|
||||
|
||||
Almost every difference between these brokers follows from one sentence about where a message
|
||||
lives.
|
||||
|
||||
- **Kafka** is an append-only log, partitioned. The broker keeps every record for the retention
|
||||
period and remembers nothing about individual consumers except an offset. Reading does not
|
||||
remove anything.
|
||||
- **RabbitMQ** is a router with queues. The broker owns each message until a consumer
|
||||
acknowledges it, and then deletes it. It can route, expire, and dead-letter on its own.
|
||||
- **Pulsar** is a log too, but the cursor and the storage are separate services: brokers are
|
||||
stateless and BookKeeper holds the data. Acknowledgement moves a cursor, and a message is
|
||||
deleted once every subscription has passed it — unless a retention policy says otherwise.
|
||||
|
||||
Read those three sentences again before reading any comparison table, including the one in
|
||||
[chapter 7](07-the-decision-table.md). They predict most of it.
|
||||
|
||||
| | Kafka | RabbitMQ | Pulsar |
|
||||
|---|---|---|---|
|
||||
| unit of parallelism | partition | queue | subscription |
|
||||
| set when | topic is created | queue is declared | consumer subscribes |
|
||||
| consuming | moves an offset | deletes the message | moves a cursor |
|
||||
| a second reader | new consumer group | another queue, bound in advance | new subscription |
|
||||
|
||||
The last row is the one people underestimate. In Kafka and Pulsar you can add a reader that sees
|
||||
history you have already processed. In RabbitMQ you cannot add one after the fact at all — the
|
||||
messages are gone — so the decision to have a second consumer has to be made *before* the
|
||||
messages arrive.
|
||||
|
||||
## What was measured, and how
|
||||
|
||||
Three questions, asked identically of all three brokers by driving the client libraries directly
|
||||
rather than through three different Spring abstractions:
|
||||
|
||||
| Question | Chapter |
|
||||
|---|---|
|
||||
| What ordering survives when you add a second consumer? | [02](02-ordering.md) |
|
||||
| Can you read the same message twice? | [03](03-replay.md) |
|
||||
| How far do consumers scale, and what stops them? | [04](04-consumer-scaling.md) |
|
||||
| What does each one cost to run? | [05](05-operational-footprint.md) |
|
||||
|
||||
Versions are the ones a Spring Boot 4.1.1 application gets: kafka-clients 4.2.1, amqp-client
|
||||
5.30.0, pulsar-client 4.2.4, all read from `spring-boot-dependencies-4.1.1.pom`. The brokers are
|
||||
Kafka 4.2.1, RabbitMQ 3.10.25 and Pulsar 4.2.4, each started from its own distribution by a script
|
||||
in `scripts/`.
|
||||
|
||||
next: [Ordering](02-ordering.md)
|
||||
Reference in New Issue
Block a user