diff --git a/README.md b/README.md index e001608..d85f4a9 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,13 @@ module's `scripts/run-all.sh`, never typed by hand. | Module | Article | What it demonstrates | |---|---|---| | [`kafka-basics/`](kafka-basics/README.md) | [Spring Boot 4.1 and Apache Kafka: Producer, Consumer and Serialisation from Scratch](https://ankurm.com/spring-boot-4-1-kafka-producer-consumer-serialisation/) | The on-ramp: what the starter gives you, the two Jackson serializer families, where a key lands and why, and which defaults are Kafka's rather than Spring's | +| [`rabbitmq/`](rabbitmq/README.md) | [Spring Boot and RabbitMQ: Exchanges, Queues, Bindings and a Working Dead-Letter Queue](https://ankurm.com/spring-boot-rabbitmq-exchanges-dead-letter-queue/) | All four exchange types against a real broker, manual acknowledgement, and a dead-letter path exercised through both rejection and TTL expiry | + +The two brokers make an instructive pair. Kafka's consumer holds an offset and the broker +remembers nothing about individual records; RabbitMQ's broker owns the message until it is +acknowledged and can route, expire and dead-letter it on its own. Almost every difference in how +you handle failure follows from that one sentence — which is why Kafka needs a retry topic +to do what RabbitMQ does with a queue argument. They are meant to be read in order. `kafka-basics` establishes that the default acknowledgement mode is `BATCH` and therefore that delivery is at-least-once; everything the error-handling diff --git a/rabbitmq/README.md b/rabbitmq/README.md new file mode 100644 index 0000000..7951038 --- /dev/null +++ b/rabbitmq/README.md @@ -0,0 +1,80 @@ +# `rabbitmq` — exchanges, bindings and a dead-letter queue that works + +Companion project for +[**Spring Boot and RabbitMQ: Exchanges, Queues, Bindings and a Working Dead-Letter Queue**](https://ankurm.com/spring-boot-rabbitmq-exchanges-dead-letter-queue/) +on ankurm.com. + +Ten tests against a **real RabbitMQ broker**. All four exchange types, manual acknowledgement, +a dead-letter path exercised through `basicNack` and through TTL expiry, and the three ways a +topology loses work quietly. + +## Versions + +| | Version | Notes | +|---|---|---| +| JDK | 25 (Temurin 25.0.4.1+1) | current LTS | +| Spring Boot | 4.1.1 | | +| Spring AMQP / Spring Rabbit | 4.1.1 | Boot-managed | +| `com.rabbitmq:amqp-client` | **5.30.0** | Boot-managed. Central has 5.35.0 | +| Testcontainers | 2.0.5 | artifact is `testcontainers-rabbitmq`, not `rabbitmq` | +| **Broker for the committed transcripts** | **RabbitMQ 3.10.25** | see the note below | + +Client-side versions were read from `repo1.maven.org/.../maven-metadata.xml` and Boot's +`spring-boot-dependencies` POM. + +> **About the broker version.** The machine that regenerates `docs/output/` has no Docker daemon +> and no root, so `scripts/broker.sh` starts a generic-unix RabbitMQ against an extracted Erlang +> 24 runtime, and the newest release that pairs with Erlang 24 is 3.10.25. Everything exercised +> here — the four exchange types, `x-dead-letter-exchange`, `x-dead-letter-routing-key`, +> `x-message-ttl`, `x-death`, manual ack, `mandatory` returns, `PRECONDITION_FAILED` on +> inequivalent arguments — is AMQP 0-9-1 behaviour that is unchanged in RabbitMQ 4.x. The one +> broker-version difference worth knowing for 4.x is that **quorum queues are the default queue +> type** and classic mirrored queues are gone; nothing in this module declares a queue type, so +> the topology is valid on both. If you have Docker, run the same tests against +> `rabbitmq:4.1-management` with +> [`TestcontainersConfiguration`](src/test/java/com/ankurm/rabbit/TestcontainersConfiguration.java). + +## Quickstart + +```bash +# with Docker: point the tests at Testcontainers, or run your own broker on 5672 +mvn test + +# without Docker: +export ERL_ROOT=/path/to/erlang RABBITMQ_HOME=/path/to/rabbitmq_server-3.10.25 +./scripts/run-all.sh +``` + +## Documentation + +1. [The on-ramp](docs/01-the-on-ramp.md) +2. [Four exchange types](docs/02-exchanges.md) +3. [The message that goes nowhere and says nothing](docs/03-the-silent-drop.md) +4. [A dead-letter queue that actually works](docs/04-dead-lettering.md) +5. [Acknowledgement](docs/05-acknowledgement.md) +6. [Changing your mind about a queue](docs/06-changing-your-mind.md) + +## Captured output + +| File | Shows | +|---|---| +| [`topic-wildcards.txt`](docs/output/topic-wildcards.txt) | `*` vs `#` over five routing keys | +| [`dead-letter.txt`](docs/output/dead-letter.txt) | `x-death` for `rejected` and for `expired` | +| [`requeue-loop.txt`](docs/output/requeue-loop.txt) | 199 redeliveries, 0 dead-lettered | +| [`unroutable.txt`](docs/output/unroutable.txt) | `312 NO_ROUTE` | +| [`precondition-failed.txt`](docs/output/precondition-failed.txt) | `406` on an inequivalent argument | +| [`tests.txt`](docs/output/tests.txt) | 10 tests | + +## Five things this module exists to prove + +1. **An unroutable message is discarded silently**, and finding out needs `publisher-returns` + *and* `mandatory` — two settings in two different places. Setting only `mandatory` does + nothing. +2. **`requeue=true` never dead-letters.** 200 attempts, 199 redeliveries, an empty DLQ and a + queue depth that stays at 1 the whole time. +3. **`x-death.reason` distinguishes `rejected` from `expired`**, which is the difference between + a consumer that refused the work and a consumer that never got to it. +4. **Queue arguments are immutable** — `406 PRECONDITION_FAILED`, wrapped in an exception whose + own message is the string `java.io.IOException`. +5. **Boot auto-configures no JSON converter for RabbitMQ**, and the converter you want is + `JacksonJsonMessageConverter`, not `Jackson2JsonMessageConverter`. diff --git a/rabbitmq/docs/01-the-on-ramp.md b/rabbitmq/docs/01-the-on-ramp.md new file mode 100644 index 0000000..1d98cee --- /dev/null +++ b/rabbitmq/docs/01-the-on-ramp.md @@ -0,0 +1,59 @@ +[Module README](../README.md) · [Exchanges →](02-exchanges.md) + +# 1. The on-ramp + +Two dependency decisions decide whether anything works, and neither produces a helpful error. + +## Use the starter, not `spring-rabbit` + +Boot 4 moved every auto-configuration out of `spring-boot-autoconfigure` into a per-technology +module. RabbitMQ's lives in `spring-boot-amqp`, package +`org.springframework.boot.amqp.autoconfigure`, and a bare `org.springframework.amqp:spring-rabbit` +dependency does not bring it. You get no `RabbitTemplate`, no `RabbitAdmin`, no listener +container factory — and a context that starts cleanly. + +```xml + + org.springframework.boot + spring-boot-starter-amqp + +``` + +## Boot does not give you a JSON converter + +Spring Kafka defaults to serializers you configure. Spring AMQP defaults to +`SimpleMessageConverter`, which handles `String`, `byte[]` and `Serializable` and refuses +everything else: + +``` +IllegalArgumentException: SimpleMessageConverter only supports String, byte[] and +Serializable payloads, received: com.ankurm.rabbit.OrderMessage +``` + +One bean fixes both directions, because the auto-configured `RabbitTemplate` and the listener +container factory both look for a `MessageConverter`: + +```java +@Bean +MessageConverter messageConverter() { + return new JacksonJsonMessageConverter(); +} +``` + +**Note the class name.** Spring AMQP 4.1 ships `Jackson2JsonMessageConverter` (Jackson 2) and +`JacksonJsonMessageConverter` (Jackson 3) side by side — exactly as Spring Kafka ships +`JsonSerializer` and `JacksonJsonSerializer`. Boot 4 is a Jackson 3 application. The rule across +both stacks is the same: **if the class name contains a `2`, it belongs to the previous major +version of Jackson.** + +## Versions + +Boot 4.1.1 manages Spring AMQP **4.1.1** and `com.rabbitmq:amqp-client` **5.30.0**. Maven Central +has amqp-client 5.35.0; overriding the managed version to reach it is a change you should have a +reason for. + +One API change to know about: `RabbitAdmin.QUEUE_MESSAGE_COUNT` now holds a `Long`. Every older +example casts it to `Integer`, which is a `ClassCastException` at runtime and not at compile +time. + +[Exchanges →](02-exchanges.md) diff --git a/rabbitmq/docs/02-exchanges.md b/rabbitmq/docs/02-exchanges.md new file mode 100644 index 0000000..dbe7938 --- /dev/null +++ b/rabbitmq/docs/02-exchanges.md @@ -0,0 +1,71 @@ +[← The on-ramp](01-the-on-ramp.md) · [Module README](../README.md) · [The silent drop →](03-the-silent-drop.md) + +# 2. Four exchange types + +A producer never publishes to a queue. It publishes to an **exchange** with a **routing key**, +and bindings decide where that lands. The whole topology is in +[`Topology.java`](../src/main/java/com/ankurm/rabbit/Topology.java) as beans; `RabbitAdmin` +declares them when the connection opens. + +## Direct — exact match + +Binding key `new` receives routing key `new`. Nothing else. This is the workhorse: one queue per +command type. + +## Fanout — routing key ignored entirely + +Every bound queue gets a copy. `audit.all` and `analytics.all` both receive it, and the routing +key you passed is not consulted at all. Use it for broadcast; use it knowing that adding a queue +adds a full copy of the traffic. + +## Topic — wildcards over dot-separated words + +`*` is **exactly one word**. `#` is **zero or more**. The distinction is the one people get +wrong, so here it is against a real broker +([`docs/output/topic-wildcards.txt`](output/topic-wildcards.txt)): + +``` +routing key orders.eu orders.high note +order.eu.high true true matches both +order.eu.low true false matches order.eu.* only +order.us.high false true matches order.#.high only +order.eu.west.high false true matches order.#.high only - * is one word +order.high false true matches order.#.high - # can be zero words +``` + +Bindings are `order.eu.*` and `order.#.high`. Two rows are worth pausing on: + +- `order.eu.west.high` does **not** match `order.eu.*`, because `*` matches one word and `west.high` + is two. Regex intuition says otherwise. +- `order.high` **does** match `order.#.high`, because `#` matches zero words. So a binding you + wrote to mean "something in the middle" also matches "nothing in the middle". + +Design routing keys most-general-to-most-specific (`order.eu.west.high`, not +`high.order.eu.west`), because `#` and `*` work left to right and a hierarchy you can bind +usefully is one that starts broad. + +## Headers — match a map, ignore the routing key + +`x-match=all` requires every named header to be present **and equal**. `x-match=any` requires +one. From `RoutingTest`: + +| Message headers | `x-match=all` queue | `x-match=any` queue | +|---|---|---| +| `priority=high, region=eu` | yes | yes | +| `priority=high` | no | yes | +| `priority=high, region=us` | no | yes | + +The third row is the one to remember: `all` matches on **value**, not on presence. A header that +is there with the wrong value fails the same way a missing one does. + +Headers exchanges are slower than topic exchanges and much less common. Reach for them when the +routing criteria are genuinely multi-dimensional and do not compose into a hierarchy. + +## The default exchange + +Publishing to the empty exchange name `""` routes by **queue name**, using the routing key as the +queue name. Every queue is implicitly bound to it. That is how +`convertAndSend("", "orders.work", message)` works, and it is the one piece of AMQP that behaves +like a magic constant. + +[The silent drop →](03-the-silent-drop.md) diff --git a/rabbitmq/docs/03-the-silent-drop.md b/rabbitmq/docs/03-the-silent-drop.md new file mode 100644 index 0000000..c906477 --- /dev/null +++ b/rabbitmq/docs/03-the-silent-drop.md @@ -0,0 +1,78 @@ +[← Exchanges](02-exchanges.md) · [Module README](../README.md) · [Dead-lettering →](04-dead-lettering.md) + +# 3. The message that goes nowhere and says nothing + +Publish to `orders.direct` with routing key `amend`. No binding matches. The broker discards the +message. `convertAndSend` returns normally. Nothing is logged, by the broker or by Spring, and +the message is gone. + +This is correct AMQP behaviour and it is the single most expensive default in RabbitMQ, because +the symptom is "the consumer isn't running" and the cause is three services away. + +**Two settings are required, and they live in different places:** + +```yaml +spring: + rabbitmq: + publisher-returns: true # on the connection factory + template: + mandatory: true # on the template +``` + +`publisher-returns` puts the connection into a mode where returns are listened for at all. +`mandatory` asks the broker to return *this* publish if it is unroutable. Setting only +`mandatory` — which is what most examples show — changes nothing, and the test in this module +was written that way first and failed. + +Then register a callback: + +```java +template.setReturnsCallback((returned) -> log.error("unroutable: exchange={} key={} {} {}", + returned.getExchange(), returned.getRoutingKey(), + returned.getReplyCode(), returned.getReplyText())); +``` + +From [`docs/output/unroutable.txt`](output/unroutable.txt): + +``` +replyCode 312 +replyText NO_ROUTE +exchange orders.direct +routingKey amend +``` + +`312 NO_ROUTE` is the code worth putting in an alert. + +## What returns still do not give you + +The callback fires **asynchronously, on the connection's thread**. `convertAndSend` had already +returned successfully by then. So a return tells you a message was lost; it does not let you +stop the caller from believing it succeeded. + +If the caller must know, you need **publisher confirms** as well: + +```yaml +spring: + rabbitmq: + publisher-confirm-type: correlated +``` + +Confirms and returns answer different questions, and you usually want both: + +| | answers | +|---|---| +| **return** | the broker had nowhere to route this | +| **confirm** | the broker took responsibility for this (`ack`), or refused it (`nack`) | + +A message that is unroutable gets a **return followed by an `ack`** — the broker successfully did +nothing with it. So a confirm alone will not tell you the message vanished, which is the trap +inside the trap. + +## The alternate exchange + +The topology-level version of the same fix: give an exchange an `alternate-exchange` argument and +unroutable messages go there instead of being dropped, with no publisher-side configuration at +all. It costs one exchange and one queue, and it catches the publishes made by services that +forgot to set `mandatory`. + +[Dead-lettering →](04-dead-lettering.md) diff --git a/rabbitmq/docs/04-dead-lettering.md b/rabbitmq/docs/04-dead-lettering.md new file mode 100644 index 0000000..d6f5dc0 --- /dev/null +++ b/rabbitmq/docs/04-dead-lettering.md @@ -0,0 +1,112 @@ +[← The silent drop](03-the-silent-drop.md) · [Module README](../README.md) · [Acknowledgement →](05-acknowledgement.md) + +# 4. A dead-letter queue that actually works + +There is no "send to DLQ" operation in AMQP. A dead-letter exchange is a **queue argument**, and +messages arrive there as a side effect of three specific events. + +```java +QueueBuilder.durable("orders.work") + .deadLetterExchange("orders.dlx") + .deadLetterRoutingKey("failed") + .build(); +``` + +`deadLetterRoutingKey` matters more than it looks. Without it, the message is republished with +its **original** routing key, so your DLQ binding has to anticipate every routing key the +upstream might have used. Setting it to a constant means one binding catches everything. + +## The three triggers + +| Event | `x-death` reason | +|---|---| +| `basicNack` / `basicReject` with `requeue=false` | `rejected` | +| message TTL expires (`x-message-ttl` or per-message) | `expired` | +| queue length or byte limit exceeded (`x-max-length`) | `maxlen` | + +And the one that is **not** a trigger: `requeue=true`. More on that below. + +## What arrives + +Both captured from real runs, in +[`docs/output/dead-letter.txt`](output/dead-letter.txt): + +``` +=== x-death after basicNack(requeue=false) === + reason rejected + count 1 + exchange + time Sat Aug 29 09:53:17 IST 2026 + routing-keys [orders.work] + queue orders.work + +=== x-death after x-message-ttl expiry === + reason expired + count 1 + queue orders.ttl +``` + +Same destination queue, different `reason`. That field is the whole diagnostic: `rejected` means +a consumer looked at the work and refused it; `expired` means nobody got to it in time. Those are +different incidents with different fixes, and they are indistinguishable without reading the +header. + +The third trigger has a detail worth knowing: when `x-max-length` is exceeded RabbitMQ drops from +the **head**, so the message that gets dead-lettered is the **oldest** one already queued, not the +one that just arrived: + +``` +=== x-death after x-max-length overflow === + reason maxlen + queue orders.bounded + body {"orderId":"m-1","detail":"detail for m-1"} +``` + +Three messages into a queue that holds two, and `m-1` is the one on the DLQ. A bounded queue +under sustained overload therefore dead-letters your *backlog* while continuing to accept new +work — which is usually what you want for telemetry and exactly wrong for orders. + +`x-death` is a **list**, not a map — one entry per queue the message has been dead-lettered from, +and `count` accumulates. That is how you build a retry limit: read +`x-death[0].count`, and stop republishing past a threshold. + +## The infinite loop + +```java +channel.basicNack(deliveryTag, false, true); // requeue = true +``` + +From [`docs/output/requeue-loop.txt`](output/requeue-loop.txt), 200 delivery attempts: + +``` +redelivered 199 times +dead-lettered 0 +still on queue 1 +``` + +The dead-letter exchange is never consulted. `x-death` is never written. There is no counter to +exhaust and no backoff. One poison message with an unconditional `requeue=true` handler will +saturate a consumer indefinitely, and the queue depth stays at 1 the whole time — so a depth +alarm will not fire either. The only visible signal is CPU. + +**`requeue=true` is only correct when the failure is transient and you have a delay.** With no +delay, a message that fails because a database is down comes straight back while the database is +still down. The `redelivered` flag on the envelope is your one piece of state: + +```java +boolean retried = response.getEnvelope().isRedeliver(); +channel.basicNack(tag, false, !retried); // one requeue, then dead-letter +``` + +That is a one-attempt retry with no extra infrastructure. For anything more, use a delay queue: +a queue with a TTL and a DLX pointing back at the work exchange turns "expire after 30 seconds" +into "retry in 30 seconds". + +## Do not point the DLQ's DLX at its own source + +If `orders.dlq` dead-letters to an exchange bound back to `orders.work`, a message that fails +consistently cycles between the two forever, gaining an `x-death` entry each time until the +header itself is the biggest thing in the message. The DLQ in this module deliberately has no +dead-letter exchange at all. + +[Acknowledgement →](05-acknowledgement.md) diff --git a/rabbitmq/docs/05-acknowledgement.md b/rabbitmq/docs/05-acknowledgement.md new file mode 100644 index 0000000..1d9ccd7 --- /dev/null +++ b/rabbitmq/docs/05-acknowledgement.md @@ -0,0 +1,82 @@ +[← Dead-lettering](04-dead-lettering.md) · [Module README](../README.md) · [Changing your mind →](06-changing-your-mind.md) + +# 5. Acknowledgement + +```yaml +spring: + rabbitmq: + listener: + simple: + acknowledge-mode: manual + prefetch: 1 +``` + +Three modes, and the default is not the one you want for work that matters. + +| Mode | Behaviour | +|---|---| +| `auto` (default) | the container acks after the listener returns, nacks on exception | +| `manual` | your code calls `basicAck` / `basicNack`; nothing else does | +| `none` | the broker forgets the message the moment it is delivered | + +(Boot publishes no default for `acknowledge-mode` either; the field initialiser in +`AbstractMessageListenerContainer` is `AcknowledgeMode.AUTO`.) + +`none` is genuinely fire-and-forget: no redelivery, ever, and no flow control either, because +`prefetch` is meaningless without acknowledgement. It is the right choice for metrics and the +wrong choice for everything else. + +`auto` is a reasonable default and its name is misleading: it does not mean "auto-ack" in the +AMQP sense (that is `none`). It means the container decides, and it decides by whether your +listener threw. + +`manual` is what you want when the acknowledgement must happen after some other durable side +effect — a database commit, a downstream call. In `manual` mode the listener takes a `Channel` +and a delivery tag: + +```java +@RabbitListener(queues = "orders.work") +void onOrder(OrderMessage message, Channel channel, + @Header(AmqpHeaders.DELIVERY_TAG) long tag) throws IOException { + try { + process(message); + channel.basicAck(tag, false); + } + catch (TransientFailure ex) { + channel.basicNack(tag, false, !alreadyRedelivered); + } + catch (PermanentFailure ex) { + channel.basicNack(tag, false, false); // dead-letter it + } +} +``` + +The two catch blocks are the point. A single `catch (Exception)` that nacks with `requeue=true` +is the loop from [chapter 4](04-dead-lettering.md); a single one that nacks with `requeue=false` +dead-letters transient failures that would have succeeded on a retry. Deciding which kind of +failure you had is work you cannot delegate to the container. + +## `prefetch` is the flow-control knob + +`prefetch` is how many un-acknowledged messages the broker will send a consumer. Spring Boot sets +no default, so `AbstractMessageListenerContainer.DEFAULT_PREFETCH_COUNT` applies, and that +constant is **250** — throughput-oriented: a slow consumer holds 250 messages that no +other consumer can take, so one stuck worker parks 250 messages behind it. + +Set it to 1 when messages are expensive and unevenly sized — it gives you round-robin +distribution across consumers. Leave it high when messages are small and uniform and you want +throughput. This is the single most effective RabbitMQ tuning parameter and it is almost never +touched. + +## The forgotten ack + +Not calling `basicAck` does not lose the message and does not throw. The message stays +un-acknowledged, counting against `prefetch`, until the channel closes. With `prefetch: 1` the +consumer stops after exactly one message; with the default it stops after 250. Both look like the +consumer "just stopped", hours apart in wall-clock time depending on traffic. + +`rabbitmqctl list_queues name messages_ready messages_unacknowledged` is the command that +distinguishes them. A large `messages_unacknowledged` with a live consumer means somebody forgot +an ack. + +[Changing your mind →](06-changing-your-mind.md) diff --git a/rabbitmq/docs/06-changing-your-mind.md b/rabbitmq/docs/06-changing-your-mind.md new file mode 100644 index 0000000..cf2cf66 --- /dev/null +++ b/rabbitmq/docs/06-changing-your-mind.md @@ -0,0 +1,58 @@ +[← Acknowledgement](05-acknowledgement.md) · [Module README](../README.md) + +# 6. Changing your mind about a queue + +Queue arguments are immutable. There is no `ALTER QUEUE`. Redeclaring an existing queue with +different arguments is a channel-level error +([`docs/output/precondition-failed.txt`](output/precondition-failed.txt)): + +``` +PRECONDITION_FAILED - inequivalent arg 'x-message-ttl' for queue 'orders.ttl' in vhost '/': +received '9999' but current is '1500', class-id=50, method-id=10 +``` + +Reply code **406**. The message is precise and helpful, and you will probably never see it, +because of what Spring wraps it in: + +``` +AmqpIOException: java.io.IOException + caused by: null + caused by: channel error; protocol method: #method(reply-code=406, ...) +``` + +`AmqpIOException`'s own message is the string `java.io.IOException`. The next cause's message is +`null`. The useful text is two levels down. This is why the failure gets reported as "some +IOException from RabbitMQ" and why it is worth writing a log statement that walks the cause +chain. + +## What this means operationally + +Changing a TTL, a max-length, a dead-letter exchange or a queue type on a live queue is not a +config edit. It is a migration: + +1. declare the new queue under a new name +2. bind it alongside the old one +3. move consumers over +4. drain the old queue +5. delete it + +Plan the rename into the change from the start — `orders.work.v2` is not ugly, it is honest. + +## Related failure modes + +**A failed declaration stops the ones after it.** `RabbitAdmin` declares beans on connection, and +a `PRECONDITION_FAILED` closes the channel. Declarations queued behind it on that channel do not +happen. So one stale queue argument can leave half your topology undeclared, and the symptom is a +*different* queue being missing. + +**Declarations happen on connection, not on context refresh.** A broken topology does not fail +startup; it fails the first publish. If you want it to fail at boot, force a connection early — +`connectionFactory.createConnection()` in an `ApplicationRunner` is enough. + +**`missing-queues-fatal` defaults differently for the two container types.** It is `true` for +`spring.rabbitmq.listener.simple` and `false` for `spring.rabbitmq.listener.direct`. So the same +missing queue kills a simple container and is quietly tolerated by a direct one, which makes +"it works in that service" an unhelpful data point. Combined with the point above, a topology +error can present as a listener startup problem in one service and as silence in another. + +[Module README](../README.md) diff --git a/rabbitmq/docs/output/dead-letter.txt b/rabbitmq/docs/output/dead-letter.txt new file mode 100644 index 0000000..15c8172 --- /dev/null +++ b/rabbitmq/docs/output/dead-letter.txt @@ -0,0 +1,22 @@ +=== x-death after basicNack(requeue=false) === + reason rejected + count 1 + exchange + time Sat Aug 29 10:30:04 IST 2026 + routing-keys [orders.work] + queue orders.work +=== x-death after x-message-ttl expiry === + reason expired + count 1 + exchange + time Sat Aug 29 10:30:06 IST 2026 + routing-keys [orders.ttl] + queue orders.ttl +=== x-death after x-max-length overflow === + reason maxlen + count 1 + exchange orders.direct + time Sat Aug 29 10:30:06 IST 2026 + routing-keys [bounded] + queue orders.bounded + body {"orderId":"m-1","detail":"detail for m-1"} diff --git a/rabbitmq/docs/output/precondition-failed.txt b/rabbitmq/docs/output/precondition-failed.txt new file mode 100644 index 0000000..49340ef --- /dev/null +++ b/rabbitmq/docs/output/precondition-failed.txt @@ -0,0 +1,2 @@ +=== redeclaring orders.ttl with x-message-ttl=9999 === + java.io.IOException | null | channel error; protocol method: #method(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'x-message-ttl' for queue 'orders.ttl' in vhost '/': received '9999' but current is '1500', class-id=50, method-id=10) | channel error; protocol method: #method(reply-code=406, reply-text=PRECONDITION_FAILED - inequivalent arg 'x-message-ttl' for queue 'orders.ttl' in vhost '/': received '9999' but current is '1500', class-id=50, method-id=10) | diff --git a/rabbitmq/docs/output/requeue-loop.txt b/rabbitmq/docs/output/requeue-loop.txt new file mode 100644 index 0000000..cacaf06 --- /dev/null +++ b/rabbitmq/docs/output/requeue-loop.txt @@ -0,0 +1,4 @@ +=== basicNack(requeue=true), 200 attempts === + redelivered 199 times + dead-lettered 0 + still on queue 1 diff --git a/rabbitmq/docs/output/tests.txt b/rabbitmq/docs/output/tests.txt new file mode 100644 index 0000000..5f192f3 --- /dev/null +++ b/rabbitmq/docs/output/tests.txt @@ -0,0 +1,4 @@ +Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.132 s -- in com.ankurm.rabbit.TopologyTrapsTest +Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.714 s -- in com.ankurm.rabbit.DeadLetterTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.061 s -- in com.ankurm.rabbit.MaxLengthTest +Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.476 s -- in com.ankurm.rabbit.RoutingTest diff --git a/rabbitmq/docs/output/topic-wildcards.txt b/rabbitmq/docs/output/topic-wildcards.txt new file mode 100644 index 0000000..cdac8fd --- /dev/null +++ b/rabbitmq/docs/output/topic-wildcards.txt @@ -0,0 +1,7 @@ +=== topic exchange === +routing key orders.eu orders.high note +order.eu.high true true matches both +order.eu.low true false matches order.eu.* only +order.us.high false true matches order.#.high only +order.eu.west.high false true matches order.#.high only - * is one word +order.high false true matches order.#.high - # can be zero words diff --git a/rabbitmq/docs/output/unroutable.txt b/rabbitmq/docs/output/unroutable.txt new file mode 100644 index 0000000..ba9ead6 --- /dev/null +++ b/rabbitmq/docs/output/unroutable.txt @@ -0,0 +1,5 @@ +=== returned message === + replyCode 312 + replyText NO_ROUTE + exchange orders.direct + routingKey amend diff --git a/rabbitmq/pom.xml b/rabbitmq/pom.xml new file mode 100644 index 0000000..57ba551 --- /dev/null +++ b/rabbitmq/pom.xml @@ -0,0 +1,67 @@ + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.1.1 + + + + com.ankurm + rabbitmq-exchanges-and-dlq + 1.0 + jar + + + 25 + UTF-8 + + + + + + org.springframework.boot + spring-boot-starter-amqp + + + org.springframework.boot + spring-boot-starter-jackson + + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.amqp + spring-rabbit-test + test + + + + org.testcontainers + testcontainers-rabbitmq + test + + + org.springframework.boot + spring-boot-testcontainers + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + diff --git a/rabbitmq/scripts/broker.sh b/rabbitmq/scripts/broker.sh new file mode 100755 index 0000000..87ae2b1 --- /dev/null +++ b/rabbitmq/scripts/broker.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +# Start a real RabbitMQ broker with no Docker and no root. +# +# RabbitMQ is an Erlang application, so it needs an Erlang runtime and epmd on the path - that +# is the whole dependency. Point ERL_ROOT at an Erlang installation and RABBITMQ_HOME at an +# unpacked rabbitmq-server-generic-unix tarball, and it starts in about fifteen seconds. +# +# If you have Docker, use Testcontainers instead: see src/test/java/.../TestcontainersConfiguration.java +set -eu +: "${ERL_ROOT:?set ERL_ROOT to an Erlang installation directory}" +: "${RABBITMQ_HOME:?set RABBITMQ_HOME to an unpacked rabbitmq-server-generic-unix directory}" + +export PATH="$ERL_ROOT/bin:$ERL_ROOT/erts-"*/bin":$PATH" +export RABBITMQ_MNESIA_BASE="${RABBITMQ_MNESIA_BASE:-/tmp/rmq/data}" +export RABBITMQ_LOG_BASE="${RABBITMQ_LOG_BASE:-/tmp/rmq/log}" +export RABBITMQ_NODENAME="${RABBITMQ_NODENAME:-rabbit@localhost}" +export HOME="${HOME:-/tmp/rmq}" +mkdir -p "$RABBITMQ_MNESIA_BASE" "$RABBITMQ_LOG_BASE" + +epmd -daemon 2>/dev/null || true +sleep 1 +setsid nohup "$RABBITMQ_HOME/sbin/rabbitmq-server" > /tmp/rmq/boot.log 2>&1 < /dev/null & +for _ in $(seq 1 60); do + if (echo > /dev/tcp/127.0.0.1/5672) 2>/dev/null; then echo "broker ready on 5672"; exit 0; fi + sleep 1 +done +echo "broker did not start; see /tmp/rmq/boot.log" >&2 +exit 1 diff --git a/rabbitmq/scripts/run-all.sh b/rabbitmq/scripts/run-all.sh new file mode 100755 index 0000000..68f1461 --- /dev/null +++ b/rabbitmq/scripts/run-all.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +# Regenerate every file under docs/output/. +# +# Starts a real RabbitMQ broker first. Set ERL_ROOT and RABBITMQ_HOME (see scripts/broker.sh), +# or point spring.rabbitmq.host at a broker you already have and skip the first line. +set -eu +cd "$(dirname "$0")/.." +./scripts/broker.sh +mvn -B test 2>&1 | tee /tmp/rabbit-test.log > /dev/null + +sed -n '/=== topic exchange ===/,/^order\.high/p' /tmp/rabbit-test.log > docs/output/topic-wildcards.txt +sed -n '/=== x-death after basicNack/,/^ queue/p' /tmp/rabbit-test.log > docs/output/dead-letter.txt +sed -n '/=== x-death after x-message-ttl/,/^ queue/p' /tmp/rabbit-test.log >> docs/output/dead-letter.txt +sed -n '/=== x-death after x-max-length/,/^ body/p' /tmp/rabbit-test.log >> docs/output/dead-letter.txt +sed -n '/=== basicNack(requeue=true)/,/still on queue/p' /tmp/rabbit-test.log > docs/output/requeue-loop.txt +sed -n '/=== returned message ===/,/routingKey/p' /tmp/rabbit-test.log > docs/output/unroutable.txt +sed -n '/=== redeclaring orders.ttl/,+1p' /tmp/rabbit-test.log > docs/output/precondition-failed.txt +grep -E 'Tests run:.*in com\.ankurm' /tmp/rabbit-test.log | sed 's/^\[INFO\] //' > docs/output/tests.txt +echo "regenerated:"; ls -1 docs/output/ diff --git a/rabbitmq/src/main/java/com/ankurm/rabbit/ConverterConfiguration.java b/rabbitmq/src/main/java/com/ankurm/rabbit/ConverterConfiguration.java new file mode 100644 index 0000000..4686a63 --- /dev/null +++ b/rabbitmq/src/main/java/com/ankurm/rabbit/ConverterConfiguration.java @@ -0,0 +1,36 @@ +package com.ankurm.rabbit; + +import org.springframework.amqp.support.converter.JacksonJsonMessageConverter; +import org.springframework.amqp.support.converter.MessageConverter; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Spring Boot does not auto-configure a JSON message converter for RabbitMQ. The default + * is {@code SimpleMessageConverter}, which handles {@code String}, {@code byte[]} and + * {@code Serializable} and nothing else: + * + *
+ * IllegalArgumentException: SimpleMessageConverter only supports String, byte[] and
+ * Serializable payloads, received: com.ankurm.rabbit.OrderMessage
+ * 
+ * + *

Declaring one {@code MessageConverter} bean fixes both directions — the auto-configured + * {@code RabbitTemplate} and the listener container factory both pick it up. + * + *

Note the class name. Spring AMQP 4.1 ships {@code Jackson2JsonMessageConverter} (Jackson 2) + * and {@code JacksonJsonMessageConverter} (Jackson 3) side by side, exactly as Spring Kafka + * ships {@code JsonSerializer} and {@code JacksonJsonSerializer}. Boot 4 is a Jackson 3 + * application; pick the one without the 2. + * + * @see docs/01-the-on-ramp.md + */ +@Configuration(proxyBeanMethods = false) +public class ConverterConfiguration { + + @Bean + MessageConverter messageConverter() { + return new JacksonJsonMessageConverter(); + } + +} diff --git a/rabbitmq/src/main/java/com/ankurm/rabbit/OrderMessage.java b/rabbitmq/src/main/java/com/ankurm/rabbit/OrderMessage.java new file mode 100644 index 0000000..665e7f2 --- /dev/null +++ b/rabbitmq/src/main/java/com/ankurm/rabbit/OrderMessage.java @@ -0,0 +1,13 @@ +package com.ankurm.rabbit; + +/** + * The payload. Kept to a String field so the tests are about routing and acknowledgement rather + * than about converters. + */ +public record OrderMessage(String orderId, String detail) { + + public static OrderMessage of(String orderId) { + return new OrderMessage(orderId, "detail for " + orderId); + } + +} diff --git a/rabbitmq/src/main/java/com/ankurm/rabbit/RabbitDemoApplication.java b/rabbitmq/src/main/java/com/ankurm/rabbit/RabbitDemoApplication.java new file mode 100644 index 0000000..b7047fc --- /dev/null +++ b/rabbitmq/src/main/java/com/ankurm/rabbit/RabbitDemoApplication.java @@ -0,0 +1,20 @@ +package com.ankurm.rabbit; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +/** + * All four AMQP 0-9-1 exchange types, a working dead-letter path, and the traps that make + * RabbitMQ feel unpredictable until you know them. + * + *

Everything here runs against a real broker. See the module README for how the scripts start + * one without Docker. + */ +@SpringBootApplication +public class RabbitDemoApplication { + + public static void main(String[] args) { + SpringApplication.run(RabbitDemoApplication.class, args); + } + +} diff --git a/rabbitmq/src/main/java/com/ankurm/rabbit/Topology.java b/rabbitmq/src/main/java/com/ankurm/rabbit/Topology.java new file mode 100644 index 0000000..bf5acba --- /dev/null +++ b/rabbitmq/src/main/java/com/ankurm/rabbit/Topology.java @@ -0,0 +1,217 @@ +package com.ankurm.rabbit; + +import org.springframework.amqp.core.Binding; +import org.springframework.amqp.core.BindingBuilder; +import org.springframework.amqp.core.DirectExchange; +import org.springframework.amqp.core.FanoutExchange; +import org.springframework.amqp.core.HeadersExchange; +import org.springframework.amqp.core.Queue; +import org.springframework.amqp.core.QueueBuilder; +import org.springframework.amqp.core.TopicExchange; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +import java.util.Map; + +/** + * The whole topology as beans. {@code RabbitAdmin} declares every {@code Exchange}, + * {@code Queue} and {@code Binding} bean when the connection is first opened. + * + *

Two things about that are worth knowing before you rely on it: + *

    + *
  • Declaration is idempotent only if the arguments match. Redeclaring an existing + * queue with different arguments is a channel-level {@code PRECONDITION_FAILED} — see + * docs/06-changing-your-mind.md.
  • + *
  • Declarations happen on connection, not on context refresh, so a topology error surfaces + * at the first publish rather than at startup.
  • + *
+ * + * @see docs/02-exchanges.md + */ +@Configuration(proxyBeanMethods = false) +public class Topology { + + public static final String DIRECT = "orders.direct"; + + public static final String FANOUT = "orders.fanout"; + + public static final String TOPIC = "orders.topic"; + + public static final String HEADERS = "orders.headers"; + + public static final String DLX = "orders.dlx"; + + public static final String Q_NEW = "orders.new"; + + public static final String Q_CANCEL = "orders.cancel"; + + public static final String Q_AUDIT = "audit.all"; + + public static final String Q_ANALYTICS = "analytics.all"; + + public static final String Q_EU = "orders.eu"; + + public static final String Q_HIGH = "orders.high"; + + public static final String Q_PRIORITY = "orders.priority"; + + public static final String Q_ANY = "orders.any"; + + public static final String Q_WORK = "orders.work"; + + public static final String Q_TTL = "orders.ttl"; + + public static final String Q_DLQ = "orders.dlq"; + + // --- direct: routing key must match the binding key exactly ----------------------------- + + @Bean + DirectExchange directExchange() { + return new DirectExchange(DIRECT); + } + + @Bean + Queue newQueue() { + return QueueBuilder.durable(Q_NEW).build(); + } + + @Bean + Queue cancelQueue() { + return QueueBuilder.durable(Q_CANCEL).build(); + } + + @Bean + Binding bindNew(DirectExchange directExchange, Queue newQueue) { + return BindingBuilder.bind(newQueue).to(directExchange).with("new"); + } + + @Bean + Binding bindCancel(DirectExchange directExchange, Queue cancelQueue) { + return BindingBuilder.bind(cancelQueue).to(directExchange).with("cancel"); + } + + // --- fanout: routing key ignored entirely ----------------------------------------------- + + @Bean + FanoutExchange fanoutExchange() { + return new FanoutExchange(FANOUT); + } + + @Bean + Queue auditQueue() { + return QueueBuilder.durable(Q_AUDIT).build(); + } + + @Bean + Queue analyticsQueue() { + return QueueBuilder.durable(Q_ANALYTICS).build(); + } + + @Bean + Binding bindAudit(FanoutExchange fanoutExchange, Queue auditQueue) { + return BindingBuilder.bind(auditQueue).to(fanoutExchange); + } + + @Bean + Binding bindAnalytics(FanoutExchange fanoutExchange, Queue analyticsQueue) { + return BindingBuilder.bind(analyticsQueue).to(fanoutExchange); + } + + // --- topic: '*' is exactly one word, '#' is zero or more ------------------------------- + + @Bean + TopicExchange topicExchange() { + return new TopicExchange(TOPIC); + } + + @Bean + Queue euQueue() { + return QueueBuilder.durable(Q_EU).build(); + } + + @Bean + Queue highQueue() { + return QueueBuilder.durable(Q_HIGH).build(); + } + + @Bean + Binding bindEu(TopicExchange topicExchange, Queue euQueue) { + return BindingBuilder.bind(euQueue).to(topicExchange).with("order.eu.*"); + } + + @Bean + Binding bindHigh(TopicExchange topicExchange, Queue highQueue) { + return BindingBuilder.bind(highQueue).to(topicExchange).with("order.#.high"); + } + + // --- headers: routing key ignored, header map matched ------------------------------------ + + @Bean + HeadersExchange headersExchange() { + return new HeadersExchange(HEADERS); + } + + @Bean + Queue priorityQueue() { + return QueueBuilder.durable(Q_PRIORITY).build(); + } + + @Bean + Queue anyQueue() { + return QueueBuilder.durable(Q_ANY).build(); + } + + /** x-match=all: every header must match. */ + @Bean + Binding bindPriority(HeadersExchange headersExchange, Queue priorityQueue) { + return BindingBuilder.bind(priorityQueue).to(headersExchange) + .whereAll(Map.of("priority", "high", "region", "eu")).match(); + } + + /** x-match=any: one is enough. */ + @Bean + Binding bindAny(HeadersExchange headersExchange, Queue anyQueue) { + return BindingBuilder.bind(anyQueue).to(headersExchange) + .whereAny(Map.of("priority", "high", "region", "eu")).match(); + } + + // --- the dead-letter path ---------------------------------------------------------------- + + @Bean + DirectExchange deadLetterExchange() { + return new DirectExchange(DLX); + } + + @Bean + Queue deadLetterQueue() { + // Deliberately NO dead-letter-exchange on the DLQ itself. Pointing a DLQ's DLX back at + // the exchange that feeds it is the classic infinite loop. + return QueueBuilder.durable(Q_DLQ).build(); + } + + @Bean + Binding bindDlq(DirectExchange deadLetterExchange, Queue deadLetterQueue) { + // The routing key used when dead-lettering is the message's ORIGINAL routing key unless + // deadLetterRoutingKey() overrides it. Both work queues below set it explicitly to + // "failed", so this one binding catches everything. + return BindingBuilder.bind(deadLetterQueue).to(deadLetterExchange).with("failed"); + } + + @Bean + Queue workQueue() { + return QueueBuilder.durable(Q_WORK) + .deadLetterExchange(DLX) + .deadLetterRoutingKey("failed") + .build(); + } + + @Bean + Queue ttlQueue() { + return QueueBuilder.durable(Q_TTL) + .ttl(1500) + .deadLetterExchange(DLX) + .deadLetterRoutingKey("failed") + .build(); + } + +} diff --git a/rabbitmq/src/main/resources/application.yaml b/rabbitmq/src/main/resources/application.yaml new file mode 100644 index 0000000..7879b37 --- /dev/null +++ b/rabbitmq/src/main/resources/application.yaml @@ -0,0 +1,31 @@ +spring: + application: + name: rabbitmq-exchanges-and-dlq + main: + banner-mode: off + rabbitmq: + host: localhost + port: 5672 + username: guest + password: guest + # BOTH of these are required to find out about an unroutable message, and they are set in + # two different places. publisher-returns puts the connection factory into a mode where it + # listens for returns at all; mandatory asks the broker to return this particular publish. + # Setting only mandatory - which is what most examples show - changes nothing at all. + # See docs/03-the-silent-drop.md. + publisher-returns: true + template: + mandatory: true + listener: + simple: + # MANUAL means nothing is removed from the queue until your code says so - and that + # basicNack(requeue=false) is what feeds the dead-letter exchange. + acknowledge-mode: manual + prefetch: 1 + # Listener containers are not started automatically here; each test starts the one it + # needs, so that a redelivery loop in one test cannot consume another test's messages. + auto-startup: false +logging: + level: + root: WARN + com.ankurm: INFO diff --git a/rabbitmq/src/test/java/com/ankurm/rabbit/DeadLetterTest.java b/rabbitmq/src/test/java/com/ankurm/rabbit/DeadLetterTest.java new file mode 100644 index 0000000..efbacbe --- /dev/null +++ b/rabbitmq/src/test/java/com/ankurm/rabbit/DeadLetterTest.java @@ -0,0 +1,143 @@ +package com.ankurm.rabbit; + +import com.rabbitmq.client.GetResponse; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.amqp.rabbit.core.RabbitAdmin; +import org.springframework.amqp.rabbit.core.RabbitTemplate; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; + +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * The dead-letter path, driven with the AMQP primitives directly rather than through a listener + * container. {@code basicGet} + {@code basicNack} is what a container does underneath, and doing + * it by hand removes all the timing from the test. + * + * @see docs/04-dead-lettering.md + */ +@SpringBootTest +class DeadLetterTest { + + @Autowired + RabbitTemplate template; + + @Autowired + RabbitAdmin admin; + + @BeforeEach + void drain() { + List.of(Topology.Q_WORK, Topology.Q_TTL, Topology.Q_DLQ) + .forEach((q) -> this.admin.purgeQueue(q, false)); + } + + private int depth(String queue) { + Properties properties = this.admin.getQueueProperties(queue); + return (properties == null) ? -1 + : ((Number) properties.get(RabbitAdmin.QUEUE_MESSAGE_COUNT)).intValue(); + } + + private int settled(String queue, int expected) throws InterruptedException { + for (int i = 0; i < 150; i++) { + if (depth(queue) == expected) { + return expected; + } + Thread.sleep(20); + } + return depth(queue); + } + + @Test + void nackWithoutRequeueDeadLettersAndStampsTheReason() throws Exception { + this.template.convertAndSend(Topology.DIRECT, "ignored", OrderMessage.of("o-1"), + (m) -> m, null); + // Publish straight to the work queue via the default exchange: the empty exchange name + // routes by queue name, which is the one piece of AMQP that behaves like a magic + // constant and is worth knowing. + this.template.convertAndSend("", Topology.Q_WORK, OrderMessage.of("o-1")); + assertThat(settled(Topology.Q_WORK, 1)).isEqualTo(1); + + this.template.execute((channel) -> { + GetResponse response = channel.basicGet(Topology.Q_WORK, false); + assertThat(response).isNotNull(); + // requeue=false is the entire dead-letter trigger. There is no separate "send to + // DLQ" call in AMQP. + channel.basicNack(response.getEnvelope().getDeliveryTag(), false, false); + return null; + }); + + assertThat(settled(Topology.Q_DLQ, 1)).isEqualTo(1); + assertThat(depth(Topology.Q_WORK)).isZero(); + + Map death = firstDeath(Topology.Q_DLQ); + System.out.println("=== x-death after basicNack(requeue=false) ==="); + death.forEach((k, v) -> System.out.printf(" %-16s %s%n", k, v)); + assertThat(death).containsEntry("reason", "rejected"); + assertThat(death).containsEntry("queue", Topology.Q_WORK); + assertThat(((Number) death.get("count")).intValue()).isEqualTo(1); + } + + @Test + void messageTtlExpiryAlsoDeadLettersButWithADifferentReason() throws Exception { + this.template.convertAndSend("", Topology.Q_TTL, OrderMessage.of("o-ttl")); + // orders.ttl carries x-message-ttl=1500. Nobody consumes it; the broker expires it. + assertThat(settled(Topology.Q_DLQ, 1)).isEqualTo(1); + + Map death = firstDeath(Topology.Q_DLQ); + System.out.println("=== x-death after x-message-ttl expiry ==="); + death.forEach((k, v) -> System.out.printf(" %-16s %s%n", k, v)); + // Same destination, different reason. This is the field that tells an operator whether + // the consumer rejected the work or never got to it. + assertThat(death).containsEntry("reason", "expired"); + assertThat(death).containsEntry("queue", Topology.Q_TTL); + } + + @Test + void nackWithRequeueIsAnInfiniteLoopAndNeverDeadLetters() throws Exception { + this.template.convertAndSend("", Topology.Q_WORK, OrderMessage.of("o-loop")); + assertThat(settled(Topology.Q_WORK, 1)).isEqualTo(1); + + int redeliveries = this.template.execute((channel) -> { + int count = 0; + for (int i = 0; i < 200; i++) { + GetResponse response = channel.basicGet(Topology.Q_WORK, false); + if (response == null) { + break; + } + if (response.getEnvelope().isRedeliver()) { + count++; + } + // requeue=true puts it back. The dead-letter exchange is never consulted, + // x-death is never written, and the loop has no counter to exhaust. + channel.basicNack(response.getEnvelope().getDeliveryTag(), false, true); + } + return count; + }); + + System.out.println("=== basicNack(requeue=true), 200 attempts ==="); + System.out.println(" redelivered " + redeliveries + " times"); + System.out.println(" dead-lettered " + depth(Topology.Q_DLQ)); + System.out.println(" still on queue " + depth(Topology.Q_WORK)); + + assertThat(redeliveries).isGreaterThan(150); + // The message is exactly where it started, having been processed 200 times. + assertThat(depth(Topology.Q_DLQ)).isZero(); + assertThat(settled(Topology.Q_WORK, 1)).isEqualTo(1); + } + + @SuppressWarnings("unchecked") + private Map firstDeath(String queue) { + var message = this.template.receive(queue, 5000); + assertThat(message).isNotNull(); + List> deaths = + (List>) message.getMessageProperties().getHeader("x-death"); + assertThat(deaths).isNotNull().isNotEmpty(); + return deaths.get(0); + } + +} diff --git a/rabbitmq/src/test/java/com/ankurm/rabbit/MaxLengthTest.java b/rabbitmq/src/test/java/com/ankurm/rabbit/MaxLengthTest.java new file mode 100644 index 0000000..6d8b942 --- /dev/null +++ b/rabbitmq/src/test/java/com/ankurm/rabbit/MaxLengthTest.java @@ -0,0 +1,73 @@ +package com.ankurm.rabbit; + +import com.rabbitmq.client.GetResponse; +import org.junit.jupiter.api.Test; +import org.springframework.amqp.core.Binding; +import org.springframework.amqp.core.BindingBuilder; +import org.springframework.amqp.core.DirectExchange; +import org.springframework.amqp.core.Queue; +import org.springframework.amqp.core.QueueBuilder; +import org.springframework.amqp.rabbit.core.RabbitAdmin; +import org.springframework.amqp.rabbit.core.RabbitTemplate; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; + +import java.util.List; +import java.util.Map; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * The third dead-letter trigger: a queue that is full. + * + *

Included because the article states all three triggers in one table, and this was the only + * row not produced by a run. It is now. + * + * @see docs/04-dead-lettering.md + */ +@SpringBootTest +class MaxLengthTest { + + static final String Q_BOUNDED = "orders.bounded"; + + @Autowired + RabbitTemplate template; + + @Autowired + RabbitAdmin admin; + + @Test + @SuppressWarnings("unchecked") + void exceedingMaxLengthDeadLettersTheOldestWithReasonMaxlen() throws Exception { + this.admin.deleteQueue(Q_BOUNDED); + Queue bounded = QueueBuilder.durable(Q_BOUNDED) + .maxLength(2) + .deadLetterExchange(Topology.DLX) + .deadLetterRoutingKey("failed") + .build(); + this.admin.declareQueue(bounded); + Binding binding = BindingBuilder.bind(bounded) + .to(new DirectExchange(Topology.DIRECT)).with("bounded"); + this.admin.declareBinding(binding); + this.admin.purgeQueue(Topology.Q_DLQ, false); + + // Three messages into a queue that holds two. RabbitMQ drops from the HEAD, so the + // FIRST message is the one dead-lettered - the oldest, not the newest. + for (String id : List.of("m-1", "m-2", "m-3")) { + this.template.convertAndSend(Topology.DIRECT, "bounded", OrderMessage.of(id)); + } + + var message = this.template.receive(Topology.Q_DLQ, 10_000); + assertThat(message).isNotNull(); + List> deaths = + (List>) message.getMessageProperties().getHeader("x-death"); + System.out.println("=== x-death after x-max-length overflow ==="); + deaths.get(0).forEach((k, v) -> System.out.printf(" %-16s %s%n", k, v)); + System.out.println(" body " + new String(message.getBody())); + + assertThat(deaths.get(0)).containsEntry("reason", "maxlen"); + assertThat(deaths.get(0)).containsEntry("queue", Q_BOUNDED); + assertThat(new String(message.getBody())).contains("m-1"); + } + +} diff --git a/rabbitmq/src/test/java/com/ankurm/rabbit/RoutingTest.java b/rabbitmq/src/test/java/com/ankurm/rabbit/RoutingTest.java new file mode 100644 index 0000000..5d550b9 --- /dev/null +++ b/rabbitmq/src/test/java/com/ankurm/rabbit/RoutingTest.java @@ -0,0 +1,152 @@ +package com.ankurm.rabbit; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.amqp.core.MessageProperties; +import org.springframework.amqp.rabbit.core.RabbitAdmin; +import org.springframework.amqp.rabbit.core.RabbitTemplate; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; + +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * All four exchange types, driven with real publishes and counted by draining the queues. + * + * @see docs/02-exchanges.md + */ +@SpringBootTest +class RoutingTest { + + @Autowired + RabbitTemplate template; + + @Autowired + RabbitAdmin admin; + + private static final List QUEUES = List.of(Topology.Q_NEW, Topology.Q_CANCEL, + Topology.Q_AUDIT, Topology.Q_ANALYTICS, Topology.Q_EU, Topology.Q_HIGH, + Topology.Q_PRIORITY, Topology.Q_ANY); + + @BeforeEach + void drain() { + QUEUES.forEach((q) -> this.admin.purgeQueue(q, false)); + } + + /** + * Ready message count, straight from the broker. + * + *

Note the {@code Number}: {@code RabbitAdmin.QUEUE_MESSAGE_COUNT} holds a {@code Long} + * in Spring AMQP 4.1. Casting it to {@code Integer}, as every older example does, is a + * {@code ClassCastException} at runtime. + */ + private int depth(String queue) { + Properties properties = this.admin.getQueueProperties(queue); + return (properties == null) ? -1 + : ((Number) properties.get(RabbitAdmin.QUEUE_MESSAGE_COUNT)).intValue(); + } + + /** Publishing is asynchronous; give the broker a moment to route before counting. */ + private int settledDepth(String queue) { + int last = -1; + for (int i = 0; i < 50; i++) { + last = depth(queue); + if (last > 0) { + return last; + } + try { + Thread.sleep(20); + } + catch (InterruptedException ex) { + Thread.currentThread().interrupt(); + } + } + return last; + } + + @Test + void directExchangeMatchesTheRoutingKeyExactly() { + this.template.convertAndSend(Topology.DIRECT, "new", OrderMessage.of("o-1")); + this.template.convertAndSend(Topology.DIRECT, "cancel", OrderMessage.of("o-2")); + // No binding for "amend". The broker discards it - see UnroutableTest. + this.template.convertAndSend(Topology.DIRECT, "amend", OrderMessage.of("o-3")); + + assertThat(settledDepth(Topology.Q_NEW)).isEqualTo(1); + assertThat(settledDepth(Topology.Q_CANCEL)).isEqualTo(1); + } + + @Test + void fanoutIgnoresTheRoutingKeyCompletely() { + this.template.convertAndSend(Topology.FANOUT, "this-is-ignored", OrderMessage.of("o-1")); + + assertThat(settledDepth(Topology.Q_AUDIT)).isEqualTo(1); + assertThat(settledDepth(Topology.Q_ANALYTICS)).isEqualTo(1); + } + + @Test + void topicWildcardsAreWordsNotCharacters() { + // binding "order.eu.*" - exactly one word after order.eu + // binding "order.#.high" - zero or more words between order. and .high + Map routingKeys = new LinkedHashMap<>(); + routingKeys.put("order.eu.high", "matches both"); + routingKeys.put("order.eu.low", "matches order.eu.* only"); + routingKeys.put("order.us.high", "matches order.#.high only"); + routingKeys.put("order.eu.west.high", "matches order.#.high only - * is one word"); + routingKeys.put("order.high", "matches order.#.high - # can be zero words"); + + System.out.println("=== topic exchange ==="); + System.out.printf("%-24s %-12s %-12s %s%n", "routing key", "orders.eu", "orders.high", "note"); + for (Map.Entry entry : routingKeys.entrySet()) { + this.admin.purgeQueue(Topology.Q_EU, false); + this.admin.purgeQueue(Topology.Q_HIGH, false); + this.template.convertAndSend(Topology.TOPIC, entry.getKey(), OrderMessage.of("o")); + System.out.printf("%-24s %-12s %-12s %s%n", entry.getKey(), settledDepth(Topology.Q_EU) == 1, + settledDepth(Topology.Q_HIGH) == 1, entry.getValue()); + } + + this.admin.purgeQueue(Topology.Q_EU, false); + this.admin.purgeQueue(Topology.Q_HIGH, false); + // The one everybody gets wrong: '*' is one WORD, so it does not match two. + this.template.convertAndSend(Topology.TOPIC, "order.eu.west.high", OrderMessage.of("o")); + assertThat(depth(Topology.Q_EU)).isEqualTo(0); + assertThat(settledDepth(Topology.Q_HIGH)).isEqualTo(1); + + this.admin.purgeQueue(Topology.Q_HIGH, false); + // And '#' really does match zero words. + this.template.convertAndSend(Topology.TOPIC, "order.high", OrderMessage.of("o")); + assertThat(settledDepth(Topology.Q_HIGH)).isEqualTo(1); + } + + @Test + void headersExchangeMatchesTheHeaderMapNotTheRoutingKey() { + send(Map.of("priority", "high", "region", "eu")); + assertThat(settledDepth(Topology.Q_PRIORITY)).isEqualTo(1); // x-match=all: both present + assertThat(settledDepth(Topology.Q_ANY)).isEqualTo(1); // x-match=any: either is enough + + drain(); + send(Map.of("priority", "high")); + assertThat(depth(Topology.Q_PRIORITY)).isEqualTo(0); // all: region missing + assertThat(settledDepth(Topology.Q_ANY)).isEqualTo(1); + + drain(); + send(Map.of("priority", "high", "region", "us")); + // x-match=all needs every header to match by VALUE, not merely to be present. + assertThat(depth(Topology.Q_PRIORITY)).isEqualTo(0); + assertThat(settledDepth(Topology.Q_ANY)).isEqualTo(1); + } + + private void send(Map headers) { + this.template.convertAndSend(Topology.HEADERS, "routing-key-is-ignored", + OrderMessage.of("o-1"), (message) -> { + MessageProperties properties = message.getMessageProperties(); + headers.forEach(properties::setHeader); + return message; + }); + } + +} diff --git a/rabbitmq/src/test/java/com/ankurm/rabbit/TestcontainersConfiguration.java b/rabbitmq/src/test/java/com/ankurm/rabbit/TestcontainersConfiguration.java new file mode 100644 index 0000000..2807d90 --- /dev/null +++ b/rabbitmq/src/test/java/com/ankurm/rabbit/TestcontainersConfiguration.java @@ -0,0 +1,33 @@ +package com.ankurm.rabbit; + +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.testcontainers.service.connection.ServiceConnection; +import org.springframework.context.annotation.Bean; +import org.testcontainers.rabbitmq.RabbitMQContainer; +import org.testcontainers.utility.DockerImageName; + +/** + * The Testcontainers route, for machines that have a Docker daemon. + * + *

{@code @ServiceConnection} supplies host, port, username and password to the + * auto-configuration, so no {@code spring.rabbitmq.*} property and no + * {@code @DynamicPropertySource} block is needed. + * + *

The Maven coordinate is {@code org.testcontainers:testcontainers-rabbitmq}. + * Testcontainers 2.x prefixed every module artifact; the old {@code org.testcontainers:rabbitmq} + * stopped at 1.21.4 and is not managed by the Boot 4.1 BOM. + * + *

The committed transcripts under {@code docs/output/} came from a broker started by + * {@code scripts/broker.sh} instead, on a machine with no Docker — see the module README + * for why that matters and what version it was. + */ +@TestConfiguration(proxyBeanMethods = false) +public class TestcontainersConfiguration { + + @Bean + @ServiceConnection + RabbitMQContainer rabbitContainer() { + return new RabbitMQContainer(DockerImageName.parse("rabbitmq:4.1-management")); + } + +} diff --git a/rabbitmq/src/test/java/com/ankurm/rabbit/TopologyTrapsTest.java b/rabbitmq/src/test/java/com/ankurm/rabbit/TopologyTrapsTest.java new file mode 100644 index 0000000..5c37e91 --- /dev/null +++ b/rabbitmq/src/test/java/com/ankurm/rabbit/TopologyTrapsTest.java @@ -0,0 +1,100 @@ +package com.ankurm.rabbit; + +import org.junit.jupiter.api.Test; +import org.springframework.amqp.AmqpIOException; +import org.springframework.amqp.core.Queue; +import org.springframework.amqp.core.QueueBuilder; +import org.springframework.amqp.core.ReturnedMessage; +import org.springframework.amqp.rabbit.core.RabbitAdmin; +import org.springframework.amqp.rabbit.core.RabbitTemplate; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; + +import java.util.concurrent.CopyOnWriteArrayList; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatExceptionOfType; + +/** + * Two ways a RabbitMQ topology loses your work quietly. + * + * @see docs/03-the-silent-drop.md + * @see docs/06-changing-your-mind.md + */ +@SpringBootTest +class TopologyTrapsTest { + + @Autowired + RabbitTemplate template; + + @Autowired + RabbitAdmin admin; + + @Test + void anUnroutableMessageIsDiscardedUnlessYouAskForItBack() throws Exception { + List returned = new CopyOnWriteArrayList<>(); + this.template.setReturnsCallback(returned::add); + + // "amend" matches no binding on orders.direct. The broker has nowhere to put it. + this.template.convertAndSend(Topology.DIRECT, "amend", OrderMessage.of("o-1")); + for (int i = 0; i < 100 && returned.isEmpty(); i++) { + Thread.sleep(20); + } + + // spring.rabbitmq.template.mandatory=true is what turns a silent discard into a return. + assertThat(this.template.isMandatoryFor(null)).isTrue(); + assertThat(returned).hasSize(1); + ReturnedMessage message = returned.get(0); + System.out.println("=== returned message ==="); + System.out.println(" replyCode " + message.getReplyCode()); + System.out.println(" replyText " + message.getReplyText()); + System.out.println(" exchange " + message.getExchange()); + System.out.println(" routingKey " + message.getRoutingKey()); + + assertThat(message.getReplyCode()).isEqualTo(312); + assertThat(message.getReplyText()).isEqualTo("NO_ROUTE"); + assertThat(message.getRoutingKey()).isEqualTo("amend"); + // Note what did NOT happen: convertAndSend returned normally. Publishing is fire and + // forget at the protocol level, so even with mandatory=true the failure arrives + // asynchronously on another thread. Nothing throws. + } + + @Test + void redeclaringAQueueWithDifferentArgumentsIsAPreconditionFailure() { + // orders.ttl already exists with x-message-ttl=1500. Same name, different argument. + Queue conflicting = QueueBuilder.durable(Topology.Q_TTL) + .ttl(9999) + .deadLetterExchange(Topology.DLX) + .deadLetterRoutingKey("failed") + .build(); + + assertThatExceptionOfType(AmqpIOException.class) + .isThrownBy(() -> this.admin.declareQueue(conflicting)) + .satisfies((ex) -> { + String detail = rootMessage(ex); + System.out.println("=== redeclaring orders.ttl with x-message-ttl=9999 ==="); + System.out.println(" " + detail); + assertThat(detail).contains("PRECONDITION_FAILED") + .contains("inequivalent arg 'x-message-ttl'"); + }); + + // Queue arguments are immutable. There is no ALTER QUEUE. Changing a TTL, a max-length + // or a dead-letter exchange on an existing queue means: declare a new queue, move the + // consumers, drain the old one, delete it. Plan the rename into the change. + } + + private static String rootMessage(Throwable throwable) { + // The useful text is on the cause. AmqpIOException's own message is just + // "java.io.IOException", which is why this failure is so often reported as "IOException" + // with no further detail. + Throwable current = throwable; + StringBuilder all = new StringBuilder(); + while (current != null) { + all.append(current.getMessage()).append(" | "); + current = current.getCause(); + } + return all.toString(); + } + +}