Add outbox module: Transactional Outbox with the Spring Modulith Event Publication Registry

This commit is contained in:
2026-10-03 19:27:32 +00:00
parent 866eceed0a
commit d815a37f2e
17 changed files with 721 additions and 0 deletions
+35
View File
@@ -0,0 +1,35 @@
$ mvn -pl outbox -am test -Dtest=DualWriteProblemTest
(NaiveBillingService: save the Invoice, then separately call kafka.send() -- no
coordinator between the two. spring.kafka.bootstrap-servers points at localhost:19999,
nothing listening there)
2026-10-04T00:52:56.429+05:30 ERROR 5271 --- [outbox] [ main] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='1' and payload='WIDGET-1:1999' to topic invoices-naive:
org.apache.kafka.common.errors.TimeoutException: Topic invoices-naive not present in metadata after 2000 ms.
Exception thrown synchronously from issueInvoiceTheNaiveWay(): org.springframework.kafka.KafkaException: Send failed
Invoice count regardless: 1 (was 0). The database write already committed inside saveInvoiceOnly() before this exception was ever thrown -- the exception came from the Kafka call, several lines later, on an already-committed row.
2026-10-04T00:52:58.495+05:30 ERROR 5271 --- [outbox] [ main] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='2' and payload='WIDGET-1:2999' to topic invoices-naive:
org.apache.kafka.common.errors.TimeoutException: Topic invoices-naive not present in metadata after 2000 ms.
Invoice count after the blocked, failed send: 2 (was 1). The row committed before the send was even attempted.
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
Two real findings here, not one. First, the obvious one: the invoice exists in the
database in both tests, and Kafka never received either message -- that's the dual-write
problem, reproduced with nothing faked beyond pointing Kafka at a port with no listener.
Second, a more specific one that contradicts the usual "fire-and-forget send() fails
silently" framing: when the broker can't be reached to fetch topic metadata at all,
Spring's KafkaTemplate.send() throws synchronously, wrapping the Kafka client's own
TimeoutException in a KafkaException, even though the method's declared return type is
a CompletableFuture. The "silent" version of this bug is real, but it is a narrower
case than total broker unavailability -- it's what happens when the broker is reachable
and the topic's metadata is known, but the specific produce request times out waiting
for an acknowledgment afterward. That failure mode surfaces only through the returned
future or a callback, never as a synchronous exception, and is not reproduced in this
transcript (it needs a broker that accepts the connection and then stops responding
mid-request, not one that was never there to begin with).
+20
View File
@@ -0,0 +1,20 @@
$ mvn -pl outbox -am test -Dtest=RegistrySafetyNetTest
(real embedded Kafka broker via @EmbeddedKafka; BillingManagement.issueInvoice()
publishes InvoiceIssued, @Externalized("invoices::#{#this.invoiceId}") routes it
through spring-modulith-events-kafka)
Kafka record received -- topic=invoices key=1 value={"invoiceId":1,"sku":"GADGET-7","amountCents":4500}
EVENT_PUBLICATION_ARCHIVE rows for InvoiceIssued: 2
EVENT_PUBLICATION rows still outstanding for InvoiceIssued: 0
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
The message really arrived on the real topic, with the real routing key (the invoice's
own id, exactly as the SpEL expression after "::" in @Externalized specifies) and the
real JSON payload. Two archive rows for InvoiceIssued, not one, because this module's
test sources also declare FlakyAuditListener (see output/02) -- it is a second,
independent listener on the same event, and with completion-mode=ARCHIVE configured in
application.properties, both listeners' completed publications moved out of
EVENT_PUBLICATION and into EVENT_PUBLICATION_ARCHIVE once they succeeded. Zero rows
left outstanding in EVENT_PUBLICATION for this event type confirms neither listener is
still pending.
@@ -0,0 +1,23 @@
$ mvn -pl outbox -am test -Dtest=ReplayIncompletePublicationsTest
(FlakyAuditListener throws on its first invocation of InvoiceIssued, succeeds on every
one after -- standing in for any downstream dependency that is briefly unavailable)
Incomplete FlakyAuditListener publications after the simulated failure: 1
Incomplete FlakyAuditListener publications after resubmission: 0
Invoice 1 never changed -- the retry was purely about redelivering the event, the invoice row was correct the whole time.
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
The sequence that actually ran: BillingManagement.issueInvoice() commits the Invoice
row and the EVENT_PUBLICATION row for FlakyAuditListener in one local transaction.
FlakyAuditListener.on(InvoiceIssued) is then invoked asynchronously, throws on purpose,
and the registry leaves that row with COMPLETION_DATE still null -- "incomplete" is
exactly that: a row with no completion date, regardless of whether the listener ever
ran at all or ran and failed. Calling
IncompleteEventPublications.resubmitIncompletePublicationsOlderThan(Duration.ZERO) --
a real method on a real interface, verified against the compiled events-api 2.1.1 jar,
not assumed from documentation prose -- re-invokes the listener for every such row.
This time failNextAttempt is already false, so it succeeds, and the row's completion
date is set. The invoice itself was never touched by any of this: the retry is about
redelivering a notification, not about redoing a database write that already
succeeded the first time.