Add outbox module: Transactional Outbox with the Spring Modulith Event Publication Registry
This commit is contained in:
@@ -0,0 +1,35 @@
|
||||
$ mvn -pl outbox -am test -Dtest=DualWriteProblemTest
|
||||
(NaiveBillingService: save the Invoice, then separately call kafka.send() -- no
|
||||
coordinator between the two. spring.kafka.bootstrap-servers points at localhost:19999,
|
||||
nothing listening there)
|
||||
|
||||
2026-10-04T00:52:56.429+05:30 ERROR 5271 --- [outbox] [ main] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='1' and payload='WIDGET-1:1999' to topic invoices-naive:
|
||||
|
||||
org.apache.kafka.common.errors.TimeoutException: Topic invoices-naive not present in metadata after 2000 ms.
|
||||
|
||||
Exception thrown synchronously from issueInvoiceTheNaiveWay(): org.springframework.kafka.KafkaException: Send failed
|
||||
Invoice count regardless: 1 (was 0). The database write already committed inside saveInvoiceOnly() before this exception was ever thrown -- the exception came from the Kafka call, several lines later, on an already-committed row.
|
||||
|
||||
2026-10-04T00:52:58.495+05:30 ERROR 5271 --- [outbox] [ main] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='2' and payload='WIDGET-1:2999' to topic invoices-naive:
|
||||
|
||||
org.apache.kafka.common.errors.TimeoutException: Topic invoices-naive not present in metadata after 2000 ms.
|
||||
|
||||
Invoice count after the blocked, failed send: 2 (was 1). The row committed before the send was even attempted.
|
||||
|
||||
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
|
||||
|
||||
Two real findings here, not one. First, the obvious one: the invoice exists in the
|
||||
database in both tests, and Kafka never received either message -- that's the dual-write
|
||||
problem, reproduced with nothing faked beyond pointing Kafka at a port with no listener.
|
||||
|
||||
Second, a more specific one that contradicts the usual "fire-and-forget send() fails
|
||||
silently" framing: when the broker can't be reached to fetch topic metadata at all,
|
||||
Spring's KafkaTemplate.send() throws synchronously, wrapping the Kafka client's own
|
||||
TimeoutException in a KafkaException, even though the method's declared return type is
|
||||
a CompletableFuture. The "silent" version of this bug is real, but it is a narrower
|
||||
case than total broker unavailability -- it's what happens when the broker is reachable
|
||||
and the topic's metadata is known, but the specific produce request times out waiting
|
||||
for an acknowledgment afterward. That failure mode surfaces only through the returned
|
||||
future or a callback, never as a synchronous exception, and is not reproduced in this
|
||||
transcript (it needs a broker that accepts the connection and then stops responding
|
||||
mid-request, not one that was never there to begin with).
|
||||
@@ -0,0 +1,20 @@
|
||||
$ mvn -pl outbox -am test -Dtest=RegistrySafetyNetTest
|
||||
(real embedded Kafka broker via @EmbeddedKafka; BillingManagement.issueInvoice()
|
||||
publishes InvoiceIssued, @Externalized("invoices::#{#this.invoiceId}") routes it
|
||||
through spring-modulith-events-kafka)
|
||||
|
||||
Kafka record received -- topic=invoices key=1 value={"invoiceId":1,"sku":"GADGET-7","amountCents":4500}
|
||||
EVENT_PUBLICATION_ARCHIVE rows for InvoiceIssued: 2
|
||||
EVENT_PUBLICATION rows still outstanding for InvoiceIssued: 0
|
||||
|
||||
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
|
||||
|
||||
The message really arrived on the real topic, with the real routing key (the invoice's
|
||||
own id, exactly as the SpEL expression after "::" in @Externalized specifies) and the
|
||||
real JSON payload. Two archive rows for InvoiceIssued, not one, because this module's
|
||||
test sources also declare FlakyAuditListener (see output/02) -- it is a second,
|
||||
independent listener on the same event, and with completion-mode=ARCHIVE configured in
|
||||
application.properties, both listeners' completed publications moved out of
|
||||
EVENT_PUBLICATION and into EVENT_PUBLICATION_ARCHIVE once they succeeded. Zero rows
|
||||
left outstanding in EVENT_PUBLICATION for this event type confirms neither listener is
|
||||
still pending.
|
||||
@@ -0,0 +1,23 @@
|
||||
$ mvn -pl outbox -am test -Dtest=ReplayIncompletePublicationsTest
|
||||
(FlakyAuditListener throws on its first invocation of InvoiceIssued, succeeds on every
|
||||
one after -- standing in for any downstream dependency that is briefly unavailable)
|
||||
|
||||
Incomplete FlakyAuditListener publications after the simulated failure: 1
|
||||
Incomplete FlakyAuditListener publications after resubmission: 0
|
||||
Invoice 1 never changed -- the retry was purely about redelivering the event, the invoice row was correct the whole time.
|
||||
|
||||
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
|
||||
|
||||
The sequence that actually ran: BillingManagement.issueInvoice() commits the Invoice
|
||||
row and the EVENT_PUBLICATION row for FlakyAuditListener in one local transaction.
|
||||
FlakyAuditListener.on(InvoiceIssued) is then invoked asynchronously, throws on purpose,
|
||||
and the registry leaves that row with COMPLETION_DATE still null -- "incomplete" is
|
||||
exactly that: a row with no completion date, regardless of whether the listener ever
|
||||
ran at all or ran and failed. Calling
|
||||
IncompleteEventPublications.resubmitIncompletePublicationsOlderThan(Duration.ZERO) --
|
||||
a real method on a real interface, verified against the compiled events-api 2.1.1 jar,
|
||||
not assumed from documentation prose -- re-invokes the listener for every such row.
|
||||
This time failNextAttempt is already false, so it succeeds, and the row's completion
|
||||
date is set. The invoice itself was never touched by any of this: the retry is about
|
||||
redelivering a notification, not about redoing a database write that already
|
||||
succeeded the first time.
|
||||
Reference in New Issue
Block a user