Skip to main content
Java

Testing Asynchronous Code with Awaitility (@Async, Kafka Listeners, Schedulers)

Thread.sleep in a test is a guess about timing, not a wait. This post replaces it with Awaitility’s await() across a void @Async method, a real in-process @EmbeddedKafka listener, and a @Scheduled job — plus the pom.xml trap where Spring Boot 4.1.1 split Kafka’s autoconfiguration into its own module, leaving @KafkaListener silently unwired.

Ankur 18 min read

If you’ve ever written Thread.sleep(500) in a test, then watched that same test fail on a slower CI runner and pass on your laptop, you’ve already found the problem this post is about. The fix is not a bigger number. It’s a different kind of wait.

Versions used in this post, verified against the real jars and POMs on Maven Central, not release notes: JDK 25 (Temurin 25.0.4.1+1), Spring Boot 4.1.1, Spring Kafka 4.1.1, Awaitility 4.3.0 (released 2025‑02‑21, and already on your test classpath today if you use spring-boot-starter-test — more on that below).

Everything below is real, runnable code in a new awaitility module alongside this blog’s existing @Async and @Scheduled companion modules. Every number quoted here came out of an actual test run, including the two real failures this post shows on purpose.

Why Thread.sleep in a test is a guess, not a wait

A synchronous test asserts the instant the method it called returns. An asynchronous one can’t do that, because the work it’s checking hasn’t necessarily finished when the call that kicked it off does — a void @Async method, a message a @KafkaListener hasn’t consumed yet, a @Scheduled job that runs on its own clock. Thread.sleep(n) is the obvious patch: wait a while, then check. The number n is a guess about how long the real work takes, made once, on whichever machine happened to be running the test when it was written.

That guess has exactly two ways to be wrong, and both are common. Too short, and the test fails on a slower machine — a loaded CI runner, a container with a throttled CPU share — even though the code is correct. Too long, and the test passes every time but the suite gets slower with every one of these you add, for no benefit. There is no number that is simultaneously fast and safe, because the real duration isn’t a constant the test can know in advance.

Here’s that first failure mode, for real. This project has a void @Async method that takes 220ms to do its work; a version of its test guessed 100ms was enough:

$ mvn -B -Dtest=SleepGuessesWrongTest test
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.568 s <<< FAILURE! -- in com.ankurm.awaitility.SleepGuessesWrongTest
com.ankurm.awaitility.SleepGuessesWrongTest.confirmationArrivesWithin100ms -- Time elapsed: 1.084 s <<< FAILURE!
java.lang.AssertionError:

Expecting actual not to be null
	at com.ankurm.awaitility.SleepGuessesWrongTest.confirmationArrivesWithin100ms(SleepGuessesWrongTest.java:27)

Full transcript in 01-sleep-guesses-wrong.txt. Note this isn’t the flaky, one-in-twenty kind of failure — it fails every single run, because 100 is simply less than 220. “Flaky” implies intermittent; a wrong guess about timing is often just wrong, consistently.

Thread.sleep(100) checks once, too early sleep(100) check at 100ms: not ready yet, FAIL real work finishes at 220ms await().atMost(2s) checks repeatedly, finds it as soon as it’s ready polls every 100ms, starting at a 100ms poll delay poll at ~220ms: ready, PASS

Two different ways of finding out “is it ready yet”, drawn to the same timeline. The top one asks exactly once, at a time chosen before the test ever ran. The bottom one keeps asking, on a short fixed interval, until the answer is yes or a much longer budget runs out. The fix below is the bottom one.

The smallest fix: await().untilAsserted() against a void @Async method

The method under test here is deliberately the hard case for blocking: it’s @Async and returns void, so there is no Future to call .get() on. That’s not a contrived example — a fire-and-forget confirmation, log line, or cache invalidation is exactly this shape, and it’s exactly the shape that pushes people toward Thread.sleep because there is nothing else to wait on.

@Async
public void sendConfirmation(String orderId) {
    sleepFor(220);
    mailbox.record(orderId, "order " + orderId + " confirmed");
}

Source: AsyncGreetingService.java. Mailbox is just a thread-safe map standing in for whatever side effect you’d otherwise have to wait on. The fix is three lines, and the method it replaces was the same three lines with Thread.sleep(100) swapped in for the middle one:

await().atMost(Duration.ofSeconds(2))
        .untilAsserted(() -> assertThat(mailbox.confirmationFor("ORD-CAPTURE")).isNotNull());
mailbox.confirmationFor("ORD-CAPTURE") immediately before sendConfirmation(): null
await().atMost(2s).untilAsserted(...) returned after approximately:           307ms
mailbox.confirmationFor("ORD-CAPTURE") once await() returns:                   order ORD-CAPTURE confirmed

Source: AwaitAsyncConfirmationTest.java; output from 02-await-finds-it.txt. 307ms, not 220 — the extra ~90ms is the Spring context and the test method itself, not something to chase. The point is that the test took as long as the real work plus one poll cycle, not a fixed guess made in advance.

You probably already have this dependency. There is no <dependency> block for Awaitility anywhere in this module’s pom.xml. spring-boot-starter-test:4.1.1‘s own POM on Maven Central declares org.awaitility:awaitility:4.3.0 as a compile-scope dependency of its own. If your project uses that starter, await() is on your test classpath right now, whether or not you’ve ever imported it.

What await() actually does while it waits, with no configuration at all

Neither call above set a poll interval. It’s worth knowing exactly what runs when you don’t, because “it polls periodically” is true but not precise enough to reason about a slow condition. Reading Awaitility.java and AtMostWaitConstraint.java in the 4.3.0 sources jar rather than the user guide: the default timeout is a fixed AtMostWaitConstraint.TEN_SECONDS, and the default poll interval is a fixed 100 milliseconds — with the first poll delayed by that same 100ms, not fired immediately.

await() with zero overrides 100ms 200ms 300ms … one poll every 100ms … 10.000s ConditionTimeoutException default timeout = AtMostWaitConstraint.TEN_SECONDS, default poll interval = 100ms, first poll delayed by the same 100ms — all read from Awaitility.java, not the docs.

Two tests prove these numbers against the real jar instead of quoting them. The first lets a condition that’s always false run to completion with no atMost(...):

elapsed before ConditionTimeoutException: 10057ms (expected: ~10,000ms)

Output from 03-default-timeout-is-ten-seconds.txt. The second records the wall-clock gap between consecutive polls:

poll timestamps (ms since start): [107, 207, 308, 409, 509]
gaps between consecutive polls (ms): [100, 101, 101, 100]
average gap: 100.5ms (expected: ~100ms)

Source: DefaultTimingTest.java; output from 04-default-poll-interval-is-100ms.txt. Both tests tolerate a window rather than an exact number — 10,070ms and not 10,000ms exactly is ordinary scheduling jitter, not a bug.

Ten seconds is a generous default for a unit test. Most of the calls in this post override it down to one or two seconds with atMost(...), both to keep the suite fast and because a passing assertion should not need ten seconds of slack to begin with; if it does, that’s a sign the thing you’re waiting on, not the test, has a problem.

  • Overriding the interval for a condition you know is slow: ConditionFactory.pollInterval(Duration), same sources jar.
  • Waiting the other direction — proving something does not happen within a window — is ConditionFactory.during(...), not covered in this post.

The one assumption await() makes silently: your condition might throw

Every example so far polled a condition that returns a value or passes an assertion. A condition that throws while the thing it’s checking is still warming up is a different case, and Awaitility does not treat it the way you might expect by default. Reading Awaitility.java again: defaultExceptionIgnorer is a predicate that returns false for every exception — meaning nothing is ignored, and an exception from the very first poll propagates immediately, without waiting out any part of the atMost(...) window.

This project’s FlakyStartupResource stands in for exactly that: it throws IllegalStateException for its first 300ms, then returns normally. A test that awaits its value with no further configuration fails on the first poll, not after a timeout:

$ mvn -B -Dtest=ExceptionPropagatesImmediatelyTest test
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.940 s <<< FAILURE! -- in com.ankurm.awaitility.ExceptionPropagatesImmediatelyTest
com.ankurm.awaitility.ExceptionPropagatesImmediatelyTest.failsOnTheFirstPollInsteadOfWaiting -- Time elapsed: 0.946 s <<< ERROR!
java.lang.IllegalStateException: resource is still warming up
	at com.ankurm.awaitility.FlakyStartupResource.value(FlakyStartupResource.java:26)
	at com.ankurm.awaitility.ExceptionPropagatesImmediatelyTest.lambda$failsOnTheFirstPollInsteadOfWaiting$0(ExceptionPropagatesImmediatelyTest.java:28)

Full transcript in 08-exception-propagates-immediately.txt. “Errors: 1” rather than a ConditionTimeoutException after the full window is the tell: this failed in under a second against a one-second budget, because the exception was never retried in the first place. The fix is one call, telling Awaitility which exception means “not ready yet” rather than “broken”:

await().atMost(Duration.ofSeconds(1))
        .ignoreExceptionsInstanceOf(IllegalStateException.class)
        .untilAsserted(() -> assertThat(resource.value()).isEqualTo("ready"));
approximate elapsed time before resource.value() finally returned "ready": 302ms (expected: a little over 300ms)

Source: IgnoreExceptionsTest.java; output from 09-ignore-exceptions-waits-it-out.txt.

condition throws IllegalStateException default: propagate now fails before atMost elapses + ignoreExceptionsInstanceOf(…) treated as “still false”, keep polling until value() returns, or atMost runs out
Scope the exception type as narrowly as you mean to. ignoreExceptionsInstanceOf(IllegalStateException.class) ignores that type and its subclasses; ignoreException(Class) ignores only that exact class; ignoreExceptions() ignores everything, which also swallows a genuine bug in the condition itself (a NullPointerException from a typo, say) until the timeout quietly reports “condition never became true” instead of the real stack trace.
  • The predicate- and Hamcrest-matcher based variants, for exceptions that need more than a type check: ConditionFactory.ignoreExceptionsMatching(...) in the 4.3.0 sources.
  • The opposite control, for a test suite that globally ignores exceptions and needs one call to not: ConditionFactory.ignoreNoExceptions().

A background thread you didn’t know was there

Every await() call so far ran its polling on a thread Awaitility creates for the purpose, not on the test’s own main thread — that’s in ConditionAwaiter.java, which pulls an ExecutorService from the condition settings and submits each poll to it. One consequence of that is process-wide, not per-thread, and worth knowing before it surprises you somewhere else: with catchUncaughtExceptions at its default of true, the constructor of ConditionAwaiter calls Thread.setDefaultUncaughtExceptionHandler(this) — for the entire JVM, not just its own polling thread — for as long as that one await() call is in flight, then restores whatever handler was there before.

This test proves it rather than just quoting the source:

Thread.UncaughtExceptionHandler original = (t, e) -> { };
Thread.setDefaultUncaughtExceptionHandler(original);

AtomicReference<Thread.UncaughtExceptionHandler> handlerDuringAwait = new AtomicReference<>();

await().atMost(Duration.ofMillis(300)).until(() -> {
    handlerDuringAwait.set(Thread.getDefaultUncaughtExceptionHandler());
    return true;
});

Thread.UncaughtExceptionHandler handlerAfterAwait = Thread.getDefaultUncaughtExceptionHandler();
handler during await() is the one this test installed:  false (expected: false)
handler during await() is Awaitility's own (class org.awaitility.core.CallableCondition$1)
handler after await() is the one this test installed:   true (expected: true)

Source: UncaughtExceptionHandlerSwapTest.java; output from 07-uncaught-exception-handler-swap.txt. The handler installed before the call is not the one visible from inside the polled condition, and the original one is back once the call returns.

before await() handler = the test’s own during await() handler = Awaitility’s own after await() returns handler = the test’s own, restored A process-wide setting, swapped and restored around one call — not a per-thread one.
Why this matters in practice: if your own code installs a custom default uncaught-exception handler — to ship crash reports somewhere, say — and a background thread throws while a test elsewhere in the same JVM happens to be inside an await() call, that exception is briefly handled by Awaitility’s handler instead of yours. It restores correctly afterward, but “briefly” is a real window, not a theoretical one, in a test suite that runs classes in parallel.
  • ConditionFactory.dontCatchUncaughtExceptions() opts a single await() call out of this, if your test is sensitive to it.
  • The source this section is read from: ConditionAwaiter.java and OriginalDefaultUncaughtExceptionHandler.java in the Awaitility 4.3.0 sources jar on Maven Central.

Testing a @Scheduled job without sleeping for a guessed multiple of its rate

A @Scheduled job starts running the moment the application context comes up, on its own thread, on its own clock — not when a test method happens to start. “Sleep for three times the fixed rate, then assert three runs happened” has the same problem as every Thread.sleep in this post, plus one more: the job may already have been running for an unknown amount of time before the test method’s first line executes, especially if Spring’s test context cache reused a context across several test classes.

@Scheduled(fixedRate = 150)
public void run() {
    runs.incrementAndGet();
}

Source: ReportJob.java. The test records a baseline when it starts, then waits for at least three more runs than that — a claim that’s true regardless of how long the job had already been ticking:

int baseline = reportJob.runCount();

await().atMost(Duration.ofSeconds(2))
        .untilAsserted(() -> assertThat(reportJob.runCount()).isGreaterThanOrEqualTo(baseline + 3));
runCount() when the test started: 1
target (baseline + 3):            4
runCount() once await() returned: 4
approximate elapsed time:          510ms

Source: ScheduledJobAwaitTest.java; output from 06-scheduled-job-await.txt. The job had already run once by the time this test’s first line executed — a fixed-count assertion (“runCount() == 3”) written before that context was understood would have been wrong from the start, independent of any timing issue.

Testing a @KafkaListener against a real, in-memory broker

Sending a Kafka record and consuming it are two round trips through the broker, on two different threads, with a consumer poll loop in between — there is no return value from KafkaTemplate.send(...) that tells you a listener has consumed the record, because the whole point of a message broker is that the producer doesn’t know or care who’s listening. spring-kafka-test‘s @EmbeddedKafka starts a real, in-process broker for exactly this kind of test, with no Docker and no Testcontainers — which also makes it the only honest way to demonstrate this in a sandbox with no Docker daemon available, a constraint worth being upfront about.

@KafkaListener(topics = "orders", groupId = "awaitility-demo")
public void onOrderEvent(String payload) {
    received.add(payload);
}
producer.send("orders", "order-77-created");

await().atMost(Duration.ofSeconds(5))
        .untilAsserted(() -> assertThat(listener.received()).contains("order-77-created"));
listener.received() immediately after send() returned: (not checked -- that is the point)
listener.received() once await() returned:              [order-77-created]
approximate elapsed time:                               378ms

Source: OrderEventListener.java, KafkaListenerAwaitTest.java; output from 05-kafka-listener-await.txt.

That code is the easy part. Getting to a running listener container at all was not, and the reason is worth a full section of its own because it will hit anyone moving a Kafka integration to Spring Boot 4.1 for the first time.

The pom.xml that compiles cleanly and consumes nothing

The first version of this module’s pom.xml depended directly on org.springframework.kafka:spring-kafka — the dependency every pre-4.1 Spring Kafka tutorial tells you to add, and the one that makes @KafkaListener and KafkaTemplate resolve at compile time. It compiled without a single warning. Every test that sent a record and waited for the listener to consume it then failed the same way, not with an error, but with an empty collection and a plain ConditionTimeoutException after the full wait:

[ERROR] Errors:
[ERROR]   KafkaListenerAwaitTest.messageSentIsEventuallyConsumed:49 ? ConditionTimeout Assertion condition defined as a Lambda expression in com.ankurm.awaitility.KafkaListenerAwaitTest
Expecting ConcurrentLinkedQueue:
  []
to contain:
  ["order-42-created"]
but could not find the following element(s):
  ["order-42-created"]
 within 5 seconds.
[INFO]
[ERROR] Tests run: 11, Failures: 0, Errors: 2, Skipped: 0

Full transcript in 10-missing-kafka-starter-failure.txt. No stack trace pointing at a misconfiguration, no exception at context startup — the application context came up clean every time. The only clue was negative evidence: grepping the full test-run log for any line starting with o.s.k. (Spring Kafka’s logger prefix) returned nothing at all, not even the routine “listener container started” line a working setup always logs:

$ grep -c "o\.s\.k\." full-test-run.log
0

Autowiring KafkaListenerEndpointRegistry directly in a scratch test and asking it for its containers confirmed it: the bean didn’t exist. Neither did KafkaTemplate, ConsumerFactory, or kafkaListenerContainerFactory — every bean spring-boot-autoconfigure has created automatically from a spring-kafka dependency since Spring Boot’s earliest Kafka support.

The reason is a module split that’s easy to miss if you’ve written Spring Kafka code against any Boot version before 4.1. Querying spring-boot-dependencies-4.1.1.pom on Maven Central directly turns up an artifact that does not exist in earlier Boot BOMs:

$ grep -B3 '<artifactId>spring-boot-kafka</artifactId>' spring-boot-dependencies-4.1.1.pom
      </dependency>
      <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-kafka</artifactId>

Full transcript, including the two follow-up POM queries below, in 11-pom-diagnosis.txt. spring-boot-kafka is where the Kafka autoconfiguration classes now live — extracted out of the monolithic spring-boot-autoconfigure jar into their own module, the same way this blog’s research for a different post in this batch found spring-boot-resttestclient, spring-boot-jdbc-test, spring-boot-flyway and spring-boot-liquibase split out on their own. spring-boot-autoconfigure itself no longer carries Kafka’s wiring, at all, in Boot 4.1. Depending on spring-kafka alone now gets you the annotations and the client classes with none of the Spring Boot glue that turns a @KafkaListener method into a running container — and because @ConditionalOnClass-style autoconfiguration simply doesn’t fire rather than failing loudly, nothing at context startup tells you it’s missing.

What this module’s pom.xml had org.springframework.kafka:spring-kafka @KafkaListener compiles; no autoconfiguration runs What actually wires a running container org.springframework.boot:spring-boot-starter-kafka org.springframework.boot:spring-boot-kafka org.springframework.kafka:spring-kafka spring-boot-kafka’s own POM pulls spring-kafka transitively — so the starter alone is both necessary and sufficient. Adding spring-kafka directly is the step that’s now wrong.

The fix was a one-line change in pom.xml: replace org.springframework.kafka:spring-kafka with org.springframework.boot:spring-boot-starter-kafka (compile scope, for the @KafkaListener annotation in main sources), and org.springframework.kafka:spring-kafka-test with org.springframework.boot:spring-boot-starter-kafka-test (test scope, for @EmbeddedKafka). Querying each starter’s real POM on Maven Central confirms the chain: spring-boot-starter-kafka depends on spring-boot-starter and spring-boot-kafka; spring-boot-kafka in turn depends on spring-kafka itself. The starter is both necessary (for the autoconfiguration) and sufficient (it still brings spring-kafka along), so it fully replaces the old direct dependency rather than adding to it.

The fingerprint of this specific trap: a Spring Boot 4.1+ application with a @KafkaListener bean that the context creates without error, a KafkaTemplate.send(...) that succeeds, and a consumer that never receives anything — with nothing in the startup log mentioning Kafka’s listener containers at all. If your dependency list has org.springframework.kafka:spring-kafka rather than org.springframework.boot:spring-boot-starter-kafka, that’s almost certainly it.
  • The other Boot 4 module splits this same research pattern already turned up: spring-boot-resttestclient vs the old TestRestTemplate autoconfiguration, covered in this repository’s rest-test-client module.
  • spring-boot-starter-kafka and spring-boot-kafka‘s real POMs, queried directly: https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-starter-kafka/4.1.1/ and the sibling spring-boot-kafka path.

Should you reach for Awaitility, or fix the design instead?

Awaitility makes tests for asynchronous code correct and fast at the same time, which Thread.sleep cannot. It does not make the underlying asynchrony easier to reason about, and it is not a substitute for a design that makes the eventual result observable at all. If there is genuinely no way to find out, from outside the method, whether the work completed — no row, no counter, no returned value, nothing — the honest fix is to add one of those, even a package-private one used only by tests, rather than polling for a side effect that happens to be visible today but isn’t part of the method’s actual contract. Every example in this post polls something the production code already needed for its own reasons: a stored confirmation, a consumed-message queue, a run counter.

It’s also not a fix for a test that’s slow because the work under test is genuinely slow. await() with a ten-second default timeout will quite happily make a test take ten real seconds before telling you it failed; a condition that’s merely slow to check (an expensive query run every 100ms, say) can make a passing test slow too. The fix in that case is a bigger pollInterval, not Awaitility at all.

One more thing worth saying plainly, found while researching this post rather than built into it: this blog’s own guide to flaky JUnit 6 tests already recommends Awaitility for exactly the Thread.sleep problem this post opens with — correctly — but its code sample pins <version>4.2.1</version>, one minor release behind the 4.3.0 that’s actually current and the one Spring Boot 4.1.1 manages. Noted here rather than silently fixed there, per this blog’s own policy on incidental findings.

Further reading

No Comments yet!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.