spring-async-demo/virtual-threads-benchmark-webflux/docs/01-webflux-benchmark-methodology.md

1. WebFlux benchmark methodology and results

README | Companion module: ../virtual-threads-benchmark (platform threads vs virtual threads)

Source: ReactiveDemoController.java, CpuWork.java. Test: WebfluxLoadBenchmarkTest.java. Transcripts: docs/output/01-io-bound-webflux.txt, docs/output/02-cpu-bound-webflux.txt, docs/output/03-event-loop-starvation.txt, docs/output/04-stream-backpressure.txt.

Why this module exists, and why it's separate from ../virtual-threads-benchmark

The companion post for this module is the three-way comparison: platform threads vs virtual threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real, re-run benchmark in ../virtual-threads-benchmark, built for a different ankurm.com post and reused here rather than duplicated. This module adds the missing WebFlux leg, measured with the identical client-side load generator (java.net.http.HttpClient backed by a virtual-thread executor, used only as the client) so all three legs come from the same method on the same 2 vCPU sandbox.

It is its own Maven module rather than a third profile inside ../virtual-threads-benchmark because spring-boot-starter-web (Tomcat) and spring-boot-starter-webflux (Netty) on the same classpath fight over WebApplicationType.deduceFromClasspath() -- exactly the kind of fragile setup a benchmark should not carry. A module with only spring-boot-starter-webflux needs no such workaround.

A methodology correction made while building this module: single trials are not reliable here

While measuring the I/O-bound endpoint, identical back-to-back single trials at 600 concurrency swung from 795ms to 1247ms wall time on this shared sandbox -- a spread larger than the actual gap this benchmark exists to measure. A single HTTP load-test run on a noisy, shared 2 vCPU box is not precise enough to support a "X% faster" claim between two models that are actually close; it only looks precise because it produces one number.

The fix applied to the I/O-bound scenario and the event-loop-starvation scenario below: run the load multiple independent times and report the median across trials, not a single shot. This is a real methodology change, not cosmetic -- it changed which model's number looked better in earlier drafts of this module before the fix was applied.

This has one important consequence for how to read the numbers below. The platform-thread and virtual-thread numbers reused from ../virtual-threads-benchmark are single trials, measured for a different post before this variance was discovered there. The WebFlux numbers in this module are medians of 5 (I/O) or 3 (event-loop starvation) trials. Comparing a denoised median against a single trial is not perfectly apples-to-apples: a precise percentage gap between WebFlux and virtual threads should be read as directional, not as a figure you could reproduce to the point. A gap wide enough to swamp the observed ~50% single-trial swing -- which is the case for every comparison against platform threads in this post, and turned out to be the case for WebFlux vs. virtual threads too once de-noised -- is the part worth trusting.

I/O-bound: /io, Mono.delay(300ms), concurrency 600, median of 5 trials

webflux (netty) : total=600 success=600 wall=569ms p50=455ms p99=510ms

Mono.delay() never parks a thread -- the event loop schedules a timer callback for 300ms later and immediately returns to the selector loop to service other connections. Against the single-trial platform/virtual-thread numbers in ../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt (platform wall=1325ms p50=733ms p99=1201ms, virtual wall=1042ms p50=658ms p99=720ms), WebFlux's median-of-5 number is faster on all three metrics: roughly 55-58% faster than platform threads and 29-45% faster than virtual threads, depending on the metric. Platform threads being clearly worse is a robust finding -- even the noisiest single WebFlux trial observed while building this module (1247ms) still beats platform threads' 1325ms. The WebFlux-vs-virtual-threads gap specifically should be read with the single-trial-vs-median caveat above in mind, and its exact size moved by roughly 5-10% between the last two verification runs of this same median-of-5 measurement -- both models avoid Tomcat's thread-pool queueing entirely and are dramatically faster than platform threads for this workload, which is the reproducible part of this result.

CPU-bound: /cpu (naive) vs /cpu-offloaded, concurrency 60

This is the section worth reading slowly, because the first, most intuitive prediction -- "blocking the event loop must be dramatically worse" -- is not what the isolated benchmark shows on this hardware, and the reason why is a real, checkable fact about this JVM rather than noise.

LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2

webflux naive (Mono.fromCallable, no subscribeOn)      : total=60 success=60 wall=302ms p50=145ms p99=288ms
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=271ms p50=142ms p99=261ms

/cpu wraps the identical 20,000x SHA-256 loop from ../virtual-threads-benchmark's DemoController in Mono.fromCallable() with no subscribeOn(...) -- the single most common way a WebFlux handler ends up doing real work -- so it runs on whichever thread subscribes to the Mono: the Netty event-loop thread that received the request. /cpu-offloaded moves the identical work to Schedulers.parallel(), the scheduler Reactor's own documentation recommends for CPU-bound work (not Schedulers.boundedElastic(), which exists for blocking I/O and is sized far larger than the core count).

Reactor Netty's default event-loop pool (LoopResources.DEFAULT_IO_WORKER_COUNT, confirmed by printing the constant directly rather than reading it off documentation) is max(availableProcessors(), 4) -- 4 threads on this 2-core box. Schedulers.parallel()'s default pool is sized to availableProcessors() -- 2 threads on the same box, confirmed the same way. The endpoint that "incorrectly" runs on the event loop has more worker threads available to it, at this concurrency, than the "correctly offloaded" one. That is why offloaded is only roughly 2-10% faster across these three metrics rather than showing a dramatic gap -- not because the advice to offload CPU work is wrong.

Offloaded is faster here on every metric, consistent with the advice, but the margin alone understates why the naive version is still wrong. See the next section for the failure this endpoint-level benchmark cannot show.

What the isolated CPU benchmark hides: event-loop starvation of other traffic

/io alone (baseline, no concurrent CPU load)         : total=20 success=20 wall=347ms p50=332ms p99=346ms
/io while 150x /cpu (naive) run concurrently         : total=20 success=20 wall=995ms p50=742ms p99=950ms
/io while 150x /cpu-offloaded run concurrently        : total=20 success=20 wall=731ms p50=693ms p99=697ms

(median of 3 trials each; see the methodology correction above for why)

This is the real cost of the naive endpoint, and benchmarking /cpu by itself cannot show it: /io and /cpu share the same small event-loop pool. Firing concurrent /cpu requests and, while they are still in flight, firing 20 unrelated /io requests at the same server measures what happens to traffic that has nothing to do with the CPU-bound endpoint.

Getting a reproducible number here took two fixes, not one. This test's first cut used only 8 concurrent /cpu requests and produced an inconsistent, sometimes-inverted result across repeated runs: 8 requests drain through 4 event-loop threads in two short rounds, finishing well before the /io measurement window was over. Raising the load to 60 concurrent requests (matching the CPU-bound benchmark's own concurrency) fixed the under-loading problem but still left a margin thin enough to flake once during verification (372ms vs 374ms p99 -- a real tie, not a real result). The final fix was 150 concurrent /cpu requests, median of 3 trials -- both changes, not a threshold that happened to pass once.

A second, more interesting finding came out of raising the load this high: at 150 concurrent CPU-bound requests -- far beyond the 2 physical cores available -- even the offloaded case now degrades /io noticeably (baseline p99 346ms vs. offloaded-load p99 697ms, roughly 2x). Offloading moves the CPU work off the event loop's 4 threads onto Schedulers.parallel()'s 2 threads, which stops it from directly starving /io's access to the event loop -- but it does not stop it from saturating the 2 physical cores those event-loop threads still need CPU time on. Naive is still clearly worse than offloaded (roughly 7-36% worse across wall/p50/p99, varying by metric and by trial), which is the mechanism this test is built to demonstrate; it is just not a free pass to "offloaded means no impact at all" once concurrent CPU load exceeds what the box can actually run at once. At higher production concurrency, both the direct mechanism (naive holding event-loop threads) and this indirect one (any CPU-bound load competing for the same physical cores) matter.

• If you're auditing your own WebFlux service: grep for Mono.fromCallable, Flux.fromIterable wrapping computation, or any synchronous call inside a reactive chain with no subscribeOn(...) after it -- each one runs on the event loop by default.
• Schedulers.parallel() for CPU-bound work, Schedulers.boundedElastic() for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the wrong one either wastes threads or defeats the point of offloading.
• Reactor Netty event-loop sizing: projectreactor.io/docs/netty

Backpressure is structural, not a benchmark number

/stream (ReactiveDemoController.streamResults()) and StreamBackpressureTest exist to check a claim this kind of post usually just asserts: that a Flux never emits faster than its subscriber requests. The test drives the endpoint's Flux with StepVerifier, requesting 3 items, then 2 more, then the remaining 45, and asserts no item ever arrives ahead of a pending request:

StepVerifier.create(controller.streamResults(), 3)
  .expectNext("event-0", "event-1", "event-2")
  .expectNoEvent(Duration.ofMillis(80))   -- no 4th item arrives without a request
  .thenRequest(2).expectNext("event-3", "event-4")
  .thenRequest(45).expectNextCount(45)
  .expectComplete()

(docs/output/04-stream-backpressure.txt) The endpoint is built on Flux.range() rather than the more "realistic-looking" Flux.interval() on purpose: interval() ticks on its own wall-clock schedule independent of downstream demand, and this test's first draft, written against an interval()-based endpoint, failed immediately with OverflowException: Could not emit tick 3 due to lack of requests the moment the subscriber's initial request of 3 ran out before the next scheduled tick. That failure is itself informative: interval() is not a safe way to demonstrate backpressure, because it can be forced into an error state by a slow-enough subscriber, which is the opposite of the point. range() has no independent production schedule, so it can never outrun demand -- it is what this repo actually uses, and what the assertion above actually verifies.

Back to README.