spring-async-demo/virtual-threads-benchmark-webflux/docs/01-webflux-benchmark-methodology.md

1. WebFlux benchmark methodology and results

README | Companion module: ../virtual-threads-benchmark (platform threads vs virtual threads)

Source: ReactiveDemoController.java, CpuWork.java. Test: WebfluxLoadBenchmarkTest.java. Transcripts: docs/output/01-io-bound-webflux.txt, docs/output/02-cpu-bound-webflux.txt, docs/output/03-event-loop-starvation.txt, docs/output/04-stream-backpressure.txt.

Why this module exists, and why it's separate from ../virtual-threads-benchmark

The companion post for this module is the three-way comparison: platform threads vs virtual threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real, re-run benchmark in ../virtual-threads-benchmark, built for a different ankurm.com post and reused here rather than duplicated. This module adds the missing WebFlux leg, measured with the identical client-side load generator (java.net.http.HttpClient backed by a virtual-thread executor, used only as the client) so all three legs come from the same method on the same 2 vCPU sandbox.

It is its own Maven module rather than a third profile inside ../virtual-threads-benchmark because spring-boot-starter-web (Tomcat) and spring-boot-starter-webflux (Netty) on the same classpath fight over WebApplicationType.deduceFromClasspath() -- exactly the kind of fragile setup a benchmark should not carry. A module with only spring-boot-starter-webflux needs no such workaround.

I/O-bound: /io, Mono.delay(300ms), concurrency 600

Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread /io benchmark:

webflux (netty) : total=600 success=600 wall=1023ms p50=639ms p99=835ms

Mono.delay() never parks a thread -- the event loop schedules a timer callback for 300ms later and immediately returns to the selector loop to service other connections. Wall time for 600 concurrent requests stays close to a single 300ms round, the same shape virtual threads show in ../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt (wall=1042ms p50=658ms p99=720ms on that run) -- both models avoid the platform-thread pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a worker while it waits.

CPU-bound: /cpu (naive) vs /cpu-offloaded, concurrency 60

This is the section worth reading slowly, because the first, most intuitive prediction -- "blocking the event loop must be dramatically worse" -- is not what the isolated benchmark shows on this hardware, and the reason why is a real, checkable fact about this JVM rather than noise.

LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2

webflux naive (Mono.fromCallable, no subscribeOn)      : total=60 success=60 wall=403ms p50=243ms p99=378ms
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=298ms p50=167ms p99=282ms

/cpu wraps the identical 20,000x SHA-256 loop from ../virtual-threads-benchmark's DemoController in Mono.fromCallable() with no subscribeOn(...) -- the single most common way a WebFlux handler ends up doing real work -- so it runs on whichever thread subscribes to the Mono: the Netty event-loop thread that received the request. /cpu-offloaded moves the identical work to Schedulers.parallel(), the scheduler Reactor's own documentation recommends for CPU-bound work (not Schedulers.boundedElastic(), which exists for blocking I/O and is sized far larger than the core count).

Reactor Netty's default event-loop pool (LoopResources.DEFAULT_IO_WORKER_COUNT, confirmed by printing the constant directly rather than reading it off documentation) is max(availableProcessors(), 4) -- 4 threads on this 2-core box. Schedulers.parallel()'s default pool is sized to availableProcessors() -- 2 threads on the same box, confirmed the same way. The endpoint that "incorrectly" runs on the event loop has more worker threads available to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times above are close rather than showing a dramatic gap -- not because the advice to offload CPU work is wrong.

Offloaded is faster here, consistent with the advice, but the margin alone understates why the naive version is still wrong. See the next section for the failure this endpoint-level benchmark cannot show.

What the isolated CPU benchmark hides: event-loop starvation of other traffic

/io alone (baseline, no concurrent CPU load)         : total=20 success=20 wall=340ms p50=322ms p99=331ms
/io while 60x /cpu (naive) run concurrently          : total=20 success=20 wall=805ms p50=451ms p99=760ms
/io while 60x /cpu-offloaded run concurrently        : total=20 success=20 wall=405ms p50=389ms p99=398ms

This is the real cost of the naive endpoint, and benchmarking /cpu by itself cannot show it: /io and /cpu share the same small event-loop pool. Firing concurrent naive /cpu requests and, while they are still in flight, firing 20 unrelated /io requests at the same server measures what happens to traffic that has nothing to do with the CPU-bound endpoint.

This test's first cut used only 8 concurrent /cpu requests and produced an inconsistent, sometimes-inverted result across repeated runs -- 8 requests drain through 4 event-loop threads in two short rounds, finishing well before the /io measurement window (~300-400ms) was over, so most of that window ran with no concurrent CPU load at all. The fix was not to loosen the assertion but to fix the load: 60 concurrent /cpu requests (matching the concurrency used in the CPU-bound benchmark above) keep all 4 event-loop threads occupied for roughly the full measurement window. With that fixed, the result above is large and reproducible across repeated runs: naive /io p99 nearly triples the undisturbed baseline, while offloaded /io stays close to it. One CPU-heavy endpoint left un-offloaded is enough to degrade every other endpoint sharing its event loop -- invisible to anyone who benchmarked the CPU endpoint in isolation and called it "fine."

• If you're auditing your own WebFlux service: grep for Mono.fromCallable, Flux.fromIterable wrapping computation, or any synchronous call inside a reactive chain with no subscribeOn(...) after it -- each one runs on the event loop by default.
• Schedulers.parallel() for CPU-bound work, Schedulers.boundedElastic() for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the wrong one either wastes threads or defeats the point of offloading.
• Reactor Netty event-loop sizing: projectreactor.io/docs/netty

Backpressure is structural, not a benchmark number

/stream (ReactiveDemoController.streamResults()) and StreamBackpressureTest exist to check a claim this kind of post usually just asserts: that a Flux never emits faster than its subscriber requests. The test drives the endpoint's Flux with StepVerifier, requesting 3 items, then 2 more, then the remaining 45, and asserts no item ever arrives ahead of a pending request:

StepVerifier.create(controller.streamResults(), 3)
  .expectNext("event-0", "event-1", "event-2")
  .expectNoEvent(Duration.ofMillis(80))   -- no 4th item arrives without a request
  .thenRequest(2).expectNext("event-3", "event-4")
  .thenRequest(45).expectNextCount(45)
  .expectComplete()

(docs/output/04-stream-backpressure.txt) The endpoint is built on Flux.range() rather than the more "realistic-looking" Flux.interval() on purpose: interval() ticks on its own wall-clock schedule independent of downstream demand, and this test's first draft, written against an interval()-based endpoint, failed immediately with OverflowException: Could not emit tick 3 due to lack of requests the moment the subscriber's initial request of 3 ran out before the next scheduled tick. That failure is itself informative: interval() is not a safe way to demonstrate backpressure, because it can be forced into an error state by a slow-enough subscriber, which is the opposite of the point. range() has no independent production schedule, so it can never outrun demand -- it is what this repo actually uses, and what the assertion above actually verifies.

Back to README.