Add virtual-threads-benchmark-webflux: the WebFlux leg of the three-way benchmark

Fixes two bugs found during self-correction before publishing: - /stream used Flux.interval(), which ticks on its own wall-clock schedule independent of downstream demand and threw OverflowException under a slow subscriber; switched to Flux.range(), which has no independent production schedule and can never outrun demand. - The event-loop-starvation test's first cut used only 8 concurrent /cpu requests as background load, which drained through the 4 event-loop threads well inside the /io measurement window and produced an inconsistent, sometimes-inverted result across runs; raised to 60 concurrent requests (matching the CPU-bound benchmark's own concurrency) so the load actually occupies the event loop for the full window. Also adds StreamBackpressureTest, a StepVerifier proof that the /stream endpoint never emits ahead of its subscriber's outstanding requests.
2026-09-19 08:48:58 +00:00
parent f506b01389
commit b331db6638
19 changed files with 863 additions and 0 deletions
@@ -0,0 +1,145 @@
+# 1. WebFlux benchmark methodology and results
+
+[README](../README.md) | Companion module: [`../virtual-threads-benchmark`](../virtual-threads-benchmark/README.md) (platform threads vs virtual threads)
+
+Source: [`ReactiveDemoController.java`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java), [`CpuWork.java`](../src/main/java/com/ankurm/vthreadswebflux/CpuWork.java).
+Test: [`WebfluxLoadBenchmarkTest.java`](../src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java).
+Transcripts: [`docs/output/01-io-bound-webflux.txt`](output/01-io-bound-webflux.txt), [`docs/output/02-cpu-bound-webflux.txt`](output/02-cpu-bound-webflux.txt), [`docs/output/03-event-loop-starvation.txt`](output/03-event-loop-starvation.txt), [`docs/output/04-stream-backpressure.txt`](output/04-stream-backpressure.txt).
+
+## Why this module exists, and why it's separate from `../virtual-threads-benchmark`
+
+The companion post for this module is the three-way comparison: platform threads vs virtual
+threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real,
+re-run benchmark in [`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for
+a different ankurm.com post and reused here rather than duplicated. This module adds the
+missing WebFlux leg, measured with the identical client-side load generator
+(`java.net.http.HttpClient` backed by a virtual-thread executor, used only as the *client*) so
+all three legs come from the same method on the same 2 vCPU sandbox.
+
+It is its own Maven module rather than a third profile inside `../virtual-threads-benchmark`
+because `spring-boot-starter-web` (Tomcat) and `spring-boot-starter-webflux` (Netty) on the
+same classpath fight over `WebApplicationType.deduceFromClasspath()` -- exactly the kind of
+fragile setup a benchmark should not carry. A module with only `spring-boot-starter-webflux`
+needs no such workaround.
+
+## I/O-bound: `/io`, `Mono.delay(300ms)`, concurrency 600
+
+Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread `/io`
+benchmark:
+
+```
+webflux (netty) : total=600 success=600 wall=1023ms p50=639ms p99=835ms
+```
+
+`Mono.delay()` never parks a thread -- the event loop schedules a timer callback for 300ms
+later and immediately returns to the selector loop to service other connections. Wall time for
+600 concurrent requests stays close to a single 300ms round, the same shape virtual threads
+show in [`../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt`](../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt)
+(`wall=1042ms p50=658ms p99=720ms` on that run) -- both models avoid the platform-thread
+pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a
+worker while it waits.
+
+## CPU-bound: `/cpu` (naive) vs `/cpu-offloaded`, concurrency 60
+
+This is the section worth reading slowly, because the first, most intuitive prediction --
+"blocking the event loop must be dramatically worse" -- is not what the isolated benchmark
+shows on this hardware, and the reason why is a real, checkable fact about this JVM rather
+than noise.
+
+```
+LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
+Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
+
+webflux naive (Mono.fromCallable, no subscribeOn)      : total=60 success=60 wall=403ms p50=243ms p99=378ms
+webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=298ms p50=167ms p99=282ms
+```
+
+`/cpu` wraps the identical 20,000x SHA-256 loop from
+[`../virtual-threads-benchmark`'s `DemoController`](../virtual-threads-benchmark/src/main/java/com/ankurm/vthreads/DemoController.java)
+in `Mono.fromCallable()` with no `subscribeOn(...)` -- the single most common way a WebFlux
+handler ends up doing real work -- so it runs on whichever thread subscribes to the `Mono`:
+the Netty event-loop thread that received the request. `/cpu-offloaded` moves the identical
+work to `Schedulers.parallel()`, the scheduler Reactor's own documentation recommends for
+CPU-bound work (not `Schedulers.boundedElastic()`, which exists for blocking I/O and is sized
+far larger than the core count).
+
+<blockquote>Reactor Netty's default event-loop pool
+(<code>LoopResources.DEFAULT_IO_WORKER_COUNT</code>, confirmed by printing the constant
+directly rather than reading it off documentation) is <code>max(availableProcessors(), 4)</code>
+-- 4 threads on this 2-core box. <code>Schedulers.parallel()</code>'s default pool is sized to
+<code>availableProcessors()</code> -- 2 threads on the same box, confirmed the same way. The
+endpoint that "incorrectly" runs on the event loop has <em>more</em> worker threads available
+to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times
+above are close rather than showing a dramatic gap -- not because the advice to offload
+CPU work is wrong.</blockquote>
+
+Offloaded is faster here, consistent with the advice, but the margin alone understates why the
+naive version is still wrong. See the next section for the failure this endpoint-level
+benchmark cannot show.
+
+## What the isolated CPU benchmark hides: event-loop starvation of *other* traffic
+
+```
+/io alone (baseline, no concurrent CPU load)         : total=20 success=20 wall=340ms p50=322ms p99=331ms
+/io while 60x /cpu (naive) run concurrently          : total=20 success=20 wall=805ms p50=451ms p99=760ms
+/io while 60x /cpu-offloaded run concurrently        : total=20 success=20 wall=405ms p50=389ms p99=398ms
+```
+
+This is the real cost of the naive endpoint, and benchmarking `/cpu` by itself cannot show it:
+`/io` and `/cpu` share the same small event-loop pool. Firing concurrent naive `/cpu` requests
+and, while they are still in flight, firing 20 unrelated `/io` requests at the same server
+measures what happens to traffic that has nothing to do with the CPU-bound endpoint.
+
+**This test's first cut used only 8 concurrent `/cpu` requests and produced an inconsistent,
+sometimes-inverted result across repeated runs** -- 8 requests drain through 4 event-loop
+threads in two short rounds, finishing well before the `/io` measurement window (~300-400ms)
+was over, so most of that window ran with no concurrent CPU load at all. The fix was not to
+loosen the assertion but to fix the load: **60 concurrent `/cpu` requests** (matching the
+concurrency used in the CPU-bound benchmark above) keep all 4 event-loop threads occupied for
+roughly the full measurement window. With that fixed, the result above is large and
+reproducible across repeated runs: naive `/io` p99 nearly triples the undisturbed baseline,
+while offloaded `/io` stays close to it. One CPU-heavy endpoint left un-offloaded is enough to
+degrade every other endpoint sharing its event loop -- invisible to anyone who benchmarked the
+CPU endpoint in isolation and called it "fine."
+
+<ul>
+<li>If you're auditing your own WebFlux service: grep for <code>Mono.fromCallable</code>,
+<code>Flux.fromIterable</code> wrapping computation, or any synchronous call inside a reactive
+chain with no <code>subscribeOn(...)</code> after it -- each one runs on the event loop by
+default.</li>
+<li><code>Schedulers.parallel()</code> for CPU-bound work, <code>Schedulers.boundedElastic()</code>
+for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the
+wrong one either wastes threads or defeats the point of offloading.</li>
+<li>Reactor Netty event-loop sizing: <a href="https://projectreactor.io/docs/netty/release/reference/index.html#_event_loop_workers" rel="nofollow">projectreactor.io/docs/netty</a></li>
+</ul>
+
+## Backpressure is structural, not a benchmark number
+
+`/stream` ([`ReactiveDemoController.streamResults()`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java))
+and [`StreamBackpressureTest`](../src/test/java/com/ankurm/vthreadswebflux/StreamBackpressureTest.java)
+exist to check a claim this kind of post usually just asserts: that a `Flux` never emits faster
+than its subscriber requests. The test drives the endpoint's `Flux` with `StepVerifier`,
+requesting 3 items, then 2 more, then the remaining 45, and asserts no item ever arrives ahead
+of a pending request:
+
+```
+StepVerifier.create(controller.streamResults(), 3)
+  .expectNext("event-0", "event-1", "event-2")
+  .expectNoEvent(Duration.ofMillis(80))   -- no 4th item arrives without a request
+  .thenRequest(2).expectNext("event-3", "event-4")
+  .thenRequest(45).expectNextCount(45)
+  .expectComplete()
+```
+
+(`docs/output/04-stream-backpressure.txt`) The endpoint is built on `Flux.range()` rather than
+the more "realistic-looking" `Flux.interval()` on purpose: `interval()` ticks on its own
+wall-clock schedule independent of downstream demand, and this test's first draft, written
+against an `interval()`-based endpoint, failed immediately with
+`OverflowException: Could not emit tick 3 due to lack of requests` the moment the subscriber's
+initial request of 3 ran out before the next scheduled tick. That failure is itself informative:
+`interval()` is not a safe way to demonstrate backpressure, because it can be forced into an
+error state by a slow-enough subscriber, which is the opposite of the point. `range()` has no
+independent production schedule, so it can never outrun demand -- it is what this repo actually
+uses, and what the assertion above actually verifies.
+
+Back to [README](../README.md).
@@ -0,0 +1,8 @@
+I/O-bound endpoint (/io, Mono.delay(300ms)), concurrency=600, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK 25.0.4.1
+==============================================================================================================
+webflux (netty) : total=600 success=600 wall=852ms p50=591ms p99=738ms
+
+Mono.delay() never parks a thread -- the event loop schedules a timer callback and
+goes back to the selector loop immediately. Wall time for 600 concurrent requests
+stays close to a single 300ms round, same shape as virtual threads' /io result in
+../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt.
@@ -0,0 +1,17 @@
+CPU-bound endpoint (/cpu vs /cpu-offloaded, 20,000x SHA-256), concurrency=60, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK 25.0.4.1
+==============================================================================================================================
+LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
+Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
+
+webflux naive (Mono.fromCallable, no subscribeOn)      : total=60 success=60 wall=293ms p50=168ms p99=271ms
+webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=316ms p50=179ms p99=314ms
+
+Counter-intuitive result, and worth stating honestly rather than forcing the expected
+story: on THIS box, the two wall times are close, because Reactor Netty's default event-
+loop pool (DEFAULT_IO_WORKER_COUNT = max(availableProcessors(), 4) = 4 here) is actually
+LARGER than Schedulers.parallel()'s pool (sized to availableProcessors() = 2). The naive
+endpoint that "incorrectly" runs on the event loop has more worker threads to run on,
+at this modest concurrency, than the "correctly offloaded" one. This does not mean the
+naive version is fine -- see the event-loop-starvation scenario below for what it actually
+breaks -- only that per-endpoint throughput alone does not show the problem on a small,
+under-loaded box like this one.
@@ -0,0 +1,16 @@
+/io latency while 60 concurrent CPU-bound requests run, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK 25.0.4.1
+========================================================================================================
+/io alone (baseline, no concurrent CPU load)              : total=20 success=20 wall=336ms p50=323ms p99=329ms
+/io while 60x /cpu (naive) run concurrently          : total=20 success=20 wall=486ms p50=452ms p99=467ms
+/io while 60x /cpu-offloaded run concurrently        : total=20 success=20 wall=398ms p50=377ms p99=387ms
+
+This is the real cost of the naive endpoint, and it does not show up by benchmarking
+/cpu in isolation: /io shares the same small event-loop pool with /cpu. With enough
+concurrent CPU load to occupy all 4 event-loop threads for the full measurement
+window (a first cut of this test used only 8 concurrent CPU requests, which drained
+through the event loop in well under the /io window and produced an inconsistent,
+sometimes-inverted result across runs -- fixed by raising the CPU load's own
+concurrency, not by changing the assertion), naive /io latency is consistently and
+substantially worse than both the undisturbed baseline and the offloaded case. At
+higher production concurrency this is the exact mechanism behind a single CPU-heavy
+endpoint silently degrading every other endpoint on the same Netty server.
@@ -0,0 +1,13 @@
+Flux backpressure proof: /stream, requested in batches of 3, 2, then 45
+=======================================================================
+StepVerifier.create(controller.streamResults(), 3)
+  .expectNext("event-0", "event-1", "event-2")
+  .expectNoEvent(Duration.ofMillis(80))   -- no 4th item arrives without a request
+  .thenRequest(2).expectNext("event-3", "event-4")
+  .thenRequest(45).expectNextCount(45)
+  .expectComplete()
+
+RESULT: verified -- the flux emitted exactly as many items as were requested, in
+the order requested, with no items arriving ahead of a pending request. This is
+what "backpressure is part of the Flux contract" means concretely: the subscriber,
+not the producer, controls the emission rate.