spring-async-demo/virtual-threads-benchmark-webflux/docs/01-webflux-benchmark-methodology.md

# 1. WebFlux benchmark methodology and results

[README](../README.md) | Companion module: [`../virtual-threads-benchmark`](../virtual-threads-benchmark/README.md) (platform threads vs virtual threads)

Source: [`ReactiveDemoController.java`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java), [`CpuWork.java`](../src/main/java/com/ankurm/vthreadswebflux/CpuWork.java).
Test: [`WebfluxLoadBenchmarkTest.java`](../src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java).
Transcripts: [`docs/output/01-io-bound-webflux.txt`](output/01-io-bound-webflux.txt), [`docs/output/02-cpu-bound-webflux.txt`](output/02-cpu-bound-webflux.txt), [`docs/output/03-event-loop-starvation.txt`](output/03-event-loop-starvation.txt), [`docs/output/04-stream-backpressure.txt`](output/04-stream-backpressure.txt).

## Why this module exists, and why it's separate from `../virtual-threads-benchmark`

The companion post for this module is the three-way comparison: platform threads vs virtual
threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real,
re-run benchmark in [`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for
a different ankurm.com post and reused here rather than duplicated. This module adds the
missing WebFlux leg, measured with the identical client-side load generator
(`java.net.http.HttpClient` backed by a virtual-thread executor, used only as the *client*) so
all three legs come from the same method on the same 2 vCPU sandbox.

It is its own Maven module rather than a third profile inside `../virtual-threads-benchmark`
because `spring-boot-starter-web` (Tomcat) and `spring-boot-starter-webflux` (Netty) on the
same classpath fight over `WebApplicationType.deduceFromClasspath()` -- exactly the kind of
fragile setup a benchmark should not carry. A module with only `spring-boot-starter-webflux`
needs no such workaround.

## I/O-bound: `/io`, `Mono.delay(300ms)`, concurrency 600

Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread `/io`
benchmark:

```
webflux (netty) : total=600 success=600 wall=1023ms p50=639ms p99=835ms
```

`Mono.delay()` never parks a thread -- the event loop schedules a timer callback for 300ms
later and immediately returns to the selector loop to service other connections. Wall time for
600 concurrent requests stays close to a single 300ms round, the same shape virtual threads
show in [`../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt`](../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt)
(`wall=1042ms p50=658ms p99=720ms` on that run) -- both models avoid the platform-thread
pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a
worker while it waits.

## CPU-bound: `/cpu` (naive) vs `/cpu-offloaded`, concurrency 60

This is the section worth reading slowly, because the first, most intuitive prediction --
"blocking the event loop must be dramatically worse" -- is not what the isolated benchmark
shows on this hardware, and the reason why is a real, checkable fact about this JVM rather
than noise.

```
LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2

webflux naive (Mono.fromCallable, no subscribeOn)      : total=60 success=60 wall=403ms p50=243ms p99=378ms
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=298ms p50=167ms p99=282ms
```

`/cpu` wraps the identical 20,000x SHA-256 loop from
[`../virtual-threads-benchmark`'s `DemoController`](../virtual-threads-benchmark/src/main/java/com/ankurm/vthreads/DemoController.java)
in `Mono.fromCallable()` with no `subscribeOn(...)` -- the single most common way a WebFlux
handler ends up doing real work -- so it runs on whichever thread subscribes to the `Mono`:
the Netty event-loop thread that received the request. `/cpu-offloaded` moves the identical
work to `Schedulers.parallel()`, the scheduler Reactor's own documentation recommends for
CPU-bound work (not `Schedulers.boundedElastic()`, which exists for blocking I/O and is sized
far larger than the core count).

<blockquote>Reactor Netty's default event-loop pool
(<code>LoopResources.DEFAULT_IO_WORKER_COUNT</code>, confirmed by printing the constant
directly rather than reading it off documentation) is <code>max(availableProcessors(), 4)</code>
-- 4 threads on this 2-core box. <code>Schedulers.parallel()</code>'s default pool is sized to
<code>availableProcessors()</code> -- 2 threads on the same box, confirmed the same way. The
endpoint that "incorrectly" runs on the event loop has <em>more</em> worker threads available
to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times
above are close rather than showing a dramatic gap -- not because the advice to offload
CPU work is wrong.</blockquote>

Offloaded is faster here, consistent with the advice, but the margin alone understates why the
naive version is still wrong. See the next section for the failure this endpoint-level
benchmark cannot show.

## What the isolated CPU benchmark hides: event-loop starvation of *other* traffic

```
/io alone (baseline, no concurrent CPU load)         : total=20 success=20 wall=340ms p50=322ms p99=331ms
/io while 60x /cpu (naive) run concurrently          : total=20 success=20 wall=805ms p50=451ms p99=760ms
/io while 60x /cpu-offloaded run concurrently        : total=20 success=20 wall=405ms p50=389ms p99=398ms
```

This is the real cost of the naive endpoint, and benchmarking `/cpu` by itself cannot show it:
`/io` and `/cpu` share the same small event-loop pool. Firing concurrent naive `/cpu` requests
and, while they are still in flight, firing 20 unrelated `/io` requests at the same server
measures what happens to traffic that has nothing to do with the CPU-bound endpoint.

**This test's first cut used only 8 concurrent `/cpu` requests and produced an inconsistent,
sometimes-inverted result across repeated runs** -- 8 requests drain through 4 event-loop
threads in two short rounds, finishing well before the `/io` measurement window (~300-400ms)
was over, so most of that window ran with no concurrent CPU load at all. The fix was not to
loosen the assertion but to fix the load: **60 concurrent `/cpu` requests** (matching the
concurrency used in the CPU-bound benchmark above) keep all 4 event-loop threads occupied for
roughly the full measurement window. With that fixed, the result above is large and
reproducible across repeated runs: naive `/io` p99 nearly triples the undisturbed baseline,
while offloaded `/io` stays close to it. One CPU-heavy endpoint left un-offloaded is enough to
degrade every other endpoint sharing its event loop -- invisible to anyone who benchmarked the
CPU endpoint in isolation and called it "fine."

<ul>
<li>If you're auditing your own WebFlux service: grep for <code>Mono.fromCallable</code>,
<code>Flux.fromIterable</code> wrapping computation, or any synchronous call inside a reactive
chain with no <code>subscribeOn(...)</code> after it -- each one runs on the event loop by
default.</li>
<li><code>Schedulers.parallel()</code> for CPU-bound work, <code>Schedulers.boundedElastic()</code>
for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the
wrong one either wastes threads or defeats the point of offloading.</li>
<li>Reactor Netty event-loop sizing: <a href="https://projectreactor.io/docs/netty/release/reference/index.html#_event_loop_workers" rel="nofollow">projectreactor.io/docs/netty</a></li>
</ul>

## Backpressure is structural, not a benchmark number

`/stream` ([`ReactiveDemoController.streamResults()`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java))
and [`StreamBackpressureTest`](../src/test/java/com/ankurm/vthreadswebflux/StreamBackpressureTest.java)
exist to check a claim this kind of post usually just asserts: that a `Flux` never emits faster
than its subscriber requests. The test drives the endpoint's `Flux` with `StepVerifier`,
requesting 3 items, then 2 more, then the remaining 45, and asserts no item ever arrives ahead
of a pending request:

```
StepVerifier.create(controller.streamResults(), 3)
  .expectNext("event-0", "event-1", "event-2")
  .expectNoEvent(Duration.ofMillis(80))   -- no 4th item arrives without a request
  .thenRequest(2).expectNext("event-3", "event-4")
  .thenRequest(45).expectNextCount(45)
  .expectComplete()
```

(`docs/output/04-stream-backpressure.txt`) The endpoint is built on `Flux.range()` rather than
the more "realistic-looking" `Flux.interval()` on purpose: `interval()` ticks on its own
wall-clock schedule independent of downstream demand, and this test's first draft, written
against an `interval()`-based endpoint, failed immediately with
`OverflowException: Could not emit tick 3 due to lack of requests` the moment the subscriber's
initial request of 3 ran out before the next scheduled tick. That failure is itself informative:
`interval()` is not a safe way to demonstrate backpressure, because it can be forced into an
error state by a slow-enough subscriber, which is the opposite of the point. `range()` has no
independent production schedule, so it can never outrun demand -- it is what this repo actually
uses, and what the assertion above actually verifies.

Back to [README](../README.md).