Add virtual-threads-benchmark-webflux: the WebFlux leg of the three-way benchmark
Fixes two bugs found during self-correction before publishing: - /stream used Flux.interval(), which ticks on its own wall-clock schedule independent of downstream demand and threw OverflowException under a slow subscriber; switched to Flux.range(), which has no independent production schedule and can never outrun demand. - The event-loop-starvation test's first cut used only 8 concurrent /cpu requests as background load, which drained through the 4 event-loop threads well inside the /io measurement window and produced an inconsistent, sometimes-inverted result across runs; raised to 60 concurrent requests (matching the CPU-bound benchmark's own concurrency) so the load actually occupies the event loop for the full window. Also adds StreamBackpressureTest, a StepVerifier proof that the /stream endpoint never emits ahead of its subscriber's outstanding requests.
This commit is contained in:
Claude
@@ -0,0 +1,145 @@
|
||||
# 1. WebFlux benchmark methodology and results
|
||||
|
||||
[README](../README.md) | Companion module: [`../virtual-threads-benchmark`](../virtual-threads-benchmark/README.md) (platform threads vs virtual threads)
|
||||
|
||||
Source: [`ReactiveDemoController.java`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java), [`CpuWork.java`](../src/main/java/com/ankurm/vthreadswebflux/CpuWork.java).
|
||||
Test: [`WebfluxLoadBenchmarkTest.java`](../src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java).
|
||||
Transcripts: [`docs/output/01-io-bound-webflux.txt`](output/01-io-bound-webflux.txt), [`docs/output/02-cpu-bound-webflux.txt`](output/02-cpu-bound-webflux.txt), [`docs/output/03-event-loop-starvation.txt`](output/03-event-loop-starvation.txt), [`docs/output/04-stream-backpressure.txt`](output/04-stream-backpressure.txt).
|
||||
|
||||
## Why this module exists, and why it's separate from `../virtual-threads-benchmark`
|
||||
|
||||
The companion post for this module is the three-way comparison: platform threads vs virtual
|
||||
threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real,
|
||||
re-run benchmark in [`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for
|
||||
a different ankurm.com post and reused here rather than duplicated. This module adds the
|
||||
missing WebFlux leg, measured with the identical client-side load generator
|
||||
(`java.net.http.HttpClient` backed by a virtual-thread executor, used only as the *client*) so
|
||||
all three legs come from the same method on the same 2 vCPU sandbox.
|
||||
|
||||
It is its own Maven module rather than a third profile inside `../virtual-threads-benchmark`
|
||||
because `spring-boot-starter-web` (Tomcat) and `spring-boot-starter-webflux` (Netty) on the
|
||||
same classpath fight over `WebApplicationType.deduceFromClasspath()` -- exactly the kind of
|
||||
fragile setup a benchmark should not carry. A module with only `spring-boot-starter-webflux`
|
||||
needs no such workaround.
|
||||
|
||||
## I/O-bound: `/io`, `Mono.delay(300ms)`, concurrency 600
|
||||
|
||||
Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread `/io`
|
||||
benchmark:
|
||||
|
||||
```
|
||||
webflux (netty) : total=600 success=600 wall=1023ms p50=639ms p99=835ms
|
||||
```
|
||||
|
||||
`Mono.delay()` never parks a thread -- the event loop schedules a timer callback for 300ms
|
||||
later and immediately returns to the selector loop to service other connections. Wall time for
|
||||
600 concurrent requests stays close to a single 300ms round, the same shape virtual threads
|
||||
show in [`../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt`](../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt)
|
||||
(`wall=1042ms p50=658ms p99=720ms` on that run) -- both models avoid the platform-thread
|
||||
pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a
|
||||
worker while it waits.
|
||||
|
||||
## CPU-bound: `/cpu` (naive) vs `/cpu-offloaded`, concurrency 60
|
||||
|
||||
This is the section worth reading slowly, because the first, most intuitive prediction --
|
||||
"blocking the event loop must be dramatically worse" -- is not what the isolated benchmark
|
||||
shows on this hardware, and the reason why is a real, checkable fact about this JVM rather
|
||||
than noise.
|
||||
|
||||
```
|
||||
LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
|
||||
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
|
||||
|
||||
webflux naive (Mono.fromCallable, no subscribeOn) : total=60 success=60 wall=403ms p50=243ms p99=378ms
|
||||
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=298ms p50=167ms p99=282ms
|
||||
```
|
||||
|
||||
`/cpu` wraps the identical 20,000x SHA-256 loop from
|
||||
[`../virtual-threads-benchmark`'s `DemoController`](../virtual-threads-benchmark/src/main/java/com/ankurm/vthreads/DemoController.java)
|
||||
in `Mono.fromCallable()` with no `subscribeOn(...)` -- the single most common way a WebFlux
|
||||
handler ends up doing real work -- so it runs on whichever thread subscribes to the `Mono`:
|
||||
the Netty event-loop thread that received the request. `/cpu-offloaded` moves the identical
|
||||
work to `Schedulers.parallel()`, the scheduler Reactor's own documentation recommends for
|
||||
CPU-bound work (not `Schedulers.boundedElastic()`, which exists for blocking I/O and is sized
|
||||
far larger than the core count).
|
||||
|
||||
<blockquote>Reactor Netty's default event-loop pool
|
||||
(<code>LoopResources.DEFAULT_IO_WORKER_COUNT</code>, confirmed by printing the constant
|
||||
directly rather than reading it off documentation) is <code>max(availableProcessors(), 4)</code>
|
||||
-- 4 threads on this 2-core box. <code>Schedulers.parallel()</code>'s default pool is sized to
|
||||
<code>availableProcessors()</code> -- 2 threads on the same box, confirmed the same way. The
|
||||
endpoint that "incorrectly" runs on the event loop has <em>more</em> worker threads available
|
||||
to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times
|
||||
above are close rather than showing a dramatic gap -- not because the advice to offload
|
||||
CPU work is wrong.</blockquote>
|
||||
|
||||
Offloaded is faster here, consistent with the advice, but the margin alone understates why the
|
||||
naive version is still wrong. See the next section for the failure this endpoint-level
|
||||
benchmark cannot show.
|
||||
|
||||
## What the isolated CPU benchmark hides: event-loop starvation of *other* traffic
|
||||
|
||||
```
|
||||
/io alone (baseline, no concurrent CPU load) : total=20 success=20 wall=340ms p50=322ms p99=331ms
|
||||
/io while 60x /cpu (naive) run concurrently : total=20 success=20 wall=805ms p50=451ms p99=760ms
|
||||
/io while 60x /cpu-offloaded run concurrently : total=20 success=20 wall=405ms p50=389ms p99=398ms
|
||||
```
|
||||
|
||||
This is the real cost of the naive endpoint, and benchmarking `/cpu` by itself cannot show it:
|
||||
`/io` and `/cpu` share the same small event-loop pool. Firing concurrent naive `/cpu` requests
|
||||
and, while they are still in flight, firing 20 unrelated `/io` requests at the same server
|
||||
measures what happens to traffic that has nothing to do with the CPU-bound endpoint.
|
||||
|
||||
**This test's first cut used only 8 concurrent `/cpu` requests and produced an inconsistent,
|
||||
sometimes-inverted result across repeated runs** -- 8 requests drain through 4 event-loop
|
||||
threads in two short rounds, finishing well before the `/io` measurement window (~300-400ms)
|
||||
was over, so most of that window ran with no concurrent CPU load at all. The fix was not to
|
||||
loosen the assertion but to fix the load: **60 concurrent `/cpu` requests** (matching the
|
||||
concurrency used in the CPU-bound benchmark above) keep all 4 event-loop threads occupied for
|
||||
roughly the full measurement window. With that fixed, the result above is large and
|
||||
reproducible across repeated runs: naive `/io` p99 nearly triples the undisturbed baseline,
|
||||
while offloaded `/io` stays close to it. One CPU-heavy endpoint left un-offloaded is enough to
|
||||
degrade every other endpoint sharing its event loop -- invisible to anyone who benchmarked the
|
||||
CPU endpoint in isolation and called it "fine."
|
||||
|
||||
<ul>
|
||||
<li>If you're auditing your own WebFlux service: grep for <code>Mono.fromCallable</code>,
|
||||
<code>Flux.fromIterable</code> wrapping computation, or any synchronous call inside a reactive
|
||||
chain with no <code>subscribeOn(...)</code> after it -- each one runs on the event loop by
|
||||
default.</li>
|
||||
<li><code>Schedulers.parallel()</code> for CPU-bound work, <code>Schedulers.boundedElastic()</code>
|
||||
for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the
|
||||
wrong one either wastes threads or defeats the point of offloading.</li>
|
||||
<li>Reactor Netty event-loop sizing: <a href="https://projectreactor.io/docs/netty/release/reference/index.html#_event_loop_workers" rel="nofollow">projectreactor.io/docs/netty</a></li>
|
||||
</ul>
|
||||
|
||||
## Backpressure is structural, not a benchmark number
|
||||
|
||||
`/stream` ([`ReactiveDemoController.streamResults()`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java))
|
||||
and [`StreamBackpressureTest`](../src/test/java/com/ankurm/vthreadswebflux/StreamBackpressureTest.java)
|
||||
exist to check a claim this kind of post usually just asserts: that a `Flux` never emits faster
|
||||
than its subscriber requests. The test drives the endpoint's `Flux` with `StepVerifier`,
|
||||
requesting 3 items, then 2 more, then the remaining 45, and asserts no item ever arrives ahead
|
||||
of a pending request:
|
||||
|
||||
```
|
||||
StepVerifier.create(controller.streamResults(), 3)
|
||||
.expectNext("event-0", "event-1", "event-2")
|
||||
.expectNoEvent(Duration.ofMillis(80)) -- no 4th item arrives without a request
|
||||
.thenRequest(2).expectNext("event-3", "event-4")
|
||||
.thenRequest(45).expectNextCount(45)
|
||||
.expectComplete()
|
||||
```
|
||||
|
||||
(`docs/output/04-stream-backpressure.txt`) The endpoint is built on `Flux.range()` rather than
|
||||
the more "realistic-looking" `Flux.interval()` on purpose: `interval()` ticks on its own
|
||||
wall-clock schedule independent of downstream demand, and this test's first draft, written
|
||||
against an `interval()`-based endpoint, failed immediately with
|
||||
`OverflowException: Could not emit tick 3 due to lack of requests` the moment the subscriber's
|
||||
initial request of 3 ran out before the next scheduled tick. That failure is itself informative:
|
||||
`interval()` is not a safe way to demonstrate backpressure, because it can be forced into an
|
||||
error state by a slow-enough subscriber, which is the opposite of the point. `range()` has no
|
||||
independent production schedule, so it can never outrun demand -- it is what this repo actually
|
||||
uses, and what the assertion above actually verifies.
|
||||
|
||||
Back to [README](../README.md).
|
||||
Reference in New Issue
Block a user