Add virtual-threads-benchmark-webflux: the WebFlux leg of the three-way benchmark
Re-uses the platform/virtual-thread transcripts already committed in virtual-threads-benchmark rather than re-measuring them, and adds a WebFlux module measured the same way (client-side java.net.http.HttpClient load generator against a real Netty server). Covers I/O-bound throughput, a naive-vs-offloaded CPU-bound comparison that turned up a real, counter- intuitive fact about this sandbox's default thread-pool sizes (LoopResources.DEFAULT_IO_WORKER_COUNT=4 vs Schedulers.parallel()'s availableProcessors()=2), and an event-loop-starvation scenario showing what the naive endpoint actually costs unrelated /io traffic on the same server. Companion code for the ankurm.com post "Virtual Threads vs WebFlux vs Platform Threads: Benchmarks and a Decision Framework." Also fixes the root README's module table, which was missing the virtual-threads-benchmark row from an earlier commit.
This commit is contained in:
Claude
@@ -0,0 +1,112 @@
|
||||
# 1. WebFlux benchmark methodology and results
|
||||
|
||||
[README](../README.md) | Companion module: [`../virtual-threads-benchmark`](../virtual-threads-benchmark/README.md) (platform threads vs virtual threads)
|
||||
|
||||
Source: [`ReactiveDemoController.java`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java), [`CpuWork.java`](../src/main/java/com/ankurm/vthreadswebflux/CpuWork.java).
|
||||
Test: [`WebfluxLoadBenchmarkTest.java`](../src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java).
|
||||
Transcripts: [`docs/output/01-io-bound-webflux.txt`](output/01-io-bound-webflux.txt), [`docs/output/02-cpu-bound-webflux.txt`](output/02-cpu-bound-webflux.txt), [`docs/output/03-event-loop-starvation.txt`](output/03-event-loop-starvation.txt).
|
||||
|
||||
## Why this module exists, and why it's separate from `../virtual-threads-benchmark`
|
||||
|
||||
The companion post for this module is the three-way comparison: platform threads vs virtual
|
||||
threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real,
|
||||
re-run benchmark in [`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for
|
||||
a different ankurm.com post and reused here rather than duplicated. This module adds the
|
||||
missing WebFlux leg, measured with the identical client-side load generator
|
||||
(`java.net.http.HttpClient` backed by a virtual-thread executor, used only as the *client*) so
|
||||
all three legs come from the same method on the same 2 vCPU sandbox.
|
||||
|
||||
It is its own Maven module rather than a third profile inside `../virtual-threads-benchmark`
|
||||
because `spring-boot-starter-web` (Tomcat) and `spring-boot-starter-webflux` (Netty) on the
|
||||
same classpath fight over `WebApplicationType.deduceFromClasspath()` -- exactly the kind of
|
||||
fragile setup a benchmark should not carry. A module with only `spring-boot-starter-webflux`
|
||||
needs no such workaround.
|
||||
|
||||
## I/O-bound: `/io`, `Mono.delay(300ms)`, concurrency 600
|
||||
|
||||
Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread `/io`
|
||||
benchmark:
|
||||
|
||||
```
|
||||
webflux (netty) : total=600 success=600 wall=897ms p50=596ms p99=780ms
|
||||
```
|
||||
|
||||
`Mono.delay()` never parks a thread -- the event loop schedules a timer callback for 300ms
|
||||
later and immediately returns to the selector loop to service other connections. Wall time for
|
||||
600 concurrent requests stays close to a single 300ms round, the same shape virtual threads
|
||||
show in [`../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt`](../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt)
|
||||
(`wall=1042ms p50=658ms p99=720ms` on that run) -- both models avoid the platform-thread
|
||||
pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a
|
||||
worker while it waits.
|
||||
|
||||
## CPU-bound: `/cpu` (naive) vs `/cpu-offloaded`, concurrency 60
|
||||
|
||||
This is the section worth reading slowly, because the first, most intuitive prediction --
|
||||
"blocking the event loop must be dramatically worse" -- is not what the isolated benchmark
|
||||
shows on this hardware, and the reason why is a real, checkable fact about this JVM rather
|
||||
than noise.
|
||||
|
||||
```
|
||||
LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
|
||||
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
|
||||
|
||||
webflux naive (Mono.fromCallable, no subscribeOn) : total=60 success=60 wall=358ms p50=209ms p99=330ms
|
||||
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=324ms p50=171ms p99=283ms
|
||||
```
|
||||
|
||||
`/cpu` wraps the identical 20,000x SHA-256 loop from
|
||||
[`../virtual-threads-benchmark`'s `DemoController`](../virtual-threads-benchmark/src/main/java/com/ankurm/vthreads/DemoController.java)
|
||||
in `Mono.fromCallable()` with no `subscribeOn(...)` -- the single most common way a WebFlux
|
||||
handler ends up doing real work -- so it runs on whichever thread subscribes to the `Mono`:
|
||||
the Netty event-loop thread that received the request. `/cpu-offloaded` moves the identical
|
||||
work to `Schedulers.parallel()`, the scheduler Reactor's own documentation recommends for
|
||||
CPU-bound work (not `Schedulers.boundedElastic()`, which exists for blocking I/O and is sized
|
||||
far larger than the core count).
|
||||
|
||||
<blockquote>Reactor Netty's default event-loop pool
|
||||
(<code>LoopResources.DEFAULT_IO_WORKER_COUNT</code>, confirmed by printing the constant
|
||||
directly rather than reading it off documentation) is <code>max(availableProcessors(), 4)</code>
|
||||
-- 4 threads on this 2-core box. <code>Schedulers.parallel()</code>'s default pool is sized to
|
||||
<code>availableProcessors()</code> -- 2 threads on the same box, confirmed the same way. The
|
||||
endpoint that "incorrectly" runs on the event loop has <em>more</em> worker threads available
|
||||
to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times
|
||||
above are close rather than showing a dramatic gap -- not because the advice to offload
|
||||
CPU work is wrong.</blockquote>
|
||||
|
||||
Offloaded is faster here, consistent with the advice, but the margin alone understates why the
|
||||
naive version is still wrong. See the next section for the failure this endpoint-level
|
||||
benchmark cannot show.
|
||||
|
||||
## What the isolated CPU benchmark hides: event-loop starvation of *other* traffic
|
||||
|
||||
```
|
||||
/io alone (baseline, no concurrent CPU load) : total=20 success=20 wall=341ms p50=329ms p99=340ms
|
||||
/io while 8x /cpu (naive) run concurrently : total=20 success=20 wall=382ms p50=359ms p99=374ms
|
||||
/io while 8x /cpu-offloaded run concurrently : total=20 success=20 wall=344ms p50=328ms p99=338ms
|
||||
```
|
||||
|
||||
This is the real cost of the naive endpoint, and benchmarking `/cpu` by itself cannot show it:
|
||||
`/io` and `/cpu` share the same small event-loop pool. Firing 8 concurrent naive `/cpu`
|
||||
requests and, while they are still in flight, firing 20 unrelated `/io` requests at the same
|
||||
server measures what happens to traffic that has nothing to do with the CPU-bound endpoint.
|
||||
Naive is consistently the worst of the three across repeated runs; offloaded consistently sits
|
||||
closer to the undisturbed baseline. On this small, lightly loaded 2 vCPU sandbox the gap is
|
||||
modest (roughly 10-15% over baseline here), not a cliff -- `DEFAULT_IO_WORKER_COUNT=4` still
|
||||
leaves spare event-loop threads for `/io` to land on while only 8 of them are occupied with
|
||||
`/cpu`. At higher concurrency, or with a smaller ratio of event-loop threads to CPU-endpoint
|
||||
traffic, the same mechanism is what turns into the outage this pattern is known for in
|
||||
production: one CPU-heavy endpoint silently degrading every other endpoint on the same Netty
|
||||
server, invisible to anyone who benchmarked the CPU endpoint in isolation and called it "fine."
|
||||
|
||||
<ul>
|
||||
<li>If you're auditing your own WebFlux service: grep for <code>Mono.fromCallable</code>,
|
||||
<code>Flux.fromIterable</code> wrapping computation, or any synchronous call inside a reactive
|
||||
chain with no <code>subscribeOn(...)</code> after it -- each one runs on the event loop by
|
||||
default.</li>
|
||||
<li><code>Schedulers.parallel()</code> for CPU-bound work, <code>Schedulers.boundedElastic()</code>
|
||||
for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the
|
||||
wrong one either wastes threads or defeats the point of offloading.</li>
|
||||
<li>Reactor Netty event-loop sizing: <a href="https://projectreactor.io/docs/netty/release/reference/index.html#_event_loop_workers" rel="nofollow">projectreactor.io/docs/netty</a></li>
|
||||
</ul>
|
||||
|
||||
Back to [README](../README.md).
|
||||
Reference in New Issue
Block a user