|
|
|
@@ -0,0 +1,112 @@
|
|
|
|
|
# 1. WebFlux benchmark methodology and results
|
|
|
|
|
|
|
|
|
|
[README](../README.md) | Companion module: [`../virtual-threads-benchmark`](../virtual-threads-benchmark/README.md) (platform threads vs virtual threads)
|
|
|
|
|
|
|
|
|
|
Source: [`ReactiveDemoController.java`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java), [`CpuWork.java`](../src/main/java/com/ankurm/vthreadswebflux/CpuWork.java).
|
|
|
|
|
Test: [`WebfluxLoadBenchmarkTest.java`](../src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java).
|
|
|
|
|
Transcripts: [`docs/output/01-io-bound-webflux.txt`](output/01-io-bound-webflux.txt), [`docs/output/02-cpu-bound-webflux.txt`](output/02-cpu-bound-webflux.txt), [`docs/output/03-event-loop-starvation.txt`](output/03-event-loop-starvation.txt).
|
|
|
|
|
|
|
|
|
|
## Why this module exists, and why it's separate from `../virtual-threads-benchmark`
|
|
|
|
|
|
|
|
|
|
The companion post for this module is the three-way comparison: platform threads vs virtual
|
|
|
|
|
threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real,
|
|
|
|
|
re-run benchmark in [`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for
|
|
|
|
|
a different ankurm.com post and reused here rather than duplicated. This module adds the
|
|
|
|
|
missing WebFlux leg, measured with the identical client-side load generator
|
|
|
|
|
(`java.net.http.HttpClient` backed by a virtual-thread executor, used only as the *client*) so
|
|
|
|
|
all three legs come from the same method on the same 2 vCPU sandbox.
|
|
|
|
|
|
|
|
|
|
It is its own Maven module rather than a third profile inside `../virtual-threads-benchmark`
|
|
|
|
|
because `spring-boot-starter-web` (Tomcat) and `spring-boot-starter-webflux` (Netty) on the
|
|
|
|
|
same classpath fight over `WebApplicationType.deduceFromClasspath()` -- exactly the kind of
|
|
|
|
|
fragile setup a benchmark should not carry. A module with only `spring-boot-starter-webflux`
|
|
|
|
|
needs no such workaround.
|
|
|
|
|
|
|
|
|
|
## I/O-bound: `/io`, `Mono.delay(300ms)`, concurrency 600
|
|
|
|
|
|
|
|
|
|
Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread `/io`
|
|
|
|
|
benchmark:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
webflux (netty) : total=600 success=600 wall=897ms p50=596ms p99=780ms
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`Mono.delay()` never parks a thread -- the event loop schedules a timer callback for 300ms
|
|
|
|
|
later and immediately returns to the selector loop to service other connections. Wall time for
|
|
|
|
|
600 concurrent requests stays close to a single 300ms round, the same shape virtual threads
|
|
|
|
|
show in [`../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt`](../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt)
|
|
|
|
|
(`wall=1042ms p50=658ms p99=720ms` on that run) -- both models avoid the platform-thread
|
|
|
|
|
pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a
|
|
|
|
|
worker while it waits.
|
|
|
|
|
|
|
|
|
|
## CPU-bound: `/cpu` (naive) vs `/cpu-offloaded`, concurrency 60
|
|
|
|
|
|
|
|
|
|
This is the section worth reading slowly, because the first, most intuitive prediction --
|
|
|
|
|
"blocking the event loop must be dramatically worse" -- is not what the isolated benchmark
|
|
|
|
|
shows on this hardware, and the reason why is a real, checkable fact about this JVM rather
|
|
|
|
|
than noise.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
|
|
|
|
|
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
|
|
|
|
|
|
|
|
|
|
webflux naive (Mono.fromCallable, no subscribeOn) : total=60 success=60 wall=358ms p50=209ms p99=330ms
|
|
|
|
|
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=324ms p50=171ms p99=283ms
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
`/cpu` wraps the identical 20,000x SHA-256 loop from
|
|
|
|
|
[`../virtual-threads-benchmark`'s `DemoController`](../virtual-threads-benchmark/src/main/java/com/ankurm/vthreads/DemoController.java)
|
|
|
|
|
in `Mono.fromCallable()` with no `subscribeOn(...)` -- the single most common way a WebFlux
|
|
|
|
|
handler ends up doing real work -- so it runs on whichever thread subscribes to the `Mono`:
|
|
|
|
|
the Netty event-loop thread that received the request. `/cpu-offloaded` moves the identical
|
|
|
|
|
work to `Schedulers.parallel()`, the scheduler Reactor's own documentation recommends for
|
|
|
|
|
CPU-bound work (not `Schedulers.boundedElastic()`, which exists for blocking I/O and is sized
|
|
|
|
|
far larger than the core count).
|
|
|
|
|
|
|
|
|
|
<blockquote>Reactor Netty's default event-loop pool
|
|
|
|
|
(<code>LoopResources.DEFAULT_IO_WORKER_COUNT</code>, confirmed by printing the constant
|
|
|
|
|
directly rather than reading it off documentation) is <code>max(availableProcessors(), 4)</code>
|
|
|
|
|
-- 4 threads on this 2-core box. <code>Schedulers.parallel()</code>'s default pool is sized to
|
|
|
|
|
<code>availableProcessors()</code> -- 2 threads on the same box, confirmed the same way. The
|
|
|
|
|
endpoint that "incorrectly" runs on the event loop has <em>more</em> worker threads available
|
|
|
|
|
to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times
|
|
|
|
|
above are close rather than showing a dramatic gap -- not because the advice to offload
|
|
|
|
|
CPU work is wrong.</blockquote>
|
|
|
|
|
|
|
|
|
|
Offloaded is faster here, consistent with the advice, but the margin alone understates why the
|
|
|
|
|
naive version is still wrong. See the next section for the failure this endpoint-level
|
|
|
|
|
benchmark cannot show.
|
|
|
|
|
|
|
|
|
|
## What the isolated CPU benchmark hides: event-loop starvation of *other* traffic
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
/io alone (baseline, no concurrent CPU load) : total=20 success=20 wall=341ms p50=329ms p99=340ms
|
|
|
|
|
/io while 8x /cpu (naive) run concurrently : total=20 success=20 wall=382ms p50=359ms p99=374ms
|
|
|
|
|
/io while 8x /cpu-offloaded run concurrently : total=20 success=20 wall=344ms p50=328ms p99=338ms
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This is the real cost of the naive endpoint, and benchmarking `/cpu` by itself cannot show it:
|
|
|
|
|
`/io` and `/cpu` share the same small event-loop pool. Firing 8 concurrent naive `/cpu`
|
|
|
|
|
requests and, while they are still in flight, firing 20 unrelated `/io` requests at the same
|
|
|
|
|
server measures what happens to traffic that has nothing to do with the CPU-bound endpoint.
|
|
|
|
|
Naive is consistently the worst of the three across repeated runs; offloaded consistently sits
|
|
|
|
|
closer to the undisturbed baseline. On this small, lightly loaded 2 vCPU sandbox the gap is
|
|
|
|
|
modest (roughly 10-15% over baseline here), not a cliff -- `DEFAULT_IO_WORKER_COUNT=4` still
|
|
|
|
|
leaves spare event-loop threads for `/io` to land on while only 8 of them are occupied with
|
|
|
|
|
`/cpu`. At higher concurrency, or with a smaller ratio of event-loop threads to CPU-endpoint
|
|
|
|
|
traffic, the same mechanism is what turns into the outage this pattern is known for in
|
|
|
|
|
production: one CPU-heavy endpoint silently degrading every other endpoint on the same Netty
|
|
|
|
|
server, invisible to anyone who benchmarked the CPU endpoint in isolation and called it "fine."
|
|
|
|
|
|
|
|
|
|
<ul>
|
|
|
|
|
<li>If you're auditing your own WebFlux service: grep for <code>Mono.fromCallable</code>,
|
|
|
|
|
<code>Flux.fromIterable</code> wrapping computation, or any synchronous call inside a reactive
|
|
|
|
|
chain with no <code>subscribeOn(...)</code> after it -- each one runs on the event loop by
|
|
|
|
|
default.</li>
|
|
|
|
|
<li><code>Schedulers.parallel()</code> for CPU-bound work, <code>Schedulers.boundedElastic()</code>
|
|
|
|
|
for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the
|
|
|
|
|
wrong one either wastes threads or defeats the point of offloading.</li>
|
|
|
|
|
<li>Reactor Netty event-loop sizing: <a href="https://projectreactor.io/docs/netty/release/reference/index.html#_event_loop_workers" rel="nofollow">projectreactor.io/docs/netty</a></li>
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
Back to [README](../README.md).
|
Claude