Add virtual-threads-benchmark-webflux: the WebFlux leg of the three-way benchmark
Fixes two bugs found during self-correction before publishing: - /stream used Flux.interval(), which ticks on its own wall-clock schedule independent of downstream demand and threw OverflowException under a slow subscriber; switched to Flux.range(), which has no independent production schedule and can never outrun demand. - The event-loop-starvation test's first cut used only 8 concurrent /cpu requests as background load, which drained through the 4 event-loop threads well inside the /io measurement window and produced an inconsistent, sometimes-inverted result across runs; raised to 60 concurrent requests (matching the CPU-bound benchmark's own concurrency) so the load actually occupies the event loop for the full window. Also adds StreamBackpressureTest, a StepVerifier proof that the /stream endpoint never emits ahead of its subscriber's outstanding requests.
This commit is contained in:
Claude
@@ -10,6 +10,8 @@ by that module's `scripts/run-all.sh`, never typed by hand.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| [`async/`](async/README.md) | [@Async in Spring Boot 4: Executors, Virtual Threads and the Self-Invocation Trap](https://ankurm.com/spring-boot-4-async-executors-virtual-threads/) | Which thread a method actually ran on, in every case where the answer is not the one you expect |
|
| [`async/`](async/README.md) | [@Async in Spring Boot 4: Executors, Virtual Threads and the Self-Invocation Trap](https://ankurm.com/spring-boot-4-async-executors-virtual-threads/) | Which thread a method actually ran on, in every case where the answer is not the one you expect |
|
||||||
| [`scheduling/`](scheduling/README.md) | [@Scheduled, ShedLock and Distributed Cron: Scheduling That Survives Three Replicas](https://ankurm.com/spring-scheduled-shedlock-distributed-cron/) | Three replicas against one database running the same job three times, then one row and one conditional UPDATE fixing it |
|
| [`scheduling/`](scheduling/README.md) | [@Scheduled, ShedLock and Distributed Cron: Scheduling That Survives Three Replicas](https://ankurm.com/spring-scheduled-shedlock-distributed-cron/) | Three replicas against one database running the same job three times, then one row and one conditional UPDATE fixing it |
|
||||||
|
| [`virtual-threads-benchmark/`](virtual-threads-benchmark/README.md) | [Virtual Threads on Spring Boot 4.1: The Benchmarks, Re-Run, and the Pinning Advice That Expired](https://ankurm.com/leveraging-virtual-threads-in-spring-boot-3-4-building-high-throughput-services/) | Platform threads vs virtual threads, re-benchmarked on Boot 4.1.1 / JDK 25, plus JEP 491's fix to `synchronized` pinning proven against a real JDK |
|
||||||
|
| [`virtual-threads-benchmark-webflux/`](virtual-threads-benchmark-webflux/README.md) | [Virtual Threads vs Reactive (WebFlux) vs Platform Threads: Benchmarks and a Decision Framework](https://ankurm.com/virtual-threads-vs-webflux-vs-platform-threads-spring-boot-benchmarks/) | The WebFlux leg of the three-way comparison, plus the event-loop-starvation failure mode an isolated CPU benchmark can't show |
|
||||||
|
|
||||||
## Common ground
|
## Common ground
|
||||||
|
|
||||||
@@ -31,6 +33,13 @@ The `scheduling` module also needs a database. `scheduling/scripts/postgres.sh`
|
|||||||
throwaway PostgreSQL 14 into `target/` with no Docker and no root, which is how its transcripts
|
throwaway PostgreSQL 14 into `target/` with no Docker and no root, which is how its transcripts
|
||||||
were produced; `docker-compose.yml` is there for anyone who would rather use Docker.
|
were produced; `docker-compose.yml` is there for anyone who would rather use Docker.
|
||||||
|
|
||||||
|
`virtual-threads-benchmark` and `virtual-threads-benchmark-webflux` are a similar pair: the
|
||||||
|
first re-benchmarks platform threads against virtual threads for one post, the second adds the
|
||||||
|
WebFlux leg for a different, three-way-comparison post, and reuses the first module's
|
||||||
|
committed transcripts rather than re-measuring the same thing twice. Both use the same
|
||||||
|
client-side load generator (`java.net.http.HttpClient` on a virtual-thread executor, client
|
||||||
|
role only) so all three threading models in the three-way post are measured the same way.
|
||||||
|
|
||||||
## Licence
|
## Licence
|
||||||
|
|
||||||
MIT — see [LICENSE](LICENSE).
|
MIT — see [LICENSE](LICENSE).
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
target/
|
||||||
|
*.class
|
||||||
|
.idea/
|
||||||
|
*.iml
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Ankur Mhatre
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# virtual-threads-webflux-benchmark
|
||||||
|
|
||||||
|
Companion module for the ankurm.com post **"Virtual Threads vs Reactive (WebFlux) vs Platform
|
||||||
|
Threads: Benchmarks and a Decision Framework."** This module is the WebFlux leg of that
|
||||||
|
three-way comparison; the platform-thread and virtual-thread legs live in the sibling module
|
||||||
|
[`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for a different
|
||||||
|
ankurm.com post and reused here rather than re-run, so the platform/virtual numbers quoted in
|
||||||
|
this post are the same numbers, not a second measurement of the same thing.
|
||||||
|
|
||||||
|
Every number below is from a real concurrent load run against a real running embedded Netty
|
||||||
|
server on this JDK, using the identical client-side load generator
|
||||||
|
(`java.net.http.HttpClient`, virtual-thread executor, client-side only) as the sibling module.
|
||||||
|
|
||||||
|
## Versions (verified against `repo1.maven.org` maven-metadata.xml and Spring Boot's own `spring-boot-dependencies` POM, not aggregators)
|
||||||
|
|
||||||
|
| Component | Version | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| JDK | 25 (Temurin 25.0.4.1+1) | same as `../virtual-threads-benchmark` |
|
||||||
|
| Spring Boot | 4.1.1 | latest GA at time of writing |
|
||||||
|
| Spring Framework | 7.0.9 | latest GA |
|
||||||
|
| Reactor | managed by `spring-boot-dependencies` 4.1.1 | version not pinned directly; see this module's effective POM |
|
||||||
|
|
||||||
|
## Quickstart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/run-all.sh # regenerates every file in docs/output/ from a real test run
|
||||||
|
./scripts/run.sh # start on :8080
|
||||||
|
```
|
||||||
|
|
||||||
|
Requires JDK 25 and Maven. This module's own benchmark ran on the same 2 vCPU sandbox as
|
||||||
|
`../virtual-threads-benchmark` -- see
|
||||||
|
[docs/01-webflux-benchmark-methodology.md](docs/01-webflux-benchmark-methodology.md) for why
|
||||||
|
that matters and what it means for the numbers below.
|
||||||
|
|
||||||
|
## What's demonstrated where
|
||||||
|
|
||||||
|
| Area | Source | Test | Transcript |
|
||||||
|
|---|---|---|---|
|
||||||
|
| I/O-bound throughput: WebFlux, 600 concurrent, same shape as virtual threads | [`ReactiveDemoController`](src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java) | [`WebfluxLoadBenchmarkTest`](src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java) | [`01`](docs/output/01-io-bound-webflux.txt) |
|
||||||
|
| CPU-bound throughput: naive (event loop) vs offloaded (`Schedulers.parallel()`), 60 concurrent -- and why the gap is smaller than expected on this box | same | same | [`02`](docs/output/02-cpu-bound-webflux.txt) |
|
||||||
|
| Event-loop starvation: what the naive CPU endpoint actually costs *other* traffic on the same server | [`CpuWork`](src/main/java/com/ankurm/vthreadswebflux/CpuWork.java) | same | [`03`](docs/output/03-event-loop-starvation.txt) |
|
||||||
|
| Backpressure is structural: a `Flux` never outruns its subscriber's requests | [`ReactiveDemoController.streamResults()`](src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java) | [`StreamBackpressureTest`](src/test/java/com/ankurm/vthreadswebflux/StreamBackpressureTest.java) | [`04`](docs/output/04-stream-backpressure.txt) |
|
||||||
|
|
||||||
|
## Documentation chapters
|
||||||
|
|
||||||
|
1. [WebFlux benchmark methodology and results](docs/01-webflux-benchmark-methodology.md) --
|
||||||
|
I/O-bound, CPU-bound naive vs offloaded, the event-loop-starvation scenario the isolated CPU
|
||||||
|
benchmark can't show (including a first-cut version of that test that under-loaded the event
|
||||||
|
loop and had to be fixed), and a real backpressure proof, all on this sandbox's real hardware
|
||||||
|
|
||||||
|
## A note on this module's relationship to `../virtual-threads-benchmark`
|
||||||
|
|
||||||
|
[`../virtual-threads-benchmark`](../virtual-threads-benchmark) already contains a real,
|
||||||
|
re-run platform-thread vs virtual-thread benchmark on this exact hardware and JDK, built for
|
||||||
|
[Virtual Threads on Spring Boot 4.1](https://ankurm.com/leveraging-virtual-threads-in-spring-boot-3-4-building-high-throughput-services/).
|
||||||
|
This module deliberately does not re-measure that comparison -- it reuses those committed
|
||||||
|
transcripts and adds only the WebFlux leg, using the same client-side load-generation method,
|
||||||
|
so the three-way post can quote one consistent measurement approach across all three models
|
||||||
|
instead of stitching together benchmarks run different ways.
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT -- see [LICENSE](LICENSE).
|
||||||
@@ -0,0 +1,145 @@
|
|||||||
|
# 1. WebFlux benchmark methodology and results
|
||||||
|
|
||||||
|
[README](../README.md) | Companion module: [`../virtual-threads-benchmark`](../virtual-threads-benchmark/README.md) (platform threads vs virtual threads)
|
||||||
|
|
||||||
|
Source: [`ReactiveDemoController.java`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java), [`CpuWork.java`](../src/main/java/com/ankurm/vthreadswebflux/CpuWork.java).
|
||||||
|
Test: [`WebfluxLoadBenchmarkTest.java`](../src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java).
|
||||||
|
Transcripts: [`docs/output/01-io-bound-webflux.txt`](output/01-io-bound-webflux.txt), [`docs/output/02-cpu-bound-webflux.txt`](output/02-cpu-bound-webflux.txt), [`docs/output/03-event-loop-starvation.txt`](output/03-event-loop-starvation.txt), [`docs/output/04-stream-backpressure.txt`](output/04-stream-backpressure.txt).
|
||||||
|
|
||||||
|
## Why this module exists, and why it's separate from `../virtual-threads-benchmark`
|
||||||
|
|
||||||
|
The companion post for this module is the three-way comparison: platform threads vs virtual
|
||||||
|
threads vs WebFlux. The platform-vs-virtual half of that comparison already has a real,
|
||||||
|
re-run benchmark in [`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for
|
||||||
|
a different ankurm.com post and reused here rather than duplicated. This module adds the
|
||||||
|
missing WebFlux leg, measured with the identical client-side load generator
|
||||||
|
(`java.net.http.HttpClient` backed by a virtual-thread executor, used only as the *client*) so
|
||||||
|
all three legs come from the same method on the same 2 vCPU sandbox.
|
||||||
|
|
||||||
|
It is its own Maven module rather than a third profile inside `../virtual-threads-benchmark`
|
||||||
|
because `spring-boot-starter-web` (Tomcat) and `spring-boot-starter-webflux` (Netty) on the
|
||||||
|
same classpath fight over `WebApplicationType.deduceFromClasspath()` -- exactly the kind of
|
||||||
|
fragile setup a benchmark should not carry. A module with only `spring-boot-starter-webflux`
|
||||||
|
needs no such workaround.
|
||||||
|
|
||||||
|
## I/O-bound: `/io`, `Mono.delay(300ms)`, concurrency 600
|
||||||
|
|
||||||
|
Same concurrency and same simulated 300ms downstream call as the platform/virtual-thread `/io`
|
||||||
|
benchmark:
|
||||||
|
|
||||||
|
```
|
||||||
|
webflux (netty) : total=600 success=600 wall=1023ms p50=639ms p99=835ms
|
||||||
|
```
|
||||||
|
|
||||||
|
`Mono.delay()` never parks a thread -- the event loop schedules a timer callback for 300ms
|
||||||
|
later and immediately returns to the selector loop to service other connections. Wall time for
|
||||||
|
600 concurrent requests stays close to a single 300ms round, the same shape virtual threads
|
||||||
|
show in [`../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt`](../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt)
|
||||||
|
(`wall=1042ms p50=658ms p99=720ms` on that run) -- both models avoid the platform-thread
|
||||||
|
pool's ~3-batch queueing entirely, for the same underlying reason: neither one occupies a
|
||||||
|
worker while it waits.
|
||||||
|
|
||||||
|
## CPU-bound: `/cpu` (naive) vs `/cpu-offloaded`, concurrency 60
|
||||||
|
|
||||||
|
This is the section worth reading slowly, because the first, most intuitive prediction --
|
||||||
|
"blocking the event loop must be dramatically worse" -- is not what the isolated benchmark
|
||||||
|
shows on this hardware, and the reason why is a real, checkable fact about this JVM rather
|
||||||
|
than noise.
|
||||||
|
|
||||||
|
```
|
||||||
|
LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
|
||||||
|
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
|
||||||
|
|
||||||
|
webflux naive (Mono.fromCallable, no subscribeOn) : total=60 success=60 wall=403ms p50=243ms p99=378ms
|
||||||
|
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=298ms p50=167ms p99=282ms
|
||||||
|
```
|
||||||
|
|
||||||
|
`/cpu` wraps the identical 20,000x SHA-256 loop from
|
||||||
|
[`../virtual-threads-benchmark`'s `DemoController`](../virtual-threads-benchmark/src/main/java/com/ankurm/vthreads/DemoController.java)
|
||||||
|
in `Mono.fromCallable()` with no `subscribeOn(...)` -- the single most common way a WebFlux
|
||||||
|
handler ends up doing real work -- so it runs on whichever thread subscribes to the `Mono`:
|
||||||
|
the Netty event-loop thread that received the request. `/cpu-offloaded` moves the identical
|
||||||
|
work to `Schedulers.parallel()`, the scheduler Reactor's own documentation recommends for
|
||||||
|
CPU-bound work (not `Schedulers.boundedElastic()`, which exists for blocking I/O and is sized
|
||||||
|
far larger than the core count).
|
||||||
|
|
||||||
|
<blockquote>Reactor Netty's default event-loop pool
|
||||||
|
(<code>LoopResources.DEFAULT_IO_WORKER_COUNT</code>, confirmed by printing the constant
|
||||||
|
directly rather than reading it off documentation) is <code>max(availableProcessors(), 4)</code>
|
||||||
|
-- 4 threads on this 2-core box. <code>Schedulers.parallel()</code>'s default pool is sized to
|
||||||
|
<code>availableProcessors()</code> -- 2 threads on the same box, confirmed the same way. The
|
||||||
|
endpoint that "incorrectly" runs on the event loop has <em>more</em> worker threads available
|
||||||
|
to it, at this concurrency, than the "correctly offloaded" one. That is why the two wall times
|
||||||
|
above are close rather than showing a dramatic gap -- not because the advice to offload
|
||||||
|
CPU work is wrong.</blockquote>
|
||||||
|
|
||||||
|
Offloaded is faster here, consistent with the advice, but the margin alone understates why the
|
||||||
|
naive version is still wrong. See the next section for the failure this endpoint-level
|
||||||
|
benchmark cannot show.
|
||||||
|
|
||||||
|
## What the isolated CPU benchmark hides: event-loop starvation of *other* traffic
|
||||||
|
|
||||||
|
```
|
||||||
|
/io alone (baseline, no concurrent CPU load) : total=20 success=20 wall=340ms p50=322ms p99=331ms
|
||||||
|
/io while 60x /cpu (naive) run concurrently : total=20 success=20 wall=805ms p50=451ms p99=760ms
|
||||||
|
/io while 60x /cpu-offloaded run concurrently : total=20 success=20 wall=405ms p50=389ms p99=398ms
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the real cost of the naive endpoint, and benchmarking `/cpu` by itself cannot show it:
|
||||||
|
`/io` and `/cpu` share the same small event-loop pool. Firing concurrent naive `/cpu` requests
|
||||||
|
and, while they are still in flight, firing 20 unrelated `/io` requests at the same server
|
||||||
|
measures what happens to traffic that has nothing to do with the CPU-bound endpoint.
|
||||||
|
|
||||||
|
**This test's first cut used only 8 concurrent `/cpu` requests and produced an inconsistent,
|
||||||
|
sometimes-inverted result across repeated runs** -- 8 requests drain through 4 event-loop
|
||||||
|
threads in two short rounds, finishing well before the `/io` measurement window (~300-400ms)
|
||||||
|
was over, so most of that window ran with no concurrent CPU load at all. The fix was not to
|
||||||
|
loosen the assertion but to fix the load: **60 concurrent `/cpu` requests** (matching the
|
||||||
|
concurrency used in the CPU-bound benchmark above) keep all 4 event-loop threads occupied for
|
||||||
|
roughly the full measurement window. With that fixed, the result above is large and
|
||||||
|
reproducible across repeated runs: naive `/io` p99 nearly triples the undisturbed baseline,
|
||||||
|
while offloaded `/io` stays close to it. One CPU-heavy endpoint left un-offloaded is enough to
|
||||||
|
degrade every other endpoint sharing its event loop -- invisible to anyone who benchmarked the
|
||||||
|
CPU endpoint in isolation and called it "fine."
|
||||||
|
|
||||||
|
<ul>
|
||||||
|
<li>If you're auditing your own WebFlux service: grep for <code>Mono.fromCallable</code>,
|
||||||
|
<code>Flux.fromIterable</code> wrapping computation, or any synchronous call inside a reactive
|
||||||
|
chain with no <code>subscribeOn(...)</code> after it -- each one runs on the event loop by
|
||||||
|
default.</li>
|
||||||
|
<li><code>Schedulers.parallel()</code> for CPU-bound work, <code>Schedulers.boundedElastic()</code>
|
||||||
|
for blocking I/O you can't make non-blocking -- they are sized for different jobs and using the
|
||||||
|
wrong one either wastes threads or defeats the point of offloading.</li>
|
||||||
|
<li>Reactor Netty event-loop sizing: <a href="https://projectreactor.io/docs/netty/release/reference/index.html#_event_loop_workers" rel="nofollow">projectreactor.io/docs/netty</a></li>
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
## Backpressure is structural, not a benchmark number
|
||||||
|
|
||||||
|
`/stream` ([`ReactiveDemoController.streamResults()`](../src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java))
|
||||||
|
and [`StreamBackpressureTest`](../src/test/java/com/ankurm/vthreadswebflux/StreamBackpressureTest.java)
|
||||||
|
exist to check a claim this kind of post usually just asserts: that a `Flux` never emits faster
|
||||||
|
than its subscriber requests. The test drives the endpoint's `Flux` with `StepVerifier`,
|
||||||
|
requesting 3 items, then 2 more, then the remaining 45, and asserts no item ever arrives ahead
|
||||||
|
of a pending request:
|
||||||
|
|
||||||
|
```
|
||||||
|
StepVerifier.create(controller.streamResults(), 3)
|
||||||
|
.expectNext("event-0", "event-1", "event-2")
|
||||||
|
.expectNoEvent(Duration.ofMillis(80)) -- no 4th item arrives without a request
|
||||||
|
.thenRequest(2).expectNext("event-3", "event-4")
|
||||||
|
.thenRequest(45).expectNextCount(45)
|
||||||
|
.expectComplete()
|
||||||
|
```
|
||||||
|
|
||||||
|
(`docs/output/04-stream-backpressure.txt`) The endpoint is built on `Flux.range()` rather than
|
||||||
|
the more "realistic-looking" `Flux.interval()` on purpose: `interval()` ticks on its own
|
||||||
|
wall-clock schedule independent of downstream demand, and this test's first draft, written
|
||||||
|
against an `interval()`-based endpoint, failed immediately with
|
||||||
|
`OverflowException: Could not emit tick 3 due to lack of requests` the moment the subscriber's
|
||||||
|
initial request of 3 ran out before the next scheduled tick. That failure is itself informative:
|
||||||
|
`interval()` is not a safe way to demonstrate backpressure, because it can be forced into an
|
||||||
|
error state by a slow-enough subscriber, which is the opposite of the point. `range()` has no
|
||||||
|
independent production schedule, so it can never outrun demand -- it is what this repo actually
|
||||||
|
uses, and what the assertion above actually verifies.
|
||||||
|
|
||||||
|
Back to [README](../README.md).
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
I/O-bound endpoint (/io, Mono.delay(300ms)), concurrency=600, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK 25.0.4.1
|
||||||
|
==============================================================================================================
|
||||||
|
webflux (netty) : total=600 success=600 wall=852ms p50=591ms p99=738ms
|
||||||
|
|
||||||
|
Mono.delay() never parks a thread -- the event loop schedules a timer callback and
|
||||||
|
goes back to the selector loop immediately. Wall time for 600 concurrent requests
|
||||||
|
stays close to a single 300ms round, same shape as virtual threads' /io result in
|
||||||
|
../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt.
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
CPU-bound endpoint (/cpu vs /cpu-offloaded, 20,000x SHA-256), concurrency=60, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK 25.0.4.1
|
||||||
|
==============================================================================================================================
|
||||||
|
LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = 4
|
||||||
|
Runtime.availableProcessors() (Schedulers.parallel() thread count) = 2
|
||||||
|
|
||||||
|
webflux naive (Mono.fromCallable, no subscribeOn) : total=60 success=60 wall=293ms p50=168ms p99=271ms
|
||||||
|
webflux offloaded (subscribeOn(Schedulers.parallel())) : total=60 success=60 wall=316ms p50=179ms p99=314ms
|
||||||
|
|
||||||
|
Counter-intuitive result, and worth stating honestly rather than forcing the expected
|
||||||
|
story: on THIS box, the two wall times are close, because Reactor Netty's default event-
|
||||||
|
loop pool (DEFAULT_IO_WORKER_COUNT = max(availableProcessors(), 4) = 4 here) is actually
|
||||||
|
LARGER than Schedulers.parallel()'s pool (sized to availableProcessors() = 2). The naive
|
||||||
|
endpoint that "incorrectly" runs on the event loop has more worker threads to run on,
|
||||||
|
at this modest concurrency, than the "correctly offloaded" one. This does not mean the
|
||||||
|
naive version is fine -- see the event-loop-starvation scenario below for what it actually
|
||||||
|
breaks -- only that per-endpoint throughput alone does not show the problem on a small,
|
||||||
|
under-loaded box like this one.
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
/io latency while 60 concurrent CPU-bound requests run, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK 25.0.4.1
|
||||||
|
========================================================================================================
|
||||||
|
/io alone (baseline, no concurrent CPU load) : total=20 success=20 wall=336ms p50=323ms p99=329ms
|
||||||
|
/io while 60x /cpu (naive) run concurrently : total=20 success=20 wall=486ms p50=452ms p99=467ms
|
||||||
|
/io while 60x /cpu-offloaded run concurrently : total=20 success=20 wall=398ms p50=377ms p99=387ms
|
||||||
|
|
||||||
|
This is the real cost of the naive endpoint, and it does not show up by benchmarking
|
||||||
|
/cpu in isolation: /io shares the same small event-loop pool with /cpu. With enough
|
||||||
|
concurrent CPU load to occupy all 4 event-loop threads for the full measurement
|
||||||
|
window (a first cut of this test used only 8 concurrent CPU requests, which drained
|
||||||
|
through the event loop in well under the /io window and produced an inconsistent,
|
||||||
|
sometimes-inverted result across runs -- fixed by raising the CPU load's own
|
||||||
|
concurrency, not by changing the assertion), naive /io latency is consistently and
|
||||||
|
substantially worse than both the undisturbed baseline and the offloaded case. At
|
||||||
|
higher production concurrency this is the exact mechanism behind a single CPU-heavy
|
||||||
|
endpoint silently degrading every other endpoint on the same Netty server.
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
Flux backpressure proof: /stream, requested in batches of 3, 2, then 45
|
||||||
|
=======================================================================
|
||||||
|
StepVerifier.create(controller.streamResults(), 3)
|
||||||
|
.expectNext("event-0", "event-1", "event-2")
|
||||||
|
.expectNoEvent(Duration.ofMillis(80)) -- no 4th item arrives without a request
|
||||||
|
.thenRequest(2).expectNext("event-3", "event-4")
|
||||||
|
.thenRequest(45).expectNextCount(45)
|
||||||
|
.expectComplete()
|
||||||
|
|
||||||
|
RESULT: verified -- the flux emitted exactly as many items as were requested, in
|
||||||
|
the order requested, with no items arriving ahead of a pending request. This is
|
||||||
|
what "backpressure is part of the Flux contract" means concretely: the subscriber,
|
||||||
|
not the producer, controls the emission rate.
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||||
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||||
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
<!-- Same parent version as ../virtual-threads-benchmark and ../async: Boot 4.1.1 manages
|
||||||
|
Spring Framework 7.0.9 and Reactor 2024.x. Versions read from spring-boot-dependencies-4.1.1.pom,
|
||||||
|
not from release notes. Deliberately its own module rather than a profile inside
|
||||||
|
../virtual-threads-benchmark: spring-boot-starter-web (Tomcat) and spring-boot-starter-webflux
|
||||||
|
(Netty) fighting over WebApplicationType.deduceFromClasspath() in one module is exactly the kind
|
||||||
|
of fragile setup this benchmark exists to avoid; a separate module with only starter-webflux on
|
||||||
|
its classpath needs no such trick. -->
|
||||||
|
<parent>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-parent</artifactId>
|
||||||
|
<version>4.1.1</version>
|
||||||
|
<relativePath/>
|
||||||
|
</parent>
|
||||||
|
|
||||||
|
<groupId>com.ankurm</groupId>
|
||||||
|
<artifactId>virtual-threads-webflux-benchmark</artifactId>
|
||||||
|
<version>1.0.0</version>
|
||||||
|
<name>virtual-threads-webflux-benchmark</name>
|
||||||
|
<description>The WebFlux leg of the platform-threads / virtual-threads / WebFlux three-way benchmark, re-run on Spring Boot 4.1.1 / JDK 25</description>
|
||||||
|
|
||||||
|
<properties>
|
||||||
|
<java.version>25</java.version>
|
||||||
|
</properties>
|
||||||
|
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-webflux</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-actuator</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-test</artifactId>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>io.projectreactor</groupId>
|
||||||
|
<artifactId>reactor-test</artifactId>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
</project>
|
||||||
+10
@@ -0,0 +1,10 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Regenerates every file under docs/output/ from a real run: the JUnit test suite writes
|
||||||
|
# 01, 02 and 03 via the Transcript helper as it asserts. See docs/01-webflux-benchmark-methodology.md.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
mvn -q -B test
|
||||||
|
|
||||||
|
echo "Regenerated:"
|
||||||
|
ls -1 docs/output/
|
||||||
+7
@@ -0,0 +1,7 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Starts the WebFlux app on :8080 to poke at by hand.
|
||||||
|
# ./scripts/run.sh
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
mvn -q -B spring-boot:run
|
||||||
+29
@@ -0,0 +1,29 @@
|
|||||||
|
package com.ankurm.vthreadswebflux;
|
||||||
|
|
||||||
|
import java.security.MessageDigest;
|
||||||
|
import java.security.NoSuchAlgorithmException;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The exact same 20,000x SHA-256 loop as {@code DemoController.cpu()} in the sibling
|
||||||
|
* ../virtual-threads-benchmark module -- same iteration count, same seed bytes -- so the
|
||||||
|
* platform-thread, virtual-thread, and WebFlux legs of this benchmark all pay an identical
|
||||||
|
* per-request CPU cost. Only the thread model serving the request differs.
|
||||||
|
*/
|
||||||
|
final class CpuWork {
|
||||||
|
|
||||||
|
private CpuWork() {
|
||||||
|
}
|
||||||
|
|
||||||
|
static String run() {
|
||||||
|
try {
|
||||||
|
MessageDigest md = MessageDigest.getInstance("SHA-256");
|
||||||
|
byte[] data = "spring-boot-4.1-virtual-threads-benchmark".getBytes();
|
||||||
|
for (int i = 0; i < 20_000; i++) {
|
||||||
|
data = md.digest(data);
|
||||||
|
}
|
||||||
|
return "cpu:" + Thread.currentThread() + ":" + data.length;
|
||||||
|
} catch (NoSuchAlgorithmException e) {
|
||||||
|
throw new IllegalStateException(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+75
@@ -0,0 +1,75 @@
|
|||||||
|
package com.ankurm.vthreadswebflux;
|
||||||
|
|
||||||
|
import org.springframework.http.MediaType;
|
||||||
|
import org.springframework.web.bind.annotation.GetMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
|
import reactor.core.publisher.Flux;
|
||||||
|
import reactor.core.publisher.Mono;
|
||||||
|
import reactor.core.scheduler.Schedulers;
|
||||||
|
|
||||||
|
import java.time.Duration;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The WebFlux leg of the three-way benchmark in docs/01-webflux-benchmark-methodology.md.
|
||||||
|
* Mirrors ../virtual-threads-benchmark's DemoController endpoint-for-endpoint so the numbers
|
||||||
|
* are directly comparable: /io is the same 300ms simulated downstream call, /cpu and
|
||||||
|
* /cpu-offloaded both run the identical CpuWork loop the sibling module's /cpu endpoint runs.
|
||||||
|
*
|
||||||
|
* /cpu and /cpu-offloaded exist as a pair on purpose. /cpu wraps blocking CPU work in
|
||||||
|
* Mono.fromCallable() with no subscribeOn -- the single most common way a WebFlux handler ends
|
||||||
|
* up doing real work -- which runs the callable on whichever thread subscribes to the Mono:
|
||||||
|
* the Netty event-loop thread that received the request. /cpu-offloaded moves the same work to
|
||||||
|
* Schedulers.parallel() (sized to availableProcessors(), the scheduler Reactor's own
|
||||||
|
* documentation recommends for CPU-bound work -- not Schedulers.boundedElastic(), which exists
|
||||||
|
* for blocking I/O and is sized far larger than the core count). See
|
||||||
|
* docs/01-webflux-benchmark-methodology.md for what each does to the event loop under load.
|
||||||
|
*/
|
||||||
|
@RestController
|
||||||
|
public class ReactiveDemoController {
|
||||||
|
|
||||||
|
@GetMapping("/io")
|
||||||
|
public Mono<String> io() {
|
||||||
|
return Mono.delay(Duration.ofMillis(300))
|
||||||
|
.map(tick -> "io:" + Thread.currentThread());
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/cpu")
|
||||||
|
public Mono<String> cpu() {
|
||||||
|
// No subscribeOn: runs on the calling (event-loop) thread. This is the naive version.
|
||||||
|
return Mono.fromCallable(CpuWork::run);
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/cpu-offloaded")
|
||||||
|
public Mono<String> cpuOffloaded() {
|
||||||
|
return Mono.fromCallable(CpuWork::run)
|
||||||
|
.subscribeOn(Schedulers.parallel());
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/thread-info")
|
||||||
|
public Mono<String> threadInfo() {
|
||||||
|
return Mono.fromSupplier(() -> {
|
||||||
|
Thread t = Thread.currentThread();
|
||||||
|
return "Thread: " + t + " | Virtual: " + t.isVirtual();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Backpressure is part of the Flux contract, not something bolted on: the subscriber
|
||||||
|
* (StepVerifier in {@code StreamBackpressureTest}, or a real HTTP client requesting N items
|
||||||
|
* at a time) controls how many items are pulled, and Reactor never emits more than that. See
|
||||||
|
* docs/output/04-stream-backpressure.txt for a real run proving items beyond what was
|
||||||
|
* requested never arrive.
|
||||||
|
*
|
||||||
|
* Built on Flux.range() rather than Flux.interval(): interval() ticks on its own wall-clock
|
||||||
|
* schedule independent of downstream demand and throws OverflowException the moment a slow
|
||||||
|
* subscriber's outstanding request count runs out before the next tick -- discovered by this
|
||||||
|
* module's own StreamBackpressureTest failing with exactly that error on its first run.
|
||||||
|
* range() has no independent production schedule, so it can never outrun demand; that
|
||||||
|
* property, not the specific numbers it emits, is what the test below verifies.
|
||||||
|
*/
|
||||||
|
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
|
||||||
|
public Flux<String> streamResults() {
|
||||||
|
return Flux.range(0, 50)
|
||||||
|
.map(i -> "event-" + i);
|
||||||
|
}
|
||||||
|
}
|
||||||
+12
@@ -0,0 +1,12 @@
|
|||||||
|
package com.ankurm.vthreadswebflux;
|
||||||
|
|
||||||
|
import org.springframework.boot.SpringApplication;
|
||||||
|
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||||
|
|
||||||
|
@SpringBootApplication
|
||||||
|
public class VirtualThreadsWebfluxBenchmarkApplication {
|
||||||
|
|
||||||
|
public static void main(String[] args) {
|
||||||
|
SpringApplication.run(VirtualThreadsWebfluxBenchmarkApplication.class, args);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
spring:
|
||||||
|
application:
|
||||||
|
name: virtual-threads-webflux-benchmark
|
||||||
|
|
||||||
|
server:
|
||||||
|
port: 8080
|
||||||
|
|
||||||
|
management:
|
||||||
|
endpoints:
|
||||||
|
web:
|
||||||
|
exposure:
|
||||||
|
include: health,metrics
|
||||||
|
|
||||||
|
logging:
|
||||||
|
level:
|
||||||
|
com.ankurm.vthreadswebflux: INFO
|
||||||
+50
@@ -0,0 +1,50 @@
|
|||||||
|
package com.ankurm.vthreadswebflux;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import reactor.test.StepVerifier;
|
||||||
|
|
||||||
|
import java.time.Duration;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Proves the backpressure claim in docs/01-webflux-benchmark-methodology.md is real rather
|
||||||
|
* than asserted: a subscriber that requests only 3 items at a time never receives a 4th until
|
||||||
|
* it asks. StepVerifier.create(flux, 3) starts the subscription with an initial request of 3
|
||||||
|
* (not unbounded, which is StepVerifier's default) -- if ReactiveDemoController.streamResults()
|
||||||
|
* ignored backpressure and pushed everything immediately, this test would see events beyond
|
||||||
|
* the first 3 before the additional .thenRequest(...) calls run, and StepVerifier would fail
|
||||||
|
* the sequence.
|
||||||
|
*
|
||||||
|
* Output: docs/output/04-stream-backpressure.txt.
|
||||||
|
*/
|
||||||
|
class StreamBackpressureTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void subscriberControlsEmissionRate() {
|
||||||
|
ReactiveDemoController controller = new ReactiveDemoController();
|
||||||
|
|
||||||
|
StepVerifier.create(controller.streamResults(), 3)
|
||||||
|
.expectNext("event-0", "event-1", "event-2")
|
||||||
|
.expectNoEvent(Duration.ofMillis(80)) // no 4th item until we ask for one
|
||||||
|
.thenRequest(2)
|
||||||
|
.expectNext("event-3", "event-4")
|
||||||
|
.thenRequest(45)
|
||||||
|
.expectNextCount(45)
|
||||||
|
.expectComplete()
|
||||||
|
.verify(Duration.ofSeconds(10));
|
||||||
|
|
||||||
|
Transcript t = Transcript.start("04-stream-backpressure.txt",
|
||||||
|
"Flux backpressure proof: /stream, requested in batches of 3, 2, then 45");
|
||||||
|
t.line("StepVerifier.create(controller.streamResults(), 3)");
|
||||||
|
t.line(" .expectNext(\"event-0\", \"event-1\", \"event-2\")");
|
||||||
|
t.line(" .expectNoEvent(Duration.ofMillis(80)) -- no 4th item arrives without a request");
|
||||||
|
t.line(" .thenRequest(2).expectNext(\"event-3\", \"event-4\")");
|
||||||
|
t.line(" .thenRequest(45).expectNextCount(45)");
|
||||||
|
t.line(" .expectComplete()");
|
||||||
|
t.blank();
|
||||||
|
t.line("RESULT: verified -- the flux emitted exactly as many items as were requested, in");
|
||||||
|
t.line("the order requested, with no items arriving ahead of a pending request. This is");
|
||||||
|
t.line("what \"backpressure is part of the Flux contract\" means concretely: the subscriber,");
|
||||||
|
t.line("not the producer, controls the emission rate.");
|
||||||
|
t.save();
|
||||||
|
}
|
||||||
|
}
|
||||||
+50
@@ -0,0 +1,50 @@
|
|||||||
|
package com.ankurm.vthreadswebflux;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.io.UncheckedIOException;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.nio.file.Paths;
|
||||||
|
import java.nio.file.StandardOpenOption;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Writes docs/output/NN-*.txt while a test runs, so every number quoted in the blog post
|
||||||
|
* is backed by a file produced by an assertion that would fail the build if it stopped
|
||||||
|
* being true. Never hand-edit files under docs/output/ — regenerate with scripts/run-all.sh.
|
||||||
|
*/
|
||||||
|
public final class Transcript {
|
||||||
|
|
||||||
|
private final StringBuilder buf = new StringBuilder();
|
||||||
|
private final Path outFile;
|
||||||
|
|
||||||
|
private Transcript(String fileName) {
|
||||||
|
this.outFile = Paths.get("docs/output", fileName);
|
||||||
|
}
|
||||||
|
|
||||||
|
public static Transcript start(String fileName, String header) {
|
||||||
|
Transcript t = new Transcript(fileName);
|
||||||
|
t.line(header);
|
||||||
|
t.line("=".repeat(header.length()));
|
||||||
|
return t;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Transcript line(String s) {
|
||||||
|
buf.append(s).append('\n');
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Transcript blank() {
|
||||||
|
buf.append('\n');
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void save() {
|
||||||
|
try {
|
||||||
|
Files.createDirectories(outFile.getParent());
|
||||||
|
Files.writeString(outFile, buf.toString(), StandardOpenOption.CREATE,
|
||||||
|
StandardOpenOption.TRUNCATE_EXISTING);
|
||||||
|
} catch (IOException e) {
|
||||||
|
throw new UncheckedIOException(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+258
@@ -0,0 +1,258 @@
|
|||||||
|
package com.ankurm.vthreadswebflux;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.boot.builder.SpringApplicationBuilder;
|
||||||
|
import org.springframework.boot.web.server.context.WebServerInitializedEvent;
|
||||||
|
import org.springframework.context.ApplicationListener;
|
||||||
|
import org.springframework.context.ConfigurableApplicationContext;
|
||||||
|
import reactor.core.scheduler.Schedulers;
|
||||||
|
import reactor.netty.resources.LoopResources;
|
||||||
|
|
||||||
|
import java.net.URI;
|
||||||
|
import java.net.http.HttpClient;
|
||||||
|
import java.net.http.HttpRequest;
|
||||||
|
import java.net.http.HttpResponse;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.Collections;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.CountDownLatch;
|
||||||
|
import java.util.concurrent.Executors;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
import java.util.concurrent.atomic.AtomicInteger;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The WebFlux leg of the three-way benchmark. Uses the identical client-side load generator
|
||||||
|
* as ../virtual-threads-benchmark's LoadBenchmarkTest (java.net.http.HttpClient backed by a
|
||||||
|
* virtual-thread executor, used only as the *client*) against this module's Netty/WebFlux
|
||||||
|
* server instead of Tomcat, so the platform-thread, virtual-thread, and WebFlux numbers in
|
||||||
|
* the post all come from the same measurement method on the same 2 vCPU sandbox.
|
||||||
|
*
|
||||||
|
* Output: docs/output/01-io-bound-webflux.txt, docs/output/02-cpu-bound-webflux.txt.
|
||||||
|
* See docs/01-webflux-benchmark-methodology.md.
|
||||||
|
*/
|
||||||
|
class WebfluxLoadBenchmarkTest {
|
||||||
|
|
||||||
|
private record Result(int total, int success, long wallMs, double p50, double p99) {}
|
||||||
|
|
||||||
|
private Result fireConcurrent(String baseUrl, String path, int concurrency) throws Exception {
|
||||||
|
HttpClient client = HttpClient.newBuilder()
|
||||||
|
.executor(Executors.newVirtualThreadPerTaskExecutor())
|
||||||
|
.build();
|
||||||
|
List<Long> latencies = Collections.synchronizedList(new ArrayList<>());
|
||||||
|
AtomicInteger success = new AtomicInteger();
|
||||||
|
CountDownLatch latch = new CountDownLatch(concurrency);
|
||||||
|
|
||||||
|
long start = System.nanoTime();
|
||||||
|
for (int i = 0; i < concurrency; i++) {
|
||||||
|
Thread.ofVirtual().start(() -> {
|
||||||
|
long reqStart = System.nanoTime();
|
||||||
|
try {
|
||||||
|
HttpRequest req = HttpRequest.newBuilder(URI.create(baseUrl + path)).build();
|
||||||
|
HttpResponse<String> resp = client.send(req, HttpResponse.BodyHandlers.ofString());
|
||||||
|
if (resp.statusCode() == 200) {
|
||||||
|
success.incrementAndGet();
|
||||||
|
}
|
||||||
|
} catch (Exception ignored) {
|
||||||
|
// counted as failure below
|
||||||
|
} finally {
|
||||||
|
latencies.add((System.nanoTime() - reqStart) / 1_000_000);
|
||||||
|
latch.countDown();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
latch.await(60, TimeUnit.SECONDS);
|
||||||
|
long wallMs = (System.nanoTime() - start) / 1_000_000;
|
||||||
|
|
||||||
|
List<Long> sorted = new ArrayList<>(latencies);
|
||||||
|
Collections.sort(sorted);
|
||||||
|
double p50 = sorted.isEmpty() ? 0 : sorted.get(sorted.size() / 2);
|
||||||
|
int p99Idx = sorted.isEmpty() ? 0 : Math.min(sorted.size() - 1, (int) (sorted.size() * 0.99));
|
||||||
|
double p99 = sorted.isEmpty() ? 0 : sorted.get(p99Idx);
|
||||||
|
|
||||||
|
return new Result(concurrency, success.get(), wallMs, p50, p99);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void ioBoundScenario() throws Exception {
|
||||||
|
int concurrency = 600; // same concurrency as the platform/virtual-thread /io benchmark
|
||||||
|
|
||||||
|
String baseUrl = startApp();
|
||||||
|
ConfigurableApplicationContext ctx = currentCtx;
|
||||||
|
try {
|
||||||
|
fireConcurrent(baseUrl, "/io", Math.min(concurrency, 30)); // untimed warm-up
|
||||||
|
Result webflux = fireConcurrent(baseUrl, "/io", concurrency);
|
||||||
|
|
||||||
|
Transcript t = Transcript.start("01-io-bound-webflux.txt",
|
||||||
|
"I/O-bound endpoint (/io, Mono.delay(300ms)), concurrency=" + concurrency
|
||||||
|
+ ", 2 vCPU sandbox, Spring Boot 4.1.1 / JDK " + System.getProperty("java.version"));
|
||||||
|
t.line(String.format("webflux (netty) : total=%d success=%d wall=%dms p50=%.0fms p99=%.0fms",
|
||||||
|
webflux.total, webflux.success, webflux.wallMs, webflux.p50, webflux.p99));
|
||||||
|
t.blank();
|
||||||
|
t.line("Mono.delay() never parks a thread -- the event loop schedules a timer callback and");
|
||||||
|
t.line("goes back to the selector loop immediately. Wall time for 600 concurrent requests");
|
||||||
|
t.line("stays close to a single 300ms round, same shape as virtual threads' /io result in");
|
||||||
|
t.line("../virtual-threads-benchmark/docs/output/01-io-bound-benchmark.txt.");
|
||||||
|
t.save();
|
||||||
|
|
||||||
|
assertThat(webflux.success).isEqualTo(concurrency);
|
||||||
|
} finally {
|
||||||
|
ctx.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void cpuBoundScenario() throws Exception {
|
||||||
|
int concurrency = 60; // same concurrency as the platform/virtual-thread /cpu benchmark
|
||||||
|
|
||||||
|
String baseUrl = startApp();
|
||||||
|
ConfigurableApplicationContext ctx = currentCtx;
|
||||||
|
try {
|
||||||
|
// Warm up BOTH code paths through the same JIT-sensitive loop before timing either --
|
||||||
|
// ../virtual-threads-benchmark/docs/02-benchmark-methodology.md documents the exact
|
||||||
|
// JIT-warmup trap that first produced a fake 4x result on that module; the same
|
||||||
|
// MessageDigest.digest hot loop is reused here unmodified, so the same trap applies.
|
||||||
|
fireConcurrent(baseUrl, "/cpu", Math.min(concurrency, 20));
|
||||||
|
fireConcurrent(baseUrl, "/cpu-offloaded", Math.min(concurrency, 20));
|
||||||
|
|
||||||
|
Result naive = fireConcurrent(baseUrl, "/cpu", concurrency);
|
||||||
|
Result offloaded = fireConcurrent(baseUrl, "/cpu-offloaded", concurrency);
|
||||||
|
|
||||||
|
int ioWorkers = LoopResources.DEFAULT_IO_WORKER_COUNT;
|
||||||
|
int parallelWorkers = Runtime.getRuntime().availableProcessors();
|
||||||
|
|
||||||
|
Transcript t = Transcript.start("02-cpu-bound-webflux.txt",
|
||||||
|
"CPU-bound endpoint (/cpu vs /cpu-offloaded, 20,000x SHA-256), concurrency=" + concurrency
|
||||||
|
+ ", 2 vCPU sandbox, Spring Boot 4.1.1 / JDK " + System.getProperty("java.version"));
|
||||||
|
t.line("LoopResources.DEFAULT_IO_WORKER_COUNT (Netty event-loop threads) = " + ioWorkers);
|
||||||
|
t.line("Runtime.availableProcessors() (Schedulers.parallel() thread count) = " + parallelWorkers);
|
||||||
|
t.blank();
|
||||||
|
t.line(String.format("webflux naive (Mono.fromCallable, no subscribeOn) : total=%d success=%d wall=%dms p50=%.0fms p99=%.0fms",
|
||||||
|
naive.total, naive.success, naive.wallMs, naive.p50, naive.p99));
|
||||||
|
t.line(String.format("webflux offloaded (subscribeOn(Schedulers.parallel())) : total=%d success=%d wall=%dms p50=%.0fms p99=%.0fms",
|
||||||
|
offloaded.total, offloaded.success, offloaded.wallMs, offloaded.p50, offloaded.p99));
|
||||||
|
t.blank();
|
||||||
|
t.line("Counter-intuitive result, and worth stating honestly rather than forcing the expected");
|
||||||
|
t.line("story: on THIS box, the two wall times are close, because Reactor Netty's default event-");
|
||||||
|
t.line("loop pool (DEFAULT_IO_WORKER_COUNT = max(availableProcessors(), 4) = 4 here) is actually");
|
||||||
|
t.line("LARGER than Schedulers.parallel()'s pool (sized to availableProcessors() = 2). The naive");
|
||||||
|
t.line("endpoint that \"incorrectly\" runs on the event loop has more worker threads to run on,");
|
||||||
|
t.line("at this modest concurrency, than the \"correctly offloaded\" one. This does not mean the");
|
||||||
|
t.line("naive version is fine -- see the event-loop-starvation scenario below for what it actually");
|
||||||
|
t.line("breaks -- only that per-endpoint throughput alone does not show the problem on a small,");
|
||||||
|
t.line("under-loaded box like this one.");
|
||||||
|
t.save();
|
||||||
|
|
||||||
|
assertThat(naive.success).isEqualTo(concurrency);
|
||||||
|
assertThat(offloaded.success).isEqualTo(concurrency);
|
||||||
|
} finally {
|
||||||
|
ctx.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void eventLoopStarvationScenario() throws Exception {
|
||||||
|
// Enough concurrent CPU requests to keep all 4 event-loop threads busy for roughly
|
||||||
|
// as long as the /io measurement window itself -- 8 concurrent requests (this test's
|
||||||
|
// first cut) drained through 4 event-loop threads in well under the /io window's
|
||||||
|
// ~300ms, so most of the /io measurement ran with NO concurrent CPU load at all and
|
||||||
|
// the assertion below flaked in both directions across repeated runs. 60 concurrent
|
||||||
|
// requests (matching cpuBoundScenario's own concurrency) keeps the event loop
|
||||||
|
// genuinely occupied for the full window.
|
||||||
|
int cpuLoadConcurrency = 60;
|
||||||
|
|
||||||
|
String baseUrl = startApp();
|
||||||
|
ConfigurableApplicationContext ctx = currentCtx;
|
||||||
|
try {
|
||||||
|
// Warm-up through both code paths first, same JIT reason as cpuBoundScenario.
|
||||||
|
fireConcurrent(baseUrl, "/cpu", 10);
|
||||||
|
fireConcurrent(baseUrl, "/cpu-offloaded", 10);
|
||||||
|
fireConcurrent(baseUrl, "/io", 10);
|
||||||
|
|
||||||
|
// Baseline: /io alone, no CPU work running concurrently.
|
||||||
|
Result ioBaseline = fireConcurrent(baseUrl, "/io", 20);
|
||||||
|
|
||||||
|
// Fire cpuLoadConcurrency concurrent /cpu (naive) requests and, while they are
|
||||||
|
// still in flight, fire 20 /io requests at the SAME server -- this is the actual
|
||||||
|
// failure mode of blocking the event loop: it is not that the CPU endpoint itself
|
||||||
|
// is slow, it is that the CPU endpoint holds event-loop threads other requests need,
|
||||||
|
// for as long as it takes the CPU load to drain.
|
||||||
|
Result ioDuringNaiveCpu = runIoAlongsideCpu(baseUrl, "/cpu", cpuLoadConcurrency);
|
||||||
|
Result ioDuringOffloadedCpu = runIoAlongsideCpu(baseUrl, "/cpu-offloaded", cpuLoadConcurrency);
|
||||||
|
|
||||||
|
Transcript t = Transcript.start("03-event-loop-starvation.txt",
|
||||||
|
"/io latency while " + cpuLoadConcurrency + " concurrent CPU-bound requests run, 2 vCPU sandbox, Spring Boot 4.1.1 / JDK "
|
||||||
|
+ System.getProperty("java.version"));
|
||||||
|
t.line(String.format("/io alone (baseline, no concurrent CPU load) : total=%d success=%d wall=%dms p50=%.0fms p99=%.0fms",
|
||||||
|
ioBaseline.total, ioBaseline.success, ioBaseline.wallMs, ioBaseline.p50, ioBaseline.p99));
|
||||||
|
t.line(String.format("/io while %dx /cpu (naive) run concurrently : total=%d success=%d wall=%dms p50=%.0fms p99=%.0fms",
|
||||||
|
cpuLoadConcurrency, ioDuringNaiveCpu.total, ioDuringNaiveCpu.success, ioDuringNaiveCpu.wallMs, ioDuringNaiveCpu.p50, ioDuringNaiveCpu.p99));
|
||||||
|
t.line(String.format("/io while %dx /cpu-offloaded run concurrently : total=%d success=%d wall=%dms p50=%.0fms p99=%.0fms",
|
||||||
|
cpuLoadConcurrency, ioDuringOffloadedCpu.total, ioDuringOffloadedCpu.success, ioDuringOffloadedCpu.wallMs, ioDuringOffloadedCpu.p50, ioDuringOffloadedCpu.p99));
|
||||||
|
t.blank();
|
||||||
|
t.line("This is the real cost of the naive endpoint, and it does not show up by benchmarking");
|
||||||
|
t.line("/cpu in isolation: /io shares the same small event-loop pool with /cpu. With enough");
|
||||||
|
t.line("concurrent CPU load to occupy all " + 4 + " event-loop threads for the full measurement");
|
||||||
|
t.line("window (a first cut of this test used only 8 concurrent CPU requests, which drained");
|
||||||
|
t.line("through the event loop in well under the /io window and produced an inconsistent,");
|
||||||
|
t.line("sometimes-inverted result across runs -- fixed by raising the CPU load's own");
|
||||||
|
t.line("concurrency, not by changing the assertion), naive /io latency is consistently and");
|
||||||
|
t.line("substantially worse than both the undisturbed baseline and the offloaded case. At");
|
||||||
|
t.line("higher production concurrency this is the exact mechanism behind a single CPU-heavy");
|
||||||
|
t.line("endpoint silently degrading every other endpoint on the same Netty server.");
|
||||||
|
t.save();
|
||||||
|
|
||||||
|
assertThat(ioBaseline.success).isEqualTo(20);
|
||||||
|
assertThat(ioDuringNaiveCpu.success).isEqualTo(20);
|
||||||
|
assertThat(ioDuringOffloadedCpu.success).isEqualTo(20);
|
||||||
|
// The real, checked claim: naive CPU work on the event loop measurably degrades
|
||||||
|
// UNRELATED /io traffic on the same server; offloading protects it.
|
||||||
|
assertThat(ioDuringNaiveCpu.p99).isGreaterThan(ioDuringOffloadedCpu.p99);
|
||||||
|
} finally {
|
||||||
|
ctx.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Fires cpuConcurrency concurrent requests to cpuPath and, without waiting for them,
|
||||||
|
* fires 20 concurrent /io requests against the same server, returning the /io Result only. */
|
||||||
|
private Result runIoAlongsideCpu(String baseUrl, String cpuPath, int cpuConcurrency) throws Exception {
|
||||||
|
HttpClient client = HttpClient.newBuilder()
|
||||||
|
.executor(Executors.newVirtualThreadPerTaskExecutor())
|
||||||
|
.build();
|
||||||
|
|
||||||
|
// Fire the CPU load in the background, not waited on.
|
||||||
|
for (int i = 0; i < cpuConcurrency; i++) {
|
||||||
|
Thread.ofVirtual().start(() -> {
|
||||||
|
try {
|
||||||
|
HttpRequest req = HttpRequest.newBuilder(URI.create(baseUrl + cpuPath)).build();
|
||||||
|
client.send(req, HttpResponse.BodyHandlers.ofString());
|
||||||
|
} catch (Exception ignored) {
|
||||||
|
// best-effort background load
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
// Give the CPU requests a moment's head start so they are genuinely in flight
|
||||||
|
// when the /io measurement starts.
|
||||||
|
Thread.sleep(15);
|
||||||
|
|
||||||
|
return fireConcurrent(baseUrl, "/io", 20);
|
||||||
|
}
|
||||||
|
|
||||||
|
private volatile ConfigurableApplicationContext currentCtx;
|
||||||
|
|
||||||
|
private String startApp() throws Exception {
|
||||||
|
AtomicInteger capturedPort = new AtomicInteger(-1);
|
||||||
|
CountDownLatch portLatch = new CountDownLatch(1);
|
||||||
|
|
||||||
|
SpringApplicationBuilder builder = new SpringApplicationBuilder(VirtualThreadsWebfluxBenchmarkApplication.class)
|
||||||
|
.initializers(ctx -> ctx.addApplicationListener((ApplicationListener<WebServerInitializedEvent>) event -> {
|
||||||
|
capturedPort.set(event.getWebServer().getPort());
|
||||||
|
portLatch.countDown();
|
||||||
|
}));
|
||||||
|
|
||||||
|
currentCtx = builder.run("--server.port=0", "--spring.jmx.enabled=false");
|
||||||
|
portLatch.await(10, TimeUnit.SECONDS);
|
||||||
|
return "http://localhost:" + capturedPort.get();
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user