Add virtual-threads-benchmark-webflux: the WebFlux leg of the three-way benchmark
Fixes found during self-correction before publishing: - /stream used Flux.interval(), which ticks on its own wall-clock schedule independent of downstream demand and threw OverflowException under a slow subscriber; switched to Flux.range(), which has no independent production schedule and can never outrun demand. - Single-trial HTTP load tests on this shared sandbox swung by more than 50% run to run (795ms-1247ms observed on the identical /io endpoint back to back) -- large enough to flip which threading model looked faster. Fixed by taking the median of 5 independent trials for the I/O-bound benchmark and the median of 3 for the event-loop-starvation benchmark, rather than reporting a single noisy run as if it were precise. - The event-loop-starvation test's first cut used only 8 concurrent /cpu requests as background load, which drained through the 4 event-loop threads well inside the /io measurement window and produced an inconsistent, sometimes-inverted result across runs; raising to 60 fixed the under-loading problem but still flaked once during verification (372ms vs 374ms p99, a real tie). Final fix: 150 concurrent requests plus the median-of-3 trials above. Also adds StreamBackpressureTest, a StepVerifier proof that the /stream endpoint never emits ahead of its subscriber's outstanding requests, and updates the module's docs to report the de-noised numbers with an explicit methodology note on how they compare to the single-trial platform/virtual- thread numbers reused from a different post.
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
# virtual-threads-webflux-benchmark
|
||||
|
||||
Companion module for the ankurm.com post **"Virtual Threads vs Reactive (WebFlux) vs Platform
|
||||
Threads: Benchmarks and a Decision Framework."** This module is the WebFlux leg of that
|
||||
three-way comparison; the platform-thread and virtual-thread legs live in the sibling module
|
||||
[`../virtual-threads-benchmark`](../virtual-threads-benchmark), built for a different
|
||||
ankurm.com post and reused here rather than re-run, so the platform/virtual numbers quoted in
|
||||
this post are the same numbers, not a second measurement of the same thing.
|
||||
|
||||
Every number below is from a real concurrent load run against a real running embedded Netty
|
||||
server on this JDK, using the identical client-side load generator
|
||||
(`java.net.http.HttpClient`, virtual-thread executor, client-side only) as the sibling module.
|
||||
|
||||
## Versions (verified against `repo1.maven.org` maven-metadata.xml and Spring Boot's own `spring-boot-dependencies` POM, not aggregators)
|
||||
|
||||
| Component | Version | Notes |
|
||||
|---|---|---|
|
||||
| JDK | 25 (Temurin 25.0.4.1+1) | same as `../virtual-threads-benchmark` |
|
||||
| Spring Boot | 4.1.1 | latest GA at time of writing |
|
||||
| Spring Framework | 7.0.9 | latest GA |
|
||||
| Reactor | managed by `spring-boot-dependencies` 4.1.1 | version not pinned directly; see this module's effective POM |
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
./scripts/run-all.sh # regenerates every file in docs/output/ from a real test run
|
||||
./scripts/run.sh # start on :8080
|
||||
```
|
||||
|
||||
Requires JDK 25 and Maven. This module's own benchmark ran on the same 2 vCPU sandbox as
|
||||
`../virtual-threads-benchmark` -- see
|
||||
[docs/01-webflux-benchmark-methodology.md](docs/01-webflux-benchmark-methodology.md) for why
|
||||
that matters and what it means for the numbers below.
|
||||
|
||||
## What's demonstrated where
|
||||
|
||||
| Area | Source | Test | Transcript |
|
||||
|---|---|---|---|
|
||||
| I/O-bound throughput: WebFlux, 600 concurrent, median of 5 trials (single trials swung 50%+ on this box) | [`ReactiveDemoController`](src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java) | [`WebfluxLoadBenchmarkTest`](src/test/java/com/ankurm/vthreadswebflux/WebfluxLoadBenchmarkTest.java) | [`01`](docs/output/01-io-bound-webflux.txt) |
|
||||
| CPU-bound throughput: naive (event loop) vs offloaded (`Schedulers.parallel()`), 60 concurrent -- and why the gap is smaller than expected on this box | same | same | [`02`](docs/output/02-cpu-bound-webflux.txt) |
|
||||
| Event-loop starvation: what the naive CPU endpoint actually costs *other* traffic on the same server, 150 concurrent, median of 3 trials | [`CpuWork`](src/main/java/com/ankurm/vthreadswebflux/CpuWork.java) | same | [`03`](docs/output/03-event-loop-starvation.txt) |
|
||||
| Backpressure is structural: a `Flux` never outruns its subscriber's requests | [`ReactiveDemoController.streamResults()`](src/main/java/com/ankurm/vthreadswebflux/ReactiveDemoController.java) | [`StreamBackpressureTest`](src/test/java/com/ankurm/vthreadswebflux/StreamBackpressureTest.java) | [`04`](docs/output/04-stream-backpressure.txt) |
|
||||
|
||||
## Documentation chapters
|
||||
|
||||
1. [WebFlux benchmark methodology and results](docs/01-webflux-benchmark-methodology.md) --
|
||||
I/O-bound, CPU-bound naive vs offloaded, the event-loop-starvation scenario the isolated CPU
|
||||
benchmark can't show (including a first-cut version of that test that under-loaded the event
|
||||
loop and had to be fixed), a real backpressure proof, and the discovery that single-trial
|
||||
measurements on this sandbox swing 50%+ and had to be replaced with medians of several
|
||||
trials, all on this sandbox's real hardware
|
||||
|
||||
## A note on this module's relationship to `../virtual-threads-benchmark`
|
||||
|
||||
[`../virtual-threads-benchmark`](../virtual-threads-benchmark) already contains a real,
|
||||
re-run platform-thread vs virtual-thread benchmark on this exact hardware and JDK, built for
|
||||
[Virtual Threads on Spring Boot 4.1](https://ankurm.com/leveraging-virtual-threads-in-spring-boot-3-4-building-high-throughput-services/).
|
||||
This module deliberately does not re-measure that comparison -- it reuses those committed
|
||||
transcripts and adds only the WebFlux leg, using the same client-side load-generation method,
|
||||
so the three-way post can quote one consistent measurement approach across all three models
|
||||
instead of stitching together benchmarks run different ways.
|
||||
|
||||
## License
|
||||
|
||||
MIT -- see [LICENSE](LICENSE).
|
||||
Reference in New Issue
Block a user