asmhatre
f506b01389
Companion module for the rewritten post 'Virtual Threads on Spring Boot 4.1: The Benchmarks, Re-Run, and the Pinning Advice That Expired', retitled and re-benchmarked on Boot 4.1.1 / JDK 25.0.4.1 (the original post was written against Boot 3.4 / JDK 21). Covers: I/O-bound and CPU-bound throughput (platform vs virtual threads, including a JIT-warmup benchmarking bug this build caught and fixed), JEP 491 proof that synchronized no longer pins a virtual thread's carrier across a blocking call as of JDK 24 (obsoleting the old avoid-synchronized advice), proof that -Djdk.tracePinnedThreads=full is inert on JDK 25, and JEP 506's finalized ScopedValue API (JDK 25 GA, no --enable-preview, and a different shape than the old preview API). Kept as its own module rather than a new top-level repository, alongside the existing async/ module, which already has a stronger dual-JDK JEP 491 proof that this module's docs cross-link to instead of duplicating.
4.7 KiB
2. Benchmark methodology and results
Previous: 01-enabling-virtual-threads.md | README | Next: 03-pinning-diagnosis.md
Source: DemoController.java.
Test: LoadBenchmarkTest.java.
Transcripts: docs/output/01-io-bound-benchmark.txt, docs/output/02-cpu-bound-benchmark.txt.
Why this repo doesn't reuse the old post's numbers
The version of this post it replaces benchmarked Boot 3.4.0 / JDK 21.0.3 with wrk (1,000
connections, 30s) on a 4-core/8GB box. Neither wrk nor an equivalent load generator is
available in this repo's build environment, and the environment itself is a 2 vCPU sandbox,
not a 4-core server. Rather than copy the old numbers forward with a changed version number
next to them, this repo builds its own concurrent load generator (java.net.http.HttpClient
backed by a virtual-thread executor, itself only used as the client -- see
fireConcurrent()) and reports what actually happened on this run, on this hardware, honestly
labeled as such.
I/O-bound: /io, Thread.sleep(300)
600 concurrent requests, each a 300ms simulated downstream call:
platform threads : total=600 success=600 wall=1256ms p50=762ms p99=1193ms
virtual threads : total=600 success=600 wall=675ms p50=531ms p99=654ms
600 concurrent requests against Tomcat's default 200-thread platform pool queue in roughly three sequential batches of 200; each batch pays the full 300ms, so total wall time lands around 3 x 300ms plus scheduling overhead -- which is what the platform-thread row shows. Virtual threads create one thread per request and unmount for the duration of the sleep, so wall time stays close to a single 300ms round, regardless of how far past 200 the concurrency goes. The shape matches the original 2025 benchmark; the absolute numbers are this sandbox's, not that server's.
CPU-bound: /cpu, 20,000x SHA-256 per request
This is the scenario worth reading carefully, because the first version of this benchmark was wrong, and the fix is itself worth knowing about.
First cut, concurrency=16, no warm-up round: virtual threads finished 4x faster than
platform threads. That result doesn't make physical sense -- a CPU-bound virtual thread never
unmounts, so it competes for the same physical cores a platform thread would, and should show
no structural advantage. The cause: MessageDigest.digest and the servlet dispatch path get
JIT-compiled per class, not per Spring context, and this benchmark starts a fresh
ApplicationContext for each of the two scenarios inside the same JVM. Platform threads always
ran first; by the time the virtual-thread scenario ran, the hot loop was already JIT-warmed --
a benchmarking artifact with nothing to do with thread model, caught by the numbers being
implausible rather than by inspecting the code.
Fix: an untimed warm-up round (fireConcurrent at a smaller concurrency, discarded)
through the same code path before every timed measurement, in both scenarios.
After the fix, concurrency=60 (chosen deliberately above this box's 2 vCPUs, so both thread models are forced into real core-bound saturation rather than fitting comfortably alongside each other):
platform threads : total=60 success=60 wall=166ms p50=146ms p99=162ms
virtual threads : total=60 success=60 wall=191ms p50=181ms p99=189ms
Close, with platform threads and virtual threads trading the lead by a few percent across
repeated runs (see the test's own run-to-run notes in
LoadBenchmarkTest.java) --
consistent with the "no meaningful difference for CPU-bound work" claim every virtual threads
article makes, but arrived at here by running it, catching a real measurement bug, fixing the
methodology, and re-running it, rather than by assuming the claim was self-evidently true.
If your own CPU-bound benchmark shows virtual threads winning decisively, check your warm-up before you check your thread model. JIT compilation state leaking between two scenarios measured in the same JVM run is an easy way to manufacture a result that isn't real.
- If you're benchmarking your own service: match concurrency to something past your real platform-thread pool size for the I/O case, and past your core count for the CPU case -- otherwise neither test forces the behaviour you're trying to observe.
- JEP 444 (virtual threads, JDK 21): https://openjdk.org/jeps/444 (
rel=nofollow)
Next: 03-pinning-diagnosis.md.