framework-benchmarks: Spring Boot 4.1.1 vs Quarkus 3.40.1 vs Micronaut 5.2.2, JVM + native, benchmarked
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
target/
|
||||
*.class
|
||||
*.log
|
||||
.idea/
|
||||
*.iml
|
||||
.DS_Store
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Ankur Mhatre
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,99 @@
|
||||
# framework-benchmarks
|
||||
|
||||
Three identical REST APIs - Spring Boot, Quarkus, Micronaut - built JVM and native,
|
||||
and benchmarked for cold-start time, resident memory, and throughput. Companion repo
|
||||
for the ankurm.com post "Spring Boot 4.1 vs Quarkus vs Micronaut: Startup, Memory and
|
||||
Throughput Benchmarked."
|
||||
|
||||
No `docs/` chapter folder here by design - every finding below is written up in full
|
||||
in the post itself, inside per-section `<details>` accordions. This repo carries the
|
||||
runnable code and the raw captured output those sections link to.
|
||||
|
||||
## Versions under test
|
||||
|
||||
| Component | Version | Verified against |
|
||||
|---|---|---|
|
||||
| Spring Boot | 4.1.1 | `spring-boot-dependencies` Maven Central metadata |
|
||||
| Quarkus | 3.40.1 (the 3.40 LTS stream) | `quarkus-bom` Maven Central metadata + quarkus.io/releases |
|
||||
| Micronaut | 5.2.2 | `micronaut-platform` Maven Central metadata |
|
||||
| GraalVM | CE 25.4.4.1.1 (OpenJDK 25.0.4.1.1) | graalvm/graalvm-ce-builds GitHub releases (latest at build time) |
|
||||
| JDK baseline | 25 (current LTS) | all three apps compiled and run on the same GraalVM JDK 25 |
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
# JVM mode
|
||||
cd spring-boot-app && mvn -q package -DskipTests && java -jar target/spring-boot-app.jar # :8081
|
||||
cd quarkus-app && mvn -q package -DskipTests && java -jar target/quarkus-app/quarkus-run.jar # :8082
|
||||
cd micronaut-app && mvn -q package -DskipTests && java -jar target/micronaut-app.jar # :8083
|
||||
|
||||
# Native mode (needs GraalVM's native-image on PATH; budget ~7-8 min per app)
|
||||
cd spring-boot-app && mvn -q -Pnative package -DskipTests && mvn -q -Pnative native:compile && ./target/spring-boot-app
|
||||
cd quarkus-app && mvn -q -Pnative clean package -DskipTests && ./target/quarkus-app-runner
|
||||
cd micronaut-app && mvn -q -Pnative clean package -DskipTests \
|
||||
&& mvn -q -Pnative package org.graalvm.buildtools:native-maven-plugin:compile -DskipTests \
|
||||
&& ./target/micronaut-app
|
||||
```
|
||||
|
||||
`scripts/run-all.sh` does all of the above plus every benchmark, end to end, and
|
||||
regenerates every file in `docs/output/`.
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Framework | JVM port | Build profile for native |
|
||||
|---|---|---|---|
|
||||
| `spring-boot-app` | Spring Boot 4.1.1, Tomcat, Spring MVC | 8081 | `-Pnative`, then explicit `native:compile` (see below) |
|
||||
| `quarkus-app` | Quarkus 3.40.1, Quarkus REST (RESTEasy Reactive), Vert.x | 8082 | `-Pnative` |
|
||||
| `micronaut-app` | Micronaut 5.2.2, Netty, Micronaut Serialization (compile-time, not reflection) | 8083 | `-Pnative` + explicit `native-maven-plugin:compile` (see below) |
|
||||
|
||||
All three expose the identical endpoints, same JSON shapes, same seed data, no database:
|
||||
|
||||
| Method | Path | Behaviour |
|
||||
|---|---|---|
|
||||
| `GET` | `/widgets` | list all widgets (seeded with 5) |
|
||||
| `GET` | `/widgets/{id}` | one widget, 404 if unknown |
|
||||
| `POST` | `/widgets` | create (`{"name": "...", "quantity": N}`), 400 on blank name or negative quantity |
|
||||
| `GET` | `/health` (Micronaut) / `/actuator/health` (Spring Boot) / `/q/health` (Quarkus) | trivial liveness check - deliberately each framework's own normal way to add one, not forced to be identical, since that difference is itself one of the post's findings |
|
||||
|
||||
No database on purpose: it isolates framework/runtime startup overhead from JDBC
|
||||
driver and native-image reflection-metadata concerns, which is a separate, larger
|
||||
topic than this post covers.
|
||||
|
||||
## The two real build failures this repo hit (and the fixes, committed as-is)
|
||||
|
||||
1. **Micronaut silently ignored `application.yml`.** `micronaut-yaml` (which wraps
|
||||
snakeyaml) isn't a transitive dependency of `micronaut-http-server-netty` - without
|
||||
it, Micronaut skips YAML config files with no warning and falls back to every
|
||||
default (here, port 8080 instead of the configured 8083). Fixed by using
|
||||
`application.properties` instead, matching the other two modules. See the post's
|
||||
Micronaut section for the full story.
|
||||
|
||||
2. **Micronaut's native-image build failed four times in a row** on a Netty/logback
|
||||
static-initialization clash before `--initialize-at-run-time=io.netty` (the
|
||||
whole package) fixed it. All four real `native-image` error messages are in
|
||||
`docs/output/06-micronaut-native-build-failures.txt`, verbatim. The working build
|
||||
argument lives in `micronaut-app/pom.xml`'s `native` profile.
|
||||
|
||||
Also note: neither `native-maven-plugin` nor Spring Boot's own native profile
|
||||
auto-binds `native-image` to a Maven phase in a plain multi-module layout the way
|
||||
`mvn -Pnative package` alone implies - both modules need the goal invoked explicitly
|
||||
(`native:compile`, or Micronaut's `native-maven-plugin:compile`). See `scripts/run-all.sh`
|
||||
for the exact commands that actually trigger a native build end to end.
|
||||
|
||||
## Captured output
|
||||
|
||||
| File | What it is |
|
||||
|---|---|
|
||||
| `docs/output/01-startup-memory-jvm.txt` | All 3 frameworks, JVM mode, 4 cold-start runs each, real `measure.py` JSON |
|
||||
| `docs/output/02-throughput-jvm.txt` | All 3 frameworks, JVM mode, 15s/20-concurrency throughput runs |
|
||||
| `docs/output/03-startup-memory-native.txt` | Same as 01, native mode |
|
||||
| `docs/output/04-throughput-native.txt` | Same as 02, native mode |
|
||||
| `docs/output/05-summary.txt` | Consolidated table, hand-written from the four files above |
|
||||
| `docs/output/06-micronaut-native-build-failures.txt` | The four real native-image build attempts for Micronaut, verbatim |
|
||||
|
||||
`scripts/measure.py` and `scripts/throughput.py` are dependency-free Python (stdlib
|
||||
only) - no `wrk`/`hey`/`ab` install required to reproduce any number in this repo.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,18 @@
|
||||
=== Spring Boot JVM startup+memory (4 runs) ===
|
||||
{"label": "spring-boot-jvm", "ready": true, "served_ok": true, "startup_s": 6.117, "rss_kb": 233324, "run": 1, "discarded": true}
|
||||
{"label": "spring-boot-jvm", "ready": true, "served_ok": true, "startup_s": 6.358, "rss_kb": 234664, "run": 2, "discarded": false}
|
||||
{"label": "spring-boot-jvm", "ready": true, "served_ok": true, "startup_s": 6.308, "rss_kb": 236924, "run": 3, "discarded": false}
|
||||
{"label": "spring-boot-jvm", "ready": true, "served_ok": true, "startup_s": 6.287, "rss_kb": 232612, "run": 4, "discarded": false}
|
||||
{"label": "spring-boot-jvm", "summary": true, "runs_kept": 3, "avg_startup_s": 6.318, "avg_rss_kb": 234733.3, "avg_rss_mb": 229.2}
|
||||
=== Quarkus JVM startup+memory (4 runs) ===
|
||||
{"label": "quarkus-jvm", "ready": true, "served_ok": true, "startup_s": 2.173, "rss_kb": 171968, "run": 1, "discarded": true}
|
||||
{"label": "quarkus-jvm", "ready": true, "served_ok": true, "startup_s": 2.084, "rss_kb": 175640, "run": 2, "discarded": false}
|
||||
{"label": "quarkus-jvm", "ready": true, "served_ok": true, "startup_s": 2.19, "rss_kb": 176004, "run": 3, "discarded": false}
|
||||
{"label": "quarkus-jvm", "ready": true, "served_ok": true, "startup_s": 2.282, "rss_kb": 175364, "run": 4, "discarded": false}
|
||||
{"label": "quarkus-jvm", "summary": true, "runs_kept": 3, "avg_startup_s": 2.185, "avg_rss_kb": 175669.3, "avg_rss_mb": 171.6}
|
||||
=== Micronaut JVM startup+memory (4 runs) ===
|
||||
{"label": "micronaut-jvm", "ready": true, "served_ok": true, "startup_s": 2.465, "rss_kb": 183740, "run": 1, "discarded": true}
|
||||
{"label": "micronaut-jvm", "ready": true, "served_ok": true, "startup_s": 2.648, "rss_kb": 184576, "run": 2, "discarded": false}
|
||||
{"label": "micronaut-jvm", "ready": true, "served_ok": true, "startup_s": 2.372, "rss_kb": 183996, "run": 3, "discarded": false}
|
||||
{"label": "micronaut-jvm", "ready": true, "served_ok": true, "startup_s": 2.502, "rss_kb": 184316, "run": 4, "discarded": false}
|
||||
{"label": "micronaut-jvm", "summary": true, "runs_kept": 3, "avg_startup_s": 2.507, "avg_rss_kb": 184296.0, "avg_rss_mb": 180.0}
|
||||
@@ -0,0 +1,42 @@
|
||||
=== Spring Boot JVM throughput (15s, concurrency 20) ===
|
||||
{
|
||||
"label": "spring-boot-jvm",
|
||||
"path": "/widgets",
|
||||
"concurrency": 20,
|
||||
"duration_s": 15.01,
|
||||
"requests_ok": 34158,
|
||||
"requests_err": 0,
|
||||
"requests_per_sec": 2275.7,
|
||||
"latency_ms_mean": 8.78,
|
||||
"latency_ms_p50": 5.6,
|
||||
"latency_ms_p95": 26.83,
|
||||
"latency_ms_p99": 57.06
|
||||
}
|
||||
=== Quarkus JVM throughput (15s, concurrency 20) ===
|
||||
{
|
||||
"label": "quarkus-jvm",
|
||||
"path": "/widgets",
|
||||
"concurrency": 20,
|
||||
"duration_s": 15.01,
|
||||
"requests_ok": 56912,
|
||||
"requests_err": 0,
|
||||
"requests_per_sec": 3791.4,
|
||||
"latency_ms_mean": 5.27,
|
||||
"latency_ms_p50": 4.6,
|
||||
"latency_ms_p95": 10.55,
|
||||
"latency_ms_p99": 19.49
|
||||
}
|
||||
=== Micronaut JVM throughput (15s, concurrency 20) ===
|
||||
{
|
||||
"label": "micronaut-jvm",
|
||||
"path": "/widgets",
|
||||
"concurrency": 20,
|
||||
"duration_s": 15.01,
|
||||
"requests_ok": 64180,
|
||||
"requests_err": 0,
|
||||
"requests_per_sec": 4276.2,
|
||||
"latency_ms_mean": 4.67,
|
||||
"latency_ms_p50": 3.99,
|
||||
"latency_ms_p95": 11.52,
|
||||
"latency_ms_p99": 19.7
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
=== Spring Boot NATIVE startup+memory (4 runs) ===
|
||||
{"label": "spring-boot-native", "ready": true, "served_ok": true, "startup_s": 0.174, "rss_kb": 81420, "run": 1, "discarded": true}
|
||||
{"label": "spring-boot-native", "ready": true, "served_ok": true, "startup_s": 0.133, "rss_kb": 81384, "run": 2, "discarded": false}
|
||||
{"label": "spring-boot-native", "ready": true, "served_ok": true, "startup_s": 0.121, "rss_kb": 81408, "run": 3, "discarded": false}
|
||||
{"label": "spring-boot-native", "ready": true, "served_ok": true, "startup_s": 0.143, "rss_kb": 81408, "run": 4, "discarded": false}
|
||||
{"label": "spring-boot-native", "summary": true, "runs_kept": 3, "avg_startup_s": 0.132, "avg_rss_kb": 81400.0, "avg_rss_mb": 79.5}
|
||||
=== Quarkus NATIVE startup+memory (4 runs) ===
|
||||
{"label": "quarkus-native", "ready": true, "served_ok": true, "startup_s": 0.048, "rss_kb": 34368, "run": 1, "discarded": true}
|
||||
{"label": "quarkus-native", "ready": true, "served_ok": true, "startup_s": 0.035, "rss_kb": 34368, "run": 2, "discarded": false}
|
||||
{"label": "quarkus-native", "ready": true, "served_ok": true, "startup_s": 0.036, "rss_kb": 34348, "run": 3, "discarded": false}
|
||||
{"label": "quarkus-native", "ready": true, "served_ok": true, "startup_s": 0.034, "rss_kb": 34356, "run": 4, "discarded": false}
|
||||
{"label": "quarkus-native", "summary": true, "runs_kept": 3, "avg_startup_s": 0.035, "avg_rss_kb": 34357.3, "avg_rss_mb": 33.6}
|
||||
=== Micronaut NATIVE startup+memory (4 runs) ===
|
||||
{"label": "micronaut-native", "ready": true, "served_ok": true, "startup_s": 0.049, "rss_kb": 47332, "run": 1, "discarded": true}
|
||||
{"label": "micronaut-native", "ready": true, "served_ok": true, "startup_s": 0.037, "rss_kb": 47304, "run": 2, "discarded": false}
|
||||
{"label": "micronaut-native", "ready": true, "served_ok": true, "startup_s": 0.05, "rss_kb": 47328, "run": 3, "discarded": false}
|
||||
{"label": "micronaut-native", "ready": true, "served_ok": true, "startup_s": 0.036, "rss_kb": 47324, "run": 4, "discarded": false}
|
||||
{"label": "micronaut-native", "summary": true, "runs_kept": 3, "avg_startup_s": 0.041, "avg_rss_kb": 47318.7, "avg_rss_mb": 46.2}
|
||||
@@ -0,0 +1,42 @@
|
||||
=== Spring Boot NATIVE throughput (15s, concurrency 20) ===
|
||||
{
|
||||
"label": "spring-boot-native",
|
||||
"path": "/widgets",
|
||||
"concurrency": 20,
|
||||
"duration_s": 15.0,
|
||||
"requests_ok": 71947,
|
||||
"requests_err": 0,
|
||||
"requests_per_sec": 4795.1,
|
||||
"latency_ms_mean": 4.16,
|
||||
"latency_ms_p50": 3.37,
|
||||
"latency_ms_p95": 10.17,
|
||||
"latency_ms_p99": 18.1
|
||||
}
|
||||
=== Quarkus NATIVE throughput (15s, concurrency 20) ===
|
||||
{
|
||||
"label": "quarkus-native",
|
||||
"path": "/widgets",
|
||||
"concurrency": 20,
|
||||
"duration_s": 15.01,
|
||||
"requests_ok": 63690,
|
||||
"requests_err": 0,
|
||||
"requests_per_sec": 4243.8,
|
||||
"latency_ms_mean": 4.7,
|
||||
"latency_ms_p50": 3.81,
|
||||
"latency_ms_p95": 11.56,
|
||||
"latency_ms_p99": 16.78
|
||||
}
|
||||
=== Micronaut NATIVE throughput (15s, concurrency 20) ===
|
||||
{
|
||||
"label": "micronaut-native",
|
||||
"path": "/widgets",
|
||||
"concurrency": 20,
|
||||
"duration_s": 15.0,
|
||||
"requests_ok": 74439,
|
||||
"requests_err": 0,
|
||||
"requests_per_sec": 4961.2,
|
||||
"latency_ms_mean": 4.02,
|
||||
"latency_ms_p50": 3.01,
|
||||
"latency_ms_p95": 10.41,
|
||||
"latency_ms_p99": 15.85
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
framework-benchmarks — consolidated results
|
||||
Sandbox: 2 vCPU / 7.8GiB RAM Linux container, GraalVM CE 25.4.4.1.1 (JDK 25.0.4.1.1)
|
||||
Versions under test: Spring Boot 4.1.1, Quarkus 3.40.1 (LTS), Micronaut 5.2.2
|
||||
All numbers are the average of 3 kept runs after discarding one cold-cache run (startup/memory),
|
||||
or one 15-second run at concurrency 20 (throughput). Raw per-run JSON in 01-04 in this directory.
|
||||
|
||||
STARTUP TIME (seconds, cold process to first 200 response)
|
||||
framework JVM native native is Nx faster
|
||||
spring-boot 6.318 0.132 47.9x
|
||||
quarkus 2.185 0.035 62.4x
|
||||
micronaut 2.507 0.041 61.1x
|
||||
|
||||
RESIDENT MEMORY (MB, RSS right after first successful request)
|
||||
framework JVM native native is Nx smaller
|
||||
spring-boot 229.2 79.5 2.9x
|
||||
quarkus 171.6 33.6 5.1x
|
||||
micronaut 180.0 46.2 3.9x
|
||||
|
||||
THROUGHPUT (requests/sec, GET /widgets, 20 concurrent stdlib-http.client workers, 15s)
|
||||
framework JVM req/s native req/s
|
||||
spring-boot 2275.7 4795.1
|
||||
quarkus 3791.4 4243.8
|
||||
micronaut 4276.2 4961.2
|
||||
|
||||
DEPLOYMENT ARTIFACT SIZE (MB)
|
||||
framework JVM (fat jar / fast-jar dir) native executable
|
||||
spring-boot 23 73
|
||||
quarkus 21 44
|
||||
micronaut 14 54
|
||||
|
||||
NATIVE BUILD TIME (wall clock, 2 vCPU, this sandbox — native-image is CPU-bound and
|
||||
parallelizes with available cores, so a bigger CI runner will be meaningfully faster)
|
||||
quarkus: 6m22s (succeeded on the first attempt)
|
||||
micronaut: 7m15s (succeeded on the 4th attempt — 3 earlier attempts failed in
|
||||
~42-50s each over a Netty/logback static-initialization clash;
|
||||
see docs/output/06-micronaut-native-build-failures.txt)
|
||||
spring-boot: 8m25s (succeeded on the first attempt, after routing around the
|
||||
native-maven-plugin's execution binding — see repo README)
|
||||
|
||||
Cross-checks against the independent 2025-10 benchmark by gillius.org (different
|
||||
hardware: Apple M2 Pro laptop vs this shared 2-vCPU container; different app: a
|
||||
Postgres-backed CRUD app vs this repo's deliberately DB-free one; Spring Boot 3.5.6/
|
||||
Quarkus 3.28.1/Micronaut 4.5.4 vs the current GA versions here) — see post for the full
|
||||
comparison table. Qualitatively consistent: native beats JVM mode by roughly one to two
|
||||
orders of magnitude on both startup and memory, in both benchmarks, on both machines.
|
||||
@@ -0,0 +1,62 @@
|
||||
Micronaut native-image build: the three failed attempts before the fix, verbatim
|
||||
error text from the real `native-image` invocations (trimmed to the diagnostic
|
||||
message; the full stack scan is in each build's svm_err_*.md, not committed since
|
||||
it also repeats the full classpath).
|
||||
|
||||
=== Attempt 1 (no --initialize-at-build-time / --initialize-at-run-time args at all) ===
|
||||
Failed generating 'micronaut-app' after 42.4s.
|
||||
|
||||
> com.oracle.graal.pointsto.util.ParallelExecutionException: An object of type 'ch.qos.logback.classic.Logger' was found in the image heap. This type, however, is marked for initialization at image run time for the following reason: classes are initialized at run time by default.
|
||||
This is not allowed for correctness reasons: All objects that are stored in the image heap must be initialized at build time.
|
||||
...
|
||||
Object was reached by
|
||||
scanning root constant ch.qos.logback.classic.Logger@7f585330: Logger[io.netty.util.ResourceLeakDetector] embedded in
|
||||
<unknown>
|
||||
at io.netty.util.ResourceLeakDetector.needReport() [bci: -5]
|
||||
|
||||
=== Attempt 2 (--initialize-at-build-time=ch.qos.logback.classic.Logger) ===
|
||||
Failed generating 'micronaut-app' after 46.0s.
|
||||
|
||||
> com.oracle.graal.pointsto.util.ParallelExecutionException: An object of type 'ch.qos.logback.classic.LoggerContext' was found in the image heap. This type, however, is marked for initialization at image run time ...
|
||||
Object was reached by
|
||||
reading field ch.qos.logback.classic.Logger.loggerContext of constant
|
||||
ch.qos.logback.classic.Logger@6f5cb3e9: Logger[io.netty.util.ResourceLeakDetector]
|
||||
scanning root constant ch.qos.logback.classic.Logger@6f5cb3e9: Logger[io.netty.util.ResourceLeakDetector] embedded in
|
||||
<unknown>
|
||||
at io.netty.util.ResourceLeakDetector.needReport() [bci: -5]
|
||||
|
||||
=== Attempt 3 (--initialize-at-build-time=ch.qos.logback [whole package]) ===
|
||||
Failed generating 'micronaut-app' after 49.9s.
|
||||
|
||||
> com.oracle.graal.pointsto.util.ParallelExecutionException: An object of type 'ch.qos.logback.classic.Logger' was found in the image heap. This type, however, is marked for initialization at image run time ...
|
||||
(identical trace to Attempt 1 - the package-level directive lost to a more specific
|
||||
--initialize-at-run-time directive for this exact class shipped in logback-classic's
|
||||
own META-INF/native-image/*/native-image.properties: GraalVM lets a more specific
|
||||
per-class directive override a broader package-level one, in either direction.)
|
||||
|
||||
=== Attempt 4 (--initialize-at-run-time=io.netty.util.ResourceLeakDetector,io.netty.util.ResourceLeakDetectorFactory) ===
|
||||
Failed generating 'micronaut-app' after 42.9s.
|
||||
|
||||
> com.oracle.graal.pointsto.util.ParallelExecutionException: An object of type 'io.netty.util.ResourceLeakDetector' was found in the image heap. This type, however, is marked for initialization at image run time ...
|
||||
Object was reached by
|
||||
scanning root constant io.netty.util.ResourceLeakDetector@2a2de2c0: io.netty.util.ResourceLeakDetector@2a2de2c0 embedded in
|
||||
io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:53)
|
||||
parsing method io.netty.buffer.AbstractByteBufAllocator.toLeakAwareBuffer(AbstractByteBufAllocator.java:53) reachable via the parsing context
|
||||
at static root method.(Unknown Source)
|
||||
(deferring just the leak detector to run-time wasn't enough: AbstractByteBufAllocator,
|
||||
itself build-time-initialized by default, holds a static reference to one.)
|
||||
|
||||
=== Fix that worked (--initialize-at-run-time=io.netty [whole package]) ===
|
||||
Finished generating 'micronaut-app' in 7m 15s.
|
||||
2,621 types -> 3,395 types reachable after the fix (more code stays reachable because
|
||||
initialization now genuinely happens at run time instead of being folded away), image
|
||||
heap 854MB peak RSS during the build, final executable 53.31MiB.
|
||||
|
||||
Takeaway for the post: this is a build-time ordering conflict between two framework
|
||||
defaults, not a bug in either library - Netty wants its leak detector and buffer
|
||||
allocator built eagerly (for speed), logback wants its Logger/LoggerContext run-time
|
||||
(so file handles and config aren't baked into the image), and the two defaults only
|
||||
collide when both are on the classpath AND something (here, static root scanning
|
||||
during Micronaut's own analysis) forces Netty's allocator to be build-time reachable.
|
||||
Pushing the whole io.netty package to run-time init was the fix that actually stuck;
|
||||
chasing individual classes one GraalVM error message at a time was not.
|
||||
@@ -0,0 +1,129 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>io.micronaut.platform</groupId>
|
||||
<artifactId>micronaut-parent</artifactId>
|
||||
<version>5.2.2</version>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm.benchmarks</groupId>
|
||||
<artifactId>micronaut-app</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<packaging>jar</packaging>
|
||||
<name>micronaut-app</name>
|
||||
<description>Widget REST API on Micronaut 5.2.2 - framework-benchmarks</description>
|
||||
|
||||
<properties>
|
||||
<packaging>jar</packaging>
|
||||
<jdk.version>25</jdk.version>
|
||||
<release.version>25</release.version>
|
||||
<micronaut.version>5.2.2</micronaut.version>
|
||||
<micronaut.runtime>netty</micronaut.runtime>
|
||||
<exec.mainClass>com.ankurm.benchmarks.micronaut.Application</exec.mainClass>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>io.micronaut</groupId>
|
||||
<artifactId>micronaut-http-server-netty</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.micronaut.serde</groupId>
|
||||
<artifactId>micronaut-serde-jackson</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.micronaut.validation</groupId>
|
||||
<artifactId>micronaut-validation</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>jakarta.annotation</groupId>
|
||||
<artifactId>jakarta.annotation-api</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-classic</artifactId>
|
||||
<scope>runtime</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>micronaut-app</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>io.micronaut.maven</groupId>
|
||||
<artifactId>micronaut-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<configuration>
|
||||
<annotationProcessorPaths combine.children="append">
|
||||
<path>
|
||||
<groupId>io.micronaut</groupId>
|
||||
<artifactId>micronaut-http-validation</artifactId>
|
||||
<version>${micronaut.core.version}</version>
|
||||
</path>
|
||||
<path>
|
||||
<groupId>io.micronaut.validation</groupId>
|
||||
<artifactId>micronaut-validation-processor</artifactId>
|
||||
<version>${micronaut.validation.version}</version>
|
||||
</path>
|
||||
</annotationProcessorPaths>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-shade-plugin</artifactId>
|
||||
<configuration>
|
||||
<transformers>
|
||||
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
|
||||
<mainClass>${exec.mainClass}</mainClass>
|
||||
</transformer>
|
||||
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
|
||||
</transformers>
|
||||
<filters>
|
||||
<filter>
|
||||
<artifact>*:*</artifact>
|
||||
<excludes>
|
||||
<exclude>META-INF/*.SF</exclude>
|
||||
<exclude>META-INF/*.DSA</exclude>
|
||||
<exclude>META-INF/*.RSA</exclude>
|
||||
</excludes>
|
||||
</filter>
|
||||
</filters>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<profiles>
|
||||
<profile>
|
||||
<id>native</id>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.graalvm.buildtools</groupId>
|
||||
<artifactId>native-maven-plugin</artifactId>
|
||||
<configuration>
|
||||
<imageName>micronaut-app</imageName>
|
||||
<mainClass>${exec.mainClass}</mainClass>
|
||||
<buildArgs>
|
||||
<!-- Netty's ResourceLeakDetector holds a static reference to a
|
||||
ch.qos.logback.classic.Logger; logback is run-time-init by
|
||||
default, which GraalVM refuses to embed in the image heap.
|
||||
See docs/output/04-micronaut-native-build.txt for the real
|
||||
error this fixes. -->
|
||||
<buildArg>--initialize-at-run-time=io.netty</buildArg>
|
||||
</buildArgs>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</profile>
|
||||
</profiles>
|
||||
|
||||
</project>
|
||||
@@ -0,0 +1,9 @@
|
||||
package com.ankurm.benchmarks.micronaut;
|
||||
|
||||
import io.micronaut.runtime.Micronaut;
|
||||
|
||||
public class Application {
|
||||
public static void main(String[] args) {
|
||||
Micronaut.run(Application.class, args);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package com.ankurm.benchmarks.micronaut;
|
||||
|
||||
import io.micronaut.serde.annotation.Serdeable;
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.PositiveOrZero;
|
||||
|
||||
@Serdeable
|
||||
public record CreateWidgetRequest(
|
||||
@NotBlank String name,
|
||||
@PositiveOrZero int quantity) {
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package com.ankurm.benchmarks.micronaut;
|
||||
|
||||
import io.micronaut.http.annotation.Controller;
|
||||
import io.micronaut.http.annotation.Get;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
* A plain custom endpoint rather than the micronaut-management starter, to keep the three
|
||||
* health checks comparably trivial instead of comparing three different libraries.
|
||||
*/
|
||||
@Controller("/health")
|
||||
public class HealthController {
|
||||
|
||||
@Get
|
||||
public Map<String, String> health() {
|
||||
return Map.of("status", "UP");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package com.ankurm.benchmarks.micronaut;
|
||||
|
||||
import io.micronaut.serde.annotation.Serdeable;
|
||||
|
||||
/**
|
||||
* The one domain type every app in this repo shares, field for field, across
|
||||
* Spring Boot, Quarkus and Micronaut. See the post's "the app under test" section.
|
||||
* {@code @Serdeable} is Micronaut's own compile-time (de)serialization annotation,
|
||||
* the equivalent of what Jackson's reflection does automatically at runtime for the
|
||||
* other two apps - this is itself one of the post's findings.
|
||||
*/
|
||||
@Serdeable
|
||||
public record Widget(String id, String name, int quantity) {
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package com.ankurm.benchmarks.micronaut;
|
||||
|
||||
import io.micronaut.http.HttpResponse;
|
||||
import io.micronaut.http.annotation.*;
|
||||
import io.micronaut.validation.Validated;
|
||||
import jakarta.validation.Valid;
|
||||
|
||||
import java.net.URI;
|
||||
import java.util.Collection;
|
||||
|
||||
@Controller("/widgets")
|
||||
@Validated
|
||||
public class WidgetController {
|
||||
|
||||
private final WidgetStore store;
|
||||
|
||||
public WidgetController(WidgetStore store) {
|
||||
this.store = store;
|
||||
}
|
||||
|
||||
@Get
|
||||
public Collection<Widget> all() {
|
||||
return store.findAll();
|
||||
}
|
||||
|
||||
@Get("/{id}")
|
||||
public HttpResponse<Widget> one(@PathVariable String id) {
|
||||
return store.findById(id)
|
||||
.map(HttpResponse::ok)
|
||||
.orElseGet(() -> HttpResponse.notFound());
|
||||
}
|
||||
|
||||
@Post
|
||||
public HttpResponse<Widget> create(@Valid @Body CreateWidgetRequest request) {
|
||||
Widget created = store.create(request.name(), request.quantity());
|
||||
return HttpResponse.created(created, URI.create("/widgets/" + created.id()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package com.ankurm.benchmarks.micronaut;
|
||||
|
||||
import jakarta.inject.Singleton;
|
||||
|
||||
import java.util.Collection;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
/**
|
||||
* Thread-safe in-memory store, seeded identically to the Spring Boot and Quarkus versions.
|
||||
* Deliberately no database: this repo isolates framework/runtime overhead from JDBC driver
|
||||
* and native-image reflection-metadata concerns, which is a separate, larger topic.
|
||||
*/
|
||||
@Singleton
|
||||
public class WidgetStore {
|
||||
|
||||
private final Map<String, Widget> widgets = new ConcurrentHashMap<>();
|
||||
private final AtomicInteger sequence = new AtomicInteger(0);
|
||||
|
||||
public WidgetStore() {
|
||||
seed("Hex Wrench Set", 42);
|
||||
seed("Torque Screwdriver", 17);
|
||||
seed("Bearing Puller", 5);
|
||||
seed("Digital Caliper", 23);
|
||||
seed("Bench Vise", 8);
|
||||
}
|
||||
|
||||
private void seed(String name, int quantity) {
|
||||
String id = "w-" + sequence.incrementAndGet();
|
||||
widgets.put(id, new Widget(id, name, quantity));
|
||||
}
|
||||
|
||||
public Collection<Widget> findAll() {
|
||||
return widgets.values();
|
||||
}
|
||||
|
||||
public Optional<Widget> findById(String id) {
|
||||
return Optional.ofNullable(widgets.get(id));
|
||||
}
|
||||
|
||||
public Widget create(String name, int quantity) {
|
||||
String id = "w-" + sequence.incrementAndGet();
|
||||
Widget widget = new Widget(id, name, quantity);
|
||||
widgets.put(id, widget);
|
||||
return widget;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
micronaut.application.name=micronautApp
|
||||
micronaut.server.port=8083
|
||||
endpoints.health.enabled=true
|
||||
endpoints.health.sensitive=false
|
||||
@@ -0,0 +1,10 @@
|
||||
<configuration>
|
||||
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
|
||||
<encoder>
|
||||
<pattern>%d{HH:mm:ss.SSS} %-5level %logger{36} - %msg%n</pattern>
|
||||
</encoder>
|
||||
</appender>
|
||||
<root level="WARN">
|
||||
<appender-ref ref="STDOUT"/>
|
||||
</root>
|
||||
</configuration>
|
||||
@@ -0,0 +1,87 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<groupId>com.ankurm.benchmarks</groupId>
|
||||
<artifactId>quarkus-app</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>quarkus-app</name>
|
||||
<description>Widget REST API on Quarkus 3.40.1 LTS - framework-benchmarks</description>
|
||||
|
||||
<properties>
|
||||
<maven.compiler.release>25</maven.compiler.release>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
|
||||
<quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
|
||||
<quarkus.platform.version>3.40.1</quarkus.platform.version>
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>${quarkus.platform.group-id}</groupId>
|
||||
<artifactId>${quarkus.platform.artifact-id}</artifactId>
|
||||
<version>${quarkus.platform.version}</version>
|
||||
<type>pom</type>
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-rest</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-rest-jackson</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-hibernate-validator</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-smallrye-health</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.quarkus</groupId>
|
||||
<artifactId>quarkus-arc</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>quarkus-app</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>${quarkus.platform.group-id}</groupId>
|
||||
<artifactId>quarkus-maven-plugin</artifactId>
|
||||
<version>${quarkus.platform.version}</version>
|
||||
<extensions>true</extensions>
|
||||
<executions>
|
||||
<execution>
|
||||
<goals>
|
||||
<goal>build</goal>
|
||||
<goal>generate-code</goal>
|
||||
<goal>generate-code-tests</goal>
|
||||
</goals>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<profiles>
|
||||
<profile>
|
||||
<id>native</id>
|
||||
<properties>
|
||||
<quarkus.package.jar.enabled>false</quarkus.package.jar.enabled>
|
||||
<quarkus.native.enabled>true</quarkus.native.enabled>
|
||||
</properties>
|
||||
</profile>
|
||||
</profiles>
|
||||
|
||||
</project>
|
||||
@@ -0,0 +1,9 @@
|
||||
package com.ankurm.benchmarks.quarkus;
|
||||
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.PositiveOrZero;
|
||||
|
||||
public record CreateWidgetRequest(
|
||||
@NotBlank String name,
|
||||
@PositiveOrZero int quantity) {
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
package com.ankurm.benchmarks.quarkus;
|
||||
|
||||
/**
|
||||
* The one domain type every app in this repo shares, field for field, across
|
||||
* Spring Boot, Quarkus and Micronaut. See the post's "the app under test" section.
|
||||
*/
|
||||
public record Widget(String id, String name, int quantity) {
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package com.ankurm.benchmarks.quarkus;
|
||||
|
||||
import jakarta.inject.Inject;
|
||||
import jakarta.validation.Valid;
|
||||
import jakarta.ws.rs.*;
|
||||
import jakarta.ws.rs.core.MediaType;
|
||||
import jakarta.ws.rs.core.Response;
|
||||
|
||||
import java.net.URI;
|
||||
import java.util.Collection;
|
||||
|
||||
@Path("/widgets")
|
||||
@Produces(MediaType.APPLICATION_JSON)
|
||||
@Consumes(MediaType.APPLICATION_JSON)
|
||||
public class WidgetResource {
|
||||
|
||||
@Inject
|
||||
WidgetStore store;
|
||||
|
||||
@GET
|
||||
public Collection<Widget> all() {
|
||||
return store.findAll();
|
||||
}
|
||||
|
||||
@GET
|
||||
@Path("/{id}")
|
||||
public Response one(@PathParam("id") String id) {
|
||||
return store.findById(id)
|
||||
.map(w -> Response.ok(w).build())
|
||||
.orElseGet(() -> Response.status(Response.Status.NOT_FOUND).build());
|
||||
}
|
||||
|
||||
@POST
|
||||
public Response create(@Valid CreateWidgetRequest request) {
|
||||
Widget created = store.create(request.name(), request.quantity());
|
||||
return Response.created(URI.create("/widgets/" + created.id())).entity(created).build();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package com.ankurm.benchmarks.quarkus;
|
||||
|
||||
import jakarta.enterprise.context.ApplicationScoped;
|
||||
|
||||
import java.util.Collection;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
/**
|
||||
* Thread-safe in-memory store, seeded identically to the Spring Boot and Micronaut versions.
|
||||
* Deliberately no database: this repo isolates framework/runtime overhead from JDBC driver
|
||||
* and native-image reflection-metadata concerns, which is a separate, larger topic.
|
||||
*/
|
||||
@ApplicationScoped
|
||||
public class WidgetStore {
|
||||
|
||||
private final Map<String, Widget> widgets = new ConcurrentHashMap<>();
|
||||
private final AtomicInteger sequence = new AtomicInteger(0);
|
||||
|
||||
public WidgetStore() {
|
||||
seed("Hex Wrench Set", 42);
|
||||
seed("Torque Screwdriver", 17);
|
||||
seed("Bearing Puller", 5);
|
||||
seed("Digital Caliper", 23);
|
||||
seed("Bench Vise", 8);
|
||||
}
|
||||
|
||||
private void seed(String name, int quantity) {
|
||||
String id = "w-" + sequence.incrementAndGet();
|
||||
widgets.put(id, new Widget(id, name, quantity));
|
||||
}
|
||||
|
||||
public Collection<Widget> findAll() {
|
||||
return widgets.values();
|
||||
}
|
||||
|
||||
public Optional<Widget> findById(String id) {
|
||||
return Optional.ofNullable(widgets.get(id));
|
||||
}
|
||||
|
||||
public Widget create(String name, int quantity) {
|
||||
String id = "w-" + sequence.incrementAndGet();
|
||||
Widget widget = new Widget(id, name, quantity);
|
||||
widgets.put(id, widget);
|
||||
return widget;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
quarkus.http.port=8082
|
||||
quarkus.banner.enabled=false
|
||||
quarkus.log.level=WARN
|
||||
@@ -0,0 +1,121 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
measure.py - launches one framework-benchmarks app, times cold startup, and samples RSS
|
||||
right after the app answers its first successful request.
|
||||
|
||||
No external dependencies: stdlib only, so the measurement script itself is auditable
|
||||
without installing anything. See docs/output/ for the real transcripts this script produced.
|
||||
|
||||
Usage:
|
||||
measure.py <label> <port> <ready-path> <runs> -- <command...>
|
||||
|
||||
Prints one JSON line per run to stdout, plus a final "summary" line. The first run is
|
||||
discarded (cold filesystem cache) exactly as the file-cache warm-up convention that
|
||||
independent Java framework benchmarks (e.g. gillius.org's Java 25 framework comparison)
|
||||
use, and the summary averages the remaining runs.
|
||||
"""
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
import signal
|
||||
import os
|
||||
|
||||
|
||||
def wait_ready(url, timeout_s=90):
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=0.5) as resp:
|
||||
if resp.status == 200:
|
||||
return True
|
||||
except (urllib.error.URLError, ConnectionError, OSError, TimeoutError):
|
||||
pass
|
||||
time.sleep(0.01)
|
||||
return False
|
||||
|
||||
|
||||
def rss_kb(pid):
|
||||
try:
|
||||
with open(f"/proc/{pid}/status") as f:
|
||||
for line in f:
|
||||
if line.startswith("VmRSS:"):
|
||||
return int(line.split()[1])
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def run_once(label, port, ready_path, cmd, cwd):
|
||||
url = f"http://127.0.0.1:{port}{ready_path}"
|
||||
start = time.monotonic()
|
||||
proc = subprocess.Popen(
|
||||
cmd, cwd=cwd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
start_new_session=True,
|
||||
)
|
||||
ok = wait_ready(url)
|
||||
elapsed = time.monotonic() - start
|
||||
rss = rss_kb(proc.pid) if ok else None
|
||||
# one more request after a short settle, to confirm the app is actually serving,
|
||||
# not just that the socket accepted a connection
|
||||
served_ok = False
|
||||
if ok:
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=2) as resp:
|
||||
served_ok = resp.status == 200
|
||||
except Exception:
|
||||
served_ok = False
|
||||
# terminate the whole process group (session leader), not just the Popen pid -
|
||||
# several of these apps fork a child JVM/launcher
|
||||
try:
|
||||
os.killpg(proc.pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
proc.wait(timeout=10)
|
||||
return {
|
||||
"label": label,
|
||||
"ready": ok,
|
||||
"served_ok": served_ok,
|
||||
"startup_s": round(elapsed, 3) if ok else None,
|
||||
"rss_kb": rss,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
sep = args.index("--")
|
||||
label, port, ready_path, runs = args[0], int(args[1]), args[2], int(args[3])
|
||||
cmd = args[sep + 1:]
|
||||
cwd = os.getcwd()
|
||||
|
||||
results = []
|
||||
for i in range(runs):
|
||||
r = run_once(label, port, ready_path, cmd, cwd)
|
||||
r["run"] = i + 1
|
||||
r["discarded"] = (i == 0)
|
||||
print(json.dumps(r))
|
||||
sys.stdout.flush()
|
||||
results.append(r)
|
||||
time.sleep(1.5)
|
||||
|
||||
kept = [r for r in results if not r["discarded"] and r["ready"]]
|
||||
if kept:
|
||||
avg_startup = sum(r["startup_s"] for r in kept) / len(kept)
|
||||
avg_rss = sum(r["rss_kb"] for r in kept if r["rss_kb"]) / len(kept)
|
||||
summary = {
|
||||
"label": label,
|
||||
"summary": True,
|
||||
"runs_kept": len(kept),
|
||||
"avg_startup_s": round(avg_startup, 3),
|
||||
"avg_rss_kb": round(avg_rss, 1),
|
||||
"avg_rss_mb": round(avg_rss / 1024, 1),
|
||||
}
|
||||
else:
|
||||
summary = {"label": label, "summary": True, "error": "no successful runs"}
|
||||
print(json.dumps(summary))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regenerates every file in docs/output/ from a clean build. Run from the repo root.
|
||||
#
|
||||
# Requires: GraalVM for JDK 25 on PATH (JAVA_HOME pointing at it), Maven, Python 3.
|
||||
# Builds JVM jars first, benchmarks them, then builds native images (slow - budget
|
||||
# 20-30 minutes total on a 2-vCPU machine) and benchmarks those too.
|
||||
#
|
||||
# This is exactly the sequence used to produce the committed docs/output/ files -
|
||||
# re-run it yourself and your numbers should land in the same ballpark, modulo your
|
||||
# own hardware. See README.md for the one GraalVM build-arg fix the Micronaut module
|
||||
# needed and why.
|
||||
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
echo "== 1/4: building JVM artifacts =="
|
||||
(cd spring-boot-app && mvn -q -B clean package -DskipTests)
|
||||
(cd quarkus-app && mvn -q -B clean package -DskipTests)
|
||||
(cd micronaut-app && mvn -q -B clean package -DskipTests)
|
||||
|
||||
echo "== 2/4: JVM startup + memory (4 runs each, first discarded) =="
|
||||
{
|
||||
echo "=== Spring Boot JVM startup+memory (4 runs) ==="
|
||||
python3 scripts/measure.py "spring-boot-jvm" 8081 /widgets 4 -- java -jar spring-boot-app/target/spring-boot-app.jar
|
||||
echo "=== Quarkus JVM startup+memory (4 runs) ==="
|
||||
python3 scripts/measure.py "quarkus-jvm" 8082 /widgets 4 -- java -jar quarkus-app/target/quarkus-app/quarkus-run.jar
|
||||
echo "=== Micronaut JVM startup+memory (4 runs) ==="
|
||||
python3 scripts/measure.py "micronaut-jvm" 8083 /widgets 4 -- java -jar micronaut-app/target/micronaut-app.jar
|
||||
} | tee docs/output/01-startup-memory-jvm.txt
|
||||
|
||||
echo "== 3/4: JVM throughput (15s @ concurrency 20, one app at a time) =="
|
||||
run_throughput() {
|
||||
local label="$1" port="$2" ready_path="$3"; shift 3
|
||||
"$@" > "/tmp/${label}.log" 2>&1 < /dev/null &
|
||||
local pid=$!
|
||||
disown
|
||||
for i in $(seq 1 60); do
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" "http://localhost:${port}${ready_path}" --max-time 2 || true)
|
||||
[ "$code" = "200" ] && break
|
||||
sleep 0.2
|
||||
done
|
||||
python3 scripts/throughput.py "$label" localhost "$port" "$ready_path" 15 20
|
||||
kill -9 "$pid" 2>/dev/null || true
|
||||
sleep 1
|
||||
}
|
||||
{
|
||||
echo "=== Spring Boot JVM throughput ==="
|
||||
run_throughput spring-boot-jvm 8081 /widgets java -jar spring-boot-app/target/spring-boot-app.jar
|
||||
echo "=== Quarkus JVM throughput ==="
|
||||
run_throughput quarkus-jvm 8082 /widgets java -jar quarkus-app/target/quarkus-app/quarkus-run.jar
|
||||
echo "=== Micronaut JVM throughput ==="
|
||||
run_throughput micronaut-jvm 8083 /widgets java -jar micronaut-app/target/micronaut-app.jar
|
||||
} | tee docs/output/02-throughput-jvm.txt
|
||||
|
||||
echo "== 4/4: native builds + benchmarks (slow) =="
|
||||
(cd quarkus-app && mvn -q -B -Pnative clean package -DskipTests && mvn -q -B package -DskipTests)
|
||||
(cd micronaut-app && mvn -q -B -Pnative clean package -DskipTests && \
|
||||
mvn -q -B -Pnative package org.graalvm.buildtools:native-maven-plugin:compile -DskipTests)
|
||||
(cd spring-boot-app && mvn -q -B -Pnative clean package -DskipTests && mvn -q -B -Pnative native:compile)
|
||||
|
||||
{
|
||||
echo "=== Spring Boot NATIVE startup+memory (4 runs) ==="
|
||||
python3 scripts/measure.py "spring-boot-native" 8081 /widgets 4 -- ./spring-boot-app/target/spring-boot-app
|
||||
echo "=== Quarkus NATIVE startup+memory (4 runs) ==="
|
||||
python3 scripts/measure.py "quarkus-native" 8082 /widgets 4 -- ./quarkus-app/target/quarkus-app-runner
|
||||
echo "=== Micronaut NATIVE startup+memory (4 runs) ==="
|
||||
python3 scripts/measure.py "micronaut-native" 8083 /widgets 4 -- ./micronaut-app/target/micronaut-app
|
||||
} | tee docs/output/03-startup-memory-native.txt
|
||||
|
||||
{
|
||||
echo "=== Spring Boot NATIVE throughput ==="
|
||||
run_throughput spring-boot-native 8081 /widgets ./spring-boot-app/target/spring-boot-app
|
||||
echo "=== Quarkus NATIVE throughput ==="
|
||||
run_throughput quarkus-native 8082 /widgets ./quarkus-app/target/quarkus-app-runner
|
||||
echo "=== Micronaut NATIVE throughput ==="
|
||||
run_throughput micronaut-native 8083 /widgets ./micronaut-app/target/micronaut-app
|
||||
} | tee docs/output/04-throughput-native.txt
|
||||
|
||||
echo "Done. See docs/output/05-summary.txt for the consolidated table (written by hand from the runs above - update it if your numbers differ materially)."
|
||||
@@ -0,0 +1,104 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
throughput.py - a minimal, dependency-free concurrent HTTP load generator.
|
||||
|
||||
This repo intentionally does not shell out to wrk/hey/ab (neither was available in the
|
||||
sandbox this repo was built in, and a stdlib script means the load generator itself needs
|
||||
no separate install or version callout). Each worker thread keeps one persistent
|
||||
HTTP/1.1 connection (http.client, keep-alive) and fires GET requests back-to-back for a
|
||||
fixed wall-clock duration; the main thread joins all workers and aggregates.
|
||||
|
||||
Usage:
|
||||
throughput.py <label> <host> <port> <path> <duration_s> <concurrency>
|
||||
|
||||
Prints one JSON line: requests, errors, requests_per_sec, latency_ms mean/p50/p95/p99.
|
||||
"""
|
||||
import http.client
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
|
||||
def worker(host, port, path, stop_at, latencies, counters, idx):
|
||||
conn = http.client.HTTPConnection(host, port, timeout=5)
|
||||
local_ok = 0
|
||||
local_err = 0
|
||||
local_latencies = []
|
||||
while time.monotonic() < stop_at:
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
conn.request("GET", path)
|
||||
resp = conn.getresponse()
|
||||
resp.read()
|
||||
if resp.status == 200:
|
||||
local_ok += 1
|
||||
else:
|
||||
local_err += 1
|
||||
except Exception:
|
||||
local_err += 1
|
||||
try:
|
||||
conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
conn = http.client.HTTPConnection(host, port, timeout=5)
|
||||
local_latencies.append((time.monotonic() - t0) * 1000.0)
|
||||
counters[idx] = (local_ok, local_err)
|
||||
latencies[idx] = local_latencies
|
||||
try:
|
||||
conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def main():
|
||||
label, host, port, path, duration_s, concurrency = sys.argv[1:7]
|
||||
port = int(port)
|
||||
duration_s = float(duration_s)
|
||||
concurrency = int(concurrency)
|
||||
|
||||
counters = [None] * concurrency
|
||||
latencies = [None] * concurrency
|
||||
stop_at = time.monotonic() + duration_s
|
||||
|
||||
threads = [
|
||||
threading.Thread(target=worker, args=(host, port, path, stop_at, latencies, counters, i))
|
||||
for i in range(concurrency)
|
||||
]
|
||||
wall_start = time.monotonic()
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
wall_elapsed = time.monotonic() - wall_start
|
||||
|
||||
total_ok = sum(c[0] for c in counters)
|
||||
total_err = sum(c[1] for c in counters)
|
||||
all_latencies = [x for sub in latencies for x in sub]
|
||||
all_latencies.sort()
|
||||
|
||||
def pct(p):
|
||||
if not all_latencies:
|
||||
return None
|
||||
k = min(len(all_latencies) - 1, int(len(all_latencies) * p))
|
||||
return round(all_latencies[k], 2)
|
||||
|
||||
result = {
|
||||
"label": label,
|
||||
"path": path,
|
||||
"concurrency": concurrency,
|
||||
"duration_s": round(wall_elapsed, 2),
|
||||
"requests_ok": total_ok,
|
||||
"requests_err": total_err,
|
||||
"requests_per_sec": round(total_ok / wall_elapsed, 1) if wall_elapsed > 0 else None,
|
||||
"latency_ms_mean": round(statistics.mean(all_latencies), 2) if all_latencies else None,
|
||||
"latency_ms_p50": pct(0.50),
|
||||
"latency_ms_p95": pct(0.95),
|
||||
"latency_ms_p99": pct(0.99),
|
||||
}
|
||||
print(json.dumps(result, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,66 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm.benchmarks</groupId>
|
||||
<artifactId>spring-boot-app</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>spring-boot-app</name>
|
||||
<description>Widget REST API on Spring Boot 4.1.1 - framework-benchmarks</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-web</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-validation</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-actuator</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>spring-boot-app</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<profiles>
|
||||
<profile>
|
||||
<id>native</id>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.graalvm.buildtools</groupId>
|
||||
<artifactId>native-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</profile>
|
||||
</profiles>
|
||||
|
||||
</project>
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
package com.ankurm.benchmarks.springboot;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
|
||||
@SpringBootApplication
|
||||
public class BenchmarkApplication {
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(BenchmarkApplication.class, args);
|
||||
}
|
||||
}
|
||||
+9
@@ -0,0 +1,9 @@
|
||||
package com.ankurm.benchmarks.springboot;
|
||||
|
||||
import jakarta.validation.constraints.NotBlank;
|
||||
import jakarta.validation.constraints.PositiveOrZero;
|
||||
|
||||
public record CreateWidgetRequest(
|
||||
@NotBlank String name,
|
||||
@PositiveOrZero int quantity) {
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
package com.ankurm.benchmarks.springboot;
|
||||
|
||||
/**
|
||||
* The one domain type every app in this repo shares, field for field, across
|
||||
* Spring Boot, Quarkus and Micronaut. See the post's "the app under test" section.
|
||||
*/
|
||||
public record Widget(String id, String name, int quantity) {
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package com.ankurm.benchmarks.springboot;
|
||||
|
||||
import jakarta.validation.Valid;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.*;
|
||||
|
||||
import java.net.URI;
|
||||
import java.util.Collection;
|
||||
|
||||
@RestController
|
||||
@RequestMapping("/widgets")
|
||||
public class WidgetController {
|
||||
|
||||
private final WidgetStore store;
|
||||
|
||||
public WidgetController(WidgetStore store) {
|
||||
this.store = store;
|
||||
}
|
||||
|
||||
@GetMapping
|
||||
public Collection<Widget> all() {
|
||||
return store.findAll();
|
||||
}
|
||||
|
||||
@GetMapping("/{id}")
|
||||
public ResponseEntity<Widget> one(@PathVariable String id) {
|
||||
return store.findById(id)
|
||||
.map(ResponseEntity::ok)
|
||||
.orElseGet(() -> ResponseEntity.notFound().build());
|
||||
}
|
||||
|
||||
@PostMapping
|
||||
public ResponseEntity<Widget> create(@Valid @RequestBody CreateWidgetRequest request) {
|
||||
Widget created = store.create(request.name(), request.quantity());
|
||||
return ResponseEntity.created(URI.create("/widgets/" + created.id())).body(created);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package com.ankurm.benchmarks.springboot;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.util.Collection;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
/**
|
||||
* Thread-safe in-memory store, seeded identically to the Quarkus and Micronaut versions.
|
||||
* Deliberately no database: this repo isolates framework/runtime overhead from JDBC driver
|
||||
* and native-image reflection-metadata concerns, which is a separate, larger topic.
|
||||
*/
|
||||
@Component
|
||||
public class WidgetStore {
|
||||
|
||||
private final Map<String, Widget> widgets = new ConcurrentHashMap<>();
|
||||
private final AtomicInteger sequence = new AtomicInteger(0);
|
||||
|
||||
public WidgetStore() {
|
||||
seed("Hex Wrench Set", 42);
|
||||
seed("Torque Screwdriver", 17);
|
||||
seed("Bearing Puller", 5);
|
||||
seed("Digital Caliper", 23);
|
||||
seed("Bench Vise", 8);
|
||||
}
|
||||
|
||||
private void seed(String name, int quantity) {
|
||||
String id = "w-" + sequence.incrementAndGet();
|
||||
widgets.put(id, new Widget(id, name, quantity));
|
||||
}
|
||||
|
||||
public Collection<Widget> findAll() {
|
||||
return widgets.values();
|
||||
}
|
||||
|
||||
public Optional<Widget> findById(String id) {
|
||||
return Optional.ofNullable(widgets.get(id));
|
||||
}
|
||||
|
||||
public Widget create(String name, int quantity) {
|
||||
String id = "w-" + sequence.incrementAndGet();
|
||||
Widget widget = new Widget(id, name, quantity);
|
||||
widgets.put(id, widget);
|
||||
return widget;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
server.port=8081
|
||||
spring.main.banner-mode=off
|
||||
management.endpoints.web.exposure.include=health
|
||||
logging.level.root=WARN
|
||||
Reference in New Issue
Block a user