parallel-streams: JMH benchmarks and demos for when parallel streams are slower
N x Q model, source splittability, boxing, ordered operations, common-pool starvation, and parallel-stream correctness bugs. Captured output in output/. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_017ic4b6hCTuFrQT8xKmmMrF
This commit is contained in:
@@ -0,0 +1,201 @@
|
||||
# JMH version: 1.37
|
||||
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
|
||||
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
|
||||
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
|
||||
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
|
||||
# Warmup: 3 iterations, 1 s each
|
||||
# Measurement: 5 iterations, 1 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: com.ankurm.parallelstreams.OrderedOpsBenchmark.distinct_par_ordered
|
||||
|
||||
# Run progress: 0.00% complete, ETA 00:00:48
|
||||
# Fork: 1 of 1
|
||||
# Warmup Iteration 1: 8704.466 us/op
|
||||
# Warmup Iteration 2: 9269.009 us/op
|
||||
# Warmup Iteration 3: 9734.713 us/op
|
||||
Iteration 1: 9733.449 us/op
|
||||
Iteration 2: 9639.780 us/op
|
||||
Iteration 3: 8957.786 us/op
|
||||
Iteration 4: 11247.905 us/op
|
||||
Iteration 5: 9896.392 us/op
|
||||
|
||||
|
||||
Result "com.ankurm.parallelstreams.OrderedOpsBenchmark.distinct_par_ordered":
|
||||
9895.062 ±(99.9%) 3221.657 us/op [Average]
|
||||
(min, avg, max) = (8957.786, 9895.062, 11247.905), stdev = 836.654
|
||||
CI (99.9%): [6673.405, 13116.719] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.37
|
||||
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
|
||||
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
|
||||
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
|
||||
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
|
||||
# Warmup: 3 iterations, 1 s each
|
||||
# Measurement: 5 iterations, 1 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: com.ankurm.parallelstreams.OrderedOpsBenchmark.distinct_par_unordered
|
||||
|
||||
# Run progress: 16.67% complete, ETA 00:00:41
|
||||
# Fork: 1 of 1
|
||||
# Warmup Iteration 1: 13742.908 us/op
|
||||
# Warmup Iteration 2: 7226.162 us/op
|
||||
# Warmup Iteration 3: 5339.945 us/op
|
||||
Iteration 1: 6457.032 us/op
|
||||
Iteration 2: 7957.638 us/op
|
||||
Iteration 3: 5756.201 us/op
|
||||
Iteration 4: 6078.999 us/op
|
||||
Iteration 5: 5919.050 us/op
|
||||
|
||||
|
||||
Result "com.ankurm.parallelstreams.OrderedOpsBenchmark.distinct_par_unordered":
|
||||
6433.784 ±(99.9%) 3429.350 us/op [Average]
|
||||
(min, avg, max) = (5756.201, 6433.784, 7957.638), stdev = 890.591
|
||||
CI (99.9%): [3004.434, 9863.134] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.37
|
||||
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
|
||||
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
|
||||
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
|
||||
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
|
||||
# Warmup: 3 iterations, 1 s each
|
||||
# Measurement: 5 iterations, 1 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: com.ankurm.parallelstreams.OrderedOpsBenchmark.distinct_seq
|
||||
|
||||
# Run progress: 33.33% complete, ETA 00:00:33
|
||||
# Fork: 1 of 1
|
||||
# Warmup Iteration 1: 4882.585 us/op
|
||||
# Warmup Iteration 2: 4988.734 us/op
|
||||
# Warmup Iteration 3: 4556.600 us/op
|
||||
Iteration 1: 5324.242 us/op
|
||||
Iteration 2: 5486.219 us/op
|
||||
Iteration 3: 4496.437 us/op
|
||||
Iteration 4: 4425.314 us/op
|
||||
Iteration 5: 4358.564 us/op
|
||||
|
||||
|
||||
Result "com.ankurm.parallelstreams.OrderedOpsBenchmark.distinct_seq":
|
||||
4818.155 ±(99.9%) 2083.875 us/op [Average]
|
||||
(min, avg, max) = (4358.564, 4818.155, 5486.219), stdev = 541.176
|
||||
CI (99.9%): [2734.281, 6902.030] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.37
|
||||
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
|
||||
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
|
||||
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
|
||||
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
|
||||
# Warmup: 3 iterations, 1 s each
|
||||
# Measurement: 5 iterations, 1 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: com.ankurm.parallelstreams.OrderedOpsBenchmark.limit_par_ordered
|
||||
|
||||
# Run progress: 50.00% complete, ETA 00:00:24
|
||||
# Fork: 1 of 1
|
||||
# Warmup Iteration 1: 258.706 us/op
|
||||
# Warmup Iteration 2: 228.867 us/op
|
||||
# Warmup Iteration 3: 218.050 us/op
|
||||
Iteration 1: 212.607 us/op
|
||||
Iteration 2: 215.389 us/op
|
||||
Iteration 3: 226.594 us/op
|
||||
Iteration 4: 210.350 us/op
|
||||
Iteration 5: 210.371 us/op
|
||||
|
||||
|
||||
Result "com.ankurm.parallelstreams.OrderedOpsBenchmark.limit_par_ordered":
|
||||
215.062 ±(99.9%) 26.069 us/op [Average]
|
||||
(min, avg, max) = (210.350, 215.062, 226.594), stdev = 6.770
|
||||
CI (99.9%): [188.993, 241.131] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.37
|
||||
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
|
||||
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
|
||||
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
|
||||
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
|
||||
# Warmup: 3 iterations, 1 s each
|
||||
# Measurement: 5 iterations, 1 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: com.ankurm.parallelstreams.OrderedOpsBenchmark.limit_par_unordered
|
||||
|
||||
# Run progress: 66.67% complete, ETA 00:00:16
|
||||
# Fork: 1 of 1
|
||||
# Warmup Iteration 1: 80.165 us/op
|
||||
# Warmup Iteration 2: 57.729 us/op
|
||||
# Warmup Iteration 3: 57.777 us/op
|
||||
Iteration 1: 55.429 us/op
|
||||
Iteration 2: 54.038 us/op
|
||||
Iteration 3: 59.503 us/op
|
||||
Iteration 4: 57.574 us/op
|
||||
Iteration 5: 56.395 us/op
|
||||
|
||||
|
||||
Result "com.ankurm.parallelstreams.OrderedOpsBenchmark.limit_par_unordered":
|
||||
56.588 ±(99.9%) 8.020 us/op [Average]
|
||||
(min, avg, max) = (54.038, 56.588, 59.503), stdev = 2.083
|
||||
CI (99.9%): [48.568, 64.608] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.37
|
||||
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
|
||||
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
|
||||
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
|
||||
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
|
||||
# Warmup: 3 iterations, 1 s each
|
||||
# Measurement: 5 iterations, 1 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: com.ankurm.parallelstreams.OrderedOpsBenchmark.limit_seq
|
||||
|
||||
# Run progress: 83.33% complete, ETA 00:00:08
|
||||
# Fork: 1 of 1
|
||||
# Warmup Iteration 1: 63.583 us/op
|
||||
# Warmup Iteration 2: 68.744 us/op
|
||||
# Warmup Iteration 3: 61.862 us/op
|
||||
Iteration 1: 60.503 us/op
|
||||
Iteration 2: 37.577 us/op
|
||||
Iteration 3: 38.935 us/op
|
||||
Iteration 4: 37.643 us/op
|
||||
Iteration 5: 36.974 us/op
|
||||
|
||||
|
||||
Result "com.ankurm.parallelstreams.OrderedOpsBenchmark.limit_seq":
|
||||
42.326 ±(99.9%) 39.223 us/op [Average]
|
||||
(min, avg, max) = (36.974, 42.326, 60.503), stdev = 10.186
|
||||
CI (99.9%): [3.103, 81.549] (assumes normal distribution)
|
||||
|
||||
|
||||
# Run complete. Total time: 00:00:49
|
||||
|
||||
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
|
||||
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
|
||||
experiments, perform baseline and negative tests that provide experimental control, make sure
|
||||
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
|
||||
Do not assume the numbers tell you what you want them to tell.
|
||||
|
||||
NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
|
||||
extra caution when trusting the results, look into the generated code to check the benchmark still
|
||||
works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
|
||||
different JVMs are already problematic, the performance difference caused by different Blackhole
|
||||
modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
|
||||
|
||||
Benchmark Mode Cnt Score Error Units
|
||||
OrderedOpsBenchmark.distinct_par_ordered avgt 5 9895.062 ± 3221.657 us/op
|
||||
OrderedOpsBenchmark.distinct_par_unordered avgt 5 6433.784 ± 3429.350 us/op
|
||||
OrderedOpsBenchmark.distinct_seq avgt 5 4818.155 ± 2083.875 us/op
|
||||
OrderedOpsBenchmark.limit_par_ordered avgt 5 215.062 ± 26.069 us/op
|
||||
OrderedOpsBenchmark.limit_par_unordered avgt 5 56.588 ± 8.020 us/op
|
||||
OrderedOpsBenchmark.limit_seq avgt 5 42.326 ± 39.223 us/op
|
||||
Reference in New Issue
Block a user