parallel-streams: JMH benchmarks and demos for when parallel streams are slower

N x Q model, source splittability, boxing, ordered operations, common-pool
starvation, and parallel-stream correctness bugs. Captured output in output/.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ic4b6hCTuFrQT8xKmmMrF
This commit is contained in:
Claude
2026-11-17 23:23:21 +05:30
parent f8c9e244a3
commit 4511cc6e6b
22 changed files with 1714 additions and 0 deletions
@@ -0,0 +1,263 @@
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.arrayList_par
# Run progress: 0.00% complete, ETA 00:01:04
# Fork: 1 of 1
# Warmup Iteration 1: 875.141 us/op
# Warmup Iteration 2: 1000.046 us/op
# Warmup Iteration 3: 887.776 us/op
Iteration 1: 892.010 us/op
Iteration 2: 897.553 us/op
Iteration 3: 799.533 us/op
Iteration 4: 810.756 us/op
Iteration 5: 923.902 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.arrayList_par":
864.751 ±(99.9%) 215.142 us/op [Average]
(min, avg, max) = (799.533, 864.751, 923.902), stdev = 55.872
CI (99.9%): [649.608, 1079.893] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.arrayList_seq
# Run progress: 12.50% complete, ETA 00:00:57
# Fork: 1 of 1
# Warmup Iteration 1: 1707.637 us/op
# Warmup Iteration 2: 1671.232 us/op
# Warmup Iteration 3: 1639.488 us/op
Iteration 1: 1646.433 us/op
Iteration 2: 1587.087 us/op
Iteration 3: 1617.898 us/op
Iteration 4: 1563.128 us/op
Iteration 5: 1565.962 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.arrayList_seq":
1596.102 ±(99.9%) 137.297 us/op [Average]
(min, avg, max) = (1563.128, 1596.102, 1646.433), stdev = 35.656
CI (99.9%): [1458.805, 1733.399] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.intArray_par
# Run progress: 25.00% complete, ETA 00:00:49
# Fork: 1 of 1
# Warmup Iteration 1: 844.636 us/op
# Warmup Iteration 2: 812.772 us/op
# Warmup Iteration 3: 799.070 us/op
Iteration 1: 934.718 us/op
Iteration 2: 807.934 us/op
Iteration 3: 833.626 us/op
Iteration 4: 805.721 us/op
Iteration 5: 808.579 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.intArray_par":
838.116 ±(99.9%) 212.527 us/op [Average]
(min, avg, max) = (805.721, 838.116, 934.718), stdev = 55.192
CI (99.9%): [625.589, 1050.642] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.intArray_seq
# Run progress: 37.50% complete, ETA 00:00:41
# Fork: 1 of 1
# Warmup Iteration 1: 1557.406 us/op
# Warmup Iteration 2: 1627.566 us/op
# Warmup Iteration 3: 1530.733 us/op
Iteration 1: 1672.832 us/op
Iteration 2: 1485.387 us/op
Iteration 3: 1511.766 us/op
Iteration 4: 1480.313 us/op
Iteration 5: 1506.690 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.intArray_seq":
1531.397 ±(99.9%) 308.809 us/op [Average]
(min, avg, max) = (1480.313, 1531.397, 1672.832), stdev = 80.197
CI (99.9%): [1222.588, 1840.206] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.iterate_par
# Run progress: 50.00% complete, ETA 00:00:32
# Fork: 1 of 1
# Warmup Iteration 1: 3539.271 us/op
# Warmup Iteration 2: 3281.146 us/op
# Warmup Iteration 3: 2699.139 us/op
Iteration 1: 2826.627 us/op
Iteration 2: 2769.716 us/op
Iteration 3: 2790.944 us/op
Iteration 4: 2817.520 us/op
Iteration 5: 2756.321 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.iterate_par":
2792.226 ±(99.9%) 115.852 us/op [Average]
(min, avg, max) = (2756.321, 2792.226, 2826.627), stdev = 30.086
CI (99.9%): [2676.374, 2908.078] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.iterate_seq
# Run progress: 62.50% complete, ETA 00:00:24
# Fork: 1 of 1
# Warmup Iteration 1: 2122.584 us/op
# Warmup Iteration 2: 2051.970 us/op
# Warmup Iteration 3: 1984.262 us/op
Iteration 1: 1958.438 us/op
Iteration 2: 1903.397 us/op
Iteration 3: 1902.650 us/op
Iteration 4: 2086.336 us/op
Iteration 5: 1962.841 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.iterate_seq":
1962.732 ±(99.9%) 288.329 us/op [Average]
(min, avg, max) = (1902.650, 1962.732, 2086.336), stdev = 74.878
CI (99.9%): [1674.404, 2251.061] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.linkedList_par
# Run progress: 75.00% complete, ETA 00:00:16
# Fork: 1 of 1
# Warmup Iteration 1: 1723.307 us/op
# Warmup Iteration 2: 1523.676 us/op
# Warmup Iteration 3: 1797.511 us/op
Iteration 1: 1736.234 us/op
Iteration 2: 1713.686 us/op
Iteration 3: 1867.840 us/op
Iteration 4: 1627.651 us/op
Iteration 5: 1840.748 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.linkedList_par":
1757.232 ±(99.9%) 376.977 us/op [Average]
(min, avg, max) = (1627.651, 1757.232, 1867.840), stdev = 97.900
CI (99.9%): [1380.255, 2134.208] (assumes normal distribution)
# JMH version: 1.37
# VM version: JDK 25.0.5, OpenJDK 64-Bit Server VM, 25.0.5+8-LTS
# VM invoker: /tmp/tools/jdk-25.0.5+8/bin/java
# VM options: -Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts -Djavax.net.ssl.trustStoreType=JKS -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=42575 -Dhttp.nonProxyHosts=localhost|127.0.0.1|::1|127.*|0.*|::|169.254.*|api.anthropic.com|api-staging.anthropic.com|api-pr-preview.anthropic.com|mcp-proxy.anthropic.com|mcp-proxy-staging.anthropic.com|registry.npmjs.org|jsr.io|npm.jsr.io|pypi.org|files.pythonhosted.org|index.crates.io|proxy.golang.org|host.docker.internal|10.*|172.16.*|172.17.*|172.18.*|172.19.*|172.20.*|172.21.*|172.22.*|172.23.*|172.24.*|172.25.*|172.26.*|172.27.*|172.28.*|172.29.*|172.30.*|172.31.*|192.168.*|100.64.0.0/10|*.svc.cluster.local|*.svc.cluster.local -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.http.auth.proxying.disabledSchemes=
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: 3 iterations, 1 s each
# Measurement: 5 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.ankurm.parallelstreams.SourceBenchmark.linkedList_seq
# Run progress: 87.50% complete, ETA 00:00:08
# Fork: 1 of 1
# Warmup Iteration 1: 1624.775 us/op
# Warmup Iteration 2: 1606.300 us/op
# Warmup Iteration 3: 1722.162 us/op
Iteration 1: 1703.266 us/op
Iteration 2: 1687.384 us/op
Iteration 3: 1581.179 us/op
Iteration 4: 1695.401 us/op
Iteration 5: 1660.109 us/op
Result "com.ankurm.parallelstreams.SourceBenchmark.linkedList_seq":
1665.468 ±(99.9%) 191.938 us/op [Average]
(min, avg, max) = (1581.179, 1665.468, 1703.266), stdev = 49.846
CI (99.9%): [1473.530, 1857.405] (assumes normal distribution)
# Run complete. Total time: 00:01:05
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
extra caution when trusting the results, look into the generated code to check the benchmark still
works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
different JVMs are already problematic, the performance difference caused by different Blackhole
modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
Benchmark Mode Cnt Score Error Units
SourceBenchmark.arrayList_par avgt 5 864.751 ± 215.142 us/op
SourceBenchmark.arrayList_seq avgt 5 1596.102 ± 137.297 us/op
SourceBenchmark.intArray_par avgt 5 838.116 ± 212.527 us/op
SourceBenchmark.intArray_seq avgt 5 1531.397 ± 308.809 us/op
SourceBenchmark.iterate_par avgt 5 2792.226 ± 115.852 us/op
SourceBenchmark.iterate_seq avgt 5 1962.732 ± 288.329 us/op
SourceBenchmark.linkedList_par avgt 5 1757.232 ± 376.977 us/op
SourceBenchmark.linkedList_seq avgt 5 1665.468 ± 191.938 us/op