Complete runnable Vector API (JEP 537) demo
SIMD kernels (square-and-add, masked filter, brightness clamp, FP reduction ordering), an exploratory timing harness, JMH benchmarks with Maven and Maven-free builds, run scripts, README and the captured results with their exact environment.
This commit was merged in pull request #1.
This commit is contained in:
118
README.md
118
README.md
@@ -1,3 +1,117 @@
|
||||
# vector-api-jep-537-demo
|
||||
# Java Vector API (JEP 537) — runnable demo
|
||||
|
||||
Runnable companion code for the Java Vector API (JEP 537) guide on ankurm.com: SIMD kernels, masks, the loopBound+tail pattern, and JMH benchmarks on JDK 21.
|
||||
Companion code for **[Java Vector API (JEP 537): SIMD, Auto-Vectorization, Masks, and Real Benchmarks](https://ankurm.com/java-vector-api-jep-537-simd-guide/)** on ankurm.com.
|
||||
Every code listing and every number in that article comes from this repository.
|
||||
|
||||
## What this is about
|
||||
|
||||
Your CPU has vector registers — 256 bits wide on AVX2, 512 on AVX-512, 128 on
|
||||
Arm NEON — that hold several numbers side by side, in *lanes*. A single SIMD
|
||||
instruction applies one operation to every lane at once. A plain Java `for` loop
|
||||
touches one element per iteration, so most of that hardware sits idle.
|
||||
|
||||
Java gives you two ways to use it:
|
||||
|
||||
* **Auto-vectorization (free, implicit).** HotSpot's C2 compiler has a SuperWord
|
||||
pass that fuses consecutive scalar iterations into vector ones. When it fires
|
||||
you get SIMD for nothing. It fires only when C2 can prove the transform is safe
|
||||
and profitable — no data-dependent branches, no aliasing it cannot rule out,
|
||||
regular access patterns — and when it declines, it does so silently.
|
||||
* **The Vector API (explicit).** `jdk.incubator.vector` lets you state the vector
|
||||
intent yourself: load a lane-width slice, do lane-wise arithmetic, store it
|
||||
back, and use *masks* instead of branches. It does not promise a fixed
|
||||
instruction sequence on every CPU, but it gives far more predictable access to
|
||||
SIMD than hoping C2 infers it.
|
||||
|
||||
The API is still incubating — JEP 537 is its twelfth incubation, targeted at JDK
|
||||
27 — because it is waiting on Project Valhalla's value classes. The shape of the
|
||||
API is stable; the package name will change when it is promoted.
|
||||
|
||||
This repo demonstrates both paths, and — more usefully — shows how to **tell
|
||||
which one you are on**, by running the same kernels with `-XX:-UseSuperWord`.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
timings/ exploratory timing experiments (plain javac, no dependencies)
|
||||
src/main/java/com/ankurm/vectorapi/
|
||||
Main.java runs everything, prints the environment first
|
||||
SquareAdd.java c[i] = -(a[i]^2 + b[i]^2) — the loop C2 vectorizes for free
|
||||
MaskedFilter.java sum elements above a threshold — masks vs a branch
|
||||
Brightness.java scale + bias + clamp — max/min vs two branches
|
||||
ReductionOrder.java why vector and scalar sums differ in the last bits
|
||||
Bench.java the (deliberately simple) timing harness
|
||||
Env.java, Data.java, Sink.java
|
||||
run.sh
|
||||
|
||||
jmh/ the same kernels under JMH — the numbers worth defending
|
||||
src/main/java/com/ankurm/vectorapi/
|
||||
VectorBench.java square-and-add, incl. a cautionary checksum variant
|
||||
MaskedBench.java masked filter-and-sum
|
||||
BrightnessBench.java brightness + clamp
|
||||
pom.xml
|
||||
run.sh Maven build + run
|
||||
run-without-maven.sh same thing with curl + javac, if you have no Maven
|
||||
|
||||
RESULTS.md everything the article quotes, with the exact machine it came from
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
JDK 21 or newer (anything with `jdk.incubator.vector`). Maven only for `jmh/run.sh`.
|
||||
The incubator module must be named explicitly at compile time *and* run time:
|
||||
|
||||
```bash
|
||||
javac --add-modules jdk.incubator.vector ...
|
||||
java --add-modules jdk.incubator.vector ...
|
||||
```
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
# 1. exploratory timings, C2 auto-vectorization on (the default)
|
||||
./timings/run.sh
|
||||
|
||||
# 2. the same experiments with SuperWord disabled — the interesting comparison
|
||||
./timings/run.sh -XX:-UseSuperWord
|
||||
|
||||
# 3. JMH, all benchmarks
|
||||
./jmh/run.sh
|
||||
|
||||
# ...or one class, with a short config
|
||||
./jmh/run.sh MaskedBench -wi 3 -i 5 -w 1 -r 1 -f 1
|
||||
```
|
||||
|
||||
`timings/run.sh` prints the JDK build, OS, CPU count and the preferred vector
|
||||
species before it measures anything, so a pasted result always carries the
|
||||
environment it came from.
|
||||
|
||||
## What to expect
|
||||
|
||||
On the AVX2 machine described in [RESULTS.md](RESULTS.md):
|
||||
|
||||
| Kernel | Scalar | Vector API | Ratio |
|
||||
|---|---|---|---|
|
||||
| square-and-add (JMH) | 838.6 ± 106.7 ns | 814.5 ± 10.8 ns | indistinguishable — C2 already vectorized it |
|
||||
| masked filter-and-sum (JMH) | 5189.9 ± 250.6 ns | 1036.4 ± 27.7 ns | **5.0x** |
|
||||
| brightness + clamp (JMH) | 6510.1 ± 302.6 ns | 1133.1 ± 17.8 ns | **5.7x** |
|
||||
|
||||
The pattern is the point: where C2 auto-vectorizes, explicit code buys nothing;
|
||||
where a data-dependent branch stops it, masks are worth roughly 5x. Confirm which
|
||||
case you are in with `-XX:-UseSuperWord` before rewriting anything — if the scalar
|
||||
loop does not get slower with that flag, C2 was not vectorizing it in the first
|
||||
place.
|
||||
|
||||
These ratios are specific to this JDK, CPU, data size and threshold
|
||||
distribution. Run them on your own hardware before quoting them.
|
||||
|
||||
## Further reading
|
||||
|
||||
* [The article this repo belongs to](https://ankurm.com/java-vector-api-jep-537-simd-guide/)
|
||||
* [JEP 537: Vector API (Twelfth Incubator)](https://openjdk.org/jeps/537)
|
||||
* [`jdk.incubator.vector` package docs](https://docs.oracle.com/en/java/javase/21/docs/api/jdk.incubator.vector/jdk/incubator/vector/package-summary.html)
|
||||
* [JMH](https://github.com/openjdk/jmh)
|
||||
|
||||
## License
|
||||
|
||||
MIT — see [LICENSE](LICENSE).
|
||||
|
||||
123
RESULTS.md
Normal file
123
RESULTS.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Captured results
|
||||
|
||||
Everything below was produced by the code in this repository. Your numbers will
|
||||
differ — that is the point of shipping the code rather than only the table.
|
||||
|
||||
## Environment
|
||||
|
||||
```
|
||||
JDK : OpenJDK 21.0.2+13-58 (HotSpot 64-Bit Server VM, mixed mode)
|
||||
OS : Ubuntu 22.04.5 LTS, Linux 6.8.0-124-generic, x86_64
|
||||
CPU : AMD Ryzen 5 5600U (Zen 3), 2 vCPUs (1 core / 2 threads) exposed
|
||||
Hypervisor : Microsoft Hyper-V, full virtualization
|
||||
ISA : AVX2 + FMA; no AVX-512 -> FloatVector.SPECIES_PREFERRED = 8 lanes, 256-bit
|
||||
Caches : L1d 32 KiB, L2 512 KiB, L3 16 MiB (as reported to the guest)
|
||||
JVM flags : --add-modules jdk.incubator.vector (plus -XX:-UseSuperWord where noted)
|
||||
Threads : single-threaded throughout
|
||||
Data : 8192 floats = 32 KiB per array, cache-resident
|
||||
Frequency : not controllable inside the guest (no cpufreq governor exposed)
|
||||
```
|
||||
|
||||
The last line matters: on a virtualized, frequency-scaled machine, absolute
|
||||
nanoseconds are soft. Ratios measured back-to-back in the same JVM are far more
|
||||
stable than the absolute numbers, and the JMH table is more trustworthy than the
|
||||
exploratory one.
|
||||
|
||||
## 1. Exploratory timings — `timings/run.sh`
|
||||
|
||||
Harness: 50,000 warmup calls, then 15 rounds of 5,000 calls; median reported.
|
||||
These are teaching experiments, not statistically rigorous benchmarks.
|
||||
|
||||
```
|
||||
--- kernel 1: c[i] = -(a[i]^2 + b[i]^2), 8192 floats ---
|
||||
scalar for-loop median 832.0 ns/call best 819.6 ns/call
|
||||
Vector API median 1279.9 ns/call best 1207.8 ns/call
|
||||
vector advantage 0.65x
|
||||
|
||||
--- kernel 2: masked filter-and-sum, 8192 floats ---
|
||||
scalar sum = 3084.1404
|
||||
vector sum = 3084.1448 (same value, different rounding)
|
||||
scalar (branch) median 5021.9 ns/call best 4926.9 ns/call
|
||||
Vector API (masked) median 1029.9 ns/call best 1021.4 ns/call
|
||||
vector advantage 4.88x
|
||||
|
||||
--- kernel 3: image brightness + clamp, 8192 pixels ---
|
||||
scalar (branchy clamp) median 6881.6 ns/call best 6839.2 ns/call
|
||||
Vector API (max/min) median 1243.8 ns/call best 1222.6 ns/call
|
||||
vector advantage 5.53x
|
||||
|
||||
--- floating-point reduction ordering ---
|
||||
scalar (left to right) : 4103.688965
|
||||
vector (lanes, then reduce): 4103.681641
|
||||
```
|
||||
|
||||
## 2. Same experiments with auto-vectorization off — `timings/run.sh -XX:-UseSuperWord`
|
||||
|
||||
```
|
||||
--- kernel 1 --- scalar 3427.0 ns/call vector 1230.1 ns/call -> 2.79x
|
||||
--- kernel 2 --- scalar 5040.4 ns/call vector 1031.3 ns/call -> 4.89x
|
||||
--- kernel 3 --- scalar 6883.9 ns/call vector 1364.1 ns/call -> 5.05x
|
||||
```
|
||||
|
||||
Read the two runs together — that comparison is the whole experiment:
|
||||
|
||||
| Scalar loop | SuperWord ON | SuperWord OFF | Conclusion |
|
||||
|---|---|---|---|
|
||||
| kernel 1 (straight-line math) | 832 ns | 3427 ns | C2 **was** vectorizing it; disabling SuperWord costs 4.1x |
|
||||
| kernel 2 (data-dependent branch) | 5022 ns | 5040 ns | unchanged, so C2 was **not** vectorizing it |
|
||||
| kernel 3 (branchy clamp) | 6882 ns | 6884 ns | unchanged, so C2 was **not** vectorizing it |
|
||||
|
||||
The Vector API versions are essentially unmoved by the flag in all three cases,
|
||||
which is the property the API is actually selling.
|
||||
|
||||
Run-to-run spread on this machine was a few percent for kernels 1 and 2, and up
|
||||
to ~13% for kernel 3 (5.05x–6.28x observed across runs) — another reminder to
|
||||
quote JMH rather than a stopwatch.
|
||||
|
||||
## 3. JMH — `jmh/run.sh`
|
||||
|
||||
JMH 1.37, 1 fork, 3 warmup iterations x 1 s, 5 measurement iterations x 1 s,
|
||||
single thread, average time per operation.
|
||||
|
||||
```
|
||||
Benchmark Mode Cnt Score Error Units
|
||||
VectorBench.scalar avgt 5 838.631 ± 106.657 ns/op
|
||||
VectorBench.vector avgt 5 814.531 ± 10.766 ns/op
|
||||
VectorBench.vectorChecksum avgt 5 7409.993 ± 224.753 ns/op
|
||||
MaskedBench.scalarBranch avgt 5 5189.874 ± 250.597 ns/op
|
||||
MaskedBench.vectorMasked avgt 5 1036.419 ± 27.726 ns/op
|
||||
BrightnessBench.scalarClamp avgt 5 6510.091 ± 302.638 ns/op
|
||||
BrightnessBench.vectorClamp avgt 5 1133.073 ± 17.763 ns/op
|
||||
```
|
||||
|
||||
Three readings:
|
||||
|
||||
1. **Square-and-add: no measurable difference.** 838.6 ± 106.7 vs 814.5 ± 10.8 —
|
||||
the error bars overlap. On a loop C2 already vectorizes, hand-writing the
|
||||
vector loop bought nothing measurable here. (The exploratory harness made the
|
||||
vector version look ~35% slower; that gap is harness overhead, not the kernel.
|
||||
This is exactly why the JMH number is the one to quote.)
|
||||
2. **Masked filter: 5.0x** (5189.9 / 1036.4), with tight error bars on the vector side.
|
||||
3. **Brightness clamp: 5.7x** (6510.1 / 1133.1).
|
||||
|
||||
`vectorChecksum` is a deliberate cautionary example: returning a checksum does
|
||||
prevent dead-code elimination, but the checksum loop is then *inside* the
|
||||
measurement, and the benchmark reports ~9x the time of the kernel it was meant to
|
||||
measure. Consuming the output array through a `Blackhole` keeps the measurement
|
||||
honest without adding work.
|
||||
|
||||
## What is not reproduced here
|
||||
|
||||
The article shows an excerpt of the AVX2 machine code C2 emits for the
|
||||
square-and-add loop. Reproducing that needs an `hsdis` disassembler plugin
|
||||
installed next to your JDK:
|
||||
|
||||
```
|
||||
java --add-modules jdk.incubator.vector \
|
||||
-XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly \
|
||||
-XX:CompileCommand=print,com.ankurm.vectorapi.SquareAdd::vectorComputation \
|
||||
-cp out com.ankurm.vectorapi.Main
|
||||
```
|
||||
|
||||
Without `hsdis` the JVM prints `Loading hsdis library failed` and falls back to a
|
||||
hex dump.
|
||||
85
jmh/pom.xml
Normal file
85
jmh/pom.xml
Normal file
@@ -0,0 +1,85 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>vector-api-jmh</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<packaging>jar</packaging>
|
||||
<name>Vector API JMH benchmarks</name>
|
||||
|
||||
<properties>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<maven.compiler.release>21</maven.compiler.release>
|
||||
<jmh.version>1.37</jmh.version>
|
||||
<uberjar.name>benchmarks</uberjar.name>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.openjdk.jmh</groupId>
|
||||
<artifactId>jmh-core</artifactId>
|
||||
<version>${jmh.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.openjdk.jmh</groupId>
|
||||
<artifactId>jmh-generator-annprocess</artifactId>
|
||||
<version>${jmh.version}</version>
|
||||
<scope>provided</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<version>3.13.0</version>
|
||||
<configuration>
|
||||
<compilerArgs>
|
||||
<arg>--add-modules</arg>
|
||||
<arg>jdk.incubator.vector</arg>
|
||||
</compilerArgs>
|
||||
<annotationProcessorPaths>
|
||||
<path>
|
||||
<groupId>org.openjdk.jmh</groupId>
|
||||
<artifactId>jmh-generator-annprocess</artifactId>
|
||||
<version>${jmh.version}</version>
|
||||
</path>
|
||||
</annotationProcessorPaths>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-shade-plugin</artifactId>
|
||||
<version>3.5.3</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<phase>package</phase>
|
||||
<goals><goal>shade</goal></goals>
|
||||
<configuration>
|
||||
<finalName>${uberjar.name}</finalName>
|
||||
<transformers>
|
||||
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
|
||||
<mainClass>org.openjdk.jmh.Main</mainClass>
|
||||
</transformer>
|
||||
</transformers>
|
||||
<filters>
|
||||
<filter>
|
||||
<artifact>*:*</artifact>
|
||||
<excludes>
|
||||
<exclude>META-INF/*.SF</exclude>
|
||||
<exclude>META-INF/*.DSA</exclude>
|
||||
<exclude>META-INF/*.RSA</exclude>
|
||||
</excludes>
|
||||
</filter>
|
||||
</filters>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
15
jmh/run-without-maven.sh
Normal file
15
jmh/run-without-maven.sh
Normal file
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
# Same benchmarks without Maven: fetch the four jars, compile with the JMH
|
||||
# annotation processor, run org.openjdk.jmh.Main directly.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
mkdir -p lib out
|
||||
for u in org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar \
|
||||
org/openjdk/jmh/jmh-generator-annprocess/1.37/jmh-generator-annprocess-1.37.jar \
|
||||
net/sf/jopt-simple/jopt-simple/5.0.4/jopt-simple-5.0.4.jar \
|
||||
org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.jar; do
|
||||
[ -f "lib/$(basename "$u")" ] || curl -sSL -o "lib/$(basename "$u")" "https://repo1.maven.org/maven2/$u"
|
||||
done
|
||||
CP=$(ls lib/*.jar | tr '\n' ':')
|
||||
javac --add-modules jdk.incubator.vector -cp "$CP" -proc:full -d out $(find src -name '*.java')
|
||||
java --add-modules jdk.incubator.vector -cp "out:$CP" org.openjdk.jmh.Main "$@"
|
||||
8
jmh/run.sh
Normal file
8
jmh/run.sh
Normal file
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build and run the JMH benchmarks (the numbers worth defending).
|
||||
# ./run.sh -> all benchmarks
|
||||
# ./run.sh MaskedBench -> one class
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
mvn -q -B clean package
|
||||
java --add-modules jdk.incubator.vector -jar target/benchmarks.jar "$@"
|
||||
55
jmh/src/main/java/com/ankurm/vectorapi/BrightnessBench.java
Normal file
55
jmh/src/main/java/com/ankurm/vectorapi/BrightnessBench.java
Normal file
@@ -0,0 +1,55 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
import org.openjdk.jmh.annotations.*;
|
||||
import org.openjdk.jmh.infra.Blackhole;
|
||||
import java.util.Random;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
/** JMH version of the image brightness + clamp kernel. */
|
||||
@BenchmarkMode(Mode.AverageTime)
|
||||
@OutputTimeUnit(TimeUnit.NANOSECONDS)
|
||||
@State(Scope.Thread)
|
||||
@Warmup(iterations = 5, time = 1)
|
||||
@Measurement(iterations = 5, time = 1)
|
||||
@Fork(1)
|
||||
public class BrightnessBench {
|
||||
static final VectorSpecies<Float> SP = FloatVector.SPECIES_PREFERRED;
|
||||
float[] in, out;
|
||||
float gain = 1.4f, bias = 12f;
|
||||
|
||||
@Setup
|
||||
public void setup() {
|
||||
in = new float[8192]; out = new float[8192];
|
||||
Random r = new Random(11);
|
||||
for (int i = 0; i < in.length; i++) in[i] = r.nextFloat() * 300f - 20f;
|
||||
}
|
||||
|
||||
@Benchmark
|
||||
public void scalarClamp(Blackhole bh) {
|
||||
for (int i = 0; i < in.length; i++) {
|
||||
float v = in[i] * gain + bias;
|
||||
if (v < 0f) v = 0f; else if (v > 255f) v = 255f;
|
||||
out[i] = v;
|
||||
}
|
||||
bh.consume(out);
|
||||
}
|
||||
|
||||
@Benchmark
|
||||
public void vectorClamp(Blackhole bh) {
|
||||
var vg = FloatVector.broadcast(SP, gain);
|
||||
var vb = FloatVector.broadcast(SP, bias);
|
||||
var lo = FloatVector.zero(SP);
|
||||
var hi = FloatVector.broadcast(SP, 255f);
|
||||
int i = 0, bound = SP.loopBound(in.length);
|
||||
for (; i < bound; i += SP.length()) {
|
||||
FloatVector.fromArray(SP, in, i).fma(vg, vb).max(lo).min(hi).intoArray(out, i);
|
||||
}
|
||||
for (; i < in.length; i++) {
|
||||
float v = in[i] * gain + bias;
|
||||
if (v < 0f) v = 0f; else if (v > 255f) v = 255f;
|
||||
out[i] = v;
|
||||
}
|
||||
bh.consume(out);
|
||||
}
|
||||
}
|
||||
47
jmh/src/main/java/com/ankurm/vectorapi/MaskedBench.java
Normal file
47
jmh/src/main/java/com/ankurm/vectorapi/MaskedBench.java
Normal file
@@ -0,0 +1,47 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
import org.openjdk.jmh.annotations.*;
|
||||
import java.util.Random;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
/** JMH version of the masked filter-and-sum: the headline speedup case. */
|
||||
@BenchmarkMode(Mode.AverageTime)
|
||||
@OutputTimeUnit(TimeUnit.NANOSECONDS)
|
||||
@State(Scope.Thread)
|
||||
@Warmup(iterations = 5, time = 1)
|
||||
@Measurement(iterations = 5, time = 1)
|
||||
@Fork(1)
|
||||
public class MaskedBench {
|
||||
static final VectorSpecies<Float> SP = FloatVector.SPECIES_PREFERRED;
|
||||
static final float THRESH = 0.5f;
|
||||
float[] a;
|
||||
|
||||
@Setup
|
||||
public void setup() {
|
||||
a = new float[8192];
|
||||
Random r = new Random(7);
|
||||
for (int i = 0; i < a.length; i++) a[i] = r.nextFloat();
|
||||
}
|
||||
|
||||
// Both methods return their result, so JMH consumes it and nothing is dead.
|
||||
@Benchmark
|
||||
public float scalarBranch() {
|
||||
float sum = 0;
|
||||
for (int i = 0; i < a.length; i++) if (a[i] > THRESH) sum += a[i];
|
||||
return sum;
|
||||
}
|
||||
|
||||
@Benchmark
|
||||
public float vectorMasked() {
|
||||
FloatVector acc = FloatVector.zero(SP);
|
||||
int i = 0, bound = SP.loopBound(a.length);
|
||||
for (; i < bound; i += SP.length()) {
|
||||
var v = FloatVector.fromArray(SP, a, i);
|
||||
acc = acc.add(v, v.compare(VectorOperators.GT, THRESH));
|
||||
}
|
||||
float sum = acc.reduceLanes(VectorOperators.ADD);
|
||||
for (; i < a.length; i++) if (a[i] > THRESH) sum += a[i];
|
||||
return sum;
|
||||
}
|
||||
}
|
||||
63
jmh/src/main/java/com/ankurm/vectorapi/VectorBench.java
Normal file
63
jmh/src/main/java/com/ankurm/vectorapi/VectorBench.java
Normal file
@@ -0,0 +1,63 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
import org.openjdk.jmh.annotations.*;
|
||||
import org.openjdk.jmh.infra.Blackhole;
|
||||
import java.util.Random;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
@BenchmarkMode(Mode.AverageTime)
|
||||
@OutputTimeUnit(TimeUnit.NANOSECONDS)
|
||||
@State(Scope.Thread)
|
||||
@Warmup(iterations = 5, time = 1)
|
||||
@Measurement(iterations = 5, time = 1)
|
||||
@Fork(1)
|
||||
public class VectorBench {
|
||||
static final VectorSpecies<Float> SP = FloatVector.SPECIES_PREFERRED;
|
||||
float[] a, b, c;
|
||||
|
||||
@Setup
|
||||
public void setup() {
|
||||
int n = 8192;
|
||||
a = new float[n]; b = new float[n]; c = new float[n];
|
||||
Random r = new Random(42);
|
||||
for (int i = 0; i < n; i++) { a[i] = r.nextFloat(); b[i] = r.nextFloat(); }
|
||||
}
|
||||
|
||||
// The result array is written, then handed to JMH's Blackhole so neither the
|
||||
// stores nor the arithmetic can be eliminated as dead code.
|
||||
@Benchmark
|
||||
public void scalar(Blackhole bh) {
|
||||
for (int i = 0; i < a.length; i++) c[i] = -(a[i]*a[i] + b[i]*b[i]);
|
||||
bh.consume(c);
|
||||
}
|
||||
|
||||
@Benchmark
|
||||
public void vector(Blackhole bh) {
|
||||
int i = 0, bound = SP.loopBound(a.length);
|
||||
for (; i < bound; i += SP.length()) {
|
||||
var va = FloatVector.fromArray(SP, a, i);
|
||||
var vb = FloatVector.fromArray(SP, b, i);
|
||||
va.mul(va).add(vb.mul(vb)).neg().intoArray(c, i);
|
||||
}
|
||||
for (; i < a.length; i++) c[i] = -(a[i]*a[i] + b[i]*b[i]);
|
||||
bh.consume(c);
|
||||
}
|
||||
|
||||
// Same kernel, but returning a value derived from the output. Returned values
|
||||
// are consumed by JMH automatically. Note what this costs: the checksum loop is
|
||||
// itself measured, which is why this variant reports ~9x the time of vector().
|
||||
@Benchmark
|
||||
public float vectorChecksum() {
|
||||
int i = 0, bound = SP.loopBound(a.length);
|
||||
for (; i < bound; i += SP.length()) {
|
||||
var va = FloatVector.fromArray(SP, a, i);
|
||||
var vb = FloatVector.fromArray(SP, b, i);
|
||||
va.mul(va).add(vb.mul(vb)).neg().intoArray(c, i);
|
||||
}
|
||||
for (; i < a.length; i++) c[i] = -(a[i]*a[i] + b[i]*b[i]);
|
||||
float sum = 0;
|
||||
for (float v : c) sum += v;
|
||||
return sum;
|
||||
}
|
||||
}
|
||||
9
timings/run.sh
Normal file
9
timings/run.sh
Normal file
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env bash
|
||||
# Compile and run the exploratory timing experiments from the article.
|
||||
# ./run.sh -> default run (C2 auto-vectorization ON)
|
||||
# ./run.sh -XX:-UseSuperWord -> same experiments with SuperWord disabled
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
mkdir -p out
|
||||
javac --add-modules jdk.incubator.vector -d out $(find src -name '*.java')
|
||||
java --add-modules jdk.incubator.vector "$@" -cp out com.ankurm.vectorapi.Main
|
||||
41
timings/src/main/java/com/ankurm/vectorapi/Bench.java
Normal file
41
timings/src/main/java/com/ankurm/vectorapi/Bench.java
Normal file
@@ -0,0 +1,41 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import java.util.Arrays;
|
||||
|
||||
/**
|
||||
* A deliberately small timing harness. It is NOT a statistical benchmark:
|
||||
* it warms the JIT up, then reports the median and best of several rounds so
|
||||
* that large effects (2x and up) are visible. Use JMH for anything you intend
|
||||
* to defend -- see the jmh/ module in this repository.
|
||||
*/
|
||||
public final class Bench {
|
||||
public static final int WARMUP_CALLS = 50_000;
|
||||
public static final int ROUND_CALLS = 5_000;
|
||||
public static final int ROUNDS = 15;
|
||||
|
||||
public interface Kernel { void run(); }
|
||||
|
||||
/** @return median ns/call over ROUNDS rounds (also prints best). */
|
||||
public static double measure(String label, Kernel k) {
|
||||
for (int i = 0; i < WARMUP_CALLS; i++) k.run(); // force C2 compilation
|
||||
double[] perCall = new double[ROUNDS];
|
||||
for (int r = 0; r < ROUNDS; r++) {
|
||||
long t0 = System.nanoTime();
|
||||
for (int i = 0; i < ROUND_CALLS; i++) k.run();
|
||||
long t1 = System.nanoTime();
|
||||
perCall[r] = (t1 - t0) / (double) ROUND_CALLS;
|
||||
}
|
||||
double[] sorted = perCall.clone();
|
||||
Arrays.sort(sorted);
|
||||
double median = sorted[ROUNDS / 2];
|
||||
double best = sorted[0];
|
||||
System.out.printf("%-24s median %8.1f ns/call best %8.1f ns/call%n", label, median, best);
|
||||
return median;
|
||||
}
|
||||
|
||||
public static void speedup(String label, double scalarNs, double vectorNs) {
|
||||
System.out.printf("%-24s %.2fx%n", label, scalarNs / vectorNs);
|
||||
}
|
||||
|
||||
private Bench() {}
|
||||
}
|
||||
51
timings/src/main/java/com/ankurm/vectorapi/Brightness.java
Normal file
51
timings/src/main/java/com/ankurm/vectorapi/Brightness.java
Normal file
@@ -0,0 +1,51 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
|
||||
/** Image brightness: scale, bias, clamp to [0, 255]. The clamp is two branches. */
|
||||
public final class Brightness {
|
||||
|
||||
static final VectorSpecies<Float> SP = FloatVector.SPECIES_PREFERRED;
|
||||
|
||||
// Scalar: the clamp is two data-dependent branches
|
||||
static void brightScalar(float[] in, float[] out, float gain, float bias) {
|
||||
for (int i = 0; i < in.length; i++) {
|
||||
float v = in[i] * gain + bias;
|
||||
if (v < 0f) v = 0f; else if (v > 255f) v = 255f;
|
||||
out[i] = v;
|
||||
}
|
||||
}
|
||||
|
||||
static void brightVector(float[] in, float[] out, float gain, float bias) {
|
||||
var vg = FloatVector.broadcast(SP, gain);
|
||||
var vb = FloatVector.broadcast(SP, bias);
|
||||
var lo = FloatVector.zero(SP);
|
||||
var hi = FloatVector.broadcast(SP, 255f);
|
||||
int i = 0, bound = SP.loopBound(in.length);
|
||||
for (; i < bound; i += SP.length()) {
|
||||
FloatVector.fromArray(SP, in, i).fma(vg, vb).max(lo).min(hi).intoArray(out, i);
|
||||
}
|
||||
for (; i < in.length; i++) { // scalar tail
|
||||
float v = in[i] * gain + bias;
|
||||
if (v < 0f) v = 0f; else if (v > 255f) v = 255f;
|
||||
out[i] = v;
|
||||
}
|
||||
}
|
||||
|
||||
static void run(int n) {
|
||||
float[] in = Data.randomFloats(n, 4);
|
||||
for (int i = 0; i < n; i++) in[i] = in[i] * 300f - 20f; // some pixels clamp at both ends
|
||||
float[] o1 = new float[n], o2 = new float[n];
|
||||
brightScalar(in, o1, 1.4f, 12f);
|
||||
brightVector(in, o2, 1.4f, 12f);
|
||||
Data.assertSame("brightness", o1, o2);
|
||||
|
||||
System.out.println("--- kernel 3: image brightness + clamp, " + n + " pixels ---");
|
||||
double s = Bench.measure("scalar (branchy clamp)", () -> brightScalar(in, o1, 1.4f, 12f));
|
||||
double v = Bench.measure("Vector API (max/min)", () -> brightVector(in, o2, 1.4f, 12f));
|
||||
Bench.speedup("vector advantage", s, v);
|
||||
System.out.println();
|
||||
}
|
||||
|
||||
private Brightness() {}
|
||||
}
|
||||
22
timings/src/main/java/com/ankurm/vectorapi/Data.java
Normal file
22
timings/src/main/java/com/ankurm/vectorapi/Data.java
Normal file
@@ -0,0 +1,22 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import java.util.Random;
|
||||
|
||||
public final class Data {
|
||||
static float[] randomFloats(int n, long seed) {
|
||||
Random r = new Random(seed);
|
||||
float[] a = new float[n];
|
||||
for (int i = 0; i < n; i++) a[i] = r.nextFloat();
|
||||
return a;
|
||||
}
|
||||
|
||||
static void assertSame(String what, float[] x, float[] y) {
|
||||
for (int i = 0; i < x.length; i++) {
|
||||
if (Math.abs(x[i] - y[i]) > 1e-4f) {
|
||||
throw new AssertionError(what + ": mismatch at " + i + ": " + x[i] + " != " + y[i]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private Data() {}
|
||||
}
|
||||
24
timings/src/main/java/com/ankurm/vectorapi/Env.java
Normal file
24
timings/src/main/java/com/ankurm/vectorapi/Env.java
Normal file
@@ -0,0 +1,24 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
|
||||
/** Prints the exact environment a benchmark run happened on. */
|
||||
public final class Env {
|
||||
public static void print() {
|
||||
Runtime rt = Runtime.getRuntime();
|
||||
System.out.println("=== environment ===");
|
||||
System.out.println("JDK : " + System.getProperty("java.vm.name")
|
||||
+ " " + System.getProperty("java.vm.version"));
|
||||
System.out.println("java.version : " + System.getProperty("java.version"));
|
||||
System.out.println("OS : " + System.getProperty("os.name") + " "
|
||||
+ System.getProperty("os.version") + " (" + System.getProperty("os.arch") + ")");
|
||||
System.out.println("Available procs: " + rt.availableProcessors());
|
||||
System.out.println("Preferred spec : " + FloatVector.SPECIES_PREFERRED
|
||||
+ " (" + FloatVector.SPECIES_PREFERRED.length() + " float lanes, "
|
||||
+ FloatVector.SPECIES_PREFERRED.vectorBitSize() + "-bit)");
|
||||
System.out.println("Max vector size: " + IntVector.SPECIES_PREFERRED.vectorBitSize() + "-bit (int)");
|
||||
System.out.println();
|
||||
}
|
||||
|
||||
private Env() {}
|
||||
}
|
||||
25
timings/src/main/java/com/ankurm/vectorapi/Main.java
Normal file
25
timings/src/main/java/com/ankurm/vectorapi/Main.java
Normal file
@@ -0,0 +1,25 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
/**
|
||||
* Runs every timing experiment quoted in the article, on whatever machine you
|
||||
* are sitting at. These are exploratory timings, not statistically rigorous
|
||||
* benchmarks -- see the jmh/ module for the harness you would defend numbers with.
|
||||
*
|
||||
* javac --add-modules jdk.incubator.vector -d out $(find src -name '*.java')
|
||||
* java --add-modules jdk.incubator.vector -cp out com.ankurm.vectorapi.Main
|
||||
* java --add-modules jdk.incubator.vector -XX:-UseSuperWord -cp out com.ankurm.vectorapi.Main
|
||||
*/
|
||||
public final class Main {
|
||||
public static void main(String[] args) {
|
||||
int n = args.length > 0 ? Integer.parseInt(args[0]) : 8192;
|
||||
Env.print();
|
||||
System.out.println("array length : " + n + " floats (" + (n * 4 / 1024) + " KiB, cache-resident)");
|
||||
System.out.println();
|
||||
SquareAdd.run(n);
|
||||
MaskedFilter.run(n);
|
||||
Brightness.run(n);
|
||||
ReductionOrder.run();
|
||||
}
|
||||
|
||||
private Main() {}
|
||||
}
|
||||
45
timings/src/main/java/com/ankurm/vectorapi/MaskedFilter.java
Normal file
45
timings/src/main/java/com/ankurm/vectorapi/MaskedFilter.java
Normal file
@@ -0,0 +1,45 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
|
||||
/** Sum only the elements above a threshold -- a data-dependent branch. */
|
||||
public final class MaskedFilter {
|
||||
|
||||
static final float THRESH = 0.5f;
|
||||
static final VectorSpecies<Float> SP = FloatVector.SPECIES_PREFERRED;
|
||||
|
||||
// Scalar: the branch is data-dependent, so C2 generally cannot vectorize this form
|
||||
static float scalar(float[] a) {
|
||||
float sum = 0;
|
||||
for (int i = 0; i < a.length; i++)
|
||||
if (a[i] > THRESH) sum += a[i];
|
||||
return sum;
|
||||
}
|
||||
|
||||
static float vector(float[] a) {
|
||||
FloatVector acc = FloatVector.zero(SP);
|
||||
int i = 0, bound = SP.loopBound(a.length);
|
||||
for (; i < bound; i += SP.length()) {
|
||||
var v = FloatVector.fromArray(SP, a, i);
|
||||
VectorMask<Float> m = v.compare(VectorOperators.GT, THRESH);
|
||||
acc = acc.add(v, m); // add only where mask is true
|
||||
}
|
||||
float sum = acc.reduceLanes(VectorOperators.ADD); // horizontal sum of the lanes
|
||||
for (; i < a.length; i++) if (a[i] > THRESH) sum += a[i]; // masked-off tail
|
||||
return sum;
|
||||
}
|
||||
|
||||
static void run(int n) {
|
||||
float[] a = Data.randomFloats(n, 3);
|
||||
float s0 = scalar(a), v0 = vector(a);
|
||||
|
||||
System.out.println("--- kernel 2: masked filter-and-sum, " + n + " floats ---");
|
||||
System.out.printf("scalar sum = %.4f%nvector sum = %.4f (same value, different rounding)%n", s0, v0);
|
||||
double s = Bench.measure("scalar (branch)", () -> Sink.consume(scalar(a)));
|
||||
double v = Bench.measure("Vector API (masked)", () -> Sink.consume(vector(a)));
|
||||
Bench.speedup("vector advantage", s, v);
|
||||
System.out.println();
|
||||
}
|
||||
|
||||
private MaskedFilter() {}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
|
||||
/** Why a vector reduction and a scalar loop disagree in the last bits. */
|
||||
public final class ReductionOrder {
|
||||
|
||||
static final VectorSpecies<Float> SP = FloatVector.SPECIES_PREFERRED;
|
||||
|
||||
static float scalarSum(float[] a) {
|
||||
float sum = 0;
|
||||
for (float v : a) sum += v; // strictly left to right
|
||||
return sum;
|
||||
}
|
||||
|
||||
static float vectorSum(float[] a) {
|
||||
FloatVector acc = FloatVector.zero(SP);
|
||||
int i = 0, bound = SP.loopBound(a.length);
|
||||
for (; i < bound; i += SP.length()) acc = acc.add(FloatVector.fromArray(SP, a, i));
|
||||
float sum = acc.reduceLanes(VectorOperators.ADD); // lanes combined at the end
|
||||
for (; i < a.length; i++) sum += a[i];
|
||||
return sum;
|
||||
}
|
||||
|
||||
static void run() {
|
||||
float[] a = Data.randomFloats(8192, 5);
|
||||
System.out.println("--- floating-point reduction ordering ---");
|
||||
System.out.printf("scalar (left to right) : %.6f%n", scalarSum(a));
|
||||
System.out.printf("vector (lanes, then reduce): %.6f%n", vectorSum(a));
|
||||
System.out.println("Same mathematical sum; different addition order, so different rounding.");
|
||||
System.out.println();
|
||||
}
|
||||
|
||||
private ReductionOrder() {}
|
||||
}
|
||||
10
timings/src/main/java/com/ankurm/vectorapi/Sink.java
Normal file
10
timings/src/main/java/com/ankurm/vectorapi/Sink.java
Normal file
@@ -0,0 +1,10 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
/** Keeps computed values observable so the JIT cannot delete the computation. */
|
||||
public final class Sink {
|
||||
public static volatile float sink;
|
||||
|
||||
public static void consume(float v) { sink = v; }
|
||||
|
||||
private Sink() {}
|
||||
}
|
||||
50
timings/src/main/java/com/ankurm/vectorapi/SquareAdd.java
Normal file
50
timings/src/main/java/com/ankurm/vectorapi/SquareAdd.java
Normal file
@@ -0,0 +1,50 @@
|
||||
package com.ankurm.vectorapi;
|
||||
|
||||
import jdk.incubator.vector.*;
|
||||
|
||||
/** c[i] = -(a[i]^2 + b[i]^2) -- the loop C2's SuperWord pass handles well. */
|
||||
public final class SquareAdd {
|
||||
|
||||
static final VectorSpecies<Float> SPECIES = FloatVector.SPECIES_PREFERRED;
|
||||
|
||||
static void scalarComputation(float[] a, float[] b, float[] c) {
|
||||
for (int i = 0; i < a.length; i++) {
|
||||
c[i] = -(a[i] * a[i] + b[i] * b[i]);
|
||||
}
|
||||
}
|
||||
|
||||
static void vectorComputation(float[] a, float[] b, float[] c) {
|
||||
int i = 0;
|
||||
int upperBound = SPECIES.loopBound(a.length); // e.g. 8192 -> 8192, 8190 -> 8184
|
||||
|
||||
// Main loop: process SPECIES.length() elements per iteration
|
||||
for (; i < upperBound; i += SPECIES.length()) {
|
||||
var va = FloatVector.fromArray(SPECIES, a, i);
|
||||
var vb = FloatVector.fromArray(SPECIES, b, i);
|
||||
va.mul(va).add(vb.mul(vb)).neg().intoArray(c, i);
|
||||
}
|
||||
|
||||
// Scalar tail: the last (a.length % laneCount) elements
|
||||
for (; i < a.length; i++) {
|
||||
c[i] = -(a[i] * a[i] + b[i] * b[i]);
|
||||
}
|
||||
}
|
||||
|
||||
static void run(int n) {
|
||||
float[] a = Data.randomFloats(n, 1);
|
||||
float[] b = Data.randomFloats(n, 2);
|
||||
float[] c1 = new float[n], c2 = new float[n];
|
||||
|
||||
scalarComputation(a, b, c1);
|
||||
vectorComputation(a, b, c2);
|
||||
Data.assertSame("square-add", c1, c2);
|
||||
|
||||
System.out.println("--- kernel 1: c[i] = -(a[i]^2 + b[i]^2), " + n + " floats ---");
|
||||
double s = Bench.measure("scalar for-loop", () -> scalarComputation(a, b, c1));
|
||||
double v = Bench.measure("Vector API", () -> vectorComputation(a, b, c2));
|
||||
Bench.speedup("vector advantage", s, v);
|
||||
System.out.println();
|
||||
}
|
||||
|
||||
private SquareAdd() {}
|
||||
}
|
||||
Reference in New Issue
Block a user