diff --git a/README.md b/README.md index cf0845b..f9dcf5c 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,117 @@ -# vector-api-jep-537-demo +# Java Vector API (JEP 537) — runnable demo -Runnable companion code for the Java Vector API (JEP 537) guide on ankurm.com: SIMD kernels, masks, the loopBound+tail pattern, and JMH benchmarks on JDK 21. \ No newline at end of file +Companion code for **[Java Vector API (JEP 537): SIMD, Auto-Vectorization, Masks, and Real Benchmarks](https://ankurm.com/java-vector-api-jep-537-simd-guide/)** on ankurm.com. +Every code listing and every number in that article comes from this repository. + +## What this is about + +Your CPU has vector registers — 256 bits wide on AVX2, 512 on AVX-512, 128 on +Arm NEON — that hold several numbers side by side, in *lanes*. A single SIMD +instruction applies one operation to every lane at once. A plain Java `for` loop +touches one element per iteration, so most of that hardware sits idle. + +Java gives you two ways to use it: + +* **Auto-vectorization (free, implicit).** HotSpot's C2 compiler has a SuperWord + pass that fuses consecutive scalar iterations into vector ones. When it fires + you get SIMD for nothing. It fires only when C2 can prove the transform is safe + and profitable — no data-dependent branches, no aliasing it cannot rule out, + regular access patterns — and when it declines, it does so silently. +* **The Vector API (explicit).** `jdk.incubator.vector` lets you state the vector + intent yourself: load a lane-width slice, do lane-wise arithmetic, store it + back, and use *masks* instead of branches. It does not promise a fixed + instruction sequence on every CPU, but it gives far more predictable access to + SIMD than hoping C2 infers it. + +The API is still incubating — JEP 537 is its twelfth incubation, targeted at JDK +27 — because it is waiting on Project Valhalla's value classes. The shape of the +API is stable; the package name will change when it is promoted. + +This repo demonstrates both paths, and — more usefully — shows how to **tell +which one you are on**, by running the same kernels with `-XX:-UseSuperWord`. + +## Layout + +``` +timings/ exploratory timing experiments (plain javac, no dependencies) + src/main/java/com/ankurm/vectorapi/ + Main.java runs everything, prints the environment first + SquareAdd.java c[i] = -(a[i]^2 + b[i]^2) — the loop C2 vectorizes for free + MaskedFilter.java sum elements above a threshold — masks vs a branch + Brightness.java scale + bias + clamp — max/min vs two branches + ReductionOrder.java why vector and scalar sums differ in the last bits + Bench.java the (deliberately simple) timing harness + Env.java, Data.java, Sink.java + run.sh + +jmh/ the same kernels under JMH — the numbers worth defending + src/main/java/com/ankurm/vectorapi/ + VectorBench.java square-and-add, incl. a cautionary checksum variant + MaskedBench.java masked filter-and-sum + BrightnessBench.java brightness + clamp + pom.xml + run.sh Maven build + run + run-without-maven.sh same thing with curl + javac, if you have no Maven + +RESULTS.md everything the article quotes, with the exact machine it came from +``` + +## Requirements + +JDK 21 or newer (anything with `jdk.incubator.vector`). Maven only for `jmh/run.sh`. +The incubator module must be named explicitly at compile time *and* run time: + +```bash +javac --add-modules jdk.incubator.vector ... +java --add-modules jdk.incubator.vector ... +``` + +## Running it + +```bash +# 1. exploratory timings, C2 auto-vectorization on (the default) +./timings/run.sh + +# 2. the same experiments with SuperWord disabled — the interesting comparison +./timings/run.sh -XX:-UseSuperWord + +# 3. JMH, all benchmarks +./jmh/run.sh + +# ...or one class, with a short config +./jmh/run.sh MaskedBench -wi 3 -i 5 -w 1 -r 1 -f 1 +``` + +`timings/run.sh` prints the JDK build, OS, CPU count and the preferred vector +species before it measures anything, so a pasted result always carries the +environment it came from. + +## What to expect + +On the AVX2 machine described in [RESULTS.md](RESULTS.md): + +| Kernel | Scalar | Vector API | Ratio | +|---|---|---|---| +| square-and-add (JMH) | 838.6 ± 106.7 ns | 814.5 ± 10.8 ns | indistinguishable — C2 already vectorized it | +| masked filter-and-sum (JMH) | 5189.9 ± 250.6 ns | 1036.4 ± 27.7 ns | **5.0x** | +| brightness + clamp (JMH) | 6510.1 ± 302.6 ns | 1133.1 ± 17.8 ns | **5.7x** | + +The pattern is the point: where C2 auto-vectorizes, explicit code buys nothing; +where a data-dependent branch stops it, masks are worth roughly 5x. Confirm which +case you are in with `-XX:-UseSuperWord` before rewriting anything — if the scalar +loop does not get slower with that flag, C2 was not vectorizing it in the first +place. + +These ratios are specific to this JDK, CPU, data size and threshold +distribution. Run them on your own hardware before quoting them. + +## Further reading + +* [The article this repo belongs to](https://ankurm.com/java-vector-api-jep-537-simd-guide/) +* [JEP 537: Vector API (Twelfth Incubator)](https://openjdk.org/jeps/537) +* [`jdk.incubator.vector` package docs](https://docs.oracle.com/en/java/javase/21/docs/api/jdk.incubator.vector/jdk/incubator/vector/package-summary.html) +* [JMH](https://github.com/openjdk/jmh) + +## License + +MIT — see [LICENSE](LICENSE). diff --git a/RESULTS.md b/RESULTS.md new file mode 100644 index 0000000..b485b46 --- /dev/null +++ b/RESULTS.md @@ -0,0 +1,123 @@ +# Captured results + +Everything below was produced by the code in this repository. Your numbers will +differ — that is the point of shipping the code rather than only the table. + +## Environment + +``` +JDK : OpenJDK 21.0.2+13-58 (HotSpot 64-Bit Server VM, mixed mode) +OS : Ubuntu 22.04.5 LTS, Linux 6.8.0-124-generic, x86_64 +CPU : AMD Ryzen 5 5600U (Zen 3), 2 vCPUs (1 core / 2 threads) exposed +Hypervisor : Microsoft Hyper-V, full virtualization +ISA : AVX2 + FMA; no AVX-512 -> FloatVector.SPECIES_PREFERRED = 8 lanes, 256-bit +Caches : L1d 32 KiB, L2 512 KiB, L3 16 MiB (as reported to the guest) +JVM flags : --add-modules jdk.incubator.vector (plus -XX:-UseSuperWord where noted) +Threads : single-threaded throughout +Data : 8192 floats = 32 KiB per array, cache-resident +Frequency : not controllable inside the guest (no cpufreq governor exposed) +``` + +The last line matters: on a virtualized, frequency-scaled machine, absolute +nanoseconds are soft. Ratios measured back-to-back in the same JVM are far more +stable than the absolute numbers, and the JMH table is more trustworthy than the +exploratory one. + +## 1. Exploratory timings — `timings/run.sh` + +Harness: 50,000 warmup calls, then 15 rounds of 5,000 calls; median reported. +These are teaching experiments, not statistically rigorous benchmarks. + +``` +--- kernel 1: c[i] = -(a[i]^2 + b[i]^2), 8192 floats --- +scalar for-loop median 832.0 ns/call best 819.6 ns/call +Vector API median 1279.9 ns/call best 1207.8 ns/call +vector advantage 0.65x + +--- kernel 2: masked filter-and-sum, 8192 floats --- +scalar sum = 3084.1404 +vector sum = 3084.1448 (same value, different rounding) +scalar (branch) median 5021.9 ns/call best 4926.9 ns/call +Vector API (masked) median 1029.9 ns/call best 1021.4 ns/call +vector advantage 4.88x + +--- kernel 3: image brightness + clamp, 8192 pixels --- +scalar (branchy clamp) median 6881.6 ns/call best 6839.2 ns/call +Vector API (max/min) median 1243.8 ns/call best 1222.6 ns/call +vector advantage 5.53x + +--- floating-point reduction ordering --- +scalar (left to right) : 4103.688965 +vector (lanes, then reduce): 4103.681641 +``` + +## 2. Same experiments with auto-vectorization off — `timings/run.sh -XX:-UseSuperWord` + +``` +--- kernel 1 --- scalar 3427.0 ns/call vector 1230.1 ns/call -> 2.79x +--- kernel 2 --- scalar 5040.4 ns/call vector 1031.3 ns/call -> 4.89x +--- kernel 3 --- scalar 6883.9 ns/call vector 1364.1 ns/call -> 5.05x +``` + +Read the two runs together — that comparison is the whole experiment: + +| Scalar loop | SuperWord ON | SuperWord OFF | Conclusion | +|---|---|---|---| +| kernel 1 (straight-line math) | 832 ns | 3427 ns | C2 **was** vectorizing it; disabling SuperWord costs 4.1x | +| kernel 2 (data-dependent branch) | 5022 ns | 5040 ns | unchanged, so C2 was **not** vectorizing it | +| kernel 3 (branchy clamp) | 6882 ns | 6884 ns | unchanged, so C2 was **not** vectorizing it | + +The Vector API versions are essentially unmoved by the flag in all three cases, +which is the property the API is actually selling. + +Run-to-run spread on this machine was a few percent for kernels 1 and 2, and up +to ~13% for kernel 3 (5.05x–6.28x observed across runs) — another reminder to +quote JMH rather than a stopwatch. + +## 3. JMH — `jmh/run.sh` + +JMH 1.37, 1 fork, 3 warmup iterations x 1 s, 5 measurement iterations x 1 s, +single thread, average time per operation. + +``` +Benchmark Mode Cnt Score Error Units +VectorBench.scalar avgt 5 838.631 ± 106.657 ns/op +VectorBench.vector avgt 5 814.531 ± 10.766 ns/op +VectorBench.vectorChecksum avgt 5 7409.993 ± 224.753 ns/op +MaskedBench.scalarBranch avgt 5 5189.874 ± 250.597 ns/op +MaskedBench.vectorMasked avgt 5 1036.419 ± 27.726 ns/op +BrightnessBench.scalarClamp avgt 5 6510.091 ± 302.638 ns/op +BrightnessBench.vectorClamp avgt 5 1133.073 ± 17.763 ns/op +``` + +Three readings: + +1. **Square-and-add: no measurable difference.** 838.6 ± 106.7 vs 814.5 ± 10.8 — + the error bars overlap. On a loop C2 already vectorizes, hand-writing the + vector loop bought nothing measurable here. (The exploratory harness made the + vector version look ~35% slower; that gap is harness overhead, not the kernel. + This is exactly why the JMH number is the one to quote.) +2. **Masked filter: 5.0x** (5189.9 / 1036.4), with tight error bars on the vector side. +3. **Brightness clamp: 5.7x** (6510.1 / 1133.1). + +`vectorChecksum` is a deliberate cautionary example: returning a checksum does +prevent dead-code elimination, but the checksum loop is then *inside* the +measurement, and the benchmark reports ~9x the time of the kernel it was meant to +measure. Consuming the output array through a `Blackhole` keeps the measurement +honest without adding work. + +## What is not reproduced here + +The article shows an excerpt of the AVX2 machine code C2 emits for the +square-and-add loop. Reproducing that needs an `hsdis` disassembler plugin +installed next to your JDK: + +``` +java --add-modules jdk.incubator.vector \ + -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly \ + -XX:CompileCommand=print,com.ankurm.vectorapi.SquareAdd::vectorComputation \ + -cp out com.ankurm.vectorapi.Main +``` + +Without `hsdis` the JVM prints `Loading hsdis library failed` and falls back to a +hex dump. diff --git a/jmh/pom.xml b/jmh/pom.xml new file mode 100644 index 0000000..4029432 --- /dev/null +++ b/jmh/pom.xml @@ -0,0 +1,85 @@ + + + 4.0.0 + + com.ankurm + vector-api-jmh + 1.0.0 + jar + Vector API JMH benchmarks + + + UTF-8 + 21 + 1.37 + benchmarks + + + + + org.openjdk.jmh + jmh-core + ${jmh.version} + + + org.openjdk.jmh + jmh-generator-annprocess + ${jmh.version} + provided + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.13.0 + + + --add-modules + jdk.incubator.vector + + + + org.openjdk.jmh + jmh-generator-annprocess + ${jmh.version} + + + + + + org.apache.maven.plugins + maven-shade-plugin + 3.5.3 + + + package + shade + + ${uberjar.name} + + + org.openjdk.jmh.Main + + + + + *:* + + META-INF/*.SF + META-INF/*.DSA + META-INF/*.RSA + + + + + + + + + + diff --git a/jmh/run-without-maven.sh b/jmh/run-without-maven.sh new file mode 100644 index 0000000..1a88061 --- /dev/null +++ b/jmh/run-without-maven.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# Same benchmarks without Maven: fetch the four jars, compile with the JMH +# annotation processor, run org.openjdk.jmh.Main directly. +set -euo pipefail +cd "$(dirname "$0")" +mkdir -p lib out +for u in org/openjdk/jmh/jmh-core/1.37/jmh-core-1.37.jar \ + org/openjdk/jmh/jmh-generator-annprocess/1.37/jmh-generator-annprocess-1.37.jar \ + net/sf/jopt-simple/jopt-simple/5.0.4/jopt-simple-5.0.4.jar \ + org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.jar; do + [ -f "lib/$(basename "$u")" ] || curl -sSL -o "lib/$(basename "$u")" "https://repo1.maven.org/maven2/$u" +done +CP=$(ls lib/*.jar | tr '\n' ':') +javac --add-modules jdk.incubator.vector -cp "$CP" -proc:full -d out $(find src -name '*.java') +java --add-modules jdk.incubator.vector -cp "out:$CP" org.openjdk.jmh.Main "$@" diff --git a/jmh/run.sh b/jmh/run.sh new file mode 100644 index 0000000..236dc2d --- /dev/null +++ b/jmh/run.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Build and run the JMH benchmarks (the numbers worth defending). +# ./run.sh -> all benchmarks +# ./run.sh MaskedBench -> one class +set -euo pipefail +cd "$(dirname "$0")" +mvn -q -B clean package +java --add-modules jdk.incubator.vector -jar target/benchmarks.jar "$@" diff --git a/jmh/src/main/java/com/ankurm/vectorapi/BrightnessBench.java b/jmh/src/main/java/com/ankurm/vectorapi/BrightnessBench.java new file mode 100644 index 0000000..e66a16f --- /dev/null +++ b/jmh/src/main/java/com/ankurm/vectorapi/BrightnessBench.java @@ -0,0 +1,55 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; +import org.openjdk.jmh.annotations.*; +import org.openjdk.jmh.infra.Blackhole; +import java.util.Random; +import java.util.concurrent.TimeUnit; + +/** JMH version of the image brightness + clamp kernel. */ +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.NANOSECONDS) +@State(Scope.Thread) +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 5, time = 1) +@Fork(1) +public class BrightnessBench { + static final VectorSpecies SP = FloatVector.SPECIES_PREFERRED; + float[] in, out; + float gain = 1.4f, bias = 12f; + + @Setup + public void setup() { + in = new float[8192]; out = new float[8192]; + Random r = new Random(11); + for (int i = 0; i < in.length; i++) in[i] = r.nextFloat() * 300f - 20f; + } + + @Benchmark + public void scalarClamp(Blackhole bh) { + for (int i = 0; i < in.length; i++) { + float v = in[i] * gain + bias; + if (v < 0f) v = 0f; else if (v > 255f) v = 255f; + out[i] = v; + } + bh.consume(out); + } + + @Benchmark + public void vectorClamp(Blackhole bh) { + var vg = FloatVector.broadcast(SP, gain); + var vb = FloatVector.broadcast(SP, bias); + var lo = FloatVector.zero(SP); + var hi = FloatVector.broadcast(SP, 255f); + int i = 0, bound = SP.loopBound(in.length); + for (; i < bound; i += SP.length()) { + FloatVector.fromArray(SP, in, i).fma(vg, vb).max(lo).min(hi).intoArray(out, i); + } + for (; i < in.length; i++) { + float v = in[i] * gain + bias; + if (v < 0f) v = 0f; else if (v > 255f) v = 255f; + out[i] = v; + } + bh.consume(out); + } +} diff --git a/jmh/src/main/java/com/ankurm/vectorapi/MaskedBench.java b/jmh/src/main/java/com/ankurm/vectorapi/MaskedBench.java new file mode 100644 index 0000000..77ed0f9 --- /dev/null +++ b/jmh/src/main/java/com/ankurm/vectorapi/MaskedBench.java @@ -0,0 +1,47 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; +import org.openjdk.jmh.annotations.*; +import java.util.Random; +import java.util.concurrent.TimeUnit; + +/** JMH version of the masked filter-and-sum: the headline speedup case. */ +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.NANOSECONDS) +@State(Scope.Thread) +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 5, time = 1) +@Fork(1) +public class MaskedBench { + static final VectorSpecies SP = FloatVector.SPECIES_PREFERRED; + static final float THRESH = 0.5f; + float[] a; + + @Setup + public void setup() { + a = new float[8192]; + Random r = new Random(7); + for (int i = 0; i < a.length; i++) a[i] = r.nextFloat(); + } + + // Both methods return their result, so JMH consumes it and nothing is dead. + @Benchmark + public float scalarBranch() { + float sum = 0; + for (int i = 0; i < a.length; i++) if (a[i] > THRESH) sum += a[i]; + return sum; + } + + @Benchmark + public float vectorMasked() { + FloatVector acc = FloatVector.zero(SP); + int i = 0, bound = SP.loopBound(a.length); + for (; i < bound; i += SP.length()) { + var v = FloatVector.fromArray(SP, a, i); + acc = acc.add(v, v.compare(VectorOperators.GT, THRESH)); + } + float sum = acc.reduceLanes(VectorOperators.ADD); + for (; i < a.length; i++) if (a[i] > THRESH) sum += a[i]; + return sum; + } +} diff --git a/jmh/src/main/java/com/ankurm/vectorapi/VectorBench.java b/jmh/src/main/java/com/ankurm/vectorapi/VectorBench.java new file mode 100644 index 0000000..c617532 --- /dev/null +++ b/jmh/src/main/java/com/ankurm/vectorapi/VectorBench.java @@ -0,0 +1,63 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; +import org.openjdk.jmh.annotations.*; +import org.openjdk.jmh.infra.Blackhole; +import java.util.Random; +import java.util.concurrent.TimeUnit; + +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.NANOSECONDS) +@State(Scope.Thread) +@Warmup(iterations = 5, time = 1) +@Measurement(iterations = 5, time = 1) +@Fork(1) +public class VectorBench { + static final VectorSpecies SP = FloatVector.SPECIES_PREFERRED; + float[] a, b, c; + + @Setup + public void setup() { + int n = 8192; + a = new float[n]; b = new float[n]; c = new float[n]; + Random r = new Random(42); + for (int i = 0; i < n; i++) { a[i] = r.nextFloat(); b[i] = r.nextFloat(); } + } + + // The result array is written, then handed to JMH's Blackhole so neither the + // stores nor the arithmetic can be eliminated as dead code. + @Benchmark + public void scalar(Blackhole bh) { + for (int i = 0; i < a.length; i++) c[i] = -(a[i]*a[i] + b[i]*b[i]); + bh.consume(c); + } + + @Benchmark + public void vector(Blackhole bh) { + int i = 0, bound = SP.loopBound(a.length); + for (; i < bound; i += SP.length()) { + var va = FloatVector.fromArray(SP, a, i); + var vb = FloatVector.fromArray(SP, b, i); + va.mul(va).add(vb.mul(vb)).neg().intoArray(c, i); + } + for (; i < a.length; i++) c[i] = -(a[i]*a[i] + b[i]*b[i]); + bh.consume(c); + } + + // Same kernel, but returning a value derived from the output. Returned values + // are consumed by JMH automatically. Note what this costs: the checksum loop is + // itself measured, which is why this variant reports ~9x the time of vector(). + @Benchmark + public float vectorChecksum() { + int i = 0, bound = SP.loopBound(a.length); + for (; i < bound; i += SP.length()) { + var va = FloatVector.fromArray(SP, a, i); + var vb = FloatVector.fromArray(SP, b, i); + va.mul(va).add(vb.mul(vb)).neg().intoArray(c, i); + } + for (; i < a.length; i++) c[i] = -(a[i]*a[i] + b[i]*b[i]); + float sum = 0; + for (float v : c) sum += v; + return sum; + } +} diff --git a/timings/run.sh b/timings/run.sh new file mode 100644 index 0000000..57e2366 --- /dev/null +++ b/timings/run.sh @@ -0,0 +1,9 @@ +#!/usr/bin/env bash +# Compile and run the exploratory timing experiments from the article. +# ./run.sh -> default run (C2 auto-vectorization ON) +# ./run.sh -XX:-UseSuperWord -> same experiments with SuperWord disabled +set -euo pipefail +cd "$(dirname "$0")" +mkdir -p out +javac --add-modules jdk.incubator.vector -d out $(find src -name '*.java') +java --add-modules jdk.incubator.vector "$@" -cp out com.ankurm.vectorapi.Main diff --git a/timings/src/main/java/com/ankurm/vectorapi/Bench.java b/timings/src/main/java/com/ankurm/vectorapi/Bench.java new file mode 100644 index 0000000..3b60325 --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/Bench.java @@ -0,0 +1,41 @@ +package com.ankurm.vectorapi; + +import java.util.Arrays; + +/** + * A deliberately small timing harness. It is NOT a statistical benchmark: + * it warms the JIT up, then reports the median and best of several rounds so + * that large effects (2x and up) are visible. Use JMH for anything you intend + * to defend -- see the jmh/ module in this repository. + */ +public final class Bench { + public static final int WARMUP_CALLS = 50_000; + public static final int ROUND_CALLS = 5_000; + public static final int ROUNDS = 15; + + public interface Kernel { void run(); } + + /** @return median ns/call over ROUNDS rounds (also prints best). */ + public static double measure(String label, Kernel k) { + for (int i = 0; i < WARMUP_CALLS; i++) k.run(); // force C2 compilation + double[] perCall = new double[ROUNDS]; + for (int r = 0; r < ROUNDS; r++) { + long t0 = System.nanoTime(); + for (int i = 0; i < ROUND_CALLS; i++) k.run(); + long t1 = System.nanoTime(); + perCall[r] = (t1 - t0) / (double) ROUND_CALLS; + } + double[] sorted = perCall.clone(); + Arrays.sort(sorted); + double median = sorted[ROUNDS / 2]; + double best = sorted[0]; + System.out.printf("%-24s median %8.1f ns/call best %8.1f ns/call%n", label, median, best); + return median; + } + + public static void speedup(String label, double scalarNs, double vectorNs) { + System.out.printf("%-24s %.2fx%n", label, scalarNs / vectorNs); + } + + private Bench() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/Brightness.java b/timings/src/main/java/com/ankurm/vectorapi/Brightness.java new file mode 100644 index 0000000..09cc70a --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/Brightness.java @@ -0,0 +1,51 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; + +/** Image brightness: scale, bias, clamp to [0, 255]. The clamp is two branches. */ +public final class Brightness { + + static final VectorSpecies SP = FloatVector.SPECIES_PREFERRED; + + // Scalar: the clamp is two data-dependent branches + static void brightScalar(float[] in, float[] out, float gain, float bias) { + for (int i = 0; i < in.length; i++) { + float v = in[i] * gain + bias; + if (v < 0f) v = 0f; else if (v > 255f) v = 255f; + out[i] = v; + } + } + + static void brightVector(float[] in, float[] out, float gain, float bias) { + var vg = FloatVector.broadcast(SP, gain); + var vb = FloatVector.broadcast(SP, bias); + var lo = FloatVector.zero(SP); + var hi = FloatVector.broadcast(SP, 255f); + int i = 0, bound = SP.loopBound(in.length); + for (; i < bound; i += SP.length()) { + FloatVector.fromArray(SP, in, i).fma(vg, vb).max(lo).min(hi).intoArray(out, i); + } + for (; i < in.length; i++) { // scalar tail + float v = in[i] * gain + bias; + if (v < 0f) v = 0f; else if (v > 255f) v = 255f; + out[i] = v; + } + } + + static void run(int n) { + float[] in = Data.randomFloats(n, 4); + for (int i = 0; i < n; i++) in[i] = in[i] * 300f - 20f; // some pixels clamp at both ends + float[] o1 = new float[n], o2 = new float[n]; + brightScalar(in, o1, 1.4f, 12f); + brightVector(in, o2, 1.4f, 12f); + Data.assertSame("brightness", o1, o2); + + System.out.println("--- kernel 3: image brightness + clamp, " + n + " pixels ---"); + double s = Bench.measure("scalar (branchy clamp)", () -> brightScalar(in, o1, 1.4f, 12f)); + double v = Bench.measure("Vector API (max/min)", () -> brightVector(in, o2, 1.4f, 12f)); + Bench.speedup("vector advantage", s, v); + System.out.println(); + } + + private Brightness() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/Data.java b/timings/src/main/java/com/ankurm/vectorapi/Data.java new file mode 100644 index 0000000..48ae17d --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/Data.java @@ -0,0 +1,22 @@ +package com.ankurm.vectorapi; + +import java.util.Random; + +public final class Data { + static float[] randomFloats(int n, long seed) { + Random r = new Random(seed); + float[] a = new float[n]; + for (int i = 0; i < n; i++) a[i] = r.nextFloat(); + return a; + } + + static void assertSame(String what, float[] x, float[] y) { + for (int i = 0; i < x.length; i++) { + if (Math.abs(x[i] - y[i]) > 1e-4f) { + throw new AssertionError(what + ": mismatch at " + i + ": " + x[i] + " != " + y[i]); + } + } + } + + private Data() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/Env.java b/timings/src/main/java/com/ankurm/vectorapi/Env.java new file mode 100644 index 0000000..353f229 --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/Env.java @@ -0,0 +1,24 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; + +/** Prints the exact environment a benchmark run happened on. */ +public final class Env { + public static void print() { + Runtime rt = Runtime.getRuntime(); + System.out.println("=== environment ==="); + System.out.println("JDK : " + System.getProperty("java.vm.name") + + " " + System.getProperty("java.vm.version")); + System.out.println("java.version : " + System.getProperty("java.version")); + System.out.println("OS : " + System.getProperty("os.name") + " " + + System.getProperty("os.version") + " (" + System.getProperty("os.arch") + ")"); + System.out.println("Available procs: " + rt.availableProcessors()); + System.out.println("Preferred spec : " + FloatVector.SPECIES_PREFERRED + + " (" + FloatVector.SPECIES_PREFERRED.length() + " float lanes, " + + FloatVector.SPECIES_PREFERRED.vectorBitSize() + "-bit)"); + System.out.println("Max vector size: " + IntVector.SPECIES_PREFERRED.vectorBitSize() + "-bit (int)"); + System.out.println(); + } + + private Env() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/Main.java b/timings/src/main/java/com/ankurm/vectorapi/Main.java new file mode 100644 index 0000000..add61ff --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/Main.java @@ -0,0 +1,25 @@ +package com.ankurm.vectorapi; + +/** + * Runs every timing experiment quoted in the article, on whatever machine you + * are sitting at. These are exploratory timings, not statistically rigorous + * benchmarks -- see the jmh/ module for the harness you would defend numbers with. + * + * javac --add-modules jdk.incubator.vector -d out $(find src -name '*.java') + * java --add-modules jdk.incubator.vector -cp out com.ankurm.vectorapi.Main + * java --add-modules jdk.incubator.vector -XX:-UseSuperWord -cp out com.ankurm.vectorapi.Main + */ +public final class Main { + public static void main(String[] args) { + int n = args.length > 0 ? Integer.parseInt(args[0]) : 8192; + Env.print(); + System.out.println("array length : " + n + " floats (" + (n * 4 / 1024) + " KiB, cache-resident)"); + System.out.println(); + SquareAdd.run(n); + MaskedFilter.run(n); + Brightness.run(n); + ReductionOrder.run(); + } + + private Main() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/MaskedFilter.java b/timings/src/main/java/com/ankurm/vectorapi/MaskedFilter.java new file mode 100644 index 0000000..3b0203d --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/MaskedFilter.java @@ -0,0 +1,45 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; + +/** Sum only the elements above a threshold -- a data-dependent branch. */ +public final class MaskedFilter { + + static final float THRESH = 0.5f; + static final VectorSpecies SP = FloatVector.SPECIES_PREFERRED; + + // Scalar: the branch is data-dependent, so C2 generally cannot vectorize this form + static float scalar(float[] a) { + float sum = 0; + for (int i = 0; i < a.length; i++) + if (a[i] > THRESH) sum += a[i]; + return sum; + } + + static float vector(float[] a) { + FloatVector acc = FloatVector.zero(SP); + int i = 0, bound = SP.loopBound(a.length); + for (; i < bound; i += SP.length()) { + var v = FloatVector.fromArray(SP, a, i); + VectorMask m = v.compare(VectorOperators.GT, THRESH); + acc = acc.add(v, m); // add only where mask is true + } + float sum = acc.reduceLanes(VectorOperators.ADD); // horizontal sum of the lanes + for (; i < a.length; i++) if (a[i] > THRESH) sum += a[i]; // masked-off tail + return sum; + } + + static void run(int n) { + float[] a = Data.randomFloats(n, 3); + float s0 = scalar(a), v0 = vector(a); + + System.out.println("--- kernel 2: masked filter-and-sum, " + n + " floats ---"); + System.out.printf("scalar sum = %.4f%nvector sum = %.4f (same value, different rounding)%n", s0, v0); + double s = Bench.measure("scalar (branch)", () -> Sink.consume(scalar(a))); + double v = Bench.measure("Vector API (masked)", () -> Sink.consume(vector(a))); + Bench.speedup("vector advantage", s, v); + System.out.println(); + } + + private MaskedFilter() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/ReductionOrder.java b/timings/src/main/java/com/ankurm/vectorapi/ReductionOrder.java new file mode 100644 index 0000000..74d408c --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/ReductionOrder.java @@ -0,0 +1,35 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; + +/** Why a vector reduction and a scalar loop disagree in the last bits. */ +public final class ReductionOrder { + + static final VectorSpecies SP = FloatVector.SPECIES_PREFERRED; + + static float scalarSum(float[] a) { + float sum = 0; + for (float v : a) sum += v; // strictly left to right + return sum; + } + + static float vectorSum(float[] a) { + FloatVector acc = FloatVector.zero(SP); + int i = 0, bound = SP.loopBound(a.length); + for (; i < bound; i += SP.length()) acc = acc.add(FloatVector.fromArray(SP, a, i)); + float sum = acc.reduceLanes(VectorOperators.ADD); // lanes combined at the end + for (; i < a.length; i++) sum += a[i]; + return sum; + } + + static void run() { + float[] a = Data.randomFloats(8192, 5); + System.out.println("--- floating-point reduction ordering ---"); + System.out.printf("scalar (left to right) : %.6f%n", scalarSum(a)); + System.out.printf("vector (lanes, then reduce): %.6f%n", vectorSum(a)); + System.out.println("Same mathematical sum; different addition order, so different rounding."); + System.out.println(); + } + + private ReductionOrder() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/Sink.java b/timings/src/main/java/com/ankurm/vectorapi/Sink.java new file mode 100644 index 0000000..dcaa866 --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/Sink.java @@ -0,0 +1,10 @@ +package com.ankurm.vectorapi; + +/** Keeps computed values observable so the JIT cannot delete the computation. */ +public final class Sink { + public static volatile float sink; + + public static void consume(float v) { sink = v; } + + private Sink() {} +} diff --git a/timings/src/main/java/com/ankurm/vectorapi/SquareAdd.java b/timings/src/main/java/com/ankurm/vectorapi/SquareAdd.java new file mode 100644 index 0000000..b95b81c --- /dev/null +++ b/timings/src/main/java/com/ankurm/vectorapi/SquareAdd.java @@ -0,0 +1,50 @@ +package com.ankurm.vectorapi; + +import jdk.incubator.vector.*; + +/** c[i] = -(a[i]^2 + b[i]^2) -- the loop C2's SuperWord pass handles well. */ +public final class SquareAdd { + + static final VectorSpecies SPECIES = FloatVector.SPECIES_PREFERRED; + + static void scalarComputation(float[] a, float[] b, float[] c) { + for (int i = 0; i < a.length; i++) { + c[i] = -(a[i] * a[i] + b[i] * b[i]); + } + } + + static void vectorComputation(float[] a, float[] b, float[] c) { + int i = 0; + int upperBound = SPECIES.loopBound(a.length); // e.g. 8192 -> 8192, 8190 -> 8184 + + // Main loop: process SPECIES.length() elements per iteration + for (; i < upperBound; i += SPECIES.length()) { + var va = FloatVector.fromArray(SPECIES, a, i); + var vb = FloatVector.fromArray(SPECIES, b, i); + va.mul(va).add(vb.mul(vb)).neg().intoArray(c, i); + } + + // Scalar tail: the last (a.length % laneCount) elements + for (; i < a.length; i++) { + c[i] = -(a[i] * a[i] + b[i] * b[i]); + } + } + + static void run(int n) { + float[] a = Data.randomFloats(n, 1); + float[] b = Data.randomFloats(n, 2); + float[] c1 = new float[n], c2 = new float[n]; + + scalarComputation(a, b, c1); + vectorComputation(a, b, c2); + Data.assertSame("square-add", c1, c2); + + System.out.println("--- kernel 1: c[i] = -(a[i]^2 + b[i]^2), " + n + " floats ---"); + double s = Bench.measure("scalar for-loop", () -> scalarComputation(a, b, c1)); + double v = Bench.measure("Vector API", () -> vectorComputation(a, b, c2)); + Bench.speedup("vector advantage", s, v); + System.out.println(); + } + + private SquareAdd() {} +}