Add repeated module: @RepeatedTest from JUnit 5 to JUnit 6 (6.1.3, JDK 25)
This commit is contained in:
Claude
@@ -0,0 +1,91 @@
|
|||||||
|
# repeated
|
||||||
|
|
||||||
|
Companion module for [JUnit @RepeatedTest (JUnit 5 → 6)](https://ankurm.com/) on ankurm.com.
|
||||||
|
|
||||||
|
## What this is
|
||||||
|
|
||||||
|
`@RepeatedTest` and `RepetitionInfo` are **byte-for-byte unchanged** between JUnit Jupiter 5.10.1
|
||||||
|
and 6.1.3 — confirmed here with `javap` against the real jars, not assumed from a changelog. The
|
||||||
|
"JUnit 5 → 6" part of this module's story is almost entirely a version-number story: JUnit 6
|
||||||
|
unified versioning across Platform, Jupiter and Vintage (they all now share one release number,
|
||||||
|
confirmed against the real `maven-metadata.xml` for all three artifacts) and raised the minimum
|
||||||
|
Java version to 17. Nothing about `@RepeatedTest` itself had to change for that.
|
||||||
|
|
||||||
|
What *did* need fixing is what the previous version of this post got wrong or left out, each
|
||||||
|
verified against a real jar, a real sources jar, or a real run rather than against prose:
|
||||||
|
|
||||||
|
- `{shortDisplayname}` is not a real placeholder. There are exactly three:
|
||||||
|
`DISPLAY_NAME_PLACEHOLDER`, `CURRENT_REPETITION_PLACEHOLDER`, `TOTAL_REPETITIONS_PLACEHOLDER`
|
||||||
|
(confirmed via `javap` on `RepeatedTest.class`). A `name()` pattern containing the bogus token
|
||||||
|
is accepted at compile time and left completely unresolved at run time — see
|
||||||
|
`docs/output/03-bogus-placeholder-scratch.txt`.
|
||||||
|
- `RepetitionInfo` carries **four** accessors, not two. `getFailureCount()` and
|
||||||
|
`getFailureThreshold()` have been on the interface since JUnit 5.10, the same release that
|
||||||
|
added `failureThreshold()` itself.
|
||||||
|
- `RepetitionInfo` can be injected into `@BeforeEach`/`@AfterEach`, not only into the
|
||||||
|
`@RepeatedTest` method.
|
||||||
|
- Once `failureThreshold` is reached, JUnit **skips** the remaining repetitions — it does not
|
||||||
|
merely "stop" them, and they are never invoked at all. See
|
||||||
|
`docs/output/07-failure-threshold-skips-remaining.txt` for the real console-tree output showing
|
||||||
|
`Failure threshold [2] exceeded` against repetitions 9 and 10.
|
||||||
|
|
||||||
|
**Self-correction made before this module was finished:** the "flaky test detection" use case
|
||||||
|
needs a reproducible failure, not real timing-dependent flakiness, so both demo scratch classes
|
||||||
|
fail deterministically on fixed repetition numbers (4 and 8 of 10) rather than relying on actual
|
||||||
|
race conditions. Both were run for real, their output captured, and then deleted — they were
|
||||||
|
never meant to be part of the permanent green suite.
|
||||||
|
|
||||||
|
## Versions
|
||||||
|
|
||||||
|
| Dependency | Version | Verified against |
|
||||||
|
|---|---|---|
|
||||||
|
| JUnit Jupiter | 6.1.3 | `maven-metadata.xml` (`latest`/`release`), matching `junit-bom` and `junit-platform-launcher` — JUnit 6's unified versioning |
|
||||||
|
| AssertJ | 3.27.7 | `maven-metadata.xml`, `<release>` |
|
||||||
|
| Maven Surefire | 3.6.0 | this module's own build |
|
||||||
|
| JDK | 25 (Temurin) | `java -version`, this module's own build |
|
||||||
|
|
||||||
|
`@RepeatedTest`/`RepetitionInfo` class files are identical (via `javap`) across 5.10.1 and 6.1.3.
|
||||||
|
`getFailureCount()`/`getFailureThreshold()` are absent in 5.9.3's `RepeatedTest.class` and present
|
||||||
|
from 5.10.1 onward — the old post's "introduced in 5.10" claim for `failureThreshold` itself is
|
||||||
|
correct; it just never documented these two accessors.
|
||||||
|
|
||||||
|
## Quickstart
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mvn test # 14 tests, 0 failures
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test classes
|
||||||
|
|
||||||
|
| Class | What it proves | Captured output |
|
||||||
|
|---|---|---|
|
||||||
|
| `ReliabilityTest` | The smallest `@RepeatedTest(5)`; real default display names resolved through injected `TestInfo` | `docs/output/01-basic-repetition.txt` |
|
||||||
|
| `CustomNameTest` | A custom `name()` pattern built from the three real placeholders | `docs/output/02-custom-display-name.txt` |
|
||||||
|
| `MetadataTest` | All four `RepetitionInfo` accessors, including the two the old post never covered | `docs/output/04-repetition-info-metadata.txt` |
|
||||||
|
| `LifecycleTest` | `@BeforeAll`/`@AfterAll` (once) vs `@BeforeEach`/`@AfterEach` (per repetition), plus `RepetitionInfo` injected into `@BeforeEach`/`@AfterEach` | `docs/output/05-lifecycle-order.txt` |
|
||||||
|
|
||||||
|
## Captured output
|
||||||
|
|
||||||
|
| File | Contents |
|
||||||
|
|---|---|
|
||||||
|
| `docs/output/00-full-test-run.txt` | `mvn test` — all 4 committed classes, 14 tests, 0 failures |
|
||||||
|
| `docs/output/01-basic-repetition.txt` | `ReliabilityTest` — real default display names |
|
||||||
|
| `docs/output/02-custom-display-name.txt` | `CustomNameTest` — real custom display names |
|
||||||
|
| `docs/output/03-bogus-placeholder-scratch.txt` | Scratch run proving `{shortDisplayname}` is not a real placeholder and resolves to literal text |
|
||||||
|
| `docs/output/04-repetition-info-metadata.txt` | `MetadataTest` — all four `RepetitionInfo` accessors across 3 repetitions |
|
||||||
|
| `docs/output/05-lifecycle-order.txt` | `LifecycleTest` — real call order and totals across 3 repetitions |
|
||||||
|
| `docs/output/06-flaky-detected.txt` | Scratch run: no `failureThreshold` set — all 10 repetitions run even though 2 fail |
|
||||||
|
| `docs/output/07-failure-threshold-skips-remaining.txt` | Scratch run: `failureThreshold = 2` — repetitions 9 and 10 skipped after the 2nd failure |
|
||||||
|
|
||||||
|
`docs/output/03`, `06` and `07` come from scratch classes (`ScratchPlaceholderTest`,
|
||||||
|
`FlakyDetectionDemoTest`, `FailureThresholdStopsRemainingDemoTest`) that were written, run once
|
||||||
|
for real evidence, and then deleted — they are not part of the committed suite, which stays green
|
||||||
|
at 14/14. Re-run `scripts/run-all.sh` to see exactly how each transcript was produced.
|
||||||
|
|
||||||
|
## What's sourced from documentation, not run here
|
||||||
|
|
||||||
|
The JUnit 6.0.0 release notes facts quoted in the post (minimum Java 17, Vintage deprecated,
|
||||||
|
`junit-platform-runner`/`junit-platform-jfr` removal, `MethodOrderer.Alphanumeric` removal,
|
||||||
|
`@CsvSource` moving to FastCSV) are quoted from the official JUnit 6.0.0 release notes, not
|
||||||
|
exercised in this module — none of them touch `@RepeatedTest` or `RepetitionInfo` directly.
|
||||||
|
Everything else above was compiled, run, or pulled directly from a real jar in this sitting.
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
# mvn test -- full committed suite (ReliabilityTest, CustomNameTest, LifecycleTest, MetadataTest)
|
||||||
|
|
||||||
|
[INFO] -------------------------------------------------------
|
||||||
|
[INFO] Running com.ankurm.tutorials.junit.repeated.ReliabilityTest
|
||||||
|
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.275 s -- in com.ankurm.tutorials.junit.repeated.ReliabilityTest
|
||||||
|
[INFO] Running com.ankurm.tutorials.junit.repeated.CustomNameTest
|
||||||
|
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 s -- in com.ankurm.tutorials.junit.repeated.CustomNameTest
|
||||||
|
[INFO] Running com.ankurm.tutorials.junit.repeated.LifecycleTest
|
||||||
|
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s -- in com.ankurm.tutorials.junit.repeated.LifecycleTest
|
||||||
|
[INFO] Running com.ankurm.tutorials.junit.repeated.MetadataTest
|
||||||
|
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 s -- in com.ankurm.tutorials.junit.repeated.MetadataTest
|
||||||
|
[INFO]
|
||||||
|
[INFO] Results:
|
||||||
|
[INFO]
|
||||||
|
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0
|
||||||
|
[INFO]
|
||||||
|
[INFO] ------------------------------------------------------------------------
|
||||||
|
[INFO] BUILD SUCCESS
|
||||||
|
[INFO] ------------------------------------------------------------------------
|
||||||
|
[INFO] Total time: 2.788 s
|
||||||
|
[INFO] Finished at: 2026-10-08T21:23:02+05:30
|
||||||
|
[INFO] ------------------------------------------------------------------------
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
== TestInfo.getDisplayName() for each repetition ==
|
||||||
|
|
||||||
|
repetition 1 of 5
|
||||||
|
repetition 2 of 5
|
||||||
|
repetition 3 of 5
|
||||||
|
repetition 4 of 5
|
||||||
|
repetition 5 of 5
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
== TestInfo.getDisplayName() with a custom name() pattern ==
|
||||||
|
|
||||||
|
Critical API Integration - Run 1/3
|
||||||
|
Critical API Integration - Run 2/3
|
||||||
|
Critical API Integration - Run 3/3
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
# Scratch run: {shortDisplayname} is not a real placeholder (ScratchPlaceholderTest, removed before commit)
|
||||||
|
|
||||||
|
|
||||||
|
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
|
||||||
|
|
||||||
|
╷
|
||||||
|
├─ JUnit Platform Suite ✔
|
||||||
|
├─ JUnit Jupiter ✔
|
||||||
|
│ └─ ScratchPlaceholderTest ✔
|
||||||
|
│ └─ bogusPlaceholderTest() ✔
|
||||||
|
│ ├─ {shortDisplayname} test ✔
|
||||||
|
│ └─ {shortDisplayname} test ✔
|
||||||
|
└─ JUnit Vintage ✔
|
||||||
|
|
||||||
|
Test run finished after 204 ms
|
||||||
|
[ 5 containers found ]
|
||||||
|
[ 0 containers skipped ]
|
||||||
|
[ 5 containers started ]
|
||||||
|
[ 0 containers aborted ]
|
||||||
|
[ 5 containers successful ]
|
||||||
|
[ 0 containers failed ]
|
||||||
|
[ 2 tests found ]
|
||||||
|
[ 0 tests skipped ]
|
||||||
|
[ 2 tests started ]
|
||||||
|
[ 0 tests aborted ]
|
||||||
|
[ 2 tests successful ]
|
||||||
|
[ 0 tests failed ]
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
== RepetitionInfo across all 3 repetitions ==
|
||||||
|
|
||||||
|
repetition 1 of 3 -- getFailureCount()=0 getFailureThreshold()=2
|
||||||
|
repetition 2 of 3 -- getFailureCount()=0 getFailureThreshold()=2
|
||||||
|
repetition 3 of 3 -- getFailureCount()=0 getFailureThreshold()=2
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
== Call order across 3 repetitions ==
|
||||||
|
|
||||||
|
@BeforeAll
|
||||||
|
@BeforeEach (repetition 1)
|
||||||
|
@RepeatedTest (repetition 1)
|
||||||
|
@AfterEach (repetition 1)
|
||||||
|
@BeforeEach (repetition 2)
|
||||||
|
@RepeatedTest (repetition 2)
|
||||||
|
@AfterEach (repetition 2)
|
||||||
|
@BeforeEach (repetition 3)
|
||||||
|
@RepeatedTest (repetition 3)
|
||||||
|
@AfterEach (repetition 3)
|
||||||
|
@AfterAll
|
||||||
|
|
||||||
|
Totals -- @BeforeAll: 1, @AfterAll: 1, @BeforeEach: 3, @AfterEach: 3
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# Scratch run: @RepeatedTest with no failureThreshold -- all 10 repetitions still run even though 2 fail (FlakyDetectionDemoTest, removed before commit)
|
||||||
|
|
||||||
|
|
||||||
|
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
|
||||||
|
|
||||||
|
╷
|
||||||
|
├─ JUnit Platform Suite ✔
|
||||||
|
├─ JUnit Jupiter ✔
|
||||||
|
│ └─ FlakyDetectionDemoTest ✔
|
||||||
|
│ └─ intermittentRaceCondition(RepetitionInfo) ✔
|
||||||
|
│ ├─ repetition 1 of 10 ✔
|
||||||
|
│ ├─ repetition 2 of 10 ✔
|
||||||
|
│ ├─ repetition 3 of 10 ✔
|
||||||
|
│ ├─ repetition 4 of 10 ✘ simulated race condition reproduced on repetition 4
|
||||||
|
│ ├─ repetition 5 of 10 ✔
|
||||||
|
│ ├─ repetition 6 of 10 ✔
|
||||||
|
│ ├─ repetition 7 of 10 ✔
|
||||||
|
│ ├─ repetition 8 of 10 ✘ simulated race condition reproduced on repetition 8
|
||||||
|
│ ├─ repetition 9 of 10 ✔
|
||||||
|
│ └─ repetition 10 of 10 ✔
|
||||||
|
└─ JUnit Vintage ✔
|
||||||
|
|
||||||
|
Failures (2):
|
||||||
|
JUnit Jupiter:FlakyDetectionDemoTest:intermittentRaceCondition(RepetitionInfo):repetition 4 of 10
|
||||||
|
MethodSource [className = 'com.ankurm.tutorials.junit.repeated.FlakyDetectionDemoTest', methodName = 'intermittentRaceCondition', methodParameterTypes = 'org.junit.jupiter.api.RepetitionInfo']
|
||||||
|
=> org.opentest4j.AssertionFailedError: simulated race condition reproduced on repetition 4
|
||||||
|
org.junit.jupiter.api.Assertions.fail(Assertions.java:142)
|
||||||
|
com.ankurm.tutorials.junit.repeated.FlakyDetectionDemoTest.intermittentRaceCondition(FlakyDetectionDemoTest.java:14)
|
||||||
|
JUnit Jupiter:FlakyDetectionDemoTest:intermittentRaceCondition(RepetitionInfo):repetition 8 of 10
|
||||||
|
MethodSource [className = 'com.ankurm.tutorials.junit.repeated.FlakyDetectionDemoTest', methodName = 'intermittentRaceCondition', methodParameterTypes = 'org.junit.jupiter.api.RepetitionInfo']
|
||||||
|
=> org.opentest4j.AssertionFailedError: simulated race condition reproduced on repetition 8
|
||||||
|
org.junit.jupiter.api.Assertions.fail(Assertions.java:142)
|
||||||
|
com.ankurm.tutorials.junit.repeated.FlakyDetectionDemoTest.intermittentRaceCondition(FlakyDetectionDemoTest.java:14)
|
||||||
|
|
||||||
|
Test run finished after 281 ms
|
||||||
|
[ 5 containers found ]
|
||||||
|
[ 0 containers skipped ]
|
||||||
|
[ 5 containers started ]
|
||||||
|
[ 0 containers aborted ]
|
||||||
|
[ 5 containers successful ]
|
||||||
|
[ 0 containers failed ]
|
||||||
|
[ 10 tests found ]
|
||||||
|
[ 0 tests skipped ]
|
||||||
|
[ 10 tests started ]
|
||||||
|
[ 0 tests aborted ]
|
||||||
|
[ 8 tests successful ]
|
||||||
|
[ 2 tests failed ]
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# Scratch run: @RepeatedTest(failureThreshold = 2) -- after the 2nd failure (repetition 8), repetitions 9 and 10 are SKIPPED, not invoked (FailureThresholdStopsRemainingDemoTest, removed before commit)
|
||||||
|
|
||||||
|
|
||||||
|
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
|
||||||
|
|
||||||
|
╷
|
||||||
|
├─ JUnit Platform Suite ✔
|
||||||
|
├─ JUnit Jupiter ✔
|
||||||
|
│ └─ FailureThresholdStopsRemainingDemoTest ✔
|
||||||
|
│ └─ intermittentRaceCondition(RepetitionInfo) ✔
|
||||||
|
│ ├─ repetition 1 of 10 ✔
|
||||||
|
│ ├─ repetition 2 of 10 ✔
|
||||||
|
│ ├─ repetition 3 of 10 ✔
|
||||||
|
│ ├─ repetition 4 of 10 ✘ simulated race condition reproduced on repetition 4
|
||||||
|
│ ├─ repetition 5 of 10 ✔
|
||||||
|
│ ├─ repetition 6 of 10 ✔
|
||||||
|
│ ├─ repetition 7 of 10 ✔
|
||||||
|
│ ├─ repetition 8 of 10 ✘ simulated race condition reproduced on repetition 8
|
||||||
|
│ ├─ repetition 9 of 10 ↷ Failure threshold [2] exceeded
|
||||||
|
│ └─ repetition 10 of 10 ↷ Failure threshold [2] exceeded
|
||||||
|
└─ JUnit Vintage ✔
|
||||||
|
|
||||||
|
Failures (2):
|
||||||
|
JUnit Jupiter:FailureThresholdStopsRemainingDemoTest:intermittentRaceCondition(RepetitionInfo):repetition 4 of 10
|
||||||
|
MethodSource [className = 'com.ankurm.tutorials.junit.repeated.FailureThresholdStopsRemainingDemoTest', methodName = 'intermittentRaceCondition', methodParameterTypes = 'org.junit.jupiter.api.RepetitionInfo']
|
||||||
|
=> org.opentest4j.AssertionFailedError: simulated race condition reproduced on repetition 4
|
||||||
|
org.junit.jupiter.api.Assertions.fail(Assertions.java:142)
|
||||||
|
com.ankurm.tutorials.junit.repeated.FailureThresholdStopsRemainingDemoTest.intermittentRaceCondition(FailureThresholdStopsRemainingDemoTest.java:14)
|
||||||
|
JUnit Jupiter:FailureThresholdStopsRemainingDemoTest:intermittentRaceCondition(RepetitionInfo):repetition 8 of 10
|
||||||
|
MethodSource [className = 'com.ankurm.tutorials.junit.repeated.FailureThresholdStopsRemainingDemoTest', methodName = 'intermittentRaceCondition', methodParameterTypes = 'org.junit.jupiter.api.RepetitionInfo']
|
||||||
|
=> org.opentest4j.AssertionFailedError: simulated race condition reproduced on repetition 8
|
||||||
|
org.junit.jupiter.api.Assertions.fail(Assertions.java:142)
|
||||||
|
com.ankurm.tutorials.junit.repeated.FailureThresholdStopsRemainingDemoTest.intermittentRaceCondition(FailureThresholdStopsRemainingDemoTest.java:14)
|
||||||
|
|
||||||
|
Test run finished after 292 ms
|
||||||
|
[ 5 containers found ]
|
||||||
|
[ 0 containers skipped ]
|
||||||
|
[ 5 containers started ]
|
||||||
|
[ 0 containers aborted ]
|
||||||
|
[ 5 containers successful ]
|
||||||
|
[ 0 containers failed ]
|
||||||
|
[ 10 tests found ]
|
||||||
|
[ 2 tests skipped ]
|
||||||
|
[ 8 tests started ]
|
||||||
|
[ 0 tests aborted ]
|
||||||
|
[ 6 tests successful ]
|
||||||
|
[ 2 tests failed ]
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||||
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||||
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
<groupId>com.ankurm</groupId>
|
||||||
|
<artifactId>repeated</artifactId>
|
||||||
|
<version>1.0.0</version>
|
||||||
|
<packaging>jar</packaging>
|
||||||
|
<name>repeated</name>
|
||||||
|
<description>JUnit @RepeatedTest from JUnit 5 to JUnit 6: RepetitionInfo, failureThreshold, and a
|
||||||
|
real flaky-test-detection use case, rebuilt against JUnit 6.1.3 on JDK 25.</description>
|
||||||
|
|
||||||
|
<properties>
|
||||||
|
<maven.compiler.release>25</maven.compiler.release>
|
||||||
|
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||||
|
<!-- JUnit 6 unified the version numbers across Platform, Jupiter and Vintage artifacts -
|
||||||
|
junit-jupiter and junit-platform-launcher now share this one number. -->
|
||||||
|
<junit.jupiter.version>6.1.3</junit.jupiter.version>
|
||||||
|
<assertj.version>3.27.7</assertj.version>
|
||||||
|
</properties>
|
||||||
|
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.junit.jupiter</groupId>
|
||||||
|
<artifactId>junit-jupiter</artifactId>
|
||||||
|
<version>${junit.jupiter.version}</version>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.assertj</groupId>
|
||||||
|
<artifactId>assertj-core</artifactId>
|
||||||
|
<version>${assertj.version}</version>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.apache.maven.plugins</groupId>
|
||||||
|
<artifactId>maven-surefire-plugin</artifactId>
|
||||||
|
<version>3.6.0</version>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
</project>
|
||||||
Executable
+121
@@ -0,0 +1,121 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Regenerates every file in docs/output/.
|
||||||
|
#
|
||||||
|
# docs/output/00, 01, 02, 04, 05 come straight from `mvn test` -- the four committed test
|
||||||
|
# classes write them via Capture.write() as a side effect of their own assertions.
|
||||||
|
#
|
||||||
|
# docs/output/03, 06, 07 come from three scratch classes that are deliberately NOT part of the
|
||||||
|
# committed suite (a bogus display-name placeholder, and two deterministic "flaky test" demos).
|
||||||
|
# This script recreates them in a throwaway directory, compiles and runs them against the
|
||||||
|
# JUnit Platform Console Standalone launcher, captures the real output, and cleans up -- so the
|
||||||
|
# committed suite stays green at 14/14 while every number in docs/output/ still traces back to a
|
||||||
|
# real run.
|
||||||
|
#
|
||||||
|
# Requires: JDK 17+ on PATH (JUnit 6 requires Java 17; this module was built on JDK 25), network
|
||||||
|
# access to download the console-standalone jar on first run.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
JUNIT_VERSION="6.1.3"
|
||||||
|
JAR_DIR="$(mktemp -d)"
|
||||||
|
STANDALONE_JAR="${JAR_DIR}/junit-platform-console-standalone-${JUNIT_VERSION}.jar"
|
||||||
|
trap 'rm -rf "${JAR_DIR}"' EXIT
|
||||||
|
|
||||||
|
echo "== 1/3: committed suite (mvn test) =="
|
||||||
|
mvn -q test
|
||||||
|
mvn test > "${JAR_DIR}/full-run.txt" 2>&1 || true
|
||||||
|
grep -v '^Picked up JAVA_TOOL_OPTIONS' "${JAR_DIR}/full-run.txt" | tail -20 > docs/output/00-full-test-run.txt
|
||||||
|
sed -i '1i# mvn test -- full committed suite (ReliabilityTest, CustomNameTest, LifecycleTest, MetadataTest)\n' docs/output/00-full-test-run.txt
|
||||||
|
|
||||||
|
echo "== 2/3: downloading junit-platform-console-standalone ${JUNIT_VERSION} =="
|
||||||
|
curl -fsSL -o "${STANDALONE_JAR}" \
|
||||||
|
"https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/${JUNIT_VERSION}/junit-platform-console-standalone-${JUNIT_VERSION}.jar"
|
||||||
|
|
||||||
|
echo "== 3/3: scratch classes (written, run, captured, discarded) =="
|
||||||
|
SCRATCH_SRC="${JAR_DIR}/scratch-src"
|
||||||
|
SCRATCH_CLASSES="${JAR_DIR}/scratch-classes"
|
||||||
|
PKG_DIR="${SCRATCH_SRC}/com/ankurm/tutorials/junit/repeated"
|
||||||
|
mkdir -p "${PKG_DIR}" "${SCRATCH_CLASSES}"
|
||||||
|
|
||||||
|
cat > "${PKG_DIR}/ScratchPlaceholderTest.java" <<'EOF'
|
||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
|
||||||
|
class ScratchPlaceholderTest {
|
||||||
|
|
||||||
|
@RepeatedTest(value = 2, name = "{shortDisplayname} test")
|
||||||
|
void bogusPlaceholderTest() {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
cat > "${PKG_DIR}/FlakyDetectionDemoTest.java" <<'EOF'
|
||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
import org.junit.jupiter.api.RepetitionInfo;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.fail;
|
||||||
|
|
||||||
|
class FlakyDetectionDemoTest {
|
||||||
|
|
||||||
|
@RepeatedTest(10)
|
||||||
|
void intermittentRaceCondition(RepetitionInfo info) {
|
||||||
|
int current = info.getCurrentRepetition();
|
||||||
|
if (current == 4 || current == 8) {
|
||||||
|
fail("simulated race condition reproduced on repetition " + current);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
cat > "${PKG_DIR}/FailureThresholdStopsRemainingDemoTest.java" <<'EOF'
|
||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
import org.junit.jupiter.api.RepetitionInfo;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.fail;
|
||||||
|
|
||||||
|
class FailureThresholdStopsRemainingDemoTest {
|
||||||
|
|
||||||
|
@RepeatedTest(value = 10, failureThreshold = 2)
|
||||||
|
void intermittentRaceCondition(RepetitionInfo info) {
|
||||||
|
int current = info.getCurrentRepetition();
|
||||||
|
if (current == 4 || current == 8) {
|
||||||
|
fail("simulated race condition reproduced on repetition " + current);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
javac -cp "${STANDALONE_JAR}" -d "${SCRATCH_CLASSES}" "${PKG_DIR}"/*.java
|
||||||
|
|
||||||
|
run_scratch() {
|
||||||
|
# These scratch runs are EXPECTED to report test failures (that's the point of 06 and 07),
|
||||||
|
# so capture output via command substitution with `|| true` rather than a raw pipe -- under
|
||||||
|
# `set -o pipefail` a piped non-zero exit from `java` would otherwise abort this script.
|
||||||
|
local class_name="$1" out_file="$2" header="$3" raw_out
|
||||||
|
raw_out="$(java -jar "${STANDALONE_JAR}" execute \
|
||||||
|
-cp "${SCRATCH_CLASSES}" \
|
||||||
|
--select-class="com.ankurm.tutorials.junit.repeated.${class_name}" \
|
||||||
|
--details=tree --disable-ansi-colors 2>&1 || true)"
|
||||||
|
{
|
||||||
|
echo "${header}"
|
||||||
|
echo
|
||||||
|
printf '%s\n' "${raw_out}" | grep -v '^Picked up JAVA_TOOL_OPTIONS'
|
||||||
|
} > "docs/output/${out_file}"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_scratch "ScratchPlaceholderTest" "03-bogus-placeholder-scratch.txt" \
|
||||||
|
"# Scratch run: {shortDisplayname} is not a real placeholder (ScratchPlaceholderTest, removed before commit)"
|
||||||
|
|
||||||
|
run_scratch "FlakyDetectionDemoTest" "06-flaky-detected.txt" \
|
||||||
|
"# Scratch run: @RepeatedTest with no failureThreshold -- all 10 repetitions still run even though 2 fail (FlakyDetectionDemoTest, removed before commit)"
|
||||||
|
|
||||||
|
run_scratch "FailureThresholdStopsRemainingDemoTest" "07-failure-threshold-skips-remaining.txt" \
|
||||||
|
"# Scratch run: @RepeatedTest(failureThreshold = 2) -- after the 2nd failure (repetition 8), repetitions 9 and 10 are SKIPPED, not invoked (FailureThresholdStopsRemainingDemoTest, removed before commit)"
|
||||||
|
|
||||||
|
echo "Done -- docs/output/ regenerated."
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.io.UncheckedIOException;
|
||||||
|
import java.nio.file.Files;
|
||||||
|
import java.nio.file.Path;
|
||||||
|
import java.nio.file.Paths;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Writes a real line of test output to {@code docs/output/<fileName>}, so every number quoted in
|
||||||
|
* the post traces back to a committed transcript rather than a remembered one.
|
||||||
|
*/
|
||||||
|
final class Capture {
|
||||||
|
|
||||||
|
private Capture() {
|
||||||
|
}
|
||||||
|
|
||||||
|
static void write(String fileName, String heading, String body) {
|
||||||
|
Path dir = Paths.get("docs", "output");
|
||||||
|
try {
|
||||||
|
Files.createDirectories(dir);
|
||||||
|
Files.writeString(dir.resolve(fileName), "== " + heading + " ==\n\n" + body + "\n");
|
||||||
|
} catch (IOException e) {
|
||||||
|
throw new UncheckedIOException(e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.CopyOnWriteArrayList;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.AfterAll;
|
||||||
|
import org.junit.jupiter.api.DisplayName;
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
import org.junit.jupiter.api.TestInfo;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A custom {@code name()} pattern built from {@link RepeatedTest#DISPLAY_NAME_PLACEHOLDER},
|
||||||
|
* {@link RepeatedTest#CURRENT_REPETITION_PLACEHOLDER} and
|
||||||
|
* {@link RepeatedTest#TOTAL_REPETITIONS_PLACEHOLDER}. There is no fourth placeholder for a
|
||||||
|
* "short display name" - see {@code docs/output/03-bogus-placeholder-scratch.txt} for what
|
||||||
|
* actually happens if you try one anyway.
|
||||||
|
*/
|
||||||
|
class CustomNameTest {
|
||||||
|
|
||||||
|
private static final List<String> DISPLAY_NAMES = new CopyOnWriteArrayList<>();
|
||||||
|
|
||||||
|
@RepeatedTest(value = 3, name = "{displayName} - Run {currentRepetition}/{totalRepetitions}")
|
||||||
|
@DisplayName("Critical API Integration")
|
||||||
|
void customNameTest(TestInfo testInfo) {
|
||||||
|
assertThat(1 + 1).isEqualTo(2);
|
||||||
|
DISPLAY_NAMES.add(testInfo.getDisplayName());
|
||||||
|
}
|
||||||
|
|
||||||
|
@AfterAll
|
||||||
|
static void capture() {
|
||||||
|
assertThat(DISPLAY_NAMES).containsExactly(
|
||||||
|
"Critical API Integration - Run 1/3",
|
||||||
|
"Critical API Integration - Run 2/3",
|
||||||
|
"Critical API Integration - Run 3/3");
|
||||||
|
Capture.write("02-custom-display-name.txt", "TestInfo.getDisplayName() with a custom name() pattern",
|
||||||
|
String.join("\n", DISPLAY_NAMES));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.CopyOnWriteArrayList;
|
||||||
|
import java.util.concurrent.atomic.AtomicInteger;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.AfterAll;
|
||||||
|
import org.junit.jupiter.api.AfterEach;
|
||||||
|
import org.junit.jupiter.api.BeforeAll;
|
||||||
|
import org.junit.jupiter.api.BeforeEach;
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
import org.junit.jupiter.api.RepetitionInfo;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Each repetition of a {@code @RepeatedTest} is a distinct test execution: {@code @BeforeEach}
|
||||||
|
* and {@code @AfterEach} fire for every one of them, while {@code @BeforeAll}/{@code @AfterAll}
|
||||||
|
* fire exactly once for the whole method. {@link RepetitionInfo} can also be injected into
|
||||||
|
* {@code @BeforeEach} directly - not just the test method itself - which the old version of this
|
||||||
|
* post did not mention.
|
||||||
|
*/
|
||||||
|
class LifecycleTest {
|
||||||
|
|
||||||
|
private static final AtomicInteger BEFORE_ALL_CALLS = new AtomicInteger();
|
||||||
|
private static final AtomicInteger AFTER_ALL_CALLS = new AtomicInteger();
|
||||||
|
private static final AtomicInteger BEFORE_EACH_CALLS = new AtomicInteger();
|
||||||
|
private static final AtomicInteger AFTER_EACH_CALLS = new AtomicInteger();
|
||||||
|
private static final List<String> ORDER = new CopyOnWriteArrayList<>();
|
||||||
|
|
||||||
|
@BeforeAll
|
||||||
|
static void onceBeforeAllRepetitions() {
|
||||||
|
BEFORE_ALL_CALLS.incrementAndGet();
|
||||||
|
ORDER.add("@BeforeAll");
|
||||||
|
}
|
||||||
|
|
||||||
|
@BeforeEach
|
||||||
|
void beforeEveryRepetition(RepetitionInfo info) {
|
||||||
|
BEFORE_EACH_CALLS.incrementAndGet();
|
||||||
|
ORDER.add("@BeforeEach (repetition " + info.getCurrentRepetition() + ")");
|
||||||
|
}
|
||||||
|
|
||||||
|
@RepeatedTest(3)
|
||||||
|
void everyRepetitionGetsFreshState(RepetitionInfo info) {
|
||||||
|
ORDER.add("@RepeatedTest (repetition " + info.getCurrentRepetition() + ")");
|
||||||
|
}
|
||||||
|
|
||||||
|
@AfterEach
|
||||||
|
void afterEveryRepetition(RepetitionInfo info) {
|
||||||
|
AFTER_EACH_CALLS.incrementAndGet();
|
||||||
|
ORDER.add("@AfterEach (repetition " + info.getCurrentRepetition() + ")");
|
||||||
|
}
|
||||||
|
|
||||||
|
@AfterAll
|
||||||
|
static void onceAfterAllRepetitions() {
|
||||||
|
AFTER_ALL_CALLS.incrementAndGet();
|
||||||
|
ORDER.add("@AfterAll");
|
||||||
|
|
||||||
|
assertThat(BEFORE_ALL_CALLS.get()).isEqualTo(1);
|
||||||
|
assertThat(AFTER_ALL_CALLS.get()).isEqualTo(1);
|
||||||
|
assertThat(BEFORE_EACH_CALLS.get()).isEqualTo(3);
|
||||||
|
assertThat(AFTER_EACH_CALLS.get()).isEqualTo(3);
|
||||||
|
|
||||||
|
Capture.write("05-lifecycle-order.txt", "Call order across 3 repetitions", String.join("\n", ORDER)
|
||||||
|
+ "\n\nTotals -- @BeforeAll: " + BEFORE_ALL_CALLS.get()
|
||||||
|
+ ", @AfterAll: " + AFTER_ALL_CALLS.get()
|
||||||
|
+ ", @BeforeEach: " + BEFORE_EACH_CALLS.get()
|
||||||
|
+ ", @AfterEach: " + AFTER_EACH_CALLS.get());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.CopyOnWriteArrayList;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.AfterAll;
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
import org.junit.jupiter.api.RepetitionInfo;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* {@link RepetitionInfo} carries four pieces of state, not two. The old version of this post
|
||||||
|
* covered only {@link RepetitionInfo#getCurrentRepetition()} and
|
||||||
|
* {@link RepetitionInfo#getTotalRepetitions()}. The other two -
|
||||||
|
* {@link RepetitionInfo#getFailureCount()} and {@link RepetitionInfo#getFailureThreshold()} -
|
||||||
|
* have been on the interface since JUnit 5.10, the same release that added
|
||||||
|
* {@code failureThreshold()} itself, and let a test introspect its own threshold configuration
|
||||||
|
* without duplicating the number in two places.
|
||||||
|
*/
|
||||||
|
class MetadataTest {
|
||||||
|
|
||||||
|
private static final List<String> LINES = new CopyOnWriteArrayList<>();
|
||||||
|
|
||||||
|
@RepeatedTest(value = 3, failureThreshold = 2)
|
||||||
|
void testWithMetadata(RepetitionInfo info) {
|
||||||
|
int current = info.getCurrentRepetition();
|
||||||
|
int total = info.getTotalRepetitions();
|
||||||
|
assertThat(total).isEqualTo(3);
|
||||||
|
assertThat(current).isBetween(1, total);
|
||||||
|
|
||||||
|
// No repetition here actually fails, so getFailureCount() stays 0 throughout - this
|
||||||
|
// demonstrates the accessor, not a threshold being crossed (see docs/output/07 for that).
|
||||||
|
assertThat(info.getFailureCount()).isZero();
|
||||||
|
assertThat(info.getFailureThreshold()).isEqualTo(2);
|
||||||
|
|
||||||
|
LINES.add("repetition " + current + " of " + total
|
||||||
|
+ " -- getFailureCount()=" + info.getFailureCount()
|
||||||
|
+ " getFailureThreshold()=" + info.getFailureThreshold());
|
||||||
|
}
|
||||||
|
|
||||||
|
@AfterAll
|
||||||
|
static void capture() {
|
||||||
|
assertThat(LINES).hasSize(3);
|
||||||
|
Capture.write("04-repetition-info-metadata.txt", "RepetitionInfo across all 3 repetitions",
|
||||||
|
String.join("\n", LINES));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
package com.ankurm.tutorials.junit.repeated;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.concurrent.CopyOnWriteArrayList;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.AfterAll;
|
||||||
|
import org.junit.jupiter.api.RepeatedTest;
|
||||||
|
import org.junit.jupiter.api.TestInfo;
|
||||||
|
|
||||||
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The smallest possible {@code @RepeatedTest}: a fixed number, no {@code name()} override, no
|
||||||
|
* metadata parameter. JUnit's own default display name for each invocation is
|
||||||
|
* {@link RepeatedTest#SHORT_DISPLAY_NAME} - {@code "repetition {currentRepetition} of
|
||||||
|
* {totalRepetitions}"} - resolved here through the injected {@link TestInfo}, not assumed.
|
||||||
|
*/
|
||||||
|
class ReliabilityTest {
|
||||||
|
|
||||||
|
private static final List<String> DISPLAY_NAMES = new CopyOnWriteArrayList<>();
|
||||||
|
|
||||||
|
@RepeatedTest(5)
|
||||||
|
void simpleRepeatTest(TestInfo testInfo) {
|
||||||
|
assertThat(true).isTrue();
|
||||||
|
DISPLAY_NAMES.add(testInfo.getDisplayName());
|
||||||
|
}
|
||||||
|
|
||||||
|
@AfterAll
|
||||||
|
static void capture() {
|
||||||
|
assertThat(DISPLAY_NAMES).containsExactly(
|
||||||
|
"repetition 1 of 5", "repetition 2 of 5", "repetition 3 of 5",
|
||||||
|
"repetition 4 of 5", "repetition 5 of 5");
|
||||||
|
Capture.write("01-basic-repetition.txt", "TestInfo.getDisplayName() for each repetition",
|
||||||
|
String.join("\n", DISPLAY_NAMES));
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user