In the competitive landscape of Java development, flakiness is the ultimate enemy of progress. We have all experienced the “Heisenbug” frustration: you write a test, it passes flawlessly on your local machine, but it fails intermittently once it hits the CI/CD pipeline. These non-deterministic failures erode trust in the build process and waste hours of engineering time. To combat this instability, JUnit 5 introduced a robust solution: the @RepeatedTest annotation.

Whether you are hunting down elusive race conditions, validating random data generators, or stress-testing intermittent network calls, repeating a test is a proven strategy for ensuring long-term stability. Today, we’ll dive deep into how to use @RepeatedTest to transform your test suite from “mostly reliable” to “battle-hardened.”

What is JUnit 5 @RepeatedTest?

The @RepeatedTest annotation is a specialized programming model in JUnit Jupiter that allows you to execute a single test method a specific number of times. Unlike a standard @Test annotation, which executes a method exactly once, @RepeatedTest signals the JUnit Jupiter engine to treat the method as a template. The engine then generates multiple dynamic test invocations based on that template.

Strict Rules for Usage:

To ensure the JUnit engine can discover and execute these tests properly, you must follow these constraints:

• Visibility: The method must not be private or static.
• Return Type: The return type must be void.
• Exclusivity: It replaces the standard @Test annotation. Do not use both on the same method, as this may lead to confusing results or duplicate executions.

1. Basic Usage: Simple Repetition

The most straightforward implementation requires only a numeric value representing the total number of repetitions. This is ideal for “smoke testing” a piece of logic that you suspect might be unstable.

import org.junit.jupiter.api.RepeatedTest;
import static org.junit.jupiter.api.Assertions.assertTrue;

public class ReliabilityTest {

    @RepeatedTest(5) // The test will run exactly 5 times
    void simpleRepeatTest() {
        System.out.println("Executing test...");
        assertTrue(true);
    }
}

Console Output Visualization:

repetition 1 of 5
repetition 2 of 5
repetition 3 of 5
repetition 4 of 5
repetition 5 of 5

Complete code examples are available. Click here to browse or Click here to download.

2. Customizing Display Names for Better Reporting

By default, JUnit 5 uses a generic naming pattern. However, when you are running 50 repetitions in a CI environment like Jenkins or GitHub Actions, you need better visibility. You can use the name attribute to provide context.

Available Dynamic Placeholders:

• {displayName}: The base display name of the method.
• {currentRepetition}: The index of the current execution.
• {totalRepetitions}: The total count defined in the annotation.
• {shortDisplayname}: A shortened version of the name.

@RepeatedTest(value = 3, name = "{displayName} - Run {currentRepetition}/{totalRepetitions}")
@DisplayName("Critical API Integration")
void customNameTest() {
    // Logic to verify external API stability
}

3. Accessing Metadata with RepetitionInfo

There are scenarios where the test logic itself needs to be “repetition-aware.” For example, you might want to log specific data only on the final run or use the repetition index to access different elements in an array. JUnit 5 facilitates this by allowing you to inject RepetitionInfo directly into your method parameters.

import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.RepetitionInfo;

class MetadataTest {

    @RepeatedTest(3)
    void testWithMetadata(RepetitionInfo info) {
        int current = info.getCurrentRepetition();
        int total = info.getTotalRepetitions();
        
        // Log progress or vary data inputs based on 'current'
        System.out.println("Processing batch: " + current + " of " + total);
    }
}

4. Lifecycle Behavior and State Management

Understanding the lifecycle is critical to avoiding state leakage. Each repetition of a @RepeatedTest is treated as a distinct test execution. This means the standard JUnit lifecycle hooks apply to every single iteration:

• @BeforeEach & @AfterEach: These run before and after every repetition. Use these to reset databases, clear caches, or re-initialize objects.
• @BeforeAll & @AfterAll: These run only once per class (before the first repetition and after the final one).

This ensures that “Repetition 2” is not affected by any side effects created by “Repetition 1.”

5. Optimized Debugging with failureThreshold

Introduced in JUnit 5.10, the failureThreshold attribute is a sophisticated addition for handling flaky suites. If you are running a test 100 times to find a rare bug, you might not want the test to continue if it fails immediately and repeatedly.

// If 2 out of the 10 repetitions fail, JUnit stops further executions.
@RepeatedTest(value = 10, failureThreshold = 2)
void flakyServiceTest() {
    // Logic that might fail intermittently due to thread timing
}

This saves significant time in CI/CD pipelines by failing fast once a certain threshold of instability is reached.

When Should You Use @RepeatedTest?

While it is a powerful tool, it should be applied strategically rather than globally. The best use cases include:

1. Race Conditions: Testing multi-threaded code (like CompletableFuture or Parallel Streams) where timing issues cause intermittent failures.
2. Randomness: Validating logic that relies on Math.random(), SecureRandom, or Faker libraries to ensure no specific seed causes a crash.
3. Idempotency Checks: Confirming that calling an API or a function multiple times with the same input yields the same result without unintended side effects.
4. Resource Leaks: Running a test 50 times to see if memory usage or file handles grow continuously over time.

Conclusion

The @RepeatedTest annotation is more than just a convenient loop; it is a structured framework for validating the robustness and reliability of your Java applications. By leveraging custom display names, metadata injection, and failure thresholds, you can create a suite that catches the “invisible” bugs that single-run tests often miss.