A flaky test is a test that sometimes passes and sometimes fails without any change to the code. Flaky tests are dangerous because they erode trust in the entire test suite — when developers see a red build, they think “probably just a flaky test” and ignore it, and one day that ignored red build is a real regression. This guide identifies every major root cause of flaky JUnit 6 tests and provides permanent, proven fixes for each one.
Why Flaky Tests Are Worse Than No Tests
A test that always fails is a clearly broken test — it gets fixed immediately. A test that occasionally fails trains your team to ignore failures. Once the team stops trusting the test suite, they stop acting on it, which defeats the entire purpose of automated testing. Zero tolerance for flaky tests is the correct policy.
Root Cause 1: Shared Mutable State
This is the most common cause. When tests share mutable state — a static field, a database row, a file on disk — the result depends on which test ran first.
// FLAKY: static counter shared across all test instances
private static int requestCounter = 0;
@Test
void firstRequestIncreasesCounterToOne() {
service.processRequest();
requestCounter++;
assertEquals(1, requestCounter); // FLAKY if other tests also increment
}
// FIX: reset state before each test with @BeforeEach
private int requestCounter; // instance field, NOT static
@BeforeEach
void resetCounter() {
requestCounter = 0; // fresh state for every test, guaranteed
service.reset(); // also reset the service under test
}
@Test
void firstRequestIncreasesCounterToOne() {
service.processRequest();
requestCounter++;
assertEquals(1, requestCounter); // always passes: counter starts at 0
}
Root Cause 2: Time-Dependent Tests
// FLAKY: test depends on the current time
@Test
void orderCreatedTodayShouldBeActiveForSevenDays() {
Order order = new Order(LocalDate.now()); // uses real clock
// This fails if order.isActive() checks against LocalDate.now() and
// the test runs exactly at midnight when the date rolls over
assertTrue(order.isActiveUntil(LocalDate.now().plusDays(7)));
}
// FIX: inject a Clock and control time in tests
public class Order {
private final LocalDate createdDate;
private final Clock clock;
public Order(LocalDate createdDate, Clock clock) {
this.createdDate = createdDate;
this.clock = clock;
}
public boolean isActive() {
// Uses injected clock — controllable in tests
return LocalDate.now(clock).isBefore(createdDate.plusDays(7));
}
}
@Test
void orderIsActiveForSevenDaysAfterCreation() {
// Fixed clock: always returns the same date — never flaky
Clock fixedClock = Clock.fixed(Instant.parse("2026-01-15T12:00:00Z"), ZoneOffset.UTC);
LocalDate orderDate = LocalDate.of(2026, 1, 15);
Order order = new Order(orderDate, fixedClock);
assertTrue(order.isActive(), "Order created today should be active");
}
Root Cause 3: Async / Timing-Dependent Tests
// FLAKY: Thread.sleep is never the right amount
@Test
void emailSentAfterOrderConfirmation() throws InterruptedException {
orderService.confirmOrder(orderId);
Thread.sleep(500); // Flaky: email might take 600ms on a slow CI server
verify(emailService, times(1)).send(any());
}
// FIX: use Awaitility to poll with a timeout
import static org.awaitility.Awaitility.*;
import java.time.Duration;
@Test
void emailSentAfterOrderConfirmation() {
orderService.confirmOrder(orderId);
// Poll every 100ms for up to 5 seconds — passes as soon as condition is met
// Fast on fast machines, resilient on slow CI servers
await()
.atMost(Duration.ofSeconds(5))
.pollInterval(Duration.ofMillis(100))
.untilAsserted(() ->
verify(emailService, times(1)).send(any())
);
}
// Awaitility Maven dependency
// <dependency>
// <groupId>org.awaitility</groupId>
// <artifactId>awaitility</artifactId>
// <version>4.2.1</version>
// <scope>test</scope>
// </dependency>
Root Cause 4: Port Conflicts in Integration Tests
// FLAKY: hardcoded port may already be in use in parallel CI runs
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT,
properties = "server.port=8080") // FIXED port = collisions in CI
class OrderApiTest { }
// FIX: use RANDOM_PORT — Spring Boot picks a free port automatically
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class OrderApiTest {
// @LocalServerPort injects the randomly chosen port
@LocalServerPort
private int serverPort;
@Test
void getOrderReturns200() {
String url = "http://localhost:" + serverPort + "/api/orders/1";
// Use the dynamic port — never conflicts
ResponseEntity<OrderDto> response =
new RestTemplate().getForEntity(url, OrderDto.class);
assertEquals(HttpStatus.OK, response.getStatusCode());
}
}
Root Cause 5: File System Race Conditions
// FLAKY: two parallel tests write to the same file path
@Test
void exportOrdersToCsv() throws IOException {
orderExporter.exportTo("/tmp/orders.csv"); // fixed path = collision with parallel tests
assertTrue(Files.exists(Path.of("/tmp/orders.csv")));
}
// FIX: use @TempDir for unique, test-isolated directories
@Test
void exportOrdersToCsv(@TempDir Path tempDir) throws IOException {
Path outputFile = tempDir.resolve("orders.csv"); // unique per test
orderExporter.exportTo(outputFile.toString());
assertTrue(Files.exists(outputFile),
"Export file should be created at: " + outputFile);
assertTrue(Files.size(outputFile) > 0,
"Exported file should not be empty");
}
Root Cause 6: Order-Dependent Database Tests
// FLAKY: test assumes data left by a previous test
@Test
void queryingAllOrdersReturnsFiveItems() {
// Assumes previous test inserted 4 orders, this test inserts 1 more
orderRepository.save(new Order(null, "[email protected]", 49.99));
assertEquals(5, orderRepository.count()); // depends on execution order!
}
// FIX: use @Transactional for auto-rollback, or clean up explicitly
@DataJpaTest // @DataJpaTest already wraps each test in a rollback transaction
class OrderRepositoryTest {
@Autowired private OrderRepository orderRepository;
@Test
void queryingAllOrdersAfterInsertsReturnsCorrectCount() {
// Arrange: insert OWN data — do not rely on other tests
orderRepository.saveAll(List.of(
new Order(null, "[email protected]", 10.0),
new Order(null, "[email protected]", 20.0),
new Order(null, "[email protected]", 30.0)
));
// Assert against KNOWN count, not assumed count
assertEquals(3, orderRepository.count(),
"Should have exactly 3 orders inserted by this test");
// Transaction rolls back after this test — no leftover data
}
}
Flaky Test Root Cause Reference Table
| Root cause | Symptom | Fix |
|---|---|---|
| Shared static state | Passes alone, fails in suite | Move to instance field, reset in @BeforeEach |
| Time dependency | Fails near midnight or on slow machines | Inject Clock; use fixed clock in tests |
| Thread.sleep | Fails on slow CI agents | Replace with Awaitility polling |
| Fixed network port | Fails in parallel CI runs | Use RANDOM_PORT |
| Shared file paths | Fails in parallel test runs | Use @TempDir per test |
| Order-dependent DB state | Passes in one order, fails in another | @Transactional rollback, or explicit cleanup |
| Random data without seed | Occasionally hits edge case | Use seeded Random or fixed test data |
| Uninitialized async state | Intermittent assertion failure | Awaitility or CountDownLatch synchronization |
Frequently Asked Questions (FAQs)
Q1: How do I find out if a test is truly flaky or if it has a real bug?
Use @RepeatedTest(20) to run the test 20 times in succession. If it fails consistently in the same way, it has a real bug. If it fails occasionally and in different ways, it is flaky. This also helps characterise the failure rate — “fails 1 in 10 runs” vs “fails 1 in 100 runs” tells you how urgently to fix it.
Q2: Should I quarantine flaky tests while fixing them?
Yes — immediately. Add @Disabled("FLAKY-123: intermittent failure due to shared DB state, fix in progress") and link to a tracking ticket. A disabled test with a clear reason and ticket reference is far less harmful than a flaky test that erodes team trust. Set a policy that disabled tests must have a resolution date and are reviewed weekly.
Q3: Are there tools that automatically detect flaky tests in CI?
Yes. GitHub Actions has built-in flaky test detection that flags tests that sometimes pass and sometimes fail across recent runs. BuildKite and CircleCI have similar features. For self-hosted Jenkins, the Test Results Analyzer plugin shows pass/fail history per test. Gradle Enterprise (now Develocity) has the most sophisticated flaky test detection and retry features for Gradle builds.
Q4: Is it acceptable to add retry logic to flaky tests?
Retry logic is an acceptable temporary measure for genuinely difficult-to-fix flakiness (e.g., an external API that occasionally times out in staging). Use the @RetryingTest extension pattern shown in Advanced Extensions in JUnit 6. However, retries mask the root cause and must always be accompanied by a ticket to fix the underlying issue. Never accept retry logic as a permanent solution.
Q5: How do I prevent new flaky tests from being introduced?
Add a code review rule: any test using Thread.sleep, hardcoded ports, static mutable fields, or fixed file paths is automatically blocked. Run tests with random execution order in CI (junit.jupiter.execution.order.random=true) to surface order dependencies early. Consider running every new test 5 times in the PR pipeline to catch intermittent failures before merge.
See Also
- Debugging JUnit 6 Tests: Fix Failures Like a Pro
- Parallel Test Execution in JUnit 6: Configuration and Pitfalls
- Test Data Management Strategies in JUnit 6 Projects
- Advanced Extensions in JUnit 6: Creating Custom Testing Frameworks
- JUnit 6 Tutorial: Complete Series Index
Conclusion
Every flaky test has a root cause — shared state, timing, ports, file paths, or order dependency. None of them are mysterious once you know the patterns. Identify the symptom, match it to the root cause table, apply the fix, and verify the test passes 10+ consecutive times. Zero tolerance for flaky tests is not a perfectionist stance — it is the only policy that maintains a test suite your team can actually trust.
Next: Why Your JUnit Tests Are Slow (Performance Optimization Guide) — diagnose and fix the most common causes of slow test suite execution.