Dynamic Tests in JUnit 6 using @TestFactory (Advanced Use Cases)

Dynamic tests in JUnit 6 take parameterized testing a step further: instead of declaring inputs at compile time with annotations, you generate entire test cases — with their own names, logic, and assertions — at runtime. This is the domain of @TestFactory. If you have ever needed to test against a live API response, a database result set, or a list of files on disk, dynamic tests are the right tool.

@TestFactory vs @ParameterizedTest vs @Test

@Test@ParameterizedTest@TestFactory
Test count known atCompile timeCompile time (annotations)Runtime
Names known atCompile timeCompile timeRuntime (generated dynamically)
Data sourceHardcodedAnnotation-declaredAny Java expression
ReturnsvoidvoidStream / Iterable / Collection of DynamicTest
Lifecycle hooksFullFullNo @BeforeEach/@AfterEach per dynamic test

Basic @TestFactory Example

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import java.util.stream.Stream;
import static org.junit.jupiter.api.Assertions.*;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;

class CalculatorDynamicTest {

    // @TestFactory returns a Stream of DynamicTest
    // JUnit 6 runs each DynamicTest as a separate test case
    @TestFactory
    Stream additionDynamicTests() {
        // Each record represents: displayName, addend1, addend2, expectedSum
        record AdditionCase(String name, int a, int b, int expected) {}

        return Stream.of(
            new AdditionCase("positive numbers: 1 + 2 = 3",    1,   2,   3),
            new AdditionCase("negative + positive: -5 + 5 = 0", -5,  5,   0),
            new AdditionCase("zeros: 0 + 0 = 0",               0,   0,   0),
            new AdditionCase("large numbers: 1000 + 2000 = 3000", 1000, 2000, 3000)
        ).map(testCase ->
            // dynamicTest(displayName, executable)
            dynamicTest(testCase.name(), () -> {
                Calculator calc = new Calculator();
                assertEquals(testCase.expected(),
                    calc.add(testCase.a(), testCase.b()),
                    testCase.name());
            })
        );
    }
}
Output:
  ✔ positive numbers: 1 + 2 = 3
  ✔ negative + positive: -5 + 5 = 0
  ✔ zeros: 0 + 0 = 0
  ✔ large numbers: 1000 + 2000 = 3000

Tests run: 4, Failures: 0

Real-World Use Case 1: Testing Against a Live Data Source

Suppose your application loads configuration rules from a database. You want to verify that every rule in the database parses correctly — without knowing at compile time how many rules exist:

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import java.util.List;
import java.util.stream.Stream;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import static org.junit.jupiter.api.Assertions.*;

class ConfigRuleValidationTest {

    private final ConfigRuleRepository repository = new InMemoryConfigRuleRepository();
    private final ConfigRuleParser    parser     = new ConfigRuleParser();

    @TestFactory
    @DisplayName("Every configuration rule in the database must be parseable")
    Stream allConfigRulesAreValid() {
        // Fetch rules at RUNTIME — the count is not known at compile time
        List allRules = repository.findAll();

        return allRules.stream().map(rule ->
            dynamicTest("Rule [" + rule.getId() + "]: " + rule.getName(), () -> {
                // Each rule is tested independently
                assertDoesNotThrow(
                    () -> parser.parse(rule),
                    "Rule '" + rule.getName() + "' should parse without errors"
                );
                assertNotNull(parser.parse(rule).getCompiledExpression(),
                    "Parsed rule should have a compiled expression");
            })
        );
    }
}

Real-World Use Case 2: Testing All Files in a Directory

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import java.io.IOException;
import java.nio.file.*;
import java.util.stream.Stream;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import static org.junit.jupiter.api.Assertions.*;

class JsonSchemaValidationTest {

    // Path to test JSON files in src/test/resources/json-samples/
    private static final Path JSON_SAMPLES_DIR =
        Paths.get("src", "test", "resources", "json-samples");

    @TestFactory
    @DisplayName("All JSON sample files should be valid against the schema")
    Stream allJsonFilesValidateAgainstSchema() throws IOException {
        JsonSchemaValidator validator = new JsonSchemaValidator("order-schema.json");

        // Walk the directory at RUNTIME — new files are tested automatically
        return Files.walk(JSON_SAMPLES_DIR)
            .filter(path -> path.toString().endsWith(".json"))
            .map(jsonFile ->
                dynamicTest("Validate: " + jsonFile.getFileName(), () -> {
                    String jsonContent = Files.readString(jsonFile);
                    ValidationResult result = validator.validate(jsonContent);
                    assertTrue(result.isValid(),
                        "File " + jsonFile.getFileName() + " failed schema validation: "
                        + result.getErrors());
                })
            );
    }
}

DynamicContainer — Hierarchical Dynamic Tests

DynamicContainer groups dynamic tests under a named container node, creating a hierarchical test plan similar to @Nested:

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.DynamicContainer.dynamicContainer;
import static org.junit.jupiter.api.DynamicTest.dynamicTest;
import static org.junit.jupiter.api.Assertions.*;
import java.util.stream.Stream;

class OrderStatusTransitionTest {

    private final OrderService orderService = new OrderService();

    @TestFactory
    Stream orderStatusTransitions() {
        return Stream.of(
            // Container 1: PENDING transitions
            dynamicContainer("From PENDING status", Stream.of(
                dynamicTest("can transition to PROCESSING", () ->
                    assertTrue(orderService.canTransition(PENDING, PROCESSING))),
                dynamicTest("can transition to CANCELLED", () ->
                    assertTrue(orderService.canTransition(PENDING, CANCELLED))),
                dynamicTest("cannot transition directly to DELIVERED", () ->
                    assertFalse(orderService.canTransition(PENDING, DELIVERED)))
            )),

            // Container 2: PROCESSING transitions
            dynamicContainer("From PROCESSING status", Stream.of(
                dynamicTest("can transition to SHIPPED", () ->
                    assertTrue(orderService.canTransition(PROCESSING, SHIPPED))),
                dynamicTest("cannot transition back to PENDING", () ->
                    assertFalse(orderService.canTransition(PROCESSING, PENDING)))
            ))
        );
    }
}

DynamicContainer Report Output

orderStatusTransitions()
  ├─ From PENDING status
  │   ├─ ✔ can transition to PROCESSING
  │   ├─ ✔ can transition to CANCELLED
  │   └─ ✔ cannot transition directly to DELIVERED
  └─ From PROCESSING status
      ├─ ✔ can transition to SHIPPED
      └─ ✔ cannot transition back to PENDING

Tests run: 5, Failures: 0

Supported Return Types for @TestFactory

  • Stream<DynamicTest> — most flexible, supports lazy generation
  • Collection<DynamicTest> (e.g. List, Set) — simple, eager
  • Iterable<DynamicTest> — for custom iterables
  • Iterator<DynamicTest> — one-time traversal
  • DynamicTest[] — array form
  • Stream<DynamicContainer> or mixed Stream<DynamicNode> for hierarchical tests

Frequently Asked Questions (FAQs)

Q1: Do @BeforeEach and @AfterEach run for each dynamic test?

No. @BeforeEach and @AfterEach run once for the @TestFactory method itself, not for each generated DynamicTest. If you need per-test setup, include the setup logic inside each DynamicTest executable lambda. This is one of the important differences between dynamic tests and parameterized tests.

Q2: When should I use @TestFactory instead of @ParameterizedTest?

Use @TestFactory when: (1) the test count is not known at compile time, (2) each test case has a unique name generated at runtime, (3) you need to generate tests from a live data source (database, API, filesystem), or (4) you need hierarchical grouping with DynamicContainer. Use @ParameterizedTest when inputs are known at compile time and can be declared with annotations — it has better IDE support and full lifecycle hook integration.

Q3: Can dynamic tests be filtered by tag?

Currently, individual DynamicTest instances cannot carry @Tag annotations — tags can only be applied to the @TestFactory method itself. Tagging the factory tags all the tests it generates. If you need fine-grained tag filtering on generated tests, consider using separate @TestFactory methods per tag.

Q4: Can a @TestFactory return an empty Stream?

Yes, but JUnit 6 will report a warning. Returning an empty Stream means zero tests were generated, which is usually a sign of a misconfiguration or an empty data source. If this is intentional (e.g., no records in the test database), consider using assumeTrue to skip the factory gracefully, or returning a single DynamicTest that passes with a note.

Q5: Is there a performance cost to lazy Stream-based test generation?

No — using a lazy Stream is actually more efficient than an eager List for large datasets. The DynamicTest instances are created and executed one at a time, meaning JUnit does not need to hold the entire test set in memory. For very large datasets (hundreds or thousands of test cases), prefer Stream over Collection.

See Also

Conclusion

Dynamic tests with @TestFactory unlock a level of test coverage that static annotations cannot reach. When your test data lives in a database, filesystem, or API — or when test count is only known at runtime — @TestFactory is the right tool. Use DynamicTest for individual cases and DynamicContainer for hierarchical grouping. Together they make your dynamic test suite as readable and structured as your static one.

Next: Tags and Test Suites in JUnit 6 — organise and filter your growing test base using tags, suites, and selective execution strategies.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.