TestParameterInjector gets JUnit5 support

In March 2021, we announced the open source release of TestParameterInjector: A parameterized test runner for JUnit4 (see GitHub page).

Over a year later, the Google-internal usage of TestParameterInjector has continued to rapidly grow, and is now by far the most popular parameterized test framework.

Guava's philosophy frames it nicely: "When trying to estimate the ubiquity of a feature, we frequently use the Google internal code base as a reference." We also believe that TestParameterInjector usage in Google is a decent proxy for its utility elsewhere.

As you can see on the graph above, not only did TestParameterInjector reduce the usage of the other frameworks, but it also caused a drastic increase of the total amount of parameterized tests. This suggests that TestParameterInjector reduced the threshold for parameterizing a regular unit test and Googlers are more actively using this tool to improve the quality of their tests.

JUnit5 (Jupiter) support

At Google, we use JUnit4 exclusively, but some developers outside of Google have moved on to JUnit5 (Jupiter). For those users, we have now expanded the scope of TestParameterInjector.

We've kept the API the same as much as possible:

// **************** JUnit4 **************** // @RunWith(TestParameterInjector.class) public class MyTest {   @TestParameter boolean isDryRun;   @Test public void test1(@TestParameter boolean enableFlag) { ... }   @Test public void test2(@TestParameter MyEnum myEnum) { ... }   enum MyEnum { VALUE_A, VALUE_B, VALUE_C } } // **************** JUnit5 (Jupiter) **************** // class MyTest {   @TestParameter boolean isDryRun;   @TestParameterInjectorTest   void test1(@TestParameter boolean enableFlag) {     // This method is run 4 times for all combinations of isDryRun and enableFlag   }   @TestParameterInjectorTest   void test2(@TestParameter MyEnum myEnum) {     // This method is run 6 times for all combinations of isDryRun and myEnum   }   enum MyEnum { VALUE_A, VALUE_B, VALUE_C } }

The only differences are that @RunWith / @ExtendWith are not necessary and that every test method needs a @TestParameterInjectorTest annotation.

The other features of TestParameterInjector work in a similar way with Jupiter:

class MyTest {   // **************** Defining sets of parameters **************** //   @TestParameterInjectorTest   @TestParameters(customName = "teenager", value = "{age: 17, expectIsAdult: false}")   @TestParameters(customName = "young adult", value = "{age: 22, expectIsAdult: true}")   void personIsAdult_success(int age, boolean expectIsAdult) {     assertThat(personIsAdult(age)).isEqualTo(expectIsAdult);   }   // **************** Dynamic parameter generation **************** //   @TestParameterInjectorTest   void matchesAllOf_throwsOnNull(       @TestParameter(valuesProvider = CharMatcherProvider.class) CharMatcher charMatcher) {     assertThrows(NullPointerException.class, () -> charMatcher.matchesAllOf(null));   }   private static final class CharMatcherProvider implements TestParameterValuesProvider {     @Override     public List<CharMatcher> provideValues() {       return ImmutableList.of(           CharMatcher.any(), CharMatcher.ascii(), CharMatcher.whitespace());     }   } }

Other things we've been working on

Custom names for @TestParameters
When running the following parameterized test:

@Test @TestParameters("{age: 17, expectIsAdult: false}") @TestParameters("{age: 22, expectIsAdult: true}") public void withRepeatedAnnotation(int age, boolean expectIsAdult){ ... }

the generated test names will be:

MyTest#withRepeatedAnnotation[{age: 17, expectIsAdult: false}] MyTest#withRepeatedAnnotation[{age: 22, expectIsAdult: true}]

This is fine for small parameter sets, but when the number of @TestParameters or parameters within the YAML string gets large, it quickly becomes hard to figure out what each parameter set is supposed to represent.

For those cases, we added the option to add customName:

@Test @TestParameters(customName = "teenager", value = "{age: 17, expectIsAdult: false}") @TestParameters(customName = "young adult", value = "{age: 22, expectIsAdult: true}") public void personIsAdult(int age, boolean expectIsAdult){...}

To allow this API change, we had to allow @TestParameters to be used in a different way: The original way of specifying @TestParameters sets was to specify them as a list of YAML strings inside a single @TestParameters annotation. We considered multiple options of specifying the custom name inside of these YAML strings, such as a magic _name key and an extra YAML mapping layer where the keys would be the test names. But we eventually settled on the aforementioned API, which makes @TestParameters a repeated annotation because it results in the least complex code and clearly separates the different parameter sets.

It should be noted that the original API (list of YAML strings in single annotation) still works, but it is now discouraged in favor of multiple @TestParameters annotations with a single YAML string, even when customName isn't used. The main arguments for this recommendation are:

• Consistency with the customName case, which needs a single YAML string per @TestParameters annotation
• We believe it structures the list of parameters (especially when it's long) in a more structured way

Integration with RobolectricTestRunner

Recently, we've managed to internally make a version of RobolectricTestRunner that supports TestParameterInjector annotations. There is a significant amount of work left to open source this, and we are now considering when and how to do this.

Learn more

Our GitHub README provides an overview of the framework. Let us know on GitHub if you have any questions, comments, or feature requests!

By Jens Nyman – TestParameterInjector