Improve your tests with JUnit 5

Years ago JUnit 4 was number one test library among Java developers. Since its formation software development process as well as JVM has changed. Fortunately, JUnit team knew the disadvantages of their project and made great improvements in version 5 too. Let’s have a look at them and see how they can make our tests better.

Architecture

Unlike JUnit 4, which is a monolith project, version 5 was designed to clearly separate test execution platform and testing tool itself. JUnit team finally created well defined, clearly segregated and stable APIs, making using their product easier and quicker to learn for users. We became able to choose only parts of the project we really need. Let’s see what it means by taking a closer look at JUnit 5 modules.

Platform

Main purpose of the Platform is to discover tests and arrange their execution. First of all, it defines test Engine API that needs to be implemented in order to run any type of tests we want. Such engines are then discovered and used to run tests by the second core part of the module - Platform Launcher .

Platform Launcher provides its own API that allows external build tools or IDEs to maintain and run tests easily.

Jupiter

JUnit Jupiter API contains all you need for writing tests. In fact, most times when we speak about JUnit API we actually mean Jupiter API. We can find here all basic tools to test our applications, such as Assertions class containing all types of supported assertions and annotations allowing to augment test lifecycle. We will take a closer look at that shortly.

Jupiter also contains implementation of Engine for JUnit 5 tests.

Vintage

Vintage project exists for backward compatibility. It allows to write JUnit 3 and 4 tests, and contains engine implementation to run them.

Running tests

As it was mentioned before, JUnit 5 designers wanted to allow simple integration model for external tools and nowadays all common Java build tools and IDEs support it. For demonstration purposes I created a Gradle project. The minimal required configuration might look like this:

plugins {
  id 'java'
}

group 'com._98elements'
version '1.0-SNAPSHOT'
sourceCompatibility = JavaVersion.VERSION_8

repositories {
  mavenCentral()
}

dependencies {
  implementation 'com.google.guava:guava:28.0-jre'
  testImplementation 'org.junit.jupiter:junit-jupiter-api:5.5.1'
  testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.5.1'
}

test {
  useJUnitPlatform()
}

As JUnit 5 was designed to support lambdas introduced in Java 8, it’s the minimal version we need in our project. I’m going to focus on JUnit 5 tests here, so only two Jupiter dependencies are required here: API and engine. At the end we need to tell Gradle to useJUnitPlatform() .

I’m going to add two more dependencies that will be needed in order to use JUnit parametrized tests and AssertJ library later:

testImplementation 'org.junit.jupiter:junit-jupiter-params:5.5.1'
testImplementation 'org.assertj:assertj-core:3.14.0'

Writing tests

JUnit assertions

Starting from version 5 JUnit Assertions supports lambda expressions :

assertTrue(() -> { return true; });

Combining it with assertAll method we are able to group assertions into blocks. The main advantage is that all assertions in such blocks are evaluated during test execution, so if there are multiple reasons of failure, all of them are reported:

assertAll(
  () -> assertEquals(3, list.size()),
  () -> assertTrue(list.contains("hydrogen")),
  () -> assertFalse(list.contains("radium"))
);

Cool thing about grouping is that it can be also used for dependent assertions. Nested assertAll are evaluated only if all same level assertions pass:

assertAll(
  () -> {
    assertNotNull(list);
    assertAll(
      () -> assertEquals(3, list.size()),
      () -> assertTrue(list.contains("hydrogen"))
    );
  }
);

JUnit 5 has some basic support for collections :

assertLinesMatch(list, list.subList(0, list.size()));
assertArrayEquals(array, array.clone());

And for timeouts :

Boolean someResult = assertTimeout(ofSeconds(1), () -> {
  sleep(500);
  return true;
});
assertTrue(someResult);

assertTimeout method is not only able to verify if a piece of code finishes in specific time limit, but it also allows us to make additional actions on the result of its execution. Similar assertion exists for exceptions , where additional assertions can be performed on thrown exception:

NullPointerException ex = assertThrows(NullPointerException.class, () -> ((String) null).isEmpty());
assertNotNull(ex);

3rd party libraries

Although JUnit 5 assertions are now sufficient for most common cases, we are still able to use our favorite 3rd party assertion libraries like AssertJ or Hamcrest .

import static org.assertj.core.api.Assertions.assertThat;

@Test
void testAssertion() {
  assertThat(true).isTrue();
}

And we can also mix them both e.g. using JUnit assert all with AssertJ assertions inside.

import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertAll;

@Test
void testAssertAll() {
  assertAll(
    () -> assertThat(3 > 2).isTrue(),
    () -> assertThat(2 > 3).isFalse()
  );
}

Test order

JUnit 5 default test execution order is deterministic, but the algorithm is not obvious. Typically tests shouldn’t depend on the order of their execution, but if you need such behavior, there are several ways JUnit allows to override default execution order. To archive it we need to annotate test class with @TestMethodOrder and choose required mode. There are 3 options available:

@Order

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class TestOrdering {
  @Order(value = 1)
  @Test
  void firstTest() {}
}

Test lifecycle

JUnit offers a better solution that allows plug into test execution flow. In default mode tests definition can consist of:

• @BeforeAll method executed once when test is being initialized,
• @BeforeEach method executed before each call of each test method,
• test methods ( @Test / @ParameterizedTest / @RepetedTest ),
• @AfterEach method executed after the end of execution of each test,
• @AfterAll method executed once after all tests defined in test class are finished.

@BeforeAll
static void initializeTests() {
  System.out.println("Initialize tests");
  counter = 1;
}

@BeforeEach
void beforeTest() {
  System.out.println("Test no. " + counter + " will be run...");
}

@Test
void someTest() {
  counter++;
}

@Test
void someOtherTest() {
  counter++;
}

@AfterEach
void afterTest() {
  System.out.println("Test finished.");
}

@AfterAll
static void summarizeTests() {
  System.out.println("All test finished");
}

The output of this test execution will be:

Initialize tests
Test no. 1 will be run...
Test finished.
Test no. 2 will be run...
Test finished.
All test finished

By default JUnit creates a new instance of test class for each test. But we are able to change this behavior by adding @TestInstance(TestInstance.Lifecycle.PER_CLASS) to our test class. It gives us the opportunity to share class members between tests. For example, in “class” mode we are allowed to use non-static @BeforeAll and @AfterAll methods to perform expensive / time-consuming actions that are required for all test methods.

Display name and nested classes

Test classes and methods can get custom names in JUnit 5. Name can be generated or defined during implementation. Java class and method naming convention is not very user-friendly. It may be one of the reasons Spock test framework seems more popular in BDD or TDD approach. JUnit gave us a possibility to add human readable names and descriptions to our test.

@Nested annotation allows us to arrange tests in hierarchy to represent their logical relationship. This is yet another way to improve readability and simplify the process of test management.

All test methods and classes can be annotated with @DisplayName annotation. Some additional name options exist for parametrized tests:

@DisplayName("An object")
class ObjectRepositoryTest {
  
  @Nested
  @DisplayName("when exists")
  class WhenExists {
  
    @DisplayName("should be found")
    @ParameterizedTest(name = "by id: {arguments}, test no. {index}")
    @ValueSource(strings = {"1", "2"})
    void shouldBeFoundById(String id) { }
    
    @DisplayName("should be retrieved")
    @ParameterizedTest(name = "by id: {0} with expected name: {1}")
    @CsvSource({"1,Name of first", "2,Second object's name"})
    void shouldBeRetrieved(String id, String name) { }
  }

  @Nested
  @DisplayName("when does not exist")
  class WhenDoesNotExist {
  
    @DisplayName("should not be found")
    @Test
    void shouldReturnNotFoundException() { }
  }
}

Test report will look like this:

An object
  -> when exists
    -> should be found
      -> by id: 1, test no. 1
      -> by id: 2, test no. 2
    -> should be retrieved
      -> by id: 1 with expected name: Name of first
      -> by id: 2 with expected name: Second object's name
  -> when not exist
    -> should not be found

Note that you can use nested test lifecycles to reduce code duplication between tests that need the same or similar data preparation before execution.

Conditional execution

There are several ways of making conditional assertions or even configure our test to run under certain rules. We can switch off a single test method or an entire class using @Disable annotation.

Jupiter API, along with Assertions class, offers Assumptions. We can place assumptions in our test definition, where execution of unsatisfied assumptions will be skipped. For example this test will not fail:

@Test
void testAssumption() {
  assumeTrue(7 > 8);
  fail("fail");
}

Finally we can mark selected tests with special tags , that can be later included or excluded from application build task manually or in configuration. It’s a common case that part of our tests is much slower than the others (e.g. integration tests of tests involving REST API calls). As it is time consuming to run those tests, it’s a good idea to run much quicker unit tests of our system before.

In order to do that, first of all we need to tag such test methods or classes with:

@Tag(“integration")

And then we can tell Gradle to exclude “integration” tests from standard test task:

test {
  useJUnitPlatform {
    excludeTags 'integration'
  }
}

And add new task for “integration” tests, that will be executed by Gradle after successfully finished standard test task:

task integrationTest(type: Test) {
  useJUnitPlatform {
    includeTags 'integration'
  }
  check.dependsOn it
  shouldRunAfter test
}

Data driven tests

Last but definitely not least are parametrized and repeatable tests. It’s a great feature that allows us to achieve better test coverage with little to no duplication. And JUnit 5 offers a set of tools for data driven tests.

JUnit parametrized tests allow us to run a single test many times with multiple parameters either provided explicitly or dynamically calculated. Three things need to be done in order to use this feature:

• test implementation dependencies needs to be supplemented with: org.junit.jupiter:junit-jupiter-params:5.5.1
• instead of @Test annotation we need to use @ParameterizedTest
• parameters need to be supplied.

We have a great range of possible ways of providing values to tests. The most flexible (and well known from older version of JUnit) is to use special method that returns test parameters collection (or stream):

@ParameterizedTest
@MethodSource //(value = "testMethodParameters") not necessary as same name is used
void testMethodParameters(String p1, Employee p2) {
  assertAll(
    () -> assertTrue(p1.startsWith("p")),
    () -> assertTrue(p2.getDateOfEmployment().isBefore(LocalDate.now()))
  );
}
static Stream<Arguments> testMethodParameters() {
  return Stream.of(
    Arguments.of("param", Employee.of("Joe", Department.FINANCE, LocalDate.of(2019, 02, 01))),
    Arguments.of("parameter", Employee.of("Ann", Department.IT, LocalDate.of(2018, 12, 01)))
  );
}

In addition, source method doesn’t have to return a stream of Arguments . It can be Iterable , some Collection or simply an array.

@ParameterizedTest
@MethodSource
void testMethodParameter(String p1) {
  assertTrue(p1.startsWith("a"));
}

static String[] testMethodParameter() {
  return new String[] {"a1", "a2"};
}

Alternatively we can place such method in a separate class implementing AgrumentProvider interface and use @ArgumentSource in our test method.

@ParameterizedTest
@ArgumentsSource(EmployeeSet.class)
void testMethodParametersWithProvider(String p1, Employee p2) {
  assertAll(
    () -> assertTrue(p1.startsWith("p")),
    () -> assertTrue(p2.getDateOfEmployment().isBefore(LocalDate.now()))
  );
}

class EmployeeSet implements ArgumentsProvider {
  @Override
  public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
    return Stream.of(
      Arguments.of("p1", Employee.of("Joe", Department.FINANCE, LocalDate.of(2019, 02, 01))),
      Arguments.of("p2", Employee.of("Ann", Department.IT, LocalDate.of(2018, 12, 01)))
    );
  }
}

Both source methods or argument providers are powerful tools. But in many cases it can quickly become hard to read and maintain. JUnit 5 offers a lot of simplifications for most of everyday use-cases.

In case we only need one changeable argument in our test and we already know values we want to be tested, there are much simpler solutions. If the type of argument is one of java primitives we can use @ValueSource with list of values, for example for integers:

@ParameterizedTest
@ValueSource(ints = {1, 3, 5, 7})
void testIntegerValueSource(Integer param) {
  assertAll(
    () -> assertTrue(param > 0),
    () -> assertEquals(1, param % 2)
  );
}

Starting from version 5.4 there is one new extension of this feature I found useful quite often. It allows us to add null and/or empty value to our parameter set using @EmptySource , @NullSource or its combination:

@ParameterizedTest
@NullAndEmptySource
@ValueSource(strings = {"     ", "\n"})
void testNullAndEmptySource(String value) {
  assertTrue(StringUtils.isBlank(value));
}

@EmptySource can also be used for collections and arrays.

Similarly we can test enum values:

@ParameterizedTest
@EnumSource(RoundingMode.class)
void testEnums(RoundingMode mode) {
  assertEquals(BigDecimal.valueOf(1.00).setScale(0, mode), BigDecimal.ONE);
}

It’s possible to include or exclude subset of enum values from tested range using their names directly or matching them by regular expression depending on matching mode, for example:

@ParameterizedTest
@EnumSource(value = RoundingMode.class, names = {"DOWN", "FLOOR"})
void testIncludingEnum(RoundingMode roundingMode) {
  assertEquals(BigDecimal.valueOf(77), BigDecimal.valueOf(77.7).setScale(0, roundingMode));
}

@ParameterizedTest
@EnumSource(value = RoundingMode.class, names = ".*UP", mode = EnumSource.Mode.MATCH_ALL)
void testIncludingEnumByRegex(RoundingMode roundingMode) {
  assertEquals(BigDecimal.valueOf(78), BigDecimal.valueOf(77.5).setScale(0, roundingMode));
}

But in case we need more than one parameter, JUnit 5 comes with solution allowing us to prepare parameters in form of CSV data:

@ParameterizedTest
@CsvSource({
  "2.5, HALF_UP, 3",
  "2.5, HALF_DOWN, 2",
  "2.51, HALF_DOWN, 3"})
void testMultipleParams(String testValue, String roundingMode, String expectedValue) {
assertEquals(
  new BigDecimal(expectedValue),
  (new BigDecimal(testValue)).setScale(0, RoundingMode.valueOf(roundingMode))
  );
}

CSV data can be read directly from a file as well:

@ParameterizedTest
@CsvFileSource(resources = "/testFile.csv", numLinesToSkip = 1)
void testFileParams(String testValue, String roundingMode, String expectedValue) {
  assertEquals(
    new BigDecimal(expectedValue),
    (new BigDecimal(testValue)).setScale(0, RoundingMode.valueOf(roundingMode))
  );
}

In many cases JUnit provides built-in argument converters , so we don’t have to worry about conversion, so our tests can be simplified:

@ParameterizedTest
@ValueSource(strings = {"1", "3", "5", "7"})
void testBigIntegerValueSource(BigInteger param) {
  assertEquals(BigInteger.ONE, param.mod(BigInteger.valueOf(2L)));
}
@ParameterizedTest
@CsvSource({"2, 33, 2019-12-31, true"})
void testMultipleParams(Integer p1, Long p2, LocalDate p3, Boolean p4) {
  assertAll(
    () -> assertEquals(p1, 2),
    () -> assertEquals(p2, Long.valueOf(33)),
    () -> assertTrue(p3.isBefore(LocalDate.now())),
    () -> assertTrue(p4)
  );
}

A lot of common type converters are working out of the box: all primitive types, date & time types from java.time package, Enum s, BigDecimal , UUID , Currency etc. In addition, implicit conversion will be performed on all types of objects if there is one and only public constructor method accepting only one String parameter.

public class Company {
  private String name;
  private Company(String name) { this.name = name; }
  public static Company of(String name) { return new Company(name); }
  public String getName() { return name; }
}

@ParameterizedTest
@ValueSource(strings = {"ABC", "A Company"})
void testCustomParamConversion(Company company) {
  assertTrue(company.getName().startsWith("A"));
}

In case we need non-standard data formats or not supported conversions, we are free to add our own as well. Take a look at JUnit ArgumentConverter and ArgumentsAggregator .

One more case is worth mentioning here. If we only need to run one test multiple times, but without any additional parameters, instead of parametrized test we can use repeatable.

@RepeatedTest(3)
void simpleTest(RepetitionInfo info) {
  assertAll(
    () -> assertTrue(info.getCurrentRepetition() > 0),
    () -> assertTrue(info.getCurrentRepetition() <= info.getTotalRepetitions())
  );
}

The RepetitionInfo object can be injected as a parameter of @RepeatedTest , but it’s optional and can be omitted if you don’t need it.

Conclusion

Great improvement was made since version 4 of JUnit and it keeps getting new features. Consider using nested classes to organize your test cases in a more natural way. Use test lifecycle methods to prepare the execution environment and clean it. In combination with nested classes, parametrized test methods it can reduce duplication, make test execution faster and increase test coverage and readability.