Skip to content

JUnit

This plugin provides functions that handle JUnit tests. It has been validated with JUnit 4.12 and JUnit 5.3.2 and should work with any recent version of JUnit 4 and 5.

It can be used directly in a workflow, or indirectly via generators (such as those providing access to test case managers).

A working mvn installation must be available in the targeted execution environments.

Warning

Surefire versions 3.0.0-M4 and 3.0.0-M5 contain a bug (SUREFIRE-1857) resulting in the Surefire report not containing failure messages. Because of this, the generated Allure reports will also not contain the failure messages.

Hence, it is advised to not use these Surefire releases. Any other version at or above 2.19.1 should be fine.

The functions have a junit category prefix.

JUnit 5

JUnit 5 is a very rich test framework and offers a large combination of possible configurations. This plugin uses mvn test -Dtest={testclass}#{testmethod} commands to run tests.

If you are using the JUnit 5 platform in a simple way, this command line works.

POM Examples

Here is a pom.xml file to run tests written in JUnit 5:

pom.xml
<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>…</groupId>
  <artifactId>…</artifactId>
  <version>…</version>
  <dependencies>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>5.8.2</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <version>5.8.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.2</version>
      </plugin>
    </plugins>
  </build>
</project>

Here is a pom.xml file to run tests written in JUnit 4 (in a JUnit 5 environment):

pom.xml
<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>…</groupId>
  <artifactId>…</artifactId>
  <version>…</version>
  <dependencies>
    <dependency>
      <groupId>org.junit.vintage</groupId>
      <artifactId>junit-vintage-engine</artifactId>
      <version>5.8.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

If you are using a more complex configuration, for example with junit-platform-surefire-provider or with junit-platform-runner, please check that the mvn test -Dtest=testclass#testmethod command works.

Supported types of JUnit 5 tests

The following test types are supported:

  • @Test
  • @RepeatedTest
  • @ParameterizedTest
  • @TestFactory
  • @TestTemplate
  • @Nested classes

The support of @DisplayName custom display names is still unavailable.

Functions

junit/mvntest@v1

Run a JUnit test class or test case.

The POM file is assumed to be in the current directory. If this not the case, use working-directory to specify in which directory the function is to be performed.

You can also use the extra-options input to pass additional parameters like custom settings file to the mvn command.

Inputs

The function has the following inputs:

  • test (required)

    The test reference.

  • properties (optional)

    The additional properties to use (-D option, e.g. foo: value1 would be communicated as -Dfoo=value1 to the mvn command).

  • extra-options (optional)

    Additional parameters to pass to the mvn command.

Reports

The function generates the following reports:

  • *.xml

    Surefire reports (XML). All *.xml files found in the target/surefire-reports directory.

    The Surefire reports have the application/vnd.opentestfactory.junit-surefire+xml content type.

  • *.txt

    Report formatted as text.

    All *.txt files found in the target/surefire-reports directory.

Examples

In this first example, the POM file is in the current directory:

- uses: junit/mvntest@v1
  with:
    test: class#method
    properties:
      foo: value1
      bar: value2

In this second example, the POM file is in the cloned repository:

- uses: actions/checkout@v2
  with:
    repository: https://gitlab.com/myaccount/MyProject.git
- uses: junit/mvntest@v1
  with:
    test: class#method
    properties:
      foo: value1
      bar: value2
  working-directory: MyProject

In this third example, a custom settings.xml file is specified:

- uses: junit/mvntest@v1
  with:
    test: class#method
    properties:
      foo: value1
      bar: value2
    extra-options: --settings custom/path/settings.xml

junit/execute@v1

An execute function for use by generators.

Test Reference format

  • {project}/{qualifiedClassName}[#{testMethod}]

With:

  • {project} (required): name of the project on the source code repository.
  • {qualifiedClassName} (required): qualified name of the test class.
  • {testMethod} (optional): name of the method to test in the test class.
Nested classes

If there are @Nested classes in the test class, the test reference must have one of the following formats:

  • {project}/{qualifiedClassName}[${nestedClass}]*
  • {project}/{qualifiedClassName}[${nestedClass}][#{testMethod}]

(There can be classes nested at any level, e.g. [repository]/[test_class]$[nested_class]$[sub_nested_class]$[sub_sub_nested_class]* or [repository]/[test_class]$[nested_class]$[sub_nested_class]#[test_method].)

The * wildcard

The * wildcard is necessary to execute tests in all nested classes.
It must not be used if a test method is targeted.

With:

  • {project} (required): name of the project on the source code repository.
  • {qualifiedClassName} (required): qualified name of the test class.
  • {nestedClassName} (optional): name of the nested class.
  • {testMethod} (optional): name of the method to test in the test class or nested class.

Surefire and JUnit version

A Surefire version 3.0.0-M6 or later is required to execute nested tests.
@Nested was tested using JUnit version 5.5.2 and later versions, as we encountered problems with earlier versions.

Inputs

The function has the following inputs:

  • test (required)

    The test reference.

Reports

The function generates the following reports:

  • TEST-{qualifiedClassName}.xml

    A Surefire report (XML). {qualifiedClassName} is the fully qualified name of the test class.

    The Surefire report has the application/vnd.opentestfactory.junit-surefire+xml content type.

  • {qualifiedClassName}.txt

    Report formatted as text.

  • junit-run-log.txt

    Maven full console log output in .txt format.

Customization

The JUNIT_EXTRA_OPTIONS environment variable can be used to pass additional parameters to the mvn test command.

If defined it will be appended to the end of the command line.

The following parameters are used in the provider-generated mvn test command:

mvn test \
  -f "{POM_path}" -Dtest={test_name} \
  -Dmaven.test.failure.ignore=true -DfailIfNoTests=true \
  --log-file {junit_run_log_path} $JUNIT_EXTRA_OPTIONS

You must avoid passing, via the JUNIT_EXTRA_OPTIONS variable, the command line parameters that conflict with the parameters already used, or the parameters that impact the generation or alter the path of the reports expected by the orchestrator (view “Reports” section).

Example

- uses: junit/execute@v1-
  with:
    test: path/to/test/root/qualified.testsuite.ClassName#testMethod

The path to the test root is a relative path from the step’s working directory. If not specified, this is the job’s working directory, in which case it must include the repository name.

junit/params@v1

A params function for use by generators.

Inputs

The function has the following inputs:

  • data (required)

    The data to use for the automated test.

  • format (required)

    The format to use for the automated test data.

Example

- uses: junit/params@v1
  with:
    data:
      global:
        key1: value1
        key2: value2
      test:
        key1: value3
        key3: value4
    format: format

format must so far be SQUASHTM_FORMAT (tm.squashtest.org/params@v1).

data can have two keys:

  • global for defining global parameters.
  • test for defining test parameters.

Using with inception

Please refer to “Inception” for more information on what inception is.

Preload the inception environment with at least the test execution report data.

Example

Assuming the following workflow and two existing execution reports report_1.xml and report_2.xml:

my_workflow.yaml
metadata:
  name: junit Inception
resources:
  files:
  - surefire-report1
  - surefire-report2
jobs:
  my_specific_test_suite_job:
    runs-on: inception
    steps:
    - uses: actions/prepare-inception@v1
      with:
        TEST-foo.bar.Weather: ${{ resources.files.surefire-report1 }}
    - uses: junit/execute@v1
      with:
        test: OpenWeather/foo.bar.Weather
  my_nonspecific_job:
    runs-on: inception
    steps:
    - uses: actions/prepare-inception@v1
      with:
        TEST-foo.bar.Weather.xml: ${{ resources.files.surefire-report2 }}
    - uses: junit/execute@v1
      with:
        test: OpenWeather/foo.bar.Weather#pub

You can use the following command to run it:

opentf-ctl \
    run workflow my_workflow.yaml \
    -f surefire-report1=report_1.xml \
    -f surefire-report2=report_2.xml

opentf-ctl ^
    run workflow my_workflow.yaml ^
    -f surefire-report1=report_1.xml ^
    -f surefire-report2=report_2.xml

opentf-ctl `
    run workflow my_workflow.yaml `
    -f surefire-report1=report_1.xml `
    -f surefire-report2=report_2.xml

Configuration

Hooks can be defined for the provided functions. This can be done in workflow definitions or at the orchestrator level so that they apply to all your workflows.

Configuration at the orchestrator level is done by setting the JUNIT_PROVIDER_HOOKS environment variable.

Please refer to “Common Provider Settings” for more information.