Surefire is not picking up Junit 5 tests

127
votes

I wrote a simple test method with JUnit 5:

public class SimlpeTest {
    @Test
    @DisplayName("Some description")
    void methodName() {
        // Testing logic for subject under test
    }
}

But when I run mvn test, I got:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running SimlpeTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Somehow, surefire didn't recognize that test class. My pom.xml looks like:

<properties>
    <java.version>1.8</java.version>
    <junit.version>5.0.0-SNAPSHOT</junit.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit5-api</artifactId>
        <version>${junit.version}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>snapshots-repo</id>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        <releases>
            <enabled>false</enabled>
        </releases>
        <snapshots>
            <updatePolicy>always</updatePolicy>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

<build>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>${java.version}</source>
                <target>${java.version}</target>
            </configuration>
        </plugin>
    </plugins>
</build>

Any idea how to make this work?

15
javamavenjunitmaven-surefire-pluginjunit5
Right now you have to use a special implementation of the surefire-plugin. Check the examples of the junit team here – Hendrik Jander
The question is based on an outdated version but this answer is based on the current one (as of December 2016). – Nicolai Parlog
@Nicolai Thanks for updating the answer. Anyway I would appreciate your edit on question, if you have the time. – Ali Dehghani
This error does not occur like that anymore. The most likely case for a lack of test execution is covered by this question. – Nicolai Parlog
Make sure the test files are in the right place. By default test should be a sibling of main under the src folder. – Joe Bowbeer

15 Answers

123
votes

The maven-surefire-plugin, as of today, does not have full support of JUnit 5. There is an open issue about adding this support in SUREFIRE-1206.

As such, you need to use a custom provider. One has already been developed by the JUnit team; from the user guide, you need to add the junit-platform-surefire-provider provider and the TestEngine implementation for the new API:

<build>
  <plugins>        
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <!-- latest version (2.20.1) does not work well with JUnit5 -->
      <version>2.19.1</version>
      <dependencies>
        <dependency>
          <groupId>org.junit.platform</groupId>
          <artifactId>junit-platform-surefire-provider</artifactId>
          <version>1.0.3</version>
        </dependency>
        <dependency>
          <groupId>org.junit.jupiter</groupId>
          <artifactId>junit-jupiter-engine</artifactId>
          <version>5.0.3</version>
        </dependency>
      </dependencies>
    </plugin>
  </plugins>
</build>

Also, be sure to declare the junit-jupiter-api dependency with a scope of test:

<dependencies>
  <dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.0.3</version>
    <scope>test</scope>
  </dependency>
</dependencies>
62
votes

Update 2

Issue has been fixed in Maven Surefire Plugin v2.22.0

New version is available at Maven Central Repository.

Maven

<dependency>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.0</version>
</dependency>

Gradle

compile group: 'org.apache.maven.plugins', name: 'maven-surefire-plugin', version: '2.22.0'

Update

As Marian pointed out, the latest version of JUnit 5 Platform Surefire Provider (1.2.0) supports latest version of Maven Surefire Plugin (2.21.0):

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.21.0</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.platform</groupId>
                    <artifactId>junit-platform-surefire-provider</artifactId>
                    <version>1.2.0</version>
                </dependency>
            </dependencies>
        </plugin>


Example

pom.xml

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.2.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.21.0</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.platform</groupId>
                    <artifactId>junit-platform-surefire-provider</artifactId>
                    <version>1.2.0</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

TestScenario.java

package test;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

public class TestScenario {

    @Test
    @DisplayName("Test 2 + 2 = 4")
    public void test() {
        Assertions.assertEquals(4, 2 + 2);
    }
}

Output (mvn clean install)

...
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ test --- [INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running test.TestScenario
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 s - in test.TestScenario
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
...

Simplest way till today:

    <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.19.1</version>
        <dependencies>
            <dependency>
                <groupId>org.junit.platform</groupId>
                <artifactId>junit-platform-surefire-provider</artifactId>
                <version>1.1.0</version>
            </dependency>
        </dependencies>
    </plugin>
24
votes

From the JUnit 5 documentation :

Starting with version 2.22.0, Maven Surefire provides native support for executing tests on the JUnit Platform.

Additionally you can read in the maven-surefire-plugin documentation :

Using JUnit 5 Platform

To get started with JUnit Platform, you need to add at least a single TestEngine implementation to your project. For example, if you want to write tests with Jupiter, add the test artifact junit-jupiter-engine to the dependencies in POM

So just that is enough to make run JUnit 5 tests :

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>davidxxx</groupId>
    <artifactId>minimal-pom-junit5</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <junit-jupiter.version>5.2.0</junit-jupiter.version> 
        <!--optional below but good practice to specify our java version-->
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit-jupiter.version}</version>
            <scope>test</scope>
        </dependency>

        <!--optional below -->
        <!-- add any JUnit extension you need such as -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>${junit-jupiter.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.0</version>
            </plugin>
        </plugins>
    </build>

</project>

On my GitHub space I added a working sample maven project that you can browse/clone.
URL: https://github.com/ebundy/junit5-minimal-maven-project

11
votes

I encountered the same problem in August 2019 which I asked about here: Maven silently fails to find JUnit tests to run. These answers led me in the right direction, but I found that you can solve the problem even more concisely. I copied my solution from the JUnit5 sample Maven project.

As of JUnit 5.5.1 and maven-surefire-plugin 2.22.2, you do not need to add the junit-platform-surefire-provider dependency. It is enough to have this one dependency and one plugin specified in your pom.xml:

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter</artifactId>
        <version>5.5.1</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.22.2</version>
        </plugin>
    </plugins>
</build>
9
votes

I ran into this issue with JUnit5 and Maven but also noticed that, even if only junit-jupiter-engine was added as a dependency, tests would run on some projects, not on others. And I kind of see the same pattern in the comments here: In @Alex comment above you can see he doesn't have any issue, even with earlier versions of surefire/junit/platform.

After scratching my head for some time I realized that those projects where the tests wouldn't run were those where the tests method names dit not contain the word "test". Though this isn't mandated by http://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html

In other words: just with

    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.2.0</version>
        <scope>test</scope>
    </dependency>

this

@Test
public void something() {
    assertTrue(true);
}

will NOT be run, whereas

@Test
public void testSomething() {
    assertTrue(true);
}

WILL be run !

This issue unfolds as a russian doll...

Anyway, +1 for @Mikhail Kholodkov whose updated answer fixes all the issues at once!

5
votes

Just to complement, surefire 2.22.0 + junit 5.2.0 + platform 1.2.0 also works. Attached is a working pom for your referecne:

    <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.jjhome.junit5</groupId>
    <artifactId>junit5-hone</artifactId>
    <packaging>jar</packaging>
    <version>1.0.0-SNAPSHOT</version>
    <name>junit5-home</name>

    <properties>
        <maven.compiler.source>1.6</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <junit5.version>5.2.0</junit5.version>
        <platform.version>1.2.0</platform.version>
        <surefire.version>2.22.0</surefire.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>${junit5.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>${junit5.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit5.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
            <version>${junit5.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-launcher</artifactId>
            <version>${platform.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-runner</artifactId>
            <version>${platform.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire.version}</version>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.platform</groupId>
                        <artifactId>junit-platform-surefire-provider</artifactId>
                        <version>${platform.version}</version>
                    </dependency>
                    <dependency>
                        <groupId>org.junit.jupiter</groupId>
                        <artifactId>junit-jupiter-engine</artifactId>
                        <version>${junit5.version}</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</project>
3
votes

In my case this was because of the TestNG in the classpath (SUREFIRE-1527). Groovy 2.5.5 POM have brought it with the groovy-testng module.

Manually specified test-framework provider (as it's described at the https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html) solved the problem:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.1</version>

    <dependency>
        <groupId>org.apache.maven.surefire</groupId>
        <artifactId>surefire-junit-platform</artifactId>
        <version>2.22.1</version>
    </dependency>
3
votes

I was facing the same issue both junit5 and maven-surefire tests were failing. However junit4 was running fine. Below combination worked for me, I don't add the versioning. Use junit-bom for dependency management. Using spring-boot 2.1.4

   <dependencyManagement>
    <dependencies>
            <dependency>
                <groupId>org.junit</groupId>
                <artifactId>junit-bom</artifactId>
                <version>5.6.1</version>
                <scope>import</scope>
                <type>pom</type>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <scope>test</scope>
        </dependency>

<build>
    <plugins>
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
        </plugin>
    </plugins>
</build>

Make sure to upgrade to the latest version of eclipse

1
votes

There is open issue for surefire 2.20

It is working for me with surfire 2.19 + junit-platform-* 1.0.3

1
votes

I had a similar problem also causing Surefire to recognize zero tests.

My problem turned out to be related to the following (from the JUnit 5.1.0 / maven documentation):

Due to a memory leak in Surefire 2.20 and issues running on Java 9, the junit-platform-surefire-provider currently only works with Surefire 2.19.1.

I was trying to use the latest versions of Surefire (2.21.0) and junit-platform-surefire-provider (1.1.0), and it did not work (in neither Java 8 or 9)

Switching back to Surefire 2.19.1 solved my problem.

According to this issue a fix will be included in version 1.2.0 of the junit-platform-surefire-provider (currently available as SNAPSHOT only).

1
votes

One thing I noticed that I was able to get it working:

  • Naming my test class ClinicCalendarShould does not get picked up by maven
  • Naming my test class ClinicCalendarTest DOES get picked up by maven

So, unless I am missing some sort of configuration or parameter or whatever in the surefire plugin, by default, you need to name your test classes XXXTest.

0
votes

Updating to maven-surefire-plugin:2.20 runs the Junit5 tests with no problem.

But I am using the M6 version on Junit5.

0
votes

In my case, the surefire plugin didn't get the correct version auf the jupiter-engine/api. And that was even if running Maven 3.6.1 und surefireplugin Version 2.22.2!

Now my surefire-plugin configuration looks like:

<pluginManagement>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.22.2</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.jupiter</groupId>
                    <artifactId>junit-jupiter-engine</artifactId>
                    <version>5.5.2</version>
                </dependency>
            </dependencies>
        </plugin>

        ...
    </plugins>
</pluginManagement>

Further more, I had to force these Versions:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.5.2</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.5.2</version>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-engine</artifactId>
            <version>1.5.2</version>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-commons</artifactId>
            <version>1.5.2</version>
        </dependency>
    </dependencies>
</dependencyManagement>

Looks like 5.5.2 was linked to the wrong platform version 1.3.2 instead of 1.5.2 in my case.

All JUnit5 Tests gets picked up now. Even with 2.22.0 of the surefire plugin this wasn't the case for me!

Hope that helps...

0
votes

For me the solution is adding the annotation below on top of the class.

@RunWith(JUnitPlatform.class)
public class MyTest {
 ....
}

Then even the surefire plugin is not required.

0
votes

Be careful with nested tests. I had a test class like this:

class AcrTerminalTest {
    @Nested
    class card_is_absent {
        AcrTerminal acrTerminal = new AcrTerminal(new CardAbsentFakeCardTerminal());

        @Test
        void isCardPresent_returns_false() {
            assertThat(acrTerminal.isCardPresent())
                    .isFalse();
        }
    }
}

Running just ./mvnw test -Dtest=AcrTerminalTest failed, I needed to add * after test class name like this ./mvnw test -Dtest=AcrTerminalTest\* (see the asterisk).