mvn failsafe:verify broken when using dependenciesToScan?

1
votes

I have the following failsafe setup and it works as expected when I run things "normally" (= with mvn verify, and don't use dependenciesToScan):

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.18.1</version>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <failIfNoTests>true</failIfNoTests>
    <suiteXmlFiles>
      <suiteXmlFile>src/test/resources/mySuite.xml</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
</plugin>
  • The integration tests run as expected,
  • The verify explodes as expected if no tests are specified, and
  • The build breaks in the verify phase when there are integration test failures

However, when I take exactly the same failsafe setup, and extend it with "dependenciesToScan" in a "test-runner" pom / project that just runs the tests from the jar file (so I don't recompile / rebuild the whole project every time I want to run the integration-tests):

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.18.1</version>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <failIfNoTests>true</failIfNoTests>
    <suiteXmlFiles>
      <suiteXmlFile>src/test/resources/mySuite.xml</suiteXmlFile>
    </suiteXmlFiles>
    <dependenciesToScan>
      <dependency>${test.library.groupId}:${test.library.artifactId}</dependency>
    </dependenciesToScan>
  </configuration>
</plugin>

Here's what happens:

  • The integration-tests run as expected,
  • The build fails cause it doesn't find any tests to run (in the verify phase!? - see below), and
  • When I remove the failIfNoTests, the build is reported to be successful even though there are test failures

I checked all kinds of stuff, and tried a few things, so I'm not sure what I might be missing here. The plugin configuration obviously works without any trouble if I don't use dependenciesToScan.

Is there any chance there's a bug in the failsafe plugin that breaks the verify business logic when you're using dependenciesToScan?

Tests run: 79, Failures: 6, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-failsafe-plugin:2.18.1:verify (default) @ test-runner ---
[DEBUG] Configuring mojo org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify from plugin realm ClassRealm[plugin>org.apache.maven.plugins:maven-failsafe-plugin:2.18.1, parent: sun.misc.Launcher$AppClassLoader@42a57993]
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify' with basic configurator -->
[DEBUG]   (s) basedir = /xyz/test-runner
[DEBUG]   (f) encoding = UTF-8
[DEBUG]   (s) failIfNoTests = true
[DEBUG]   (s) reportsDirectory = /xyz/test-runner/target/failsafe-reports
[DEBUG]   (s) skip = false
[DEBUG]   (f) summaryFile = /xyz/test-runner/target/failsafe-reports/failsafe-summary.xml
[DEBUG]   (s) testClassesDirectory = /xyz/test-runner/target/test-classes
[DEBUG]   (s) testFailureIgnore = false
[DEBUG] -- end configuration --
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:27 min
[INFO] Finished at: 2015-03-13T22:44:46-04:00
[INFO] Final Memory: 13M/250M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify (default) on project test-runner: No tests to run! -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify (default) on project test-runner: No tests to run!
3
javamavenmaven-failsafe-plugin

3 Answers

1
votes

Hm...first from what you have posted it looks your configuration is wrong:

You have configured it like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.18.1</version>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <failIfNoTests>true</failIfNoTests>
    <suiteXmlFiles>
      <suiteXmlFile>src/test/resources/mySuite.xml</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
  <dependenciesToScan>
    <dependency>${test.library.groupId}:${test.library.artifactId}  </dependency>
  </dependenciesToScan>
</plugin>

I'm astonished that you don't get failures from Maven...

But it should look like this. 

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.18.1</version>
  <executions>
    <execution>
      <goals>
        <goal>integration-test</goal>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <failIfNoTests>true</failIfNoTests>
    <suiteXmlFiles>
      <suiteXmlFile>src/test/resources/mySuite.xml</suiteXmlFile>
    </suiteXmlFiles>
    <dependenciesToScan>
      <dependency>${test.library.groupId}:${test.library.artifactId}  </dependency>
    </dependenciesToScan>
  </configuration>
</plugin>

cause the debugging output does not show any kind of configuration which represents the thing you wrote in your configuration (dependenciesToScan).

1
votes

I ran into the same problem. Tried to run tests in from a jar using the dependenciesToScan feature. The maven pom.xml I configured, also had <packaging>pom</packaging>

When running mvn verify -x I got this stack trace:

Caused by: org.apache.maven.plugin.MojoFailureException: No tests to run!
    at org.apache.maven.plugin.failsafe.VerifyMojo.verifyParameters (VerifyMojo.java:225)

Looking at the source code for org.apache.maven.plugin.failsafe.VerifyMojo:

if ( !getTestClassesDirectory().exists() )
{
    if ( getFailIfNoTests() != null && getFailIfNoTests() )
    {
        throw new MojoFailureException( "No tests to run!" );
    }
}

The test source directory is expected to exist.

I solved it by first creating an empty folder target/empty-test-classes, and then added this to the plugin configuration:

<testClassesDirectory>${project.build.directory}/empty-test-classes</testClassesDirectory>

An issue was opened in 2013, with other suggestions on how to work around this problem: https://issues.apache.org/jira/browse/SUREFIRE-1024

0
votes

avoid enabling failIfNoTests for the verify phase, it's enough to have it enabled for the integration-test phase:

        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <executions>
                <execution>
                    <id>default</id>
                    <goals>
                        <goal>integration-test</goal>
                    </goals>
                    <phase>integration-test</phase>
                    <configuration>
                        <failIfNoTests>true</failIfNoTests>
                    </configuration>
                </execution>
                <execution>
                    <id>verify</id>
                    <phase>verify</phase>
                    <goals>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <dependenciesToScan>
                    <dependency>${project.groupId}:cms-it</dependency>
                </dependenciesToScan>
            </configuration>
        </plugin>