maven-failsafe-plugin Failures and BUILD SUCCESS?

15
votes

I want to use maven-failsafe-plugin to run some integration tests. If any test fails, I want Maven to fail the build and not BUILD SUCCESS.

Tests run: 103, Failures: 1, Errors: 0, Skipped: 26
[INFO] BUILD SUCCESS*


how can I configure it, that build not success is?

My failsafe plugin is configured as:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>${failsafe.version}</version>
    <configuration>
        <systemProperties>
            <CI_INTEGRATION_OVERRIDE_PATH>${basedir}/..</CI_INTEGRATION_OVERRIDE_PATH>
        </systemProperties>
        <includes>
            <include>**/integration/**/*.java</include>
        </includes>
        <excludes>
            <exclude>**/integration/**/*TestSuite.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>
3
mavenmaven-pluginmaven-failsafe-plugin
Which version of maven-failsafe-plugin do you use? Which Maven version? How did you called mvn to run integration tests? – khmarbaise
See this mail thread on "Maven - Users". – Andrew Logvinov
MAVEN CALL: mvn clean install -P jars failsafe:integration-test -e <failsafe.version>2.12</failsafe.version> MAVEN 3.16 – Fawi
If i call mvn failsafe:verify then throws an error cannot find failsafe-summary.xml – Fawi

3 Answers

12
votes

As Andrew pointed out, the correct solution is to use failsafe as intended. The integration-test goal is specifically designed not to fail the build. If you want to fail the build, call mvn verify or mvn failsafe:verify

For the verify goal to work, it needs to read the results of the integration test which by default are written to ${project.build.directory}/failsafe-reports/failsafe-summary.xml so make sure that is getting generated.

Also, you have to make sure you bind your maven-failsafe-plugin configuration to both the integration-test goal and the verify goal in the executions part.

Failure to add either of those will lead to maven succeeding the build instead of failing it when integration tests fail.

2
votes

Since you are running mvn clean install both the integration-test and verify phases should be executing. According to the failsafe plugin docs the failsafe:integration-test and failsafe:verify goals are bound to those phases, so I don't believe the extra call to failsafe:integration-test is required.

That said however, I'm not sure I trust the failsafe plugin documentation. I answered a similar question for someone earlier this year. It turned out he had to explicitly bind each goal to the correct phase and then failsafe worked as expected. Might be worth a shot.

0
votes

solution.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <skip>true</skip>
  </configuration>
  <executions>
    <execution>
      <id>unit-test</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <skip>false</skip>
        <excludes>
          <exclude>**/*IntegrationTest.java</exclude>
        </excludes>
      </configuration>
      </execution>
      <execution>
        <id>integration-test</id>
        <phase>integration-test</phase>
        <goals>
          <goal>test</goal>
        </goals>
        <configuration>
          <skip>false</skip>
          <enableAssertions>false</enableAssertions>
          <includes>
            <include>**/*IntegrationTest.java</include>
          </includes>
          <systemPropertyVariables>
            <integration>${integration}</integration>
          </systemPropertyVariables>
        </configuration>
      </execution>
    </executions>
</plugin>