How to skip integration tests by default but still run them on-demand in multi-module maven project?

Question

I have a multi module project having the following structure:

Project:    
 - module1
 - module2
 - integration-test
 - parent pom

What is the correct way of achieving the following:

• run unit tests from all modules(except integration-test) using mvn clean install
• run integration tests on demand(may be by using maven-failsafe plugin or via a maven profile? )
• fail the build when integration tests fail.
• By Default integration tests should not be run using mvn clean install
• integration-test module has only the integration tests.

I have tried multiple hacks using maven-failsafe plugin and maven-sunfire-plugin(for unit tests) but not able to achieve the above in standard way.

Following is how the relevant portion of integration-test pom looks like:

<dependencies>
   <!-- dependencies required for this module-->

</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.12</version>
            <executions>
                <execution>
                    <id>add-integration-test-sources</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>add-test-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>src/test/java</source>
                        </sources>
                    </configuration>
                </execution>
                <execution>
                    <id>add-integration-test-resources</id>
                    <phase>generate-test-resources</phase>
                    <goals>
                        <goal>add-test-resource</goal>
                    </goals>
                    <configuration>
                        <resources>
                            <resource>
                                <filtering>true</filtering>
                                <directory>src/test/resources</directory>
                            </resource>
                        </resources>
                    </configuration>
                </execution>
            </executions>
        </plugin> 
    </plugins>
</build>

<profiles>
    <profile>
        <id>run-its</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <executions>
                        <execution>
                            <goals>
                                <goal>integration-test</goal>
                                <goal>verify</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

currently when i run mvn clean install it runs integration tests too. When i run mvn -Prun-its clean verify, it is running unit tests from other modules too. what am i missing?

Answer 1

You can skip execution of the Integration tests simply by setting -DskipITs=true when running your build like so:

mvn clean install -DskipITs=true

this will run all other tests but your ITs (see here for doc).

If you only want to run

mvn clean install

you can set the default for skipITs in your pom.xml

<properties>
    <skipITs>true</skipITs>
</properties>

This way you can override it on demand with

mvn clean install -DskipITs=false

To run only ITs without Unittests you can configure the -Property of the maven-surefire-plugin like so

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12.4</version>
    <configuration>
        <skip>${skipUnitTests}</skip>
    </configuration>
</plugin>

so if you run

mvn clean install -DskipITs=false -DskipUnitTests=true

Note that skipUnitTests will be false by default so no need to declare a property for that.

If you'd rather use a Profile it should work like that

    <profile>
        <id>ITs</id>
        <properties>
            <skipUnitTests>true</skipUnitTests>
            <skipITs>false</skipITs> 
        </properties>
    </profile>

and run the build like so

mvn clean install -PITs

Of course you could also use the plugin-configuration for maven-surefire-plugin with true directly in the profile so there'd be no need for the extra property, like

<profile>
    <id>ITs</id>
    <properties>
        <skipITs>false</skipITs> 
    </properties>
    <build>
       <plugins>
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.12.4</version>
                <configuration>
                   <skip>true</skip>
                </configuration>
            </plugin>
       </plugins>
    </build>
</profile>

Answer 2

I don't know if that's the best solution but you can achieve that using profiles. For example in the main pom.xml you add just the other modules in the <modules> section and then add another profile:

<modules>
... standard modules ...
</modules>
<profiles>
 <profile>
   <id>tests</id>
      <modules>
        <module>module1</module> 
... standard modules repeated (it might not be needed>...
        <module>module2</module>
        <module>module-integration-test</module>
      </modules> 
  </profile> 
</profiles>

Then you run maven with that profile if you wnat to run tests.

maven -P tests clean install

That would work if you want to run integration tests AND the other modules. If you want to run just the integration tests you can do it like that:

<modules>
  <!-- EMPTY -->
</modules>
<profiles>
 <profile>
   <id>defaultModule</id>
   <activation>
      <activeByDefault>true</activeByDefault>
   </activation>
     <modules>
        <module>module1</module> 
... standard modules repeated (it might not be needed>...
        <module>module2</module>           
      </modules> 
 </profile>
 <profile>
   <id>tests</id>
      <modules>           
        <module>module-integration-test</module>
      </modules> 
  </profile> 
</profiles>

This way with mvn clean install you will run with defaultModule (which is activeByDefault) and if you specify -P tests you will run just tests

Answer 3

The best is to make a separate module which contains the integration tests which looks like already shown in your own question. Now how to handle the integration tests being running or not ...

The integration-test module looks like this:

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>groupOfYourParant</groupId>
    <artifactId>integration-test</artifactId>
    <version>0.1.0-SNAPSHOT</version>
  </parent>

  <artifactId>integration-test</artifactId>
  <packaging>jar</packaging>

  <name>Mod-IT</name>

  <dependencies>
    <dependency>
      <groupId>groupId</groupId>
      <artifactId>TheArtifact</artifactId>
      <version>${project.version}</version>
    </dependency>
    <!-- Supplemental deps only needed in this module -->
    <dependency>
      <groupId>....</groupId>
      <artifactId>....</artifactId>
      <version>...</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <profiles>
    <profile>
      <id>run-its</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <executions>
              <execution>
                <goals>
                  <goal>integration-test</goal>
                  <goal>verify</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>
</project>

By using this you can simply control to run the integration tests via:

mvn -Prun-its clean verify

It's important that you have define the dependencies to other modules which are needed by the integration tests module or where it made be sure that those modules have to be built before this integration test module.

Furthermore you can now configure here supplemental things which are needed for integration tests and which should be run for example in the pre-integration-test and/or in post-integration-test phase.

If you put the test code into src/test/java plus optional resources into src/test/resources you can now define dependencies within your integration test module separately from any other module.

You should of course define the versions for your plugins in a pluginManagement in your parent pom to define all plugins which are being used during a build.