Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin

Question

I'm running tests with parallel execution using surefire, and that all seems to work fine.

However, there's only one testng-results.xml generated in the target/surefire-reports/ folder which ... only contains the results of the test that ran last.

I found an issue exactly for that reported for an older version of surefire, and it says "won't fix" here:

http://t161727.apache-maven-issues.apacheforum.info/surefire-1018-surefire-reports-overwrite-each-other-whenusing-forking-t161727.html

However, I doubt that I'm the first person on the planet who is trying to run unit tests in parallel with Jenkins and wants the results displayed properly using the TestNG Jenkins plugin, so I'm pretty sure there must be a solution for this, right?

Here's my surefire plugin config:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${basepom.plugin.surefire.version}</version>
      <configuration>
        <forkCount>2.5C</forkCount>
        <reuseForks>false</reuseForks>
      </configuration>
    </plugin>

Here's the link to the surefire config btw.:

http://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html

mac mac · Accepted Answer · 2015-03-06T04:38:34

I actually ended up finding a solution that worked for me.

I think the problem is trying to do it with forkCount / reuseForks, so I've set those back to the default (you could also just skip those properties if you're not trying to overwrite something from a base pom etc.).

Instead I've used parallel and threadCount. Those however only apply to TestNG, but then, I also need it for TestNG (not JUnit).

This makes tests run in parallel, but the testng-results.xml is generated correctly (without being overwritten by each test running in parallel).

More details here:

http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html

Here's the plugin configuration I have now:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <!-- those are the two default values, you can probably skip them -->
    <forkCount>1</forkCount>
    <reuseForks>true</reuseForks>
    <!-- that's what made it work -->
    <parallel>classes</parallel>
    <threadCount>10</threadCount>
  </configuration>
</plugin>

Obviously, the thread count could be lower or higher depending on what you want to do / what specs your server has, and you could change the settings depending on whether you want to run classes, or some other "level" in parallel.

Also, you could configure parallel and threadCount together with any other properties in the suite file, if you choose to use a suite file.

Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin

1 Answers