I find the surefire-report plug-in very unsuitable to my working style. I clean the project all the time and I don't want to spend 5 min to rebuild the whole site every time I want to look at the test report in my browser.
If I type mvn surefire-report:report-only, the generated report is too ugly and barely readable.
What I'm looking for is something like ant's JUnitReport task. Is there one available out there already?
This is what I do:
# Run tests and generate .xml reports
mvn test
# Convert .xml reports into .html report, but without the CSS or images
mvn surefire-report:report-only
# Put the CSS and images where they need to be without the rest of the
# time-consuming stuff
mvn site -DgenerateReports=false
go to target/site/surefire-report.html for the report.
After tests run, the rest of the two run in about 3.5 seconds for me.
Hope that helps. Enjoy!
Indeed, generating the whole site at each build is clearly not an option. But the problem is that mvn surefire-report:report-only doesn't create the the css/*.css files, hence the ugly result. This is logged in SUREFIRE-616 (doesn't mean something will happen though). Personally, I don't use HTML reports that much so I can live with that but that's not a good answer so here is a workaround based on the ant task (*sigh*):
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>test-reports</id>
<phase>test</phase>
<configuration>
<tasks>
<junitreport todir="target/surefire-reports">
<fileset dir="target/surefire-reports">
<include name="**/*.xml"/>
</fileset>
<report format="noframes" todir="target/surefire-reports"/>
</junitreport>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>ant</groupId>
<artifactId>ant-junit</artifactId>
<version>1.6.2</version>
</dependency>
</dependencies>
</plugin>
Update: My initial idea was to run the Maven AntRun plugin "on demand" to generate the reports... but that's not what I posted, I bound it to the test phase... But I didn't think about the case of failed tests (that would stop the build and prevent the execution of the AntRun plugin). So, either:
Don't bind the AntRun plugin to the test phase, move the configuration outside the execution and call mvn antrun:run on the command line to generate the reports when wanted.
or use the testFailureIgnore option of the test mojo and set it to true in the surefire plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
or set this expression from the command line using the -D parameter:
$ mvn test -Dmaven.test.failure.ignore=true
I think that Option #1 is the best option, you don't necessarily want to generate the reports (especially when the test passes) and generate them systematically may slow down the build on the long term. I'd generate them "on demand".
Thanks for Pascal, I've found an improved solution to do what I want to do:
<plugin>
<!-- Extended Maven antrun plugin -->
<!-- https://maven-antrun-extended-plugin.dev.java.net/ -->
<groupId>org.jvnet.maven-antrun-extended-plugin</groupId>
<artifactId>maven-antrun-extended-plugin</artifactId>
<executions>
<execution>
<id>test-reports</id>
<phase>test</phase>
<configuration>
<tasks>
<junitreport todir="target/surefire-reports">
<fileset dir="target/surefire-reports">
<include name="**/*.xml"/>
</fileset>
<report format="noframes" todir="target/surefire-reports"/>
</junitreport>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-junit</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-trax</artifactId>
<version>1.8.0</version>
</dependency>
</dependencies>
</plugin>
This version uses a newer version of ant and best of all. However, I still haven't found a way to generate a test report when tests fail. How should I do that?
Here is how I did it using the goal site maven-surefire:report :
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.16</version>
<configuration>
<showSuccess>false</showSuccess>
<outputDirectory>${basedir}/target/surefire-reports</outputDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.3</version>
<configuration>
<outputDirectory>${basedir}/target/surefire-reports</outputDirectory>
</configuration>
</plugin>
</plugins>
</reporting>
</project>
Run the below command
mvn clean install surefire-report:report
You can find the report in the below location
{basedir}/target/site/surefire-report.html
For more details refer the below link
http://maven.apache.org/surefire/maven-surefire-report-plugin/usage.html
Add this pom as above mentioned 1.option take out the configuration from execution phase keep outside, run mvn 'verify' and then run again mvn 'antrun:run'.. Then you are able to see failed test cases as well
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>test-reports</id>
<phase>test</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<configuration>
<tasks>
<junitreport todir="target/surefire-reports">
<fileset dir="target/surefire-reports">
<include name="**/*.xml" />
</fileset>
<report format="noframes" todir="target/surefire-reports" />
</junitreport>
</tasks>
</configuration>
<dependencies>
<dependency>
<groupId>ant</groupId>
<artifactId>ant-junit</artifactId>
<version>version</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>version</version>
</plugin>
