Wrong coverage calculation in sonarqube 6.2 on Gradle multi module project

Question

I'm currently using a gradle multi module java project with good coverage and sonarqube 6.2 with sonarJava plugin 4.10.0.1026. I'm using Gradle 4.0.1, sonarqube plugin 2.5 and jacoco 0.7.9! The code is java 8.

Because of API driven development the API tests are written as abstract tests in the API projects and called from the implementation projects providing the constructors for the tests.

When analyzing the projekt on the sonarqube server the coverage for the implementation projects is measured correctly but the API projects included in the tests of the IMPL projects are on 0.0% coverage. The coverage results for these projects are ignored.

When simply using the jacoco plugin I was able to get the same behaviour. After doing some research I found a solution to get proper jacoco reports:

task codeCoverageReport(type: JacocoReport) {
  description "Creates a unified JaCoCo test report for the project."

  // Gather execution data from all subprojects
  // (change this if you e.g. want to calculate unit test/integration test coverage separately)
  executionData fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec")

  // Add all relevant sourcesets from the subprojects 
  subprojects.each {
    sourceSets it.sourceSets.main
  }

  reports {
    xml.enabled true
    html.enabled true
    html.destination file("${buildDir}/reports/jacoco")
    csv.enabled false
  }
}

// always run the tests before generating the report
codeCoverageReport.dependsOn {
  subprojects*.test
}

My current result is the following:

JaCoCo:

1. JaCoCo (codeCoverageReport-Task)
   • 73% Instruction Coverage
   • 91% Branch Coverage
2. Sonar
   • 43.1% Line Coverage (only ~30% lines considered in calculation!)
   • 82.1% Condition Coverage (only ~20% conditions covered!)

So the coverage results in sonar are not usable. I have read an post announcing the "sonar.jacoco.reportPaths"-parameter starting with sonar 6.2 and I think java-analyzer 4.4 or sth. like that. When adding this parameter to my gradle build script, the script does not compile anymore. When adding the jacoco .exec files to sonar via sonar project administration nothing changes.

It would be great if there would be a way to manage sonar to calculate the correct coverage.

Answer 1

Thx @Lance Java! He pushed me to a cleaner solution than the one below. If all subprojects have jacoco reports this works aswell. If like me there is only a report in few projects the original solution seems to work better.

apply plugin: 'base'
apply plugin: 'org.sonarqube'
[...]
allprojects {
  apply plugin: 'java'
  apply plugin: "jacoco"
  [...]
  test {
    [...]
    jacoco {
      append=true
    }
  }
}
[...]
task jacocoMerge( type: JacocoMerge ) {
  dependsOn( subprojects.jacocoTestReport.dependsOn )
  mustRunAfter( subprojects.jacocoTestReport.mustRunAfter )
  destinationFile = file( "${buildDir}/jacoco/mergedTests.exec" )
  executionData = files( subprojects.jacocoTestReport.executionData )
                       .filter { jacocoReportFile -> jacocoReportFile.exists() }
}
tasks.sonarqube.dependsOn jacocoMerge
[...]
sonarqube {
  properties {
    [...]
    property "sonar.jacoco.reportPath", "${buildDir}/jacoco/*.exec"
  }
}

---

Original answer:

It took some time to manage to get the correct coverage data to sonar. There were multiple issues to solve. Sometimes Sonar lost track of the jacoco changes in the classes, so the tests needed the parameter:

append=true

This did not do all the work. There was still an issue in collecting the cross-project coverage. Best solution therefore was to force jacoco to write coverage data to a single .exec file and to hand this to sonar.

Final solution looks like this:

apply plugin: 'base'
apply plugin: 'org.sonarqube'
[...]
allprojects {
  apply plugin: 'java'
  apply plugin: "jacoco"
  [...]
  test {
    [...]
    jacoco {
      append=true
      destinationFile = file( "${rootProject.buildDir}/jacoco/jacocoTest.exec" ) 
    }
  }
}
[...]
sonarqube {
  properties {
    [...]
    property "sonar.jacoco.reportPath", "${buildDir}/jacoco/*.exec"
  }
}

Now sonar has the correct coverage data for my project. After adding some additional tests this is the result:

• Total Coverage 91.6%
• Line Coverage 91.7%
• Condition Coverage 91.3%
• Uncovered Lines 36
• Uncovered Conditions 11
• Lines to Cover 433
• Unit Tests 1,114
• Unit Test Errors 0
• Unit Test Failures 0
• Skipped Unit Tests 0
• Unit Test Success (%) 100.0%
• Unit Test Duration 4s

Hope this may help some of you... ;)

Answer 2

If your tests are in a different project to the sources that you want coverage reports on then you'll need to set additionalSourceDirs and additionalClassDirs. Eg:

evaluationDependsOn ':foo' 
task codeCoverageReport(type: JacocoReport) {
    additionalSourceDirs.add project(':foo').sourceSets.main.java.sourceDirectories
    additionalClassDirs.add project(':foo').sourceSets.main.output.classesDirs
    // etc
} 

Answer 3

I'm not sure I understand why it's an issue for only some projects to have jacoco and other projects not. You can use Gradle's rich API's (eg TaskCollection and Project) to find them dynamically.

Eg:

[':project1', ':project3', ':project5'].each {
    project(it) {
        apply plugin: 'java'
        apply plugin: 'jacoco'
    }
}

project(':merger') {
    Collection<Project> jacocoProjects = allprojects.findAll { it.plugins.hasPlugin('jacoco' }
    evaluationDependsOn jacocoProjects 

    task jacocoMerge(type: JacocoMerge) {
        dependsOn jacocoProjects*.tasks.withType(Test)
        executionData jacocoProjects*.tasks.withType(Test)
    }

    task mergedReport(type: JacocoReport) {
        dependsOn jacocoMerge
        executionData jacocoMerge.destinationFile
        sourceDirectories.add(files(jacocoProjects*.sourceSets*.java.srcDirs))
        classDirectories.add(files(jacocoProjects*.sourceSets*.output.classesDir))
    }
}