SonarQube coverage missing some lines covered by Jacoco report

Question

When I run unit tests with Jacoco agent, there is some discrepancy between my local Jacoco report and the coverage on SonarQube. This seems to only be affecting files that contain nested classes. The report generated locally has coverage information for the outer class and all inner classes, but the coverage data on SonarQube only includes the inner classes.

For example, Foo.java contains outer class, Foo, and inner classes, Bar and Baz.

My local report shows instruction coverage of 26% for class Foo, 46% for class Foo.Bar, and 0% for class Foo.Baz; the overall instruction coverage for Foo.java is 30%. The SonarQube coverage page gives line coverage of 15% for Foo.java. I understand that line coverage does not equal instruction coverage, but I would expect the numbers to be closer. Upon further inspection, I noticed that in the file-based coverage view of Foo.java on SonarQube, all lines in the outer class Foo are marked "Not covered by unit tests" and the only lines that are marked covered are the ones in Foo.Bar that I expected. This difference makes up the approximately 15% gap between the Jacoco report and SonarQube. I don't see any exceptions in the local scanner logs or the server analysis logs.

I am running with JaCoCo 0.7.7.201606060606, Java version 1.8.0_73, and sonar-scanner 2.8 locally. The server is running Java version 1.8.0_66-b17, SonarQube version 5.6.3, and SonarQube Java plugin version 4.2.1.6971.

I would appreciate any suggestions and would be happy to provide more details if that would be helpful.

Answer 1

Comparison of "instructions" with anything else is like comparison of apples and oranges - they don't represent the same thing. Single line of code usually contains many bytecode instructions. And so it is wrong to expect "instruction coverage" to be close to "line coverage", for example: if in total you have 100 instructions in 10 lines and cover 1 line with 20 instructions, then missed instructions 80%, but missed lines 90%.

See http://www.eclemma.org/jacoco/trunk/doc/counters.html about counters that JaCoCo provides. And http://docs.sonarqube.org/display/SONAR/Metric+Definitions about what SonarQube shows. Instructions coverage is presented only in JaCoCo.

And not clear from your question if you see difference between covered lines in Foo.java shown by SonarQube and shown by JaCoCo. If so, then please provide screenshot.

Answer 2

It turns out that the class files that used to run the unit tests (which are also used to generate my local report) differ from the ones used by sonar-scanner. This is because after compilation, unit tests, and local report generation, bnd is run over the class files and rewrites the class files of @Component classes. Because sonar-scanner is run after bnd, it sees different class files. It seems like my problem wasn't inner vs. outer classes, but rather OSGi components vs. non-components; class Foo is an OSGi component, and the inner classes are not.

When I run the scanner over the same classes files used by the Jacoco agent, the line coverage of Foo.java reported by SonarQube is 27% (instead of 15%), and the file-based coverage view matches my local report.