SonarQube Code Coverage from VSTS

3
votes

I'm using VSTS hosted agent VS2017 to run the build, tests, and SonarQube analysis. I'm using the SonarQube extension from the marketplace (https://marketplace.visualstudio.com/items?itemName=SonarSource.sonarqube). Task version in use is 4 which uses SonarQube Scanner for MSBuild 4.0.2.

The analysis works well, but it doesn't translate the code coverage results to the SonarQube. I can see the Code Coverage analysis in VSTS (build details), but not in SonarQube.

In my understanding, the problem is that VSTS generates a binary file with an extension .codecoverage. SonarQube doesn't know how to handle this file. I'm having trouble converting this to XML format as it seems I have no means to do it.

Has someone used VSTS successfully with SonarQube and got the Code Coverage results to SonarQube as well? How have you set it up?

EDIT 1
SonarQube version that I'm using is: Version 6.7 (build 33306), Community Edition

EDIT 2
The end of analysis actually generates the xml-file, like was stated in the comments below. This is from the logs: 

2018-03-06T11:14:56.4189055Z 11:14:56.417  Attempting to locate the 
CodeCoverage.exe tool...
2018-03-06T11:14:56.4210147Z 11:14:56.42  Attempting to locate the 
CodeCoverage.exe tool using setup configuration...
2018-03-06T11:14:56.4345085Z 11:14:56.433  Code coverage command line tool: 
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team 
Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
2018-03-06T11:14:56.4349101Z 11:14:56.434  Fetching code coverage report 
information from TFS...
2018-03-06T11:14:56.4363869Z 11:14:56.435  Attempting to locate a test 
results (.trx) file...
2018-03-06T11:14:57.0458245Z 11:14:57.044  Looking for TRX files in: 
D:\a\1\TestResults, D:\a\1\s\TestResults
2018-03-06T11:14:57.0462747Z 11:14:57.045  Located a test results file: 
D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09.trx
2018-03-06T11:14:57.0600587Z 11:14:57.059  One code coverage attachment was 
found in the trx file: factoryvm-az436\VssAdministrator_factoryvm-az436 
2018-03-06 11_11_34.coverage
2018-03-06T11:14:57.0602504Z 11:14:57.059  Absolute path to coverage file: 
D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09\In\factoryvm-az436\VssAdministrator_factoryvm-az436 2018-03-06 
11_11_34.coverage
2018-03-06T11:14:57.0691948Z 11:14:57.068  Executing file C:\Program Files 
(x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code 
Coverage Tools\CodeCoverage.exe
2018-03-06T11:14:57.0692731Z   Args: analyze 
/output:D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09\In\factoryvm-az436\VssAdministrator_factoryvm-az436 2018-03-06 
11_11_34.coveragexml D:\a\1\s\TestResults\VssAdministrator_factoryvm-
az436_2018-03-06_11_13_09\In\factoryvm-az436\VssAdministrator_factoryvm-
az436 2018-03-06 11_11_34.coverage 
2018-03-06T11:14:57.0694963Z   Working directory: 
D:\a\1\s\TestResults\VssAdministrator_factoryvm-az436_2018-03-
06_11_13_09\In\factoryvm-az436
2018-03-06T11:14:57.0695792Z   Timeout (ms):60000
2018-03-06T11:14:57.0699007Z   Process id: 3540
2018-03-06T11:14:58.7847582Z 11:14:58.783  Process returned exit code 0
2018-03-06T11:14:58.7858908Z 11:14:58.784  Updating project info files with 
code coverage information...
3
sonarqubeazure-devopscode-coverage
@jessehouwing it's Version 6.7 (build 33306), Community Edition. Will add it to the initial question. – Antti Simonen
Add /d:sonar.verbose=true /d:sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml to Additional Settings of Prepare the SonarQube analysis task and set System.debug variable to true, then queue build and share the log on the OneDrive. – starian chen-MSFT
The .coverage file will be convert to coveragexml during sonarqube end analysis task, so you can share the detail log of sonarqube end analysis task. – starian chen-MSFT
I've added the log to the post and it seems that xml-file is being generated correctly. It is not passed on to get SonarQube anyhow. – Antti Simonen

3 Answers

7
votes

The VSTS extension should automatically import the coverage results, but there is currently a regression that will be fixed (hopefully) soon: SONARMSBRU-339

Meanwhile, there is a workaround explained in the VSTS extension documentation in "Analysing a .NET solution": in the Additional Properties text area, add the following property: 

sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml
3
votes

I was unable to see the 'coveragexml' file even though I set 'sonar.verbose=true', 'system.debug=true'. Although, I found another way that works just fine, i.e. converting '.coverage' files into '.coveragexml' with PowerShell script:

Get-ChildItem -Path $(Agent.TempDirectory) -Include "*.coverage" -File -Recurse | % {
$outfile = "$([System.IO.Path]::GetFileNameWithoutExtension($_.FullName)).coveragexml"
$output = [System.IO.Path]::Combine([System.IO.Path]::GetDirectoryName($_.FullName), $outfile)
"Analyse '$($_.Name)' with output '$outfile'..."
.$env:USERPROFILE\.nuget\packages\microsoft.codecoverage\15.8.0\build\netstandard1.0\CodeCoverage\CodeCoverage.exe analyze /output:$output $_.FullName
}

To make it work, make sure you have 'Inline PowerShell' task installed for the Marketplace and place it before 'Run Code Analysis' step in your build pipeline.

0
votes

For those who use a self-hosted agent, you can also install Visual Studio Test Agent 2019.

It contains a CodeCoverage.exe file that the SonarQubeAnalyze@4 Azure DevOps task will find. SonarQube will be able to convert .coverage file into .coveragexml, according to this SonarQube documentation webpage.

Visual Studio Test Agent seems to be free. No need to pay for Visual Studio Enterprise.