Azure DevOps Code Coverage for last Commit

Question

I configure a custom PowerShell Task to analyze the Code Coverage of Azure DevOps repository. The steps are:

Search for specific test assemblies (*Test.dll)
Run coverlet and pass the test assemblies
Proof $LASTEXITCODE is not equal 2 (coverage lower than a threshold)
If $LASTEXITCODE is equal 2
1. Run ReportGenerator with coverlet cobertura summary
2. Send e-mail with reports to last committer (get from git git --git-dir=$git log -1 --format="%ae")

The Problem I have: The Commiter is not interested in the code coverage of the whole repository, he wants to know the coverage of his commit.

What I try to achieve: How can I check up whether a commit contains a test assembly or not? I want to analyze only the test assemblies of the last commit.

If there is no test assembly: do nothing
If there are test assemblies: analyze only this specific one and inform the developer about his code coverage

PowerShell Script:

param([string]$Root, [int]$Threshold = 80, [string]$FromMail, [string]$Output = "Report", [string[]]$Include = @("*Tests.dll"), [string[]]$Exclude)

#VARIABLES
$format = "cobertura"                                   #FORMAT OF THE GENERATED COVERAGE REPORT (json [default]/lcov/opencover/cobertura/teamcity)
$thresholdType = "line"                                 #COVERAGE TYPE TO APPLY THE THRESHOLD TO (line/branch/method)
$coverletOutput = "cobertura.xml"                       #OUTPUT OF THE GENERATED COVERAGE REPORT
$reportTypes = "HtmlInline_AzurePipelines;Cobertura"    #THE OUTPUT FORMATS AND SCOPE (SEPARATED BY SEMICOLON) (Badges/Cobertura/CsvSummary/Html/HtmlChart/HtmlInline/HtmlInline_AzurePipelines/HtmlInline_AzurePipelines_Dark/HtmlSummary/Latex/LatexSummary/MHtml/PngChart/SonarQube/TeamCitySummary/TextSummary/Xml/XmlSummary)

#CODE COVERAGE SCRIPT
#-----------------------------------------------------------------------------------------------------#
##The script should analyze the code coverage of a test assembly and create a `.xml` report.
##Requeried tools: [coverlet](https://github.com/tonerdo/coverlet/blob/master/Documentation/GlobalTool.md), [ReportGenerator](https://automationrhapsody.com/code-coverage-manual-automated-tests-opencover-net-applications/), [git](https://git-scm.com/downloads)
##Root = is the directory where the script seeks recursively for files with `$Include` patterns
##Threshold = is the threshold of the code coverage that will accept the test
##FromMail = is the mail address from which the script should send the coverage warning
##Output = is the output path of the `.xml` report file
##Include = is a pattern list for the recursive search of test assemblies which should be included for the code coverage analysis (for instance `@("*Tests.dll", "*Unit.dll")`)
##Exclude = is a pattern list of subdirectories which should be excluded for the code coverage analysis (for instance `@("*\obj\*", "*\Release\*")`)
#-----------------------------------------------------------------------------------------------------#

#JOIN INCLUDE & EXCLUDE FOR PRINTS
$includeJoin = $($Include -join "', '")
$excludeJoin = $($Exclude -join "', '")

Write-Host "Root:`t`t$Root`nThreshold:`t$Threshold`nFromMail:`t$FromMail`nOutput:`t$Output`nInclude:`t'$includeJoin'`nExclude:`t'$excludeJoin'"

#CHECK ARGUMENTS
if ($Root -eq "" -or $Threshold -lt 0 -or $FromMail -eq "" -or $Output -eq "" -or $null -eq $Include) {
    Write-Host "##vso[task.logissue type=error;][ps1] error: missing root directory, coverage threshold, output directory or include pattern list of unit test .dll," -ForegroundColor Red
    exit(-1)
}
if ($null -eq $Exclude) { $Exclude = @() }

#CHECK VALID E-MAIL
try { $_ = new-object net.mail.mailaddress($FromMail) }
catch { Write-Host "##vso[task.logissue type=error;][ps1] error: invalid mail address '$FromMail'" -ForegroundColor Red; exit(-1) }

#CHECK COMMANDS
[string[]] $cmds = "coverlet", "reportgenerator", "git"
foreach ($cmd in $cmds) {
    if (Get-Command $cmd -errorAction SilentlyContinue) { Write-Host "[$cmd] path: '$($(Get-Command $cmd).Path)'" -ForegroundColor Green }
    else { Write-Host "##vso[task.logissue type=error;][$cmd] error: '$cmd' command not exist" -ForegroundColor Red; exit(-1) }
}

#SET $PWD
Set-Location -Path $Root

#FIND GIT REPOSITORY (FOR COMMIT & E-MAIL)
$git = Get-ChildItem $pwd -Include ".git" -Recurse -Directory -Force -ErrorAction SilentlyContinue | Select-Object -First 1
if ($null -eq $git) { Write-Host "##vso[task.logissue type=error;][git] error: missing repository in directory '$($pwd.Path)' and his subdirectories" -ForegroundColor Red; exit(-1) }

#SEARCH FOR $INCLUDE FILES IN $ROOT
Write-Host "[ps1] search directory: '$Root'" -ForegroundColor Yellow
Write-Host "[ps1] search include: '$includeJoin'" -ForegroundColor Yellow
$files = Get-ChildItem -Path $Root -Include $Include -Recurse -File -Name -ErrorAction SilentlyContinue

#SEARCH FOR $EXCLUDE IN $FILES
$Exclude | Where-Object { $ex = $_; $files = $files | Where-Object { $_ -notlike $ex } }
Write-Host "[ps1] search exclude: '$excludeJoin'" -ForegroundColor Yellow
Write-Host "[ps1] search results:" -ForegroundColor Yellow
$files | Where-Object { Write-Host "`t-$_" -ForegroundColor Gray }

#CHECK FILES FOUND
if ($files.Count -eq 0) { Write-Host "##vso[task.logissue type=error;][ps1] error: error: no files with include pattern '$includeJoin' found in '$Root'" -ForegroundColor Red; exit(-1) }

#START COVERLET
foreach ($file in $files) {
    Write-Host "[coverlet] analyse: '$file'" -ForegroundColor Yellow
    $path = '"{0}"' -f $file
    coverlet $path --target "dotnet" --targetargs "vstest $path --logger:trx" --format $format --threshold $Threshold --threshold-type $thresholdType --output $coverletOutput
    $exitCoverlet = $LASTEXITCODE
    Write-Host "[coverlet] exit code for '$file': $exitCoverlet" -ForegroundColor Yellow
    if ($exitCoverlet -ne 0) { break }
}

#COVERAGE IS TO LOW (2)
if ($exitCoverlet -eq 2) {

    #START REPORT GENERATOR
    reportgenerator -reports:$coverletOutput -reporttypes:$reportTypes -targetdir:$('"{0}"' -f $Output)
    $exitReportGenerator = $LASTEXITCODE
    Write-Host "[reportgenerator] exit code: $exitReportGenerator" -ForegroundColor Yellow

    #SEND MAIL
    $from = $FromMail
    $to = git --git-dir=$git log -1 --format="%ae"
    $attachments = Get-ChildItem -Path "$Output" -Filter *.htm -Recurse | ForEach-Object { $_.FullName }
    $index = Get-ChildItem -Path "$Output" -Filter index.htm -Recurse | ForEach-Object { $_.FullName }
    $commit = git --git-dir=$git log -p $git -1 --pretty=%B
    $subject = "Code Coverage in Commit '$commit'"
    $body = "The code coverage of your commit '$commit' is under the threshold of $Threshold %.<br>Show attachments for more details.<br><br>" + $(Get-Content $index)
    $smtpServer = "smtp.server.de"
    $smtpPort = "25"
    Write-Output "##vso[task.logissue type=warning;][ps1] code coverage is to low, send mail to: $to"
    Send-MailMessage -From $from -to $to -Subject $subject -Body $body -BodyAsHtml -SmtpServer $smtpServer -port $smtpPort -Attachments $attachments
}

Azure DevOps Server Version: 17.143.28912.1 (AzureDevOps2019.0.1).

Agent: Self-Hosted Agent vsts-agent-win-x64-2.144.2.

EDIT: Code Coverage of commited Test-Assemblies

I modifie my first script with the following snippet steps:

read out all changed files from last git commit
search all project files (*csproj, *vbproj) with filter *Test*
check project file includes changed file
remove .proj extension, replace it with .dll
create assembly path with a given $OutputAssembly (bin\Release) from user

Snipped:

#GET THE LAST COMMITED FILES
$commitedFiles = git --git-dir=$GitPath diff-tree --no-commit-id --name-only -r $lastCommit

#SEARCH FOR PROJECT FILES IN $PWD WITH FILTER
$Filter = "*Test*"
$projs = Get-ChildItem -Path $pwd -Recurse -Filter $Filter -Include @("*csproj", "*vbproj")

#SEARCH FOR $EXCLUDE IN $FILES
$Exclude = @("*\obj\*")
$Exclude | Where-Object { $ex = $_; $projs = $projs | Where-Object { $_ -notlike $ex } }
Write-Host "[ps1] search exclude: '$excludeJoin'" -ForegroundColor Yellow
Write-Host "[ps1] search results:" -ForegroundColor Yellow
$projs | Where-Object { Write-Host "`t-$_" -ForegroundColor Gray }

#CHECK PROJECT FILES FOUND
if ($projs.Count -eq 0) { Write-Host "##vso[task.logissue type=error;][ps1] error: error: no projects with filter '$Filter' and include pattern '$includeJoin' found in '$Root'" -ForegroundColor Red; exit(-1) }

#ASSEMBLIES LIST
$assemblies = @()

#LOOP ALL .PROJ FILES
foreach ( $proj in $projs ) {

    #LOOP ALL LINES IN .PROJ FILE
    foreach ( $line in (Get-Content $proj) ) { 
        if ( $line -match 'Compile\s+Include="([^"]+)"' ) {

            #COMPILED FILE IN .PROJ
            $file = Split-Path $matches[1] -Leaf

            #LOOP ALL COMMITED FILES
            foreach($commitedFile in $commitedFiles){

                #GET FILE NAME
                $name = Split-Path $commitedFile -Leaf

                #ADD ASSEMBLY BASED ON .PROJ BASENAME
                if($name -eq $file) { $assemblies += $proj.BaseName + ".dll" }
            }     
        }
    }
}

#FEEDBACK CHANGED ASSEMBLIES
Write-Host "[ps1] changed assemblies:" -ForegroundColor Yellow
$assemblies | Where-Object { Write-Host "`t-$_" -ForegroundColor Gray }

#LOOP ALL ASSEMBLIES
$OutputAssembly = "bin\Release"
foreach ($assembly in $assemblies){

    $path = [IO.Path]::Combine($Root , $OutputAssembly, $assembly)

    #CHECK ASSEMBLY PATH
    if (-not (Test-Path -Path $path)) { 
        Write-Host "##vso[task.logissue type=warning;][ps1] warning: missing assembly '$assembly' at: '$path'" -ForegroundColor Yellow;
    }
    else {

        #START COVERLET
    }
}

Shayki Abramczyk Shayki Abramczyk · Accepted Answer · 2019-08-01T17:19:09

If I understand the question well, you can do something like this:

# Get the last commit SHA1
$lastCommit = "$(Build.SourceVersion)"

# Get the last commit files
$files = git diff-tree --no-commit-id --name-only -r $lastCommit

if($files -match "Test.cs")
{
     # Do something...
}
else
{
     # Do something else...
}

Because usually if you have Test.dll so the source code should be Test.cs.

Azure DevOps Code Coverage for last Commit

1 Answers