Unit Tests that use ShimContext.Create() getting error on Azure DevOps Pipeline

Question

I have some unit tests that use Fakes and ShimContext. When they run on the Visual Studio Enterprise 2017 IDE they run successfully, however, when the unit tests are run on the pipeline they are throwing the following error:

Error Message:

Test method MyTestMehodName threw exception: Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Unexpected error returned by SetDetourProvider in profiler library 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\IntelliTrace\Microsoft.IntelliTrace.Profiler.dll'.

Stack Trace:

at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.Initialize() at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider() at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext() at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create() at MySolutuionPath.MyTestMehodName() in \ProjectTestName\TestFileName.cs:line 115

I am able to reproduce this error on the Visual Studio IDE when I add anew TestSetting file to the solution and enable IntelliTrace option: Adding Test setting file to solution

Configuration of the build server:

Windows Server 2019 Datacenter
Visual Studio Enterprise 2017 Version 15.9.17

This is my test method

[TestMethod]
public void MyTestMethodName()
{
      using (ShimsContext.Create())
      {
          // Unit Test goes here with Assert
      }
}

This is how pipeline looks like: Pipeline This is the YAML configuration

steps:
 - task: VSTest@2
  displayName: 'Run Unit Tests'
  inputs:
    testAssemblyVer2: |
     **\*test*.dll
     !**\*TestAdapter.dll
     !**\obj\**
     !**\*QualityTools.UnitTestFramework.dll
     !**\*QualityTools.Testing.Fakes.dll
    searchFolder: '$(Build.BinariesDirectory)'
    vsTestVersion: toolsInstaller
    runInParallel: true
    runTestsInIsolation: false
    codeCoverageEnabled: false
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'
  continueOnError: true

Solutions I have tried:

Adding a .runsettings file to enable/disable IntelliTrace
Changing the "Test platform version" on pipeline to Visual Studio 2015/2017
Enable and Disable "Run tests in isolation" option
Enable and Disable "Code coverage enabled" option

I was wondering whether I was missing any installation on the build server. However, if the unit test run successfully inside the the IDE (Test Explorer) I assumed that I have anything missing.

Which server are you using? TFS? Or VSTS? If it is ran in VSTS, which agent it used? Hosted or privated? And which type? Based on my experience, this issue seems only occurred if the test is using shims. Did you use the Fake in your script? — Merlin Liang - MSFT
VSTS. Private self-hosted agents. I am not really sure about if I used Fake in my script, what do you mean by that? — Felipe Rodrigues
I mean the Fake framwork. Could you update with the simple sample for me to reproduce? Not sure but think this seems be the known issue on our side. Could you update thus I get more clearly about that? — Merlin Liang - MSFT
Very appreciate your script shared. Have got those script, will test it on my side and give you the result as soon as possible. For security, please delete this source link. — Merlin Liang - MSFT
Sorry for delay reply. I have tried the repos on my side and discussed with my colleague. We could reproduce the same issue if I enable the IntelliTrace in VS. But, it succeed all on Azure Devops, no matter I am using Hosted agent or private agent. Does this error can be reproduced if you are using Hosted agent? You can try and let me know the result. In addition, did you store the runsetting or testsetting file in VSTS repos? And, did you ever check its content? I did not see such file in the files you shared to me. We think this is more relative with Unit test. — Merlin Liang - MSFT

Everton Santos Everton Santos · Accepted Answer · 2019-12-16T16:35:45

According to this stack overflow post you could try to change your build agent service to logon as a Normal Privilege User, instead of system/local user.

Unit Tests that use ShimContext.Create() getting error on Azure DevOps Pipeline

1 Answers