Excluding assemblies from code coverage in Visual Studio Team Services

Question

I have a build running on Visual Studio Team Services (formerly Visual Studio Online). I want to exclude some of the assemblies from code coverage calculations. Based on a format I've read from many sources. I have created a .runsettings file as follows:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" 
          uri="datacollector://Microsoft/CodeCoverage/2.0"
          assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Exclude>
                <ModulePath>*AWSSDK*</ModulePath>
              </Exclude>
            </ModulePaths>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

But this produces the following error:

Starting test execution, please wait... Error: System.InvalidOperationException: Cannot mix synchronous and asynchronous operation on process stream. at System.Diagnostics.Process.get_StandardError() at Microsoft.VisualStudio.Coverage.Vanguard.Wait() at Microsoft.VisualStudio.Coverage.Vanguard.Start(String outputName, DataCollectionContext context) at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollectorImpl.StartVanguard(DataCollectionContext context) at Microsoft.VisualStudio.Coverage.UnitTestDataCollector.SessionStart(Object sender, SessionStartEventArgs e) at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.SessionStart(Object sender, SessionStartEventArgs e) at System.EventHandler`1.Invoke(Object sender, TEventArgs e) at WEX.TestExecution.TaefDataCollectionEvents.OnSessionStart(SessionStartEventArgs e) at WEX.TestExecution.DataCollectorTestMode.Initialize(ITestModeSettings settings, ICallbackRegistrar callbackRegistrar) Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true VSTest Test Run failed with exit code: 1

Yes, some of the tests are using async/await. The same .runsettings file works fine in Visual Studio 2015. If I do as it suggests and add the following option: /UseVsixExtensions:true, the result is the same. How can I fix this?

And with the runsettings file you provided, no testing should be found. The format should like "<ModulePath>.*AWSSDK.*</ModulePath>". — Eddie Chen - MSFT
Can you set "system.debug" variable to "true" when queue the build and then share the entire logs? — Eddie Chen - MSFT
Surprisingly, changing the content of ModulePath per your recommendation solved the problem. Strikes me that the InvalidOperationException is misleading, maybe even some bad exception handling? Here's the logs with system.debug=true for a build where ModulePath is *AWSSDK* (and fails). — Snixtor

Eddie Chen - MSFT Eddie Chen - MSFT · Accepted Answer · 2016-04-27T01:18:54

The runsettings file use following regex expressions to match the files:

Regular expressions Include and exclude nodes use regular expressions. For more information, see Using Regular Expressions in Visual Studio. Regular expressions are not the same as wildcards. In particular:

.* matches a string of any characters

. matches a dot ".")

( ) matches parentheses "( )"

\ matches a file path delimiter "\"

^ matches the start of the string

$ matches the end of the string

With your original path, all the files will be excluded since you have only "*" in the path. Refer to this link for details: Regular Expressions in Visual Studio.

For the bad exception, According to the logs your provided, you are running the build with Hosted Build Agent. I did a quick test with Hosted Build Agent and can reproduce this issue too. However this issue does not occur when I try with my own build agent. I suspect that there is some setting/configuration on Hosted Build Agent cause this issue and I have help your submit a feedback on Microsoft Connect Page. You can check this link for tracking: Invalid exception when run testing from Hosted Build Agent

Excluding assemblies from code coverage in Visual Studio Team Services

1 Answers