An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem

13
votes

I'm facing to an issue when I'm running my unit tests on a VSTS build. When I'm launching all my tests on Visual Studio locally, everything works fine. No error at all. Perfect. But when I'm launching my tests via a tests task on a VSTS build, I have this error:

An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=***********' or one of its dependencies. The system cannot find the file specified.

My test project is set like this:

  • "xunit.runner.visualstudio" on version 2.4.0 with a "targetFramework" set to "net462" (set inside the packages.config)

  • .NET Framework 4.6.2

On the build server I have:

  • Visual Studio 2017, v15.7.5

  • Visual Studio 2015, v14.0

I'm running out of clues... I'm surely missing something... Thanks for your help

PS:

  • All tests passed correctly on the VSTS build
  • On the settings of my build task, when I set "Test platform version" to "Visual Studio 2015", it works. But when I'm choosing "Visual Studio 2017" or "Latest" I have the same error...

EDIT

I have Nuget packages "FluentAssertions" v5.4.1 and "System.ValueTuple" v4.5.0. I already tried to downgrade one of them and both. It failed. But I saw that "FluentAssertions" has a dependency to "System.ValueTuple"

EDIT 2

As @PatrickLu-MSFT said, passing the vs test task version to 1 solved the issue. But when I'm looking closely, the Test platform versions available are 2015 and 2013:

Test platform version for VS test task version 1

When I'm choosing VS test task version 2, I have those choices:

Test platform version for VS test task version 2

I would like to stay on vs test task version 2 (in case of future updates, I don't want to be stuck...). What I saw is when I'm on Test platform version set to "Latest" or "Visual Studio 2017", it does not work. When I'm choosing "Visual Studio 2015", it working... So what I point is:

  • Having a vs test task version set to version 1 make it work (maybe because the latest test platform version is "Visual Studio 2015"...?)
  • Having a vs test task version set to version 2 with Test platform version set to "Visual Studio 2015" make it work

So:

  • A VS test task version set to version 1 seems to work the same way as a VS test task version set to version 2 with a "Test platform version" set to "Visual Studio 2015".
  • How can I make it work with a VS test task version set to version 2 and a "Test platform version" set to "Latest"?

EDIT 3

After connected remotely to the server of my build agent and run tests from VS2017, I compare the output of Visual Studio 2017 of the host server to the output of my local Visual Studio 2017:

  • Local Visual Studio 2017:

My local Visual Studio 2017 output

  • Remote Visual Studio 2017:

Remote Visual Studio 2017 output

Are there some issues with "xUnit.net VSTest Adapter v2.4.0 (32-bit Universal Windows)" on the host?

1
tfsbuildazure-devopsxunitxunit.net
Are you using vs test task and version 2 , you could select the version in the top left corner? If so, have you tried to use version 1, when you say everything works fine. What's the environment on locally are you using VS2015 and VS2017, suggest you make both the environment locally and server side the same, this will narrow down the issue. – PatrickLu-MSFT
I was using vs test task and version 2. After passing it to version 1, everything works fine! Thanks. But why @PatrickLu-MSFT? – Régis NIOX
I update my question @PatrickLu-MSFT (see edit 2) – Régis NIOX
Seems this works well with VS2015 not VS2017, when you test locally which version of VS are you using? "On the build server I have: Visual Studio 2017, v15.7.5/ Visual Studio 2015, v14.0" Could you remote to your build agent and run the build test directly through VS2017, this will narrow down if the issue related your environment. – PatrickLu-MSFT
When I'm testing localy (on my machine) I have Visual Studio 2017 v15.7.5 and everything works. I connected remotely to the server of my build agent and run tests from VS2017 and I had the same result from the output: "[7/26/2018 11:24:21 AM Error] An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=**********' or one of its dependencies. The system cannot find the file specified." When I'm building the project, there is no error... I have no more trace. – Régis NIOX

1 Answers

16
votes

I hit this same problem and solved it by changing the "Path to custom test adapters" to point specifically to the correct test adapter by setting it to "Source\packages\xunit.runner.visualstudio.2.4.0\build_common". (You should modify the path to account for where the packages folder is in your repository.)

This apparently happens because by default VSTS tries to be helpful and scans the entire working directory structure for test adapters and loads all of them, but the xUnit Visual Studio test adapter Nuget package installs test adaptors for .Net Core and UWP as well, so VSTS happily loads them and they blow up.