NUnit3 tests don't run on TFS build

6
votes

we are currently migrating our unit tests from using MSTest to NUnit (version 3.2.1) but are having problems running the NUnit tests from our TFS .xaml build definition. The TFS build definition is using the rule "Run tests in assemblies matching **\*.test*.dll". For my NUnit test project I have downloaded, installed and referenced the following NuGet packages:

1) NUnit v3.2.1

2) NUnit3TestAdapter v3.0.10

Executing my tests within VS (test explorer) works perfectly, and I thought that these steps would be enough to ensure they were also run as part of the CI build on TFS, however the tests are never executed. I have no error/warning/message pertaining to these tests in the diagnostic output, however I do see that the build process has detected my assembly as a candidate for unit tests seeing as it matches the aforementioned wildcard rule (**\*.test*.dll).

I also tried putting the NUnit3TestAdapter assemblies in the "Version control path to custom assemblies" folder defined in the build controller properties for TFS but to no avail.

Can anyone see if I am missing a step in the process here. From all that I have read on these forums I seem to have done everything necessary but they still don't execute.

I simplified the problem greatly by just running MSTest,exe (which is what TFS build uses) directly on the assembly containing my NUnit tests. The assembly in question has both the adapter and nunit frameworks installed as NuGet packages, and still MsTest reports that no tests are found (see below):

C:\Users\hdav>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" /testcontainer:e:\MyCode\nunit\ExpectedExceptionExample\bin\Debug\ExpectedExceptionExample.dll

Loading e:\MyCode\nunit\ExpectedExceptionExample\bin\Debug\ExpectedExceptionExample.dll...

Starting execution...

No tests to execute.

1
c#.netunit-testingtfsnunit
Could you share your build log? Are you able to see the TestAdapter restored during build? – Cece Dong - MSFT
I can't really share the build log with you but could you elaborate on what you mean by seeing the TestAdapter "restored"? I assume you mean the NUnit3.TestAdapter.dll, if so, then yes I see the following line in the build log:"Adding mapping from '$/TFSAdministration/BuildProcessTemplates/CustomAssemblies/NUnit3.TestAdapter.dll' to ......" – davies
When running tests using MsTest only the old style MsTest projects will execute. To load the new extensible test runner, you must execute vstest.console.exe instead. – jessehouwing
@jessehouwing Can you elaborate on the 'must execute vstest.console.exe' statement. I'm not sure what you mean – Neil
A new test runner was introduced with visual Studio 2012, you'll find it hidden in the visual Studio installation folder. It's called 'vstest.console.*.exe' – jessehouwing

1 Answers

0
votes

I was experiencing the same problem and found that pbd files for the test projects need to be present in order for the adapter to discover tests.

Whilst .pbd files were being generated locally, the build (in this case TF Build) was supplying the /p:DebugType=None switch to the MSBuild arguments. Removing the switch allowed the tests to be discovered and run in the build. Could you have something similar in your build definition?

The issue has been accepted as a bug on GitHub.