Run integration tests after deployment in Visual Studio Team Services

3
votes

I have added a release definition in Visual Studio Team Services to deploy an application to a Azure cloud service. I'm trying to run some integration tests after a successful deployment:

enter image description here

But unfortunately tests are not run:

Warning: No test is available in C:\a\301532a74\Build\Api.Web.IntegrationTests\Api.Web.IntegrationTests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
##[warning]No results found to publish.

I have installed the NUnit Test Adapter (nuget package) for all projects containing tests (unit tests and integration tests). The weird thing is that I am able to run the tests as part of a build but not as part of a release.

I also tried to set the Path to Custom Test Adapters but it doesn't seem to make a difference.

What I am missing here? Is there a better way to run some integration tests as part of a release?

3
azurevisual-studio-2013continuous-integrationnunitazure-devops
Is this NUnit 2 or 3? If 3, are you using the CTP9 package released this weekend? There was an issue with CTP8. – Chris
NUnit 2.6.4. I have the most recent version of NUnitTestAdapter installed. – Rui Jarimba
What build agent are you using? Hosted or your own? – Eddie Chen - MSFT
@Eddie-MSFT: I'm using the hosted agent – Rui Jarimba

3 Answers

3
votes

This is usually caused by the NUnit test adapter cannot be found by VSTest.

Usually, In "Build" environment, NUnit test adapter is restored in working folder by nuget, so VSTest can find it during the testing. But in "Release" environment, NUnit test adapter does not exist by default. So you need to update your build definition to upload the NUnit test adapter files to "drop" and the NUnit test adapter will be avaiable in "Release" environment after the artifacts is downloaded. Then you can set the "Path to Custom Test Adapters" to "$(System.DefaultWorkingDirectory)\" and try again.

If you are using your own build agent, you can also place the NUnit Test Adapter in a permanent path that the build agent can access and set the "Path to Custom Test Adapters" to it. Or install NUnit test adapter extension for VS on the build agent and enter "/UseVsixExtensions:true" in "Visual Studio Test\Other console options".

3
votes

I have found a solution for my problem, based on Eddie's suggestions:

Step 1 - Copy all NUnit assemblies to the output folder of the test project (set the Copy Local property to True)

NUnit assemblies

enter image description here

Step 2 - Create an artifact that will contain all files of the bin folder of the test project. This will contain not only the test assemblies but also the NUnit assemblies

enter image description here

Step 3 - In the test task, set the Path to Custom Test Adapters to the artifacts folder that contains the test assemblies and the NUnit adapters assemblies

enter image description here

That's it! Next time you create a release and deploy to any environment tests will be executed.

0
votes

My problem was that i didn´t have a testrunner for NUnit installed at all, since I´m using resharpers testrunner.

Therefore my solution was as simple as adding the nuget package, "NUnit3TestAdapter" version="3.4.0", to my test project.

In visualstudio.com (VS online) "Nuget restored adapters are automatically searched for." and you need not/should not specify a path for "Custom Test Adapters" - I don´t know if this also works for NUnit test adapter version 2.