Error while executing Run Functional Test task in VSTS

Question

In VSTS 2015 we have build and release pipeline setup in a project which is using NUnit test framework. We are trying to configure Integration test case execution using "Run Functional Task" configuration of the task is given in the screenshot below:

Visual Test Agent Deployment Task is used to Deploy the NUnit Test Agent in target machine.

There is a folder that is created in Target Machine where all test assemblies are getting copied i.e. C:\TestDlls and for NUnit Adapter assemblies C:\PIMPackages

The Agent deployment task is getting executed successfully but it get aborted in "Run Functional Test" with below error message:

2016-04-14T09:51:08.0151865Z DistributedTests: Test run '65' is in 'Aborted' state.
2016-04-14T09:51:18.0177458Z ##[warning]DistributedTests: Test run is aborted. Logging details of the run logs.
2016-04-14T09:51:18.2601235Z ##[warning]DistributedTests: New test run created.
2016-04-14T09:51:18.2601235Z ##[warning]Test Run queued for Project Collection Build Service (deepaksoniv).
2016-04-14T09:51:18.2611323Z ##[warning]DistributedTests: Test discovery started.
2016-04-14T09:51:18.2621353Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 65
2016-04-14T09:51:18.2631347Z ##[warning]DistributedTests: UnExpected error occured during test execution. Try again.
2016-04-14T09:51:18.2631347Z ##[warning]DistributedTests: Error : NUnit Adapter 3.0.8.0 discovering test is finished
2016-04-14T09:51:18.2641325Z ##[warning]DistributedTests: Test run aborted. Test run id: 65
2016-04-14T09:51:18.2681313Z ##[error]The test run was aborted, failing the task.

In Target Machine where Test Agent is getting deployed the DTAEXecutionHost log file abstract:

V, 3780, 11, 2016/04/14, 09:50:57.073, PIMNEWVM\DTAExecutionHost, TestCaseFilter.GetTestPropertiesInTraitDictionary : Populating trait dictionary with test case properties.
V, 3780, 11, 2016/04/14, 09:50:57.073, PIMNEWVM\DTAExecutionHost, TestCaseFilter.GetTestPropertiesInTraitDictionary : Populating trait dictionary with test case 

-------------------------------------------------------------------------------------------------

[test] 8260 ms
V, 3780, 10, 2016/04/14, 09:51:00.401, PIMNEWVM\DTAExecutionHost, TestExecutionServiceCommandQueueListener:DispatchAsync
I, 3780, 10, 2016/04/14, 09:51:00.401, PIMNEWVM\DTAExecutionHost, ExecutionTaskStateModel.ListenerCommandReceived : Requesting a stop on Stop command.
I, 3780, 10, 2016/04/14, 09:51:00.401, PIMNEWVM\DTAExecutionHost, TestExecutionServiceCommandQueueListener.DispatchAsync : Stop listening on Stop command.
I, 3780, 9, 2016/04/14, 09:51:00.401, PIMNEWVM\DTAExecutionHost, ExecutionTaskStateModel.RunStateTransitionsAsync : The task was cancelled
I, 3780, 9, 2016/04/14, 09:51:00.416, PIMNEWVM\DTAExecutionHost, Removing agent setting 

-------------------------------------------------------------------------------------------------

I, 3780, 9, 2016/04/14, 09:51:00.416, PIMNEWVM\DTAExecutionHost, Removing agent setting DtlMachineName on agent machine - 64 bit hive
I, 3780, 9, 2016/04/14, 09:51:00.416, PIMNEWVM\DTAExecutionHost, Removed agent setting DtlMachineName on agent machine - 64 bit hive
I, 3780, 9, 2016/04/14, 09:51:00.416, PIMNEWVM\DTAExecutionHost, ExecutionHost.UnregisterFromServer : Agent is unregistering from server.

Please help.

Can the test be executed successfully on your local machine? And if you run the testing manual from the target machine, can you see this issue? — Eddie Chen - MSFT
It is working in dev machine but not tried to run it in Target machine — Ravi Gupta

Patrick O'Bryan Patrick O'Bryan · Accepted Answer · 2016-06-17T22:28:54

We've run into a similar problem when trying to run tests direction from an MTM test plan using the "Run Functional Tests" build step but referencing test cases that had associated automation which linked to NUnit test cases.

It appears that the "Run Functional Tests" build step is somehow using vstest.console.exe on the back end to execute the tests on the Test Agent. vstest.console by default does not know how to run NUnit tests and if running from the command line, one would use /TestAdapterPath:. Since this isn't possible from the build, we need vstest.console to know by default how to execute NUnit tests.

To work around this problem, as part of building the test solution before deploying to the remote Test Agent machine, we've included the NuGet package for the NUntTestAdapter. This should download the following DLLs:

nunit.core.dll
nunit.core.interfaces.dll
nunit.util.dll
NUnit.VisualStudio.TestAdapter.dl

We then deploy these DLLs to the remote machine to the following location:

C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions

From that point on, you should be able to execute your tests from either vstest.console.exe directly on your test agent without having a reference to /TestAdapterPath or the build (since it's using vstest.console as part of "Run Functional Tests") will attempt to use that adapter to pickup and run NUnit tests.

Error while executing Run Functional Test task in VSTS

1 Answers