I have an Azure DevOps release pipeline setup to run my tests according to this setup:
My class file contains a unit test with 4 test cases in the following fashion:
[TestCase("test1", "arg1", "arg2")]
[TestCase("test2", "arg1", "arg2")]
[TestCase("test3", "arg1", "arg2")]
[TestCase("test4", "arg1", "arg2")]
public void Test(string name, string arg1, string arg2)
{
TestStuff();
}
When I run the tests a new release gets created and tests are run successfully however only 3 of the 4 tests are found. test2
is not found for some reason and will just get skipped. From the logs I can see that 146 tests were found in total which is accurate yet only 3 are "discovered"
2020-08-19T23:41:28.3146168Z Number of testcases discovered : 146
2020-08-19T23:41:28.3166614Z ##[debug]PERF: DiscoverTests:AddTestCases: took 0.005 ms
2020-08-19T23:41:28.3290518Z Discovered tests 146 from sources
2020-08-19T23:41:28.3291189Z
=================================================================
2020-08-19T23:41:28.3291959Z ##[debug]DiscoverTests.PerformDiscovery : Test Discovery completed.
2020-08-19T23:41:28.3295732Z ##[debug]PERF WARNING: DiscoverTests.PerformDiscovery: took 11963.8108 ms
2020-08-19T23:41:28.3607538Z [RunStatistics]This execution slice with id '3', received '4' testcases to execute out of which '3' is discovered.
What does it mean that only 3 were discovered
?
If I attempt to run that test by itself the whole thing fails with
Microsoft.VisualStudio.TestService.VstestAdapter.TestsNotFoundException: No test assemblies found on the test machine matching the source filter criteria or no tests discovered matching test filter criteria. Verify that test assemblies are present on the machine and test filter criteria is correct.
I've verified that the tests are properly associated to a test case via VS (2019) and the other 3 tests run fine it's just 1 test that doesn't get found for some reason.
assembly not found
only comes up if i try to run a single test with a test case that i know is not "discovered" so i don't think that's the issue. My real problem is that the assembly IS found but only 3/4 tests in a test suite are discovered even though they're all associated. I created a different test suite with 15 test cases and only 13/15 test cases were discovered in that run. – so cal cheesehead