Parallel option for vstest.console.exe does not work as expected

Question

I am expecting vstest.console.exe will run all test methods in parallel with /Parallel option specified. On a 4 core machine, I am expecting below test class will cost around 2~3 seconds for execution, by actually I got 8~9 seconds which means the tests are executed sequentially.

[TestClass]
public class UnitTest1
{
    [TestMethod]
    public async Task TestMethod1()
    {
        await Task.Delay(2000);
    }

    [TestMethod]
    public async Task TestMethod2()
    {
        await Task.Delay(2000);
    }

    [TestMethod]
    public async Task TestMethod3()
    {
        await Task.Delay(2000);
    }

    [TestMethod]
    public async Task TestMethod4()
    {
        await Task.Delay(2000);
    }
}

The test ouput:

Microsoft (R) Test Execution Command Line Tool Version 15.5.0 Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait... Passed TestMethod1 Passed
TestMethod2 Passed TestMethod3 Passed TestMethod4

Total tests: 4. Passed: 4. Failed: 0. Skipped: 0. Test Run Successful.

Test execution time: 8.6960 Seconds

Do you have a settings file that you're using? The complete command line (+ settings, if applicable) would be useful to make sure people can reproduce what you're seeing. — Damien_The_Unbeliever
vstest.console.exe /Parallel MyUnitTest.dll produces the above result. Noticed I got same execution time with/without /Parallel option, looks it does not have any effect. — Shuping
Not played with this much myself. But it appears when this feature was introduced that it could parallelize only across test containers - i.e. if you have multiple assemblies it will run the tests in the separate assemblies in parallel. Can't see anything that suggests that anything has changed since then. — Damien_The_Unbeliever

shrot shrot · Accepted Answer · 2018-02-09T11:09:20

You will be needing a setting file inorder to tell how many core you want to use for the execution.

command will be something like this:

vstest.console.exe /Parallel MyUnitTest.dll /Settings:C:\Settings.testsettings

And the Settings.testsettings should be something like this:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<MaxCpuCount>0</MaxCpuCount>
</RunConfiguration>
</RunSettings>

The value for MaxCpuCount has the following semantics:

‘n’ (where 1 <= n <= number of cores): upto ‘n’ processes will be launched. ‘n’ of any other value: The number of processes launched will be as many as the available cores on the machine. Typically, a value of 0 indicates that up to all of the available free cores may be used.

Parallel option for vstest.console.exe does not work as expected

1 Answers