C# - WebDriver - MSTest framework Parallel Execution

Question

I am currently working on C# - Webdriver - MSTest based testing framework. We are using testsettings file for enabling parallel execution. Also the tests are executed on a machine with 4 processors and we use Selenium Grid for executing the tests. Below are some of the challenges faced in over night execution.

• Since we have almost 100+ test cases, the time taken to complete on a single thread is around 3 hours. We wanted to reduce the execution time to nearly 15 minutes. But when we increase the thread count in testsettings file, there are lots of false failures. Is there a better way to achieve better pass rate by reducing execution time?
• I came across some blog "https://blogs.msdn.microsoft.com/devops/2016/02/08/parallel-and-context-sensitive-test-execution-with-visual-studio-2015-update-1/" which states that the number of parallel tests that can be executed is directly proportional to the number of CPU core of that machine. If this is right, with a 4 core machine, I wont be able to execute more than 4 thread at a time. Is there a way to over come this, if I use other Unit testing framework like xUnit, nUnit etc? I previously worked in Java - WebDriver using TestNG framework, and we could execute more than 15+ thread at a time.

Please help if any one is working on any C# based framework and successfully execute tests in more parallel threads.

Thanks in advance.

Answer 1

When I first did this, I encountered the same issues as you - I found that my tests ran fastest with 4 threads, though I was getting a lot of failures. The issue turned out to be that our system couldn't handle four login requests at the same time. I tried making the code thread-safe but this did not work either; in the end I ended up creating 4 accounts for our system, and using a static variable which was incremented each time a test logged in. This would cause the next test to use the next account.

If your tests require a login functionality then perhaps this could be your issue?

You can set the maximum number of threads by adding this line underneath your "using" lines at the top of your code (note: adding this line to one class is sufficient, you do not need to add it to every single class)

[assembly: Parallelize(Workers = 4, Scope = ExecutionScope.MethodLevel)]

If you change the 4 to a 0 then the maximum number of threads for your CPU will be used (this is 8 instances of chromedriver for me, though 4 was a lot faster!).