TeamCity run Nunit tests in Parallel

Question

So I was thinking that there must be a better way to run NUnit tests for a .net project via teamcity. Currently the build of the project takes about 10 minutes , and the testing step takes 30ish minutes.

I was thinking about splitting up the Nunit tests into 3 groups, assigning them each to a different agent. And then make sure they have a build dependency on the initial build before they start.

This was the best way i thought of doing it, Is there a different way I should also consider?
On a side note Is it possible to combine all the Nunit tests at the end to get one report from the tests being build on 3 different machines? I dont think this is possible unless someone thought of a clever hack.

Answer 1

NUnit users are now able to use the Parallelizable attribute at various locations to enable NUnit to run tests in parallel.

Example:

[Parallelizable(ParallelScope.All)]
[TestFixture]
public class LengthOfStayRestServiceAsyncTests
{
    [Test]
    public void Test1()
    {
        Assert.That(true, Is.True);
    }
}

Or better yet, stick this in your test project's Properties\AssemblyInfo.cs file:

[assembly: Parallelizable(ParallelScope.Fixtures)]

Source:
https://github.com/nunit/docs/wiki/Parallelizable-Attribute
https://templecoding.com/blog/2016/02/29/running-tests-in-parallel-with-nunit3

Answer 2

For parallel running Nunit test please look in to PnUnit at http://www.nunit.org/index.php?p=pnunit&r=2.5 and for reports you can configure to use Log4Net for Nunit see example here : http://www.softwarefrontier.com/2007/09/using-log4net-with-nunit.html

Answer 3

We set up a recursive MSBuild script to run the unit test dlls concurrently, which looks somelike this:

  <Target Name="UnitTestDll">
    <Message Text="Testing $(NUnitFile)" />
    <ItemGroup>
      <ThisDll Include="$(NUnitFile)"/>
    </ItemGroup>
    <NUnit ToolPath="$(NUnitFolder)" Assemblies="@(ThisDll)" OutputXmlFile="$(TestResultsDir)\%(ThisDll.FileName)-test-results.xml" ExcludeCategory="Integration,IntegrationTest,IntegrationsTest,IntegrationTests,IntegrationsTests,Integration Test,Integration Tests,Integrations Tests,Approval Tests" ContinueOnError="true" />
  </Target>

  <Target Name="UnitTest" DependsOnTargets="Clean;CompileAndPackage">
      <Message Text="Run all tests in Solution $(SolutionFileName)" />
      <CreateItem Include="$(SolutionFolder)**\bin\$(configuration)\**\*.Tests.dll" Exclude="$(SolutionFolder)\NuGet**;$(SolutionFolder)**\obj\**\*.Tests.dll;$(SolutionFolder)**\pnunit.tests.dll">
        <Output TaskParameter="Include" ItemName="NUnitFiles" />
      </CreateItem>
    <ItemGroup>
      <TempProjects Include="$(MSBuildProjectFile)">
        <Properties>NUnitFile=%(NUnitFiles.Identity)</Properties>
      </TempProjects>
    </ItemGroup>
    <RemoveDir Directories="$(TestResultsDir)" Condition = "Exists('$(TestResultsDir)')"/>
    <MakeDir Directories="$(TestResultsDir)"/>

    <MSBuild Projects="@(TempProjects)" BuildInParallel="true" Targets="UnitTestDll" />
  </Target>

You obviously still need your Compile targets (or in our case CompileAndPackage) to actually build the test dlls first.

This also messes up your NUnit results for local developers, but having hit that problem already, we wrote a tool to help with that: https://github.com/15below/NUnitMerger