With TeamCity 8, how do I produce / find a results file for an NUnit run?
We currently also run MsTest which produces a TRX file. We then use a TRX->HTML report tool to pass a report up the management food chain. How do we do the same with NUnit in TeamCity?
Right now I'm thinking I need to execute NUnit as a CommandLine build step, but that seems crazy considering there's an NUnit add-in and the MsTest add-in offers me a "Results file:" option
TeamCity executes MSTest and NUnit differently.
NUnit is not run through the NUnit console executable but instead through TeamCity's own NUnit runner. This allows TeamCity to report NUnit test results on the fly--executing test 3...4...5...of 78--and allows instant notification of failed tests, even if all tests have not yet been executed.
MSTest, on the other hand, goes directly through the MSTest executables and does not have on-the-fly reporting. There is no progress other than "in progress". Test Results, including any failures, are only reported after every tests has been run.
TeamCity requires and parses the MSTest TRX file to do its own reporting, including on any failures, so it is also made available to you. However, the NUnit reporting files are a part of the NUnit console, and not a part of the TeamCity runner, so there is no report file to provide.
If you need the report file, you will need to run the NUnit tests through the NUnit console. There are several ways of doing this, only one of which is using a Command Line step. But be aware, you will lose the on-the-fly reporting, no matter which alternative you use.
Jay's description is correct; this is the TeamCity behaviour that makes this task impossible out of the box.
There is a known workaround though:
http://devnet.jetbrains.com/message/5218450#5218450
Essentially, you invoke the TeamCity NUnit runner manually (e.g. from MSBuild). The runner can then output a result.xml file (one per test assembly). Those result files then have to be merged back into one in order to simulate the behaviour of nunit-console.
Davy Brion has even posted the MSBuild tasks for this:
http://web.archive.org/web/20080809002009/http://davybrion.com/blog/stuff/
He has since nuked his blog, so waybackmachine to the rescue. In case those links die too, here are the snippets:
NUnitMergeOutput
This task combines the output of multiple NUnit xml reports into one combined xml report.
The combined report will contain the results of each xml report that was fed to it, and it contains the total number of tests, failures, duration and overall success status of the entire test run.To define the task:
<UsingTask AssemblyFile="$(MSBuildProjectDirectory)\Libs\Brion.MSBuildTasks\Brion.MSBuildTasks.dll" TaskName="NUnitMergeOutput"/>And to use it in a target:
<CreateItem Include="TestResults\*.xml" > <Output TaskParameter="Include" ItemName="NUnitOutputXmlFiles"/> </CreateItem> <NUnitMergeOutput NUnitOutputXmlFiles="@(NUnitOutputXmlFiles)" PathOfMergedXmlFile="TestResults\TestResults.xml" />BuildTeamCityNUnitArguments
TeamCity doesn’t easily allow you to enable its integrated NUnit testing support while still keeping the NUnit output xml files around after the build. This task prepares an xml arguments file to pass to TeamCity’s NUnitLauncher task which does make it possible to keep the NUnit output xml in a directory you can specify. You can find more info on this problem here and more info on this workaround here.
To define the task:
<UsingTask AssemblyFile="$(MSBuildProjectDirectory)\Libs\Brion.MSBuildTasks\Brion.MSBuildTasks.dll" TaskName="BuildTeamCityNUnitArguments"/>And to use it in a target:
<CreateItem Include="**\Bin\Debug\*Tests*.dll" > <Output TaskParameter="Include" ItemName="TestAssemblies" /> </CreateItem> <BuildTeamCityNUnitArguments HaltOnError="true" HaltOnFirstTestFailure="true" HaltOnFailureAtEnd="true" TestAssemblies="@(TestAssemblies)" NUnitResultsOutputFolder="TestResults" PathOfNUnitArgumentsXmlFile="nunitarguments.xml" /> <Exec Command="$(teamcity_dotnet_nunitlauncher) @@ nunitarguments.xml" />
