Is there a tool to merge NUnit result files?

Question

For unclear reasons my Nunit test fixture cannot be executed in a single run, so I'm forced to execute a few tests in separate runs. However this means that the test results are splitted over multiple output files.

Is there a tool available which can merge NUnit result XML files into a single XML file?

I've tried using the existing Nunit-summary tool, but this simply sequentially parses the XML files with the given XSL file and concatenates the result as one big file.

Instead I would like it to merge/group the results for the test cases into the right namespaces/testfixtures first and then feed it to the XSLT processor. This way all test results should be displayed by fixture even though they're not gathered in a single run.

Answer 1

This is probably too late to help you, but we recently encountered a similar issue and wrote a small open source tool to help out: https://github.com/15below/NUnitMerger

From the readme:

Using in MSBuild

Load the task:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
             ToolsVersion="4.0"
             DefaultTargets="Build">
  <UsingTask AssemblyFile="$(MSBuildProjectDirectory)\..\Tools\MSBuild\15below.NUnitMerger.dll" TaskName="FifteenBelow.NUnitMerger.MSBuild.NUnitMergeTask" />
  ...

Feed it an array of files with in a target:

  <Target Name="UnitTest" DependsOnTargets="OtherThings">
    ... Generate the individual files here in $(TestResultsDir) ...

    <ItemGroup>
      <ResultsFiles Include="$(TestResultsDir)\*.xml" />
    </ItemGroup> 

    <NUnitMergeTask FilesToBeMerged="@(ResultsFiles)" OutputPath="$(MSBuildProjectDirectory)\TestResult.xml" />
  </Target>

Find the resulting combined results at OutputPath.

Using in F#

Create an F# console app and add 15below.NUnitMerger.dll, System.Xml and System.Xml.Linq as references.

open FifteenBelow.NUnitMerger.Core
open System.IO
open System.Xml.Linq

// All my files are in one directory
WriteMergedNunitResults (@"..\testdir", "*.xml", "myMergedResults.xml")

// I want files from all over the place
let myFiles = ... some filenames as a Seq

myFiles
|> Seq.map (fun fileName -> XDocument.Parse(File.ReadAllText(fileName)))
|> FoldDocs
|> CreateMerged
|> fun x -> File.WriteAllText("myOtherMergedResults.xml", x.ToString())

Answer 2

I was using the 15below NUnitMerger above for a while, but wanted to extend it and since my F# skills are not good enough to do it there, I inspected their mechanism and implemented the following class in C# to achieve the same thing. Here is my starting code which might help anyone who also want to do this kind of manipulation in C#:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Xml.Linq;
using System.Text;

namespace RunNUnitTests
{
    public static class NUnitMerger
    {
        public static bool MergeFiles(IEnumerable<string> files, string output)
        {
            XElement environment = null;
            XElement culture = null;
            var suites = new List<XElement>();

            bool finalSuccess = true;
            string finalResult = "";
            double totalTime = 0;
            int total = 0, errors = 0, failures = 0, notrun = 0, inconclusive = 0, ignored = 0, skipped = 0, invalid = 0;
            foreach (var file in files)
            {
                var doc = XDocument.Load(file);
                var tr = doc.Element("test-results");

                if (environment == null)
                    environment = tr.Element("environment");
                if (culture == null)
                    culture = tr.Element("culture-info");

                total += Convert.ToInt32(tr.Attribute("total").Value);
                errors += Convert.ToInt32(tr.Attribute("errors").Value);
                failures += Convert.ToInt32(tr.Attribute("failures").Value);
                notrun += Convert.ToInt32(tr.Attribute("not-run").Value);
                inconclusive += Convert.ToInt32(tr.Attribute("inconclusive").Value);
                ignored += Convert.ToInt32(tr.Attribute("ignored").Value);
                skipped += Convert.ToInt32(tr.Attribute("skipped").Value);
                invalid += Convert.ToInt32(tr.Attribute("invalid").Value);

                var ts = tr.Element("test-suite");
                string result = ts.Attribute("result").Value;

                if (!Convert.ToBoolean(ts.Attribute("success").Value))
                    finalSuccess = false;

                totalTime += Convert.ToDouble(ts.Attribute("time").Value);

                if (finalResult != "Failure" && (String.IsNullOrEmpty(finalResult) || result == "Failure" || finalResult == "Success"))
                    finalResult = result;

                suites.Add(ts);
            }

            if (String.IsNullOrEmpty(finalResult))
            {
                finalSuccess = false;
                finalResult = "Inconclusive";
            }

            var project = XElement.Parse(String.Format("<test-suite type=\"Test Project\" name=\"\" executed=\"True\" result=\"{0}\" success=\"{1}\" time=\"{2}\" asserts=\"0\" />", finalResult, finalSuccess ? "True" : "False", totalTime));
            var results = XElement.Parse("<results/>");
            results.Add(suites.ToArray());
            project.Add(results);

            var now = DateTime.Now;
            var trfinal = XElement.Parse(String.Format("<test-results name=\"Merged results\" total=\"{0}\" errors=\"{1}\" failures=\"{2}\" not-run=\"{3}\" inconclusive=\"{4}\" ignored=\"{5}\" skipped=\"{6}\" invalid=\"{7}\" date=\"{8}\" time=\"{9}\" />", total, errors, failures, notrun, inconclusive, ignored, skipped, invalid, now.ToString("yyyy-MM-dd"), now.ToString("HH:mm:ss")));
            trfinal.Add(new[] { environment, culture, project });
            trfinal.Save(output);

            return finalSuccess;
        }

    }
}

Answer 3

I read on the web that Nunit result files are XML so i guess you can merge the file with an ordinary merge software as WinMerge