CruiseControl.Net and NUnit - tests finish, but task doesn't?

Question

I've got a CruiseControl.Net setup using Nant to clean the previous logs, and then it kicks off a msbuild of a VS project, finally running nunit-console to execute the tests.

It seems to build for a few seconds (fine) and then hops on to running the 600 tests, which takes about a minute. However even though the log files are there, it sits there doing 'nothing' for 10 minutes, at which point the built times out and the process exits. The CruiseControl.NET webpage then shows the result as failed, with an exception:

ThoughtWorks.CruiseControl.Core.Tasks.BuilderException: Command Line Build timed out (after 600 seconds)
   at ThoughtWorks.CruiseControl.Core.Tasks.ExecutableTask.Execute(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.Tasks.TaskBase.Run(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.Project.RunTask(ITask task, IIntegrationResult result, Boolean isPublisher)
   at ThoughtWorks.CruiseControl.Core.Project.RunTasks(IIntegrationResult result, IList tasksToRun, Dictionary`2 parameterValues)
   at ThoughtWorks.CruiseControl.Core.Project.Run(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.IntegrationRunner.Build(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.IntegrationRunner.Integrate(IntegrationRequest request) BaseDirectory: , Executable: C:\Program Files\NUnit 2.5.8\bin\net-2.0\nunit-console.exe

The ccnet.config script is below. I've tried changing the timeout to 3 minutes just in case that was something to do with it, but even if that did work (it didn't) it's a dodgy hack, as by rights when the tests are finished running, they should exit gracefully!

I've run the command at the commandline and confirmed it only takes about a minute to run. Any theories?

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">

    <project name="CodeTests">
      <workingDirectory>C:\Source\Wholesale\Comp.EventControl.TestingFramework\</workingDirectory>
      <artifactDirectory>C:\Source\Wholesale\Comp.EventControl.TestingFramework\</artifactDirectory>

    <prebuild>
    <!-- clean nunit output to avoid CCNET reporting
           about previous build tests if current build fails -->
      <nant>
        <executable>C:\Nant\bin\nant.exe
        </executable>
        <baseDirectory>C:\Source\Wholesale\Comp.EventControl.TestingFramework</baseDirectory>
        <nologo>false</nologo>
        <buildFile>nant.build</buildFile>
        <targetList>
          <target>cleanNunit</target>
        </targetList>
      </nant>
    </prebuild>

    <tasks>
        <msbuild>
            <executable>C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe
            </executable>
            <workingDirectory>C:\Source\Wholesale\Comp.EventControl.TestingFramework\CodeReboot
            </workingDirectory>
        <projectFile>CodeReboot.sln</projectFile >
            <buildArgs>/noconsolelogger 
            /v:quiet 
            /noconlog 
            /p:Configuration=Debug
            /p:ReferencePath="C:\Program Files\NUnit 2.5.8\bin;C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"
            /p:AdditionalReferencePath="C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0"
            </buildArgs>
            <targets>ReBuild</targets >
            <timeout>180</timeout >
            <logger>c:\Program Files\CruiseControl.NET\server\Rodemeyer.MsBuildToCCNet.dll</logger>

        </msbuild>
        <exec>
            <executable>C:\Program Files\NUnit 2.5.8\bin\net-2.0\nunit-console.exe
            </executable >
            <buildArgs>/xml:C:\Source\Wholesale\Comp.EventControl.TestingFramework\nunit-results.xml
            /nologo C:\Source\Wholesale\Comp.EventControl.TestingFramework\CodeReboot\CodeReboot\bin\Debug\CodeReboot.dll
            </buildArgs>
        </exec>
    </tasks>
    <publishers>
      <merge>
        <files>
          <file>C:\Source\Wholesale\Comp.EventControl.TestingFramework\nunit-results.xml
          </file>
        </files>
      </merge>
      <xmllogger />
      <statistics />
      <artifactcleanup cleanUpMethod="KeepLastXBuilds"
        cleanUpValue="20" />
    </publishers>

    </project>
</cruisecontrol>

Answer 1

What version of nunit are you using? There were some problems in the 2.5.7/2.5.8 version that causes nunit-agent to hang at the end of a test. I had this problem and reverted back to an older version of nunit and the hang problem went away. In the release notes for 2.5.9 they show that "602761 nunit-agent hangs after tests complete" has been fixed. I have not upgraded to 2.5.9 yet but that may fix your problem.

Answer 2

The first thing you should do try and replicate the problem locally. Do your tests run properly and exit cleanly? Run them locally and see if the nunit process hangs around.

If your tests run fine locally, try and figure out which test the build server is stalling on or whether it is completing all of the tests. If your build scripts cannot be run locally, it's a bit harder to diagnose.

Back in the day when our tests were riddled with threaded code, it was nearly always the case that a test (or the code under test) was creating threads and then failing to shut them down, causing NUnit to hang around. Threading issues are irritating because they're non-deterministic may only be visible when run on machines with more cores (like .. build servers).

If you have tests with threading / external dependencies, I'd try disabling those first (the fact that 600 tests takes a minute to run indicates that external dependencies and/or threading is involved, as a unit test usually takes ~1ms to run).