Getting DUnit Test Coverage stats using FinalBuilder and AQTime

Question

We have a large Delphi project (1.5 million lines of code), and we're moving to using agile processes.

We already have a continous integration environment (FinalBuilder) which I've updated to include unit tests (dUnit) and code metrics (CodeHealer) in the e-mails to everyone in our development team. Our unit test coverage isn't great, so I'm now trying to get AQtime into the mix for some test coverage results on every build.

I'm using the "Execute Program" task to run the unit test executable, log the results and parse the file afterwards. I intend to use the "Run Script" task to run AQtime (via COM) and export the results to XML so I can parse through those results.

The issue I have is with AQtime running the unit test executable, I lose the ability to monitor the unit test executable directly. I'd like to get FinalBuilder to parse the results of both tasks. Does anybody know how to get access to the dUnit results when it's called from AQtime?

Answer 1

We ran down this same path, and we don't run DUnit test from AQTime,

Instead we build and run our Dunit Tests using FinalBuilder.

Our unit tests use XmlTestRunner, then we can know if the test fails or not very easily using an XPath Query on the resulting XML File.

Update: An recent update to FinalBuilder 7.0 now supports DUnit. It's through its own XMLTestrunner.pas, which is under the FinalBuilder Directory after install. This runner outputs the test results in the same format as NUnit. It then integrates really well into FinalBuilder Server.

Answer 2

You are kind of describing the setup we are slowly evolving to.

• DUnit tests are compiled as console applications using the TTextTestListener defined in TextTestRunner unit.
• CI server is a cmd script that builds all projects and executes all tests.
• The output of the tests are piped to a file.

A solution might be to have AQTime profile these console applications while still be able to pipe the results to a file that can be parsed afterwards?!

Another solution might be to implement your own TestListener object and have that object write the testresults to the eventlog, directly to a logfile, a database or wherever you like and have this picked up by FinalBuilder.

Instead of having something like this in your project file

  Application.Initialize;
  if System.IsConsole then TextTestRunner.RunRegisteredTests
  else GUITestRunner.RunRegisteredTests;

it would become something like this

  Application.Initialize;
  if System.IsConsole then OurEventLogTestRunner.RunRegisteredTests
  else GUITestRunner.RunRegisteredTests;