Running Silverlight unit tests using StatLight and TFS 2010 Build

Question

I'm currently in the processing of designing a TFS 2010 build process that can run Silverlight tests, based on this article:

http://pyxis-tech.com/blog/2011/01/24/how-to-run-silverlight-automated-tests-on-tfs-build-server/

I've got StatLight generating the results file in MSTest generic format, which is great. However, I now need to publish the results in the build so that a failed test will break the build.

The article above mentions using a custom activity to parse the StatLight output and publish failures, but doesn't actually provide the code, so I have no idea what to do here.

Ideally I would publish the test results using the standard MSTest activity which is used to run our normal tests, as I would like to avoid custom activities if at all possible, however I don't know if this is possible.

Can anyone provide some assistance in this area? I can't find anything on the web other than the article above.

Answer 1

I also did the integration in TFS as described in the Pyxis' Blog (I see my ex-colleague is still keen in using TFS).

It works fine, except that I realised I got many "categories" of SLUnitTests : - some of them could be connected to TFS straight after, - others required me some extra work (security accounts, allow process to interract, ...). Then, I was facing the same issue as you: How to EASILY STOP a check-in on error (i.e. avoid writing extra bits of C# code, etc...), and avoid manual checks from the DropFolder ?

I found an alternative solution here, based on Generic Tests integrated to Visual Studio 2010 and TFS 2010. Hence easier to configure: http://www.nielshebling.de/?p=167

Another possibility is to use a CodePlex project, which provides you templates: http://statlightteambuild.codeplex.com

Hope this helps,

Vincent THAVONEKHAM.

Answer 2

There is a similar implementation in Codeplex Project "NUnit for TeamBuild".

This project is about taking NUnit results, changing them with XSLT into MSTest - like output & then publish them onto TFS 2010.

If you grab the code & open the solution, you might find your way to your goals by reviewing-redoing the actions within Publisher2010.cs.

Redoing those actions will lead to publishing your test results in the Summary view in each of your builds.

Failing your build due to failing unit test needs some extra handling, at first to retrieve it & then to degrade your build in case you detect the fail.

NUnit reports 'fail' if its' return code is anything else than '0', so I 've placed my first step in the 'Result' of the InvokeProcess that implements the NUnit call.
Directly after that, I have placed an 'if' checking this return. If it differs from '0' (so one or more tests failed), I use the 'SetBuildProperties' activity, so I can set 'TestStatus' to BuildPhaseStatus.Failed