Code coverage with Visual Studio 2012 vs SONAR (dotcover and gallio)

0
votes

Today I am running into some discrepancies between Visual Studio 2012 and SONAR code coverage analysis.

When I execute Visual Studio 2012 code coverage analysis Test->Analyze Code Coverage->All Test it throws the below values

  • Not Covered 37%
  • Covered 63%

But when I execute this analysis through SONAR (gallio and dotcover) it throws the below values

  • Unit Test Coverage 55%

Very similar result I get from dotcover in visual studio.

  • Code coverage 53%

I am not adding any special flags in the sonar-runner properties file. Or setting something in dotcover.

1
c#visual-studio-2012code-coveragesonarqubedotcover
What is your question? - Kirk Woll
Well, for one thing, Visual Studio's coverage counts the closing brace of a unit test with the ExpectedExceptionAttribute as uncovered. I expect all code coverage tools do odd things like that somewhere, so I wouldn't worry too much about the numbers lining up. I prefer the ContinuousTests style - method coverage with risk calculations. - Magus
Kirk, my question is Why Visual Studio Analysis and SONAR are very different? Does VS do analysis in different way? It seems it does, according to the answer that I got. - JAVH
@JAVH: I don't think you can get a very good answer to this question unless someone who was on the dev team for both coverage systems exists and feels like shedding light on the subject. Ultimately, it isn't going to make much difference to software you write. - Magus

1 Answers

1
votes

Code coverage is a tool which allows you to find lines of code that your tests cannot execute. It is a fickle thing, and reaching 100% for it generally involves jumping through many hoops for little gain. Additionally, every code coverage tool takes different things into account.

My favorite test runner at the moment is ContinuousTests, which takes the strategy of checking how many times a method is called by tests and by other code, building a graph to determine the risk involved when the code is modified. This is a good metric to look at when testing something.

You can use code coverage as well, as that can help you determine how code branches and whether or not edge cases are tested, always being careful that your tests deal with the contract of a class rather than the internals.

So ultimately, yes, you're seeing different code coverage results from different tools. That's not really something to worry about.