TFS Build 2010 Code Coverage Using NUnit

I was wondering if any of you have experience reporting code coverage in TFS Build Server 2010 while running NUnit tests.

I know that this can be easily done with a packaged alternative (MSTest + inclusion of coverage in the testrunconfig file), but when using NUnit it is a bit more. I found some information here and there, pointing to NCover, but it seems to be out of date. I wonder if there are other alternatives and if someone really implemented this or not.

Here is more information about our environment / needs: - TFS Build Server 2010 - Tests are in simple class libraries (and not in test libraries, i.e. are not associated with testrunconfig files) and are implemented in NUnit. We do not have MSTests. “We are interested in launching coverage reports as part of each assembly and, if possible, setting threshold coverage requirements for pass / fail criteria.”

+4
source share
1 answer

We did this with NUnit-NCover and are very pleased with our results. NUnit execution is followed by NUnitTfs to get the test results published in the build log. NCover then launches, generating code coverage results.

One of the important things, which is a drawback, is that setting arguments for a proper NCover call was not trivial. But since I installed it, I never had to support it.

Two things can be disadvantages:

  • NUnitTfs does not work with NCover (at least I could not find a way to perform both actions at the same stage, therefore (since NCover calls NUnit), I have to run Unit tests twice: (1) to get the test results and ( 2) get NCover coverage results Naturally, this makes my builds longer.
  • Setting arguments for the correct call to NCover was not trivial. But since I installed it, I never had to support it.

In any case, the resulting reporting (especially the Trend aspect) is very useful for monitoring how our code develops over time. Especially if you work on the platform (unlike projects with short deadlines), Trend reports are of great importance.

EDIT
I will try to present briefly and dirtyly how I implemented it, I hope this can be useful. Our build server currently has NCover 3.4.12.
Our simple naming convention for our NUnit builds is that if we have a production assembly of "123.dll", then there is another assembly called "123_nunit.dll" that implements its tests. Thus, each assembly contains several * _nunit.dll collections that are of interest.

The part of the build process template in the “If you don’t disable the tests” section is the one that was redesigned to achieve our goals, in particular the section that was called “Starting MSTest for test builds”. The entire implementation is here , after some refinements, to facilitate understanding of the stream (the pic was too large to be directly inserted here).

First, some additional arguments are implemented in the assembly process template and then available for each assembly definition: enter image description here

Then we form the NUnit arguments in "Formulate nunitCommandLine":

String.Format("{0} /xml={1}\\{2}.xml", nunitDLL, TestResultsDirectory, Path.GetFileNameWithoutExtension(nunitDLL)) 

Then it is used in "Invoke NUnit"
enter image description here

In case this succeeds and we have installed the coverage for this assembly, we go to "Generate NCover NCCOV" (the coverage file for this particular assembly). To do this, we call NCover.Console.exe with the following as Args:

 String.Format("""{0}"" ""{1}"" //w ""{2}"" //x ""{3}\{4}"" //literal //ias {5} //onlywithsource //p ""{6}""", NUnitPath, Path.GetFileName(nunitDLL), Path.GetDirectoryName(nunitDLL), Path.GetDirectoryName(Path.GetDirectoryName(nunitDLL)), Path.GetFileName(nunitDLL).Replace("_nunit.dll", ".nccov"), Path.GetFileNameWithoutExtension(nunitDLL).Replace("_nunit", ""), BuildDetail.BuildNumber) 

They all start in the foreach loop "For all dll nunit". When we exit the loop, we enter “Final NCover Activities” and first the “Merge NCCovs” part, where NCover.Console.exe is executed again - this time with different arguments:

 String.Format("""{0}\*.nccov"" //s ""{0}\{1}.nccov"" //at ""{2}\{3}\{3}.trend"" //p {1} ", Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))), BuildDetail.BuildNumber, NCoverDropLocation, BuildDetail.BuildDefinition.TeamProject ) 

When this started, we reached the point where all the NCCOV files of this assembly were merged into one NCCOV file, named after the assembly + the Trend file (which controls the assembly throughout its life), was updated by the elements of this current assembly.

Now we only need to generate the final HTML report, this is done in the "Generate final NCover rep", where we call NCover.reporting with the following arguments:

 String.Format(" ""{0}\{1}.nccov"" //or FullCoverageReport //op ""{2}\{1}_NCoverReport.html"" //p ""{1}"" //at ""{3}\{4}\{4}_{5}.trend"" ", Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))), BuildDetail.BuildNumber, PathForNCoverResults, NCoverDropLocation, BuildDetail.BuildDefinition.TeamProject, BuildType ) 
+4
source

All Articles