VS2013 CodeCoverage.exe script file never parses

Question

Environment: Visual Studio 2013 Premium, Win7Ultimate, CodeCoverage.exe

Purpose: A code coverage report that excludes test project code that will later be converted to .coveragexml for reporting in SonarQube 5.1.

Annoyance I would not even know about this parsing error without adding the /verbose command to it. The only sign of failure was the .coverage file was no longer generated when I added the /config switch.

Working with files in the VS2013 IDE: The MyProject.runsettings file provides the expected output using Code Coverage Analysis in the IDE.

 Menu: Test | Test Settings | Select Test Settings File... MyProject.runsettings Menu: Test | Analyze Code Coverage | All Tests 

Trying to run the CodeCoverage.exe file to generate code coverage for my tests, I cannot use ANY *.runsettings files without errors:

 "Error: Failed to parse configuration file <configfile>.runsettings" 

Path Definitions:

 codeCoveragePath = C:\Program Files (x86)\Microsoft Visual Studio 12.0\Team Tools\Dynamic Code Coverage Tools vstestpath = C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow myProjectOutputPath = assume correct since I get results when not using /config switch 

Error getting start command (suppose paths are correct): Note. I do not show with the /verbose switch since I should not use it in the working environment

 %codeCoveragePath%\CodeCoverage.exe collect /config:MyProject.runsettings /output:CoverageOutput.coverage %vstestpath%\vstest.console.exe %myProjectOutputPath%\MyClass.Tests.Unit.dll 

Exe Works, if I DO NOT use the /config option . If I remove /config:MyProject.runsettings from the run command, I get a full report that includes a test project, but it lets me know that the rest of the command is correct, it just doesn't like the runsettings file.

I tried using the following examples:

Visual Studio 2013 script file without templates

MSDN file example

Completed empty file, no content: error

File with xml declaration only: error

File with only RunSettings Node declared: error

I even used troubleshooting tips from MSDN: no help.

MyProject.runsettings file:

 <?xml version="1.0" encoding="utf-8"?> <RunSettings> <DataCollectionRunSettings> <DataCollectors> <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"> <Configuration> <CodeCoverage> <ModulePaths> <Exclude> <ModulePath>.*\.Tests\.Unit\.dll$</ModulePath> </Exclude> </ModulePaths> </CodeCoverage> </Configuration> </DataCollector> </DataCollectors> </DataCollectionRunSettings> </RunSettings> 

The file seems correct, based on the fact that the IDE will use it and generate the correct output in the "Code coverage results" window, reporting only the MyClass code, not the MyClass.Tests.Unit code.

I am sure that the command line CodeCoverage.exe does not like the / config option or another xml scheme is used.

Update

The work gives the result I want, I just can’t specify the file location for the next step

 %vstestpath%\vstest.console.exe /Settings:MySettings.runsettings %myProjectOutputPath%\MyClass.Tests.Unit.dll 

Doesn't work. Gives the exact opposite result that I want (only test.dll coverage in the report).

 %codeCoveragePath%\CodeCoverage.exe collect /output:CoverageOutput.coverage %vstestpath%\vstest.console.exe /Settings:MySettings.runsettings %myProjectOutputPath%\MyClass.Tests.Unit.dll 

We are looking for an answer.

Answer 1

I believe that you need to specify the runsettings file at the end of vstest.console.exe using the /Settings: flag (unlike the CodeCoverage.exe configuration flag).

So your command will be:

% codeCoveragePath% \ CodeCoverage.exe collect /output:CoverageOutput.coverage% vstestpath% \ vstest.console.exe% MyProjectOutputPath% \ MyClass.Tests.Unit.dll /Settings:MyProject.runsettings

Answer 2

I had the same problem and I found your question while searching for some information about my error. My guess also was that the configuration file format was the same as the .runsettings used by vstest.console.exe, but from the parsing error after adding / verbose, I then suspected that it was a different format, so I looked, there is is there a default configuration for CodeCoverage.exe to see how it looks, and I found it on the page:

 C:\Program Files (x86)\Microsoft Visual Studio 12.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.config 

And the format looks just like the internal <CodeCoverage> part of the .runsettings format

I now have filtering to work, but I need to copy all the filter elements from the default configuration, since they no longer load, so I ended up with the following configuration:

 <CodeCoverage> <ModulePaths> <Exclude> <ModulePath>.*\\unittests.dll</ModulePath> </Exclude> </ModulePaths> <Sources> <Exclude> <!--extracted from default CodeCoverage.config --> <Source>.*\\atlmfc\\.*</Source> <Source>.*\\vctools\\.*</Source> <Source>.*\\public\\sdk\\.*</Source> <Source>.*\\externalapis\\.*</Source> <Source>.*\\microsoft sdks\\.*</Source> <Source>.*\\vc\\include\\.*</Source> </Exclude> </Sources> <Functions> <Exclude> <!--extracted from default CodeCoverage.config --> <Function>^std::.*</Function> <Function>^ATL::.*</Function> <Function>.*::__GetTestMethodInfo.*</Function> <Function>.*__CxxPureMSILEntry.*</Function> <Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function> <Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function> <Function>.*::YOU_CAN_ONLY_DESIGNATE_ONE_.*</Function> </Exclude> </Functions> </CodeCoverage> 

and command line:

 CodeCoverage collect /output:coverage.dat /config:coverage.settings vstest.console unitTests.dll /Logger:trx /Settings:test.runsettings 

Answer 3

Ok, here comes HACK !!!!

The main steps:

• Find and delete all * .coverage files
• Run the vstest command WITHOUT [codecoverage.exe collect] wrapper
• Find the new * .coverage files and send the command [codecoverage.exe analysis]

More details

I updated the build.proj file that I used to complete all this in order to complete the basic steps:

 <PropertyGroup> <SqCodeCoverageResultsFile>VisualStudio.coveragexml</SqCodeCoverageResultsFile> </PropertyGroup> <Target Name="BuildTestAssemblyList" BeforeTargets="RunAllTestsWithCodeCoverageAndConvertToXmlOutput"> <CreateItem Include="**\*.Tests.Unit.dll"> <Output TaskParameter="Include" ItemName="TestAssemblies" /> </CreateItem> </Target> <Target Name="BuildCoverageFileList" BeforeTargets="RunAllTestsWithCodeCoverageAndConvertToXmlOutput"> <CreateItem Include="**\*.coverage"> <Output TaskParameter="Include" ItemName="CoverageFiles" /> </CreateItem> </Target> <Target Name="RunAllTestsWithCodeCoverageAndConvertToXmlOutput"> <Delete Condition="Exists($(SqCodeCoverageResultsFile))" Files="$(SqCodeCoverageResultsFile)" /> <Delete Files="@(CoverageFiles)" /> <Exec Command="&quot;$(VsTestExecutable)&quot; /EnableCodeCoverage /Settings:MyProject.runsettings /inIsolation /logger:trx @(TestAssemblies->'&quot;%(FullPath)&quot;',' ') " /> <CreateItem Include="**\*.coverage"> <Output TaskParameter="Include" ItemName="NewCoverageFiles" /> </CreateItem> <Exec Command="&quot;$(VsCodeCoverageExecutable)&quot; analyze /output:&quot;$(SqCodeCoverageResultsFile)&quot; @(NewCoverageFiles->'&quot;%(FullPath)&quot;',' ') " /> </Target>` 

Now the execution of the analysis command CodeCoverage.exe with the found * .coverage files will now be output to the same file name that I tried to achieve earlier and get the results that I wanted.

MSBuild.SonarQube.Runner.exe gets what it wants, I have the results I want, and the world can start spinning again =)

Improvement:. I could use CustomTask and look for the perfect or the latest or any logic that you can come up with for this correct single file, so I would not have to delete all my others * .. I could, but I did not, because it should be running on the build server, which, in my opinion, should not have this kind of history.