Develop Reference

code coverage in gitlab ci/cd for .net applications?

I recently started working with C# and I am working on one of the legacy system we have. I am trying to figure out what is the code coverage for this legacy system. Here is my Sample.UnitTests.csproj file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="AutoFixture.AutoMoq" Version="4.2.1" />
<PackageReference Include="AutoFixture.NUnit3" Version="4.2.1" />
<PackageReference Include="coverlet.msbuild" Version="2.9.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Moq" Version="4.8.2" />
<PackageReference Include="nunit" Version="3.9.0" />
<PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
<PackageReference Include="WireMock.Net" Version="1.0.4.17" />
<PackageReference Include="Utf8Json" Version="1.3.7" />
<FrameworkReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="../Sample/Sample.csproj" />
</ItemGroup>
<ItemGroup>
<DotNetCliToolReference Include="dotnet-reportgenerator-cli" Version="4.2.10" />
</ItemGroup>
</Project>
I did some research and found out we can use coverlet which can generate cobertura style report. I followed exactly as mentioned here on my mac box and everything works fine and I can see the report being generated correctly on my console and also it generates index.html file which we can use to visualize as well.
dotnet add package coverlet.msbuild
dotnet restore
dotnet build
dotnet test /p:CollectCoverage=true
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:Exclude="[xunit*]\*" /p:CoverletOutput="./TestResults/"
dotnet reportgenerator "-reports:TestResults/coverage.cobertura.xml" "-targetdir:TestResults/html" -reporttypes:HTML;
Now since we use gitlab ci/cd pipeline for our project - Is there any way I can make this part of my .gitlab-ci.yml file so that it can generate report automatically for me whenever build happens and everybody in my team can see it successfully. Since as of now it's all manual as I need to run those above commands on my local mac box and I can see it from my console only by clicking index.html file.
These are my stages of .gitlab-ci.yml file as shown below. If needed I can provide my yml file as well but any simple example where it can demonstrate how can I do this then it will be of great help. I tried searching a lot and couldn't find this at all on how can I do it through gitlab pipeline which uses coverlet and cobertura style report for .net applications..
stages:
- test
- publish
- increment
- deploy
- integrationTests
- release
Can this be done through webhook as well if needed?

May need to use the actual GUID instead of *.
stages:
- test
- publish
- increment
- deploy
- integrationTests
- release
build-and-test:
stage: test
image: mcr.microsoft.com/dotnet/sdk:6.0
script:
- dotnet add package coverlet.msbuild
- dotnet restore
- dotnet build
- 'dotnet test --collect:"XPlat Code Coverage"'
artifacts:
reports:
cobertura: TestResults/*/coverage.cobertura.xml
GitLab can diff with previous Cobertura reports.
If you want HTML instead, simply include it among the artifacts. May also publish it to GitLab Pages.

The other answer didn't work for me because of folder confusion. Coverlet puts the test results in folders relative to the respective unit test projects. So
Either you have to tell gitlab to search for TestResults folders everywhere via wildcards, by setting path to something like ./**/TestResults/**/coverage.cobertura.xml.
Or you provide the --results-directory option (short version -r), to tell dotnet test where to put those files in the first place.
I went for the second option, gathering all results in a cobertura folder in the repo root. Here is a full, valid .gitlab-ci.yml for running tests for merge requests:
image : mcr.microsoft.com/dotnet/sdk:6.0
stages:
- test
test:
stage: test
only:
- merge_requests
script:
- 'dotnet test DotNetSolution
--collect:"XPlat Code Coverage"
-r cobertura'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: cobertura/*/coverage.cobertura.xml

Running java.exe as pre-build step in .NET Core/Standard fails with 9009

I am trying to run java.exe as a pre-build step in a C#-.NET Standard 2.0 project. The csproj file contains the following snippet:
<Target Name="Java" BeforeTargets="Build">
<Exec Command="c:\Windows\System32\java.exe"/>
</Target>
(I simplified the command line for testing.) The file java.exe exists in c:\windows\system32, but the build fails with error code 9009:
c:\Windows\System32\java.exe' is not recognized as an internal or external command,
1>operable program or batch file.
1>C:\workspace\Test.csproj(21,5): error MSB3073: The command "c:\Windows\System32\java.exe" exited with code 9009.
1>Done building project "Test.csproj" -- FAILED.
Running java.exe directly from the command line works fine.

Apparently, c:\windows\system32 is routed to another path because Visual Studio is a 32 bit application. Using the special alias "Sysnative" works:
<Target Name="Java" BeforeTargets="Build">
<Exec Command="c:\Windows\Sysnative\java.exe"/>
</Target>
More information here.

Running Nunit on console through restoring NuGet Package

I am currently working on a project in C#, and trying to get NUnit to run on my project in a Jenkins Build through restoring the package in NuGet. We do not want to install NUnit on all of our Jenkins machines so installing it directly on the box is not an option.
We've investigated the use of NUniter Runners as suggested in this link: https://peteris.rocks/blog/running-nunit-with-msbuild-on-windows-and-mono/
But, we receive an error that the tools do not exist
"C:\Users\Name\NewBranch\project\project.Tests\project.Tests.csproj(121,3): error MSB6003
: The specified task executable "cmd.exe" could not be run. The working directory "C:\Users\Name\NewBranch\project\
packages\NUnit.Runners.3.4.1\tools" does not exist."
How do I execute NUnit tests from the command line without having NUnit installed directly on the machine?

the easiest way is to dump there 2 lines in the packages.config which should be in the same folder as your project.Tests.csproj
<package id="NUnit" version="2.6.4" targetFramework="net45" />
<package id="NUnit.Runners" version="2.6.4" />
Assuming you have a target in your .csproj which is like this:
<Target Name="NugetRestore">
<Exec Command="nuget.exe restore" />
</Target>
if you use a solution level build.config, then add this instead
<Target Name="NugetRestore">
<Exec Command="nuget.exe restore $(YourSolution).sln" />
</Target>

MSBuild command line with C# 6.0 code fails to build

I am having an issue getting MSBuild to compile c# 6.0 code, it reports a failure when trying to build expression body syntax, etx.
So I've been creating build project like this for years. I have a powershell script that kicks of an msbuild with some parameters:
msbuild.exe _build\build.proj
/p:Build_Number=1.2.0
/p:Configuration=QA
/p:SolutionName=MPGCS-Api.sln
/ToolsVersion:14.0
I verified that msbuild.exe is coming from C:\Program Files (x86)\MSBuild\14.0\Bin , i did this by going into that folder and specifying the full physical path to the .proj file. I get the same exact errors.
I have been looking for an example on how to setup a MSBuild project file with C# 6.0 with no luck, so this is my basic setup (this is the build.proj file). If I had to guess I am not importing the correct targets, etc. But I am a little lost. Here is my proj file:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<PropertyGroup>
<Root>$(MSBuildStartupDirectory)</Root>
<NugetExe>$(Root)\_build\lib\nuget\nuget.exe</NugetExe>
<Build_Number>0.0.0</Build_Number>
<SolutionName></SolutionName>
<TargetFrameworkVersion>v4.5.2</TargetFrameworkVersion>
<VisualStudioVersion>14.0</VisualStudioVersion>
</PropertyGroup>
<Target Name="Clean">
<!-- Clean up -->
<ItemGroup>
<FilesToDelete Include="$(Root)\_build\Artifacts\**\*.*" />
<FilesToDelete Include="$(Root)\_build\Published\**\*.*" />
</ItemGroup>
<Delete Files="#(FilesToDelete)" ContinueOnError="false" />
<!-- Ensure directories exists -->
<MakeDir Directories="$(MSBuildProjectDirectory)\Artifacts" Condition="!Exists('$(MSBuildProjectDirectory)\Artifacts')" />
<MakeDir Directories="$(MSBuildProjectDirectory)\Published" Condition="!Exists('$(MSBuildProjectDirectory)\Artifacts')" />
</Target>
<Target Name="Debug" AfterTargets="Clean">
<!-- Diagnostics -->
<Message Text="Diagnostics:"/>
<Message Text="Build Number: $(build_number)" />
<Message Text="Configuration: $(Configuration)" />
<Message Text="VisualStudioVersion: $(VisualStudioVersion)" />
<Message Text="Project root: $(Root)" />
<!-- Restore Nuget Packages -->
<Message Text="Restoring nuget..."/>
<Exec Command="$(NugetExe) restore $(Root)\$(SolutionName)" />
</Target>
<Target Name="GenerateOctopackAPI" AfterTargets="Debug">
<ItemGroup>
<ProjectToBuild Include="$(Root)\MPG.CS.Api\MPG.CS.Api.csproj" />
</ItemGroup>
<MSBuild Projects="#(ProjectToBuild)" ContinueOnError="false" Targets="Rebuild" Properties="
RunOctoPack=true;
Configuration=$(ProjectBuildMode);
Platform=AnyCpu;
TargetFrameworkVersion=$(TargetFrameworkVersion);
VisualStudioVersion=$(VisualStudioVersion);
OctoPackPublishPackageToFileShare=$(Root)\_build\Artifacts;
OctoPackPackageVersion=$(Build_Number);
OctoPackProjectName=UI;
OutputPath=bin\$(ProjectBuildMode)" />
</Target>
</Project>
Here is some info from my "Debug" target, as you can see I am setting the VisualStudioVersion to 14.0 as per some suggestions I read online.
Diagnostics:
Build Number: 1.2.0
Configuration: QA
VisualStudioVersion: 14.0
Project root: C:\dev\mpg\MPGCS-Api
Here is an example error, it's basically failing on c# 6.0 code, if I were to remove c# 6 code, everything will compile:
TicketType.cs(19,28): error CS1002: ; expected [C:\dev\mpg\MPGCS-Api\MPG.CS.Model\MPG.CS.Model.csproj]
TicketType.cs(19,44): error CS1519: Invalid token '(' in class, struct, or interface member declaration [C:\dev\mpg\MPGCS-Api\MPG.CS.Model\MPG.CS.Model.cs proj]
TicketStatus.cs(16,36): error CS1002: ; expected [C:\dev\mpg\MPGCS-Api\MPG.CS.Model\MPG.CS.Model.csproj]
TicketStatus.cs(16,53): error CS1519: Invalid token '(' in class, struct, or interface member declaration [C:\dev\mpg\MPGCS-Api\MPG.CS.Model\MPG.CS.Model. csproj]
TicketStatus.cs(16,82): error CS1519: Invalid token '(' in class, struct, or interface member declaration [C:\dev\mpg\MPGCS-Api\MPG.CS.Model\MPG.CS.Model. csproj]
Here is a line it's failing on (C# 6 code):
public bool IsAbbreviated => (Title.ToLower() == "open" || Title.ToLower() == "closed");
I'm at a loss, I thought just using the proper msbuild.exe will allow me to take advantage of c# 6.0. Any help would be greatly appreciated.

If you're running the powershell script on a build server, separate from your dev environment, you will need to make sure that the machine itself can handle C# 6.0. Since it uses a different compiler all together, it wouldn't work if VS 2015 wasn't installed. Alternately, you can add the Microsoft.Net.Compilers nuget package as a dependency to allow VS 2012 and VS 2013 to compile it.

VS2013 CodeCoverage.exe runsettings file never parses

Environment: Visual Studio 2013 Premium, Win7Ultimate, CodeCoverage.exe
Goal: Code Coverage Report that excludes test project code to later be converted to a .coveragexml for reporting to SonarQube 5.1.
Annoyance I wouldn't even know of this parse error without adding the /verbose switch to the command. My only indication of a failure was the .coverage file was no longer being generated when I added the /config switch.
File Works in VS2013 IDE: MyProject.runsettings file provides the expected output using "Analyze Code Coverage" in the IDE.
Menu: Test | Test Settings | Select Test Settings File... MyProject.runsettings
Menu: Test | Analyze Code Coverage | All Tests
Attempting to run the CodeCoverage.exe file to generate code coverage for my tests I can't seem to use ANY *.runsettings files without getting an error:
"Error: Failed to parse configuration file <configfile>.runsettings"
Path Definitions:
codeCoveragePath = C:\Program Files (x86)\Microsoft Visual Studio 12.0\Team Tools\Dynamic Code Coverage Tools
vstestpath = C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow
myProjectOutputPath = assume correct since I get results when not using /config switch
Run Command receiving Error (assume paths are correct):
Note: I'm not showing with /verbose switch since I shouldn't be using it under working conditions
%codeCoveragePath%\CodeCoverage.exe collect /config:MyProject.runsettings /output:CoverageOutput.coverage %vstestpath%\vstest.console.exe %myProjectOutputPath%\MyClass.Tests.Unit.dll
Exe Works if I DON'T use the /config option If I remove the /config:MyProject.runsettings from the run command, I get a full report that includes the test project, but that let's me know the rest of the command is correct, it just doesn't like the runsettings file.
I've tried using the following examples:
Visual Studio 2013 runsettings Template file WITHOUT modification
MSDN's sample file
Completed blank file, no content: error
File with only the xml declaration: error
File with only RunSettings Node declared: error
I've even used the Troubleshooting tips from MSDN, too: no help.
MyProject.runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*\.Tests\.Unit\.dll$</ModulePath>
</Exclude>
</ModulePaths>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
The file seems to be correct based on the fact that the IDE will use it and generate the correct output in the "Code Coverage Results" window by only reporting the MyClass code and not any MyClass.Tests.Unit code.
I'm at the point that I think it is the CodeCoverage.exe command line doesn't like the /config option or it is using a different xml schema.
Update
Works gives the output I want, just can't specify file location for next step
%vstestpath%\vstest.console.exe /Settings:MySettings.runsettings %myProjectOutputPath%\MyClass.Tests.Unit.dll
Doesn't Work Gives exact opposite output I want (only test.dll coverage in the report).
%codeCoveragePath%\CodeCoverage.exe collect /output:CoverageOutput.coverage %vstestpath%\vstest.console.exe /Settings:MySettings.runsettings %myProjectOutputPath%\MyClass.Tests.Unit.dll
Still looking for an answer.

I believe you need to specify the runsettings file at the back of the vstest.console.exe using the /Settings: flag (as opposed to the config flag of the CodeCoverage.exe).
So that your command becomes:
%codeCoveragePath%\CodeCoverage.exe collect
/output:CoverageOutput.coverage %vstestpath%\vstest.console.exe
%myProjectOutputPath%\MyClass.Tests.Unit.dll
/Settings:MyProject.runsettings

I was having the same problem and found your question when searching for some info on my error. My assumption had been also that the config file format was the same as the .runsettings used by vstest.console.exe but from the parse error after adding /verbose I then suspected it was a different format so had a look to see if there was a default config for CodeCoverage.exe to see what it looked like, and I did find one at:
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.config
And the format appears to just be the inner <CodeCoverage> part of the .runsettings format
I have now got filtering to work but needed to copy over all the filter items from the default config as they now don't get loaded, so I ended up using a config like the following:
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*\\unittests.dll</ModulePath>
</Exclude>
</ModulePaths>
<Sources>
<Exclude>
<!--extracted from default CodeCoverage.config -->
<Source>.*\\atlmfc\\.*</Source>
<Source>.*\\vctools\\.*</Source>
<Source>.*\\public\\sdk\\.*</Source>
<Source>.*\\externalapis\\.*</Source>
<Source>.*\\microsoft sdks\\.*</Source>
<Source>.*\\vc\\include\\.*</Source>
</Exclude>
</Sources>
<Functions>
<Exclude>
<!--extracted from default CodeCoverage.config -->
<Function>^std::.*</Function>
<Function>^ATL::.*</Function>
<Function>.*::__GetTestMethodInfo.*</Function>
<Function>.*__CxxPureMSILEntry.*</Function>
<Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function>
<Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>
<Function>.*::YOU_CAN_ONLY_DESIGNATE_ONE_.*</Function>
</Exclude>
</Functions>
</CodeCoverage>
and command line:
CodeCoverage collect /output:coverage.dat /config:coverage.settings vstest.console unitTests.dll /Logger:trx /Settings:test.runsettings

Okay, Here comes the HACK!!!!
Basic steps:
Find and delete all *.coverage files
Run vstest command WITHOUT [codecoverage.exe collect] wrapper
Find the new *.coverage files and send to [codecoverage.exe analyze] command
Details
I updated the build.proj file I've been using to execute all this to do the Basic steps:
<PropertyGroup>
<SqCodeCoverageResultsFile>VisualStudio.coveragexml</SqCodeCoverageResultsFile>
</PropertyGroup>
<Target Name="BuildTestAssemblyList"
BeforeTargets="RunAllTestsWithCodeCoverageAndConvertToXmlOutput">
<CreateItem Include="**\*.Tests.Unit.dll">
<Output TaskParameter="Include" ItemName="TestAssemblies" />
</CreateItem>
</Target>
<Target Name="BuildCoverageFileList"
BeforeTargets="RunAllTestsWithCodeCoverageAndConvertToXmlOutput">
<CreateItem Include="**\*.coverage">
<Output TaskParameter="Include" ItemName="CoverageFiles" />
</CreateItem>
</Target>
<Target Name="RunAllTestsWithCodeCoverageAndConvertToXmlOutput">
<Delete Condition="Exists($(SqCodeCoverageResultsFile))" Files="$(SqCodeCoverageResultsFile)" />
<Delete Files="#(CoverageFiles)" />
<Exec Command=""$(VsTestExecutable)" /EnableCodeCoverage /Settings:MyProject.runsettings /inIsolation /logger:trx #(TestAssemblies->'"%(FullPath)"',' ') " />
<CreateItem Include="**\*.coverage">
<Output TaskParameter="Include" ItemName="NewCoverageFiles" />
</CreateItem>
<Exec Command=""$(VsCodeCoverageExecutable)" analyze /output:"$(SqCodeCoverageResultsFile)" #(NewCoverageFiles->'"%(FullPath)"',' ') " />
</Target>`
Now running the CodeCoverage.exe analyze command with those found *.coverage files will now output to the same filename I was trying to achieve before and get the results I wanted.
MSBuild.SonarQube.Runner.exe gets what it wants, I have the results I want, and the world can start to revolve again =)
Improvement: I could use a CustomTask and search for the perfect or most recent or whatever logic you can think of for that correct single file so I wouldn't have to delete all my other *.coverage files. I could, but I didn't because this is supposed to be run on a build server that shouldn't have that kind of history laying around anyways in my opinion.