I am trying to set up unit testing and code coverage for a C# project. I am using sonar runner as the analyzer, Gallio 3.4.14 OpenCover4.5.1
This is the sonar-project.properties file:
sonar.projectKey=Foo
sonar.projectName=Foo-SonarQube Runner
sonar.projectVersion=1.0
# Path to the source directories (required)
sonar.sources=.
sonar.language=cs
sonar.sourceEncoding=UTF-8
sonar.dotnet.visualstudio.solution.file=Foo.sln
sonar.silverlight.4.mscorlib.location=C:\Program Files (x86)\Reference Assemblies
\Microsoft\Framework\Silverlight\v5.0
sonar.dotnet.excludeGeneratedCode=true
sonar.dotnet.4.0.sdk.directory=C:/WIndows/Microsoft.NET/Framework/v4.0.30319
sonar.dotnet.version=4.0
sonar.gallio.runner=IsolatedProcess
sonar.gallio.coverage.tool=OpenCover
sonar.donet.visualstudio.testProjectPattern=**.Tests; **.UnitTests
sonar.opencover.installDirectory=C:/Program Files (x86)/OpenCover
sonar.dotnet.test.assemblies=FooUnitTests\bin\Release\FooUnitTests.dll
It runs successfully but in the logs I see that Gallio didnt execute:
No assembly to check with Gendarme
Skipping the non generated assembly of project : Foo
No assembly to check with NDeps
Gallio won't execute as there are no test projects.
Any help will be appreciated
Double-check your test project patterns:
sonar.donet.visualstudio.testProjectPattern=**.Tests; **.UnitTests
Try this instead:
sonar.donet.visualstudio.testProjectPattern=*Tests;*UnitTests
Also, toward the top of the sonar-runner output (within the first 20 or so lines), you should see some lines like this:
05:40:07.021 INFO - Initializing Hibernate
05:40:13.601 INFO - Load project settings
05:40:13.653 INFO - The following 'sln' file has been found and will be used: C:\TeamCity\buildAgent\work\920b95942ca1a758\Sonar\MyProject-Sonar.sln
05:40:17.276 INFO - The project 'MyProject.Tests' has been qualified as a test project.
05:40:19.235 INFO - The project 'MyOther.Project.Tests' has been qualified as a test project.
If you're not seeing this, try running sonar-runner with -X to enable debugging output. This should give you more details as to what patterns are being used to match test projects.
Additionally, any paths need to use forward slashes (/), so the following need to be changed:
sonar.silverlight.4.mscorlib.location=C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\Silverlight\v5.0
sonar.dotnet.test.assemblies=FooUnitTests\bin\Release\FooUnitTests.dll
To this:
sonar.silverlight.4.mscorlib.location=C:/Program Files (x86)/Reference Assemblies/Microsoft/Framework/Silverlight/v5.0
sonar.dotnet.test.assemblies=FooUnitTests/bin/Release/FooUnitTests.dll
Related
I have a .NET solution with bunch of projects. I wan to analyze the project with SonarQube, and show unit tests details. I'm using NUnit for tests and OpenCover to generate coverage reports. I have a simple batch file to run the required SonarQube tasks in sequence.
Now I want some of the assemblies to be excluded from the coverage analysis & calculation, and do the aggregate analysis only on a subset of assemblies. I'm using opencover filters for the purpose, but they do not seem to work.
Let's say I have assemblies like
MyProject.ViewModels
MyProject.DomainModels
MyProject.Core
MyProject.CoreTests
My.Shared.Data
MyProject.Data.Tests
MyProject.Data.Console
MyShared.Console
UIModules.Services
UIModules.Services.UnitTests
UIModules.Web
UIModules.Web.Tests
And to process them I have a batch file like the following. I have added some filter to exclude the Tests, Models & Console assemblies.
Seems like the filters are not working properly. In the code coverage shown on SonarQube, though the Test projects are excluded, still the Models & Console projects are taken into account for overall coverage calculation.
SonarQube.Scanner.MSBuild.exe begin /k:"key" /n:"My Project" /v:"1.0" /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml" /d:sonar.cs.opencover.reportsPaths="opencover.xml"
REM build the project/solution
msbuild MyProject.sln /t:rebuild
REM run NUnit tests
"C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe" /result=NUnitResults.xml Core\bin\Debug\MyProject.CoreTests.dll Data\bin\Debug\MyProject.Data.Tests.dll Services\bin\Debug\UIModules.ServicesTests.dll Web\bin\Debug\UIModules.Web.Tests.dll
REM run OpenCover coverage
"C:\Users\Me\AppData\Local\Apps\OpenCover\OpenCover.Console.exe" -register:user "-target:C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe" "-targetargs:Core\bin\Debug\MyProject.CoreTests.dll Data\bin\Debug\MyProject.Data.Tests.dll Services\bin\Debug\UIModules.ServicesTests.dll Web\bin\Debug\UIModules.Web.Tests.dll /noshadow" "-output:opencover.xml" "-filter:+[MyProject*]* +[My.Shared*]* +[UIModules*]* -[*Tests]* -[*.Console]* -[*Models]*"
REM process end analysis
SonarQube.Scanner.MSBuild.exe end
Can someone guide me to the correct direction.
Not sure where you are looking for the rules.
The coverage filter rules in the coverage tool should influence the output of the coverage.
When the coverage gets imported additional rules apply.
Sonar Qube has additional rules. And the runner can have filter rules built in as well.
Some Runners can ignore Sonar settings.
Since you are talking about coverage I assume you want to skip testing data.
For that the Testing projects names need to end in Test or Tests.
This is mentioned here:
https://docs.sonarqube.org/display/SCAN/Miscellaneous+Advanced+Usages
I also tend to exclude AssemblyInfo.cs which I set in my Sonar Host.
In Configuration -> Analysis Scope -> Global Source File Exclusions with my Pattern of **/AssemblyInfo.cs
I hope this gets you started.
I think you should leave the Other projects as not covered and list that.
If you really want to exclude per project you could try to pass additional parameters to the sonar scanner like sonar.exclusions
It is possible that sonar.exclusions is ignored by the runner.
See this answer for inspiration on different exclusion methods:
The root directory used for the resolution of the sonar.exclusions relative path is the location of each .csproj file.
I have a project with classes and methods that I want to test. I have another project with the test methods that will test the methods of my main project.
I run the tests with opencover and I generate the reports with reportgenerator, with this commands that I have in a .bet file:
..\tools\OpenCover.Console.exe -register:user -target:"C:\myDllWithTests.dll" -output:"c:\coverage\opencovertests.xml"
.\ReportGenerator.exe "-reports:c:\coverage\opencovertests.xml" "-targetdir:c:\coverage\opencovertests.xml\reports"
I am using MSTest for testing.
The problem is that in the html report, I see that the code that is covered is the tests methods, not the methods in my test main project.
How I could add the main methods in the result?
Thanks.
In target argument for OpenCover pass the path to MSTest (e.g. "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe") and specify your test assemblies (e.g. "C:\myDllWithTests.dll") in targetargs argument.
To remove test assemblies from code coverage statistics, specify them in filter argument.
Below is OpenCover command that works fine for me. Here code under test is placed in SampleApp.dll and test code is placed in SampleApp.Tests.dll.
.\OpenCover.Console.exe -register:user -mergebyhash -target:"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/testcontainer:\"d:\test\SampleApp\SampleApp.Tests\bin\Debug\SampleApp.Tests.dll\"" -output:UTResults.xml -filter:"+[SampleApp*]* -[SampleApp.Tests]*"
Result report contains only stats for SampleApp.dll assembly, SampleApp.Tests.dll is excluded:
Check this answer for some more details.
There is also a great article by Allen Conway on using OpenCover & ReportGenerator for .Net projects.
This might be quite a late answer here, but I've spent an hour or two playing with this and found the following will fix this. It's worth noting that I had the original bat script from another project that I know works, and just changed the DLL file name, so I know the script was OK.
The additional check to make is to:-
Right click the project that has the source code you want visible in the coverage report (not the unit test project) and click Properties
Select Build > Output > Advanced
Set Debugging information to Full
Rebuild solution and re-run bat file.
Works for me in Visual Studio 2019 with .NET Framework 4.7.2 projects.
I am working on a Windows Phone 8.1 app. This app has it's Unit Tests implemented using the MSTestFramework. To run the tests we need to use vstest.console.exe and also generate an .appx file for the unit test project. Now I need to use OpenCover for analyzing the tests and get a coverage report.
I am following this tutorial but so far I can't get it working.
As per the tutorial, I have created a batch file which contains the following line:
vstest.console.exe myApp_1.0.0.0_x86_Debug.appx /Settings:C:\Test\Test.runsettings /logger:trx
I then call OpenCover using the following command:
OpenCover.Console.exe -target:C:\Test\myBat.bat -register -output:out.xml
but this results in the missing PDBs exception. The above command actually kicks off all the tests and I can see that vstest.console has created a trx file and all tests passing but no report is generated by OpenCover.
I have tried to use the following command as well:
OpenCover.Console.exe -target:C:\Test\myBat.bat -register -output.xml -targetdir:<TargetDir>
In the TargetDir field I have tried giving the path of myProject\obj\x86\Debug - as this contains PDB files. After this did not work I tried giving TargetDir the path of myProject\AppPackages\myProject_x86_Debug_Test - this contains both appx and appxsym files. Finally, I tried copying all files from the Debug folder into the app packages folder and that did not work as well.
I am guessing that OpenCover isn't yet ready for providing coverage for windows phone apps. If OpenCover supports Windows Phone Apps then I would like to know how and if there is anything wrong in my approach.
I'm currently have the same problem so I can't provide the answer yet. But have you tried -register:user instead of -register?
openCover.Console.exe -target:C:\Test\myBat.bat -register:user -output:out.xml
It was a long WTF moment and I could not find any information on the issue, so I decided I put it here.
Setup: Windows 7 x86_64, sonarqube-4.3, sonar-runner-2.4, jre 1.7.0_55-b14
I followed quick start guide to configure Sonar.
I copied sonar-project.properties from a sample to my solution, put it in the directory with .sln, and made necessary adjustments:
sonar.projectKey=org.whatever.project
sonar.projectName=Project
sonar.projectVersion=2.0
sonar.sourceEncoding=UTF-8
sonar.sources=.
sonar.visualstudio.enable=true
Problem: Analysis ran without any exceptions but only C# files were analyzed.
Long story short, following line in config was causing the issue:
sonar.sources=.
After moving sonar-project.properties one level up and changing it accordingly, the problem was resolved.
sonar.sources=src
I think it actually relates to the usage of the Visual Studio Bootstrapper. Can you try the same without it?
Here is what happens: The Visual Studio bootstrapper reads all the <Compile> tags from the *.csproj files to determine which files must be imported. Javascript files are not referenced by <Compile> (as they do not need to be compiled), and will therefore not be imported in SonarQube.
See this related Jira ticket: https://jira.codehaus.org/browse/SONARVS-27
I'm currently running dotCover as a part of my MSTest build step in TeamCity. I've configured MSTest to run the assembly files matched by **.Tests.dll, otherwise everything else has default settings.
I have no idea how dotCover decides which files to track (I would like to know), but I'm noticing that a lot of third party libraries and the test projects themselves show up in the report - I would like to exclude this from being analyzed and reported on. How can I do that?
In TeamCity 7 you can specify assembly filters for dotCover. For example you can exclude a referenced assembly by adding a negative filter:
-:TheAssembly.To.Ignore.*
This would exclude any assemblies where the name started with TheAssembly.To.Ignore