Develop Reference

Displaying NUnit tests code coverage on Azure DevOps

I've setup a new pipeline on Azure DevOps which builds and run the tests of the projects. The tests are written with NUnit.
In the pipeline I'm using the VSTest#2 task to run the unit tests and I add the codeCoverageEnabled to true.
In the end the pipeline runs and when I go in the "Code Coverage" tab of the job, it allows me to download .codecoverage file but it does not display its content in the tab. My understanding was that this should happen.
How can I fix this ?
Thanks

By default, the code coverage for the VSTest Task is output to a .codecoverage file, which Azure DevOps does not know how to interpret and only provides as a downloadable file. You'll need to use a few DotNetCoreCLI tasks and coverlet to be able to display code coverage results on the code coverage tab in azure pipelines.
So, if you are on .NET CORE, There is a way how you can do that.
Step 1
add Coverlet.collector nuget Package in your test project
Step 2
Change your azure-pipelines.yml to include the following for your code coverage:
If you have any settings from a CodeCoverage.runsettings file, you can keep them too
- task: DotNetCoreCLI#2
inputs:
command: 'test'
projects: '**/*.Tests/*.csproj'
arguments: -c $(BuildConfiguration) --collect:"XPlat Code Coverage" -- RunConfiguration.DisableAppDomain=true
testRunTitle: 'Run Test and collect Coverage'
displayName: 'Running tests'
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- script: reportgenerator -reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/coverlet/reports -reporttypes:"Cobertura"
displayName: Create reports
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)/coverlet/reports/Cobertura.xml
One other thing to note for the above code is the Report Generator. Depending on which version of .net core you are using it might be required to get a different version of the tool.
More information can also be found on the Microsoft Docs

Code Coverage and Lines of Code are displayed as '-' in sonarqube console

I am using azure devops for running a test and trying to integrate sonarqube with it.The issue i am facing is that in the summary part of azure pipeline i am able to view code coverage as 22% but in the sonarqube console i am only able to view code coverage as '-'.There is a warning message that i am seeing when i run 'Run Code Analysis' task in pipeline.
The warning message is WARN: The Code Coverage report doesn't contain any coverage data for the included files.
[Please find the image to view the code coverage displayed in azure pipeline][1]
This is the yaml for dot test task
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: test
projects: '**/*Test*.csproj'
arguments: '--configuration $(BuildConfiguration) --collect "Code coverage" '
workingDirectory: '$(System.DefaultWorkingDirectory)'
This is the yaml for copy files task that i am doing right after the dot test task
steps:
- task: CopyFiles#2
displayName: 'Copy Files to: $(Common.TestResultsDirectory)'
inputs:
SourceFolder: '$(Agent.WorkFolder)\_temp'
TargetFolder: '$(Common.TestResultsDirectory)'
Please find the yaml file for Prepare analysis on sonarqube task
displayName: 'Prepare analysis on SonarQube'
inputs:
SonarQube: 'CDA-Sonarqube'
projectKey: Test
projectName: Test
extraProperties: sonar.cs.nunit.reportsPaths
Any help is appreciated.
[1]: https://i.stack.imgur.com/HbZfW.png

WARN: The Code Coverage report doesn't contain any coverage data for the included files.
For troubleshooting hints, please refer to https://docs.sonarqube.org/x/CoBh , the .coverage file will be convert to coveragexml during sonarqube end analysis task
Run Unit Tests and Save Results in file "NUnitResults.xml"
packages\NUnit.ConsoleRunner.3.7.0\tools \ nunit3-console.exe --result=NUnitResults.xml "NUnitTestProject1\bin\Debug\NUnitTestProject1.dll"
or, for older NUnit 2
"%ProgramFiles(x86)%\NUnit 2.6.4\bin \nunit-console.exe /result=NUnitResults.xml "NUnitTestProject1\bin\Debug\NUnitTestProject1.dll"
Meanwhile,there is a workaround explained in the VSTS extension documentation in "Analysing a .NET solution": in the Additional Properties text area, add the following property:
sonar.cs.vscoveragexml.reportsPaths=**/*.coveragexml
You can also refer to these cases(case1 , case2) for details.
Here is a blog about configuring Code Coverage for Dotnet Core based applications using SonarQube and Azure DevOps .

By default the test result files are in temp folder, try to copy files through Copy File task, then the .coverage file will be analysis and generate coveragexml file.
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.sourcesdirectory)\TestResults'
inputs:
SourceFolder: '$(Agent.TempDirectory)'
TargetFolder: '$(build.sourcesdirectory)\TestResults'
On the other hand, you can refer to this article to call CodeCoverage.exe:
Configure Code Coverage for Dotnet Core 2.0 based applications using SonarQube and Azure DevOps

SonarQube MSBuild Runner defaults to h2 Database

I am using Win Server 2012, Developer Command Prompt for VS2013, SonarQube v5.1.2, SonarQube Scanner for MSBuild 1.1.
I have my Sonar instance running off this same server, but connected to an Oracle database. Using the default sonar runner, i can analyze projects properly and see them appear in the dashboard using this method. That cannot be said for using the recommended MSBuild.SonarQube.Runner.exe approach.
I have tried with the csharp and vbnet examples, as well as our actual code. Oddly, i want to say each of these have worked at least once, but fail to do so now. Today, I tried using the vbnet example project for the first time and discovered and it worked as intended when using the MSBuild runner. I deleted the project from the dashboard, and ran it again. At this point i do not remember it it worked or not, but the third time for sure it stopped working and began producing the error below (wherein it tries to connect to the default h2 database instead of the oracle one that SonarQube is connected to).
I tried deleting the examples folder, extracting it again, giving it a slightly different name, and running the Sonar on it giving THOSE a different name/key/version, but now it is consistently giving me the db error.
Any idea what could be going on here, and why it would go from working to not, without me making any changes to the runner, msbuild, or the code?
Running the begin step:
c:\sonar-examples-master\projects\languages\vbnet>msbuild.sonarqube.runner.exe b
egin /v:vbnet1 /k:vbnet1 /n:vbnet1
SonarQube Scanner for MSBuild 1.1
Default properties file was found at c:\sonarqube\bin\SonarQube.Analysis.xml
Loading analysis properties from c:\sonarqube\bin\SonarQube.Analysis.xml
Pre-processing started.
Preparing working directories...
Checking for updates...
MSBuild SonarQube Runner Pre-processor 1.0.2.0
17:40:49.625 Loading analysis properties from c:\sonarqube\bin\SonarQube.Analys
is.xml
17:40:49.656 Updating build integration targets...
17:40:49.656 Fetching analysis configuration settings...
17:40:50.813 Generating rulesets...
Pre-processing succeeded.
MSBuild ran, but nothing worthwhile to post from it..
Running the end step:
c:\sonar-examples-master\projects\languages\vbnet>msbuild.sonarqube.runner.exe e
nd
SonarQube Scanner for MSBuild 1.1
Default properties file was found at c:\sonarqube\bin\SonarQube.Analysis.xml
Loading analysis properties from c:\sonarqube\bin\SonarQube.Analysis.xml
Post-processing started.
MSBuild SonarQube Runner Post-processor 1.0.2.0
WARNING: File is not under the project directory and cannot currently be analyse
d by SonarQube. File: C:\Users\ts3conusr\AppData\Local\Temp\.NETFramework,Versio
n=v4.5.AssemblyAttributes.vb, project: c:\sonar-examples-master\projects\languag
es\vbnet\ConsoleApplication1\ConsoleApplication1.vbproj
The SONAR_RUNNER_HOME environment variable is not required and will be ignored.
SONAR_RUNNER_OPTS is not configured. Setting it to the default value of -Xmx1024
m
Calling the sonar-runner...
c:\sonar-examples-master\projects\languages\vbnet\.sonarqube\bin\sonar-runner\bi
n\..
SonarQube Runner 2.4
Java 1.7.0_79 Oracle Corporation (32-bit)
Windows Server 2008 R2 6.1 x86
SONAR_RUNNER_OPTS=-Xmx1024m
INFO: Error stacktraces are turned on.
INFO: Runner configuration file: c:\sonar-examples-master\projects\languages\vbn
et\.sonarqube\bin\sonar-runner\bin\..\conf\sonar-runner.properties
INFO: Project configuration file: c:\sonar-examples-master\projects\languages\vb
net\.sonarqube\out\sonar-project.properties
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Work directory: c:\sonar-examples-master\projects\languages\vbnet\.sonarqu
be\out\.sonar
INFO: SonarQube Server 5.1.2
17:41:04.563 INFO - Load global repositories
17:41:04.751 INFO - Load global repositories (done) | time=188ms
17:41:04.751 INFO - Server id: 20160120154951
17:41:04.751 INFO - User cache: C:\Users\ts3conusr\.sonar\cache
17:41:04.766 INFO - Install plugins
17:41:04.860 INFO - Install JDBC driver
17:41:04.860 INFO - Create JDBC datasource for jdbc:h2:tcp://localhost/sonar
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
Total time: 1.047s
Final Memory: 3M/15M
INFO: ------------------------------------------------------------------------
ERROR: Error during Sonar runner execution
org.sonar.runner.impl.RunnerException: Unable to execute Sonar
Of note: I feel that it is worth mentioning that the vbnet report didn't actually have any content in it, but did at least properly get generated in the Sonar dashboard. I have tried running the commands against both that and the csharp example with now neither of them being added to the dashboard.

In the MSBuild.SonarQube.Runner install folder, there is a SonarQube.Analysis.xml. Do you have the correct server details on the below line in there?
<Property Name="sonar.host.url">http://{host}:{port}/{optionalPath}</Property>
This property value should point to your Sonar server.

Couple other things to try. Do you see tables being populated with analysis data in your oracle database? When was the last time it got any record?
Also, are you seeing below warning on your SonarQube application pages (say default dashboard)? If yes, then oracle configuration is not being picked up.

So i upgraded to the latest SonarQube version (5.3) and upgraded a few of the extensions while i was at it and preliminary testing is showing positive results. Ill try it a few more times but im closing the question for now. Thank you all that helped!

Does Coverity support xbuild (mono)?

I've a C# project: https://github.com/Pro/dkim-exchange
It uses Travis CI: https://travis-ci.org/Pro/dkim-exchange
Travis successfully builds my project.
I wanted to set up Coverity to do automatic code quality measurements. For this I configured my .travis.yml as follows:
language: objective-c
env:
global:
- EnableNuGetPackageRestore=true
# The next declaration is the encrypted COVERITY_SCAN_TOKEN, created
# via the "travis encrypt" command using the project repo's public key
- secure: "kC7O0CWm9h4g+tzCwhIZEGwcdiLrb1/1PijeOKGbIWGuWS7cIksAkj2tRNMgtxxcE9CFQr8W7xDv2YzflCIlqN1nGkFjbyD4CrNg6+V1j0fZjPOQ6ssdBBVPrfrvecsAUJ0/48Tqa9VTkEpZSlwOF/VS1sO2ob36FVyWjtxvG9s="
matrix:
- MONO_VERSION="3.10.0"
install:
# Fetch Mono
- wget "http://download.mono-project.com/archive/${MONO_VERSION}/macos-10-x86/MonoFramework-MDK-${MONO_VERSION}.macos10.xamarin.x86.pkg"
- sudo installer -pkg "MonoFramework-MDK-${MONO_VERSION}.macos10.xamarin.x86.pkg" -target /
script:
- xbuild travis.proj
addons:
coverity_scan:
project:
name: "Pro/dkim-exchange"
description: "Build submitted via Travis CI"
notification_email: mail#example.com
build_command_prepend: "xbuild /t:CleanAll travis.proj"
build_command: "xbuild /t:Build travis.proj"
branch_pattern: coverity_scan
If I execute the coverity build commands as indicated here (using msbuild): https://scan.coverity.com/download?tab=csharp the uploaded archive is analyzed correctly, but in combination with travis, the coverity analysis fails (see e.g. this build log: https://travis-ci.org/Pro/dkim-exchange/builds/42295611).
There's this warning:
[WARNING] No files were emitted. This may be due to a problem with your configuration
or because no files were actually compiled by your build command.
Please make sure you have configured the compilers actually used in the compilation.
I think this may be related to xbuild from Mono. Unfortunately Dr. Google didn't find anythin about Coverity+xbuild. Does Coverity support xbuild? If yes, how can I correctly setup the project?

When it comes to C#, Coverity actually only supports msbuild.
You can find some more official information about this in the following
http://www.coverity.com/library/pdf/CoverityStaticAnalysis.pdf
https://communities.coverity.com/message/6251#6251
The last link explicitly states
Our C# analysis only supports the Visual Studio C# compilers
So, no xbuild support as of now.
Update:
When you download the Coverity build tool, the doc/en/help/cov-build.txt explicitly states the following:
C# build capture is only supported on Windows.

How to calculate code coverage with NCover using Jenkins?

I am working with a windows application in c#.
Using Jenkins, i created a job that will do the following tasks.
1.Build the application using msbuild
2.Tests Unit Test Cases using nunit-console.exe
3.Calculate Code coverage using NCover. (Issue)
4.Later publish the application using Nant plugin
Tasks 1, 2 and 4 works fine, while 3 having issue.
Can somebody put light on this matter?
This is the batch file that i used to find out the coverage
C:\Program Files\NCover\NCover.Console.exe" "E:\Myapp\test.exe" -h //x "E:\Newfolder\coverage.xml
The batch file is executed in Jenkins and we can see the Test.exe in task manager, what i need is the code coverage in html format when executing the Nunit test cases and no need to run my text.exe
D:\Set Up\Nuint\NUnit-2.6.2\bin\nunit-console.exe" "E:\Myapp\test.sln" /xml="E:\Newfolder\TestResult.xml
This is the batch command that i used to test the test cases, i need to know the code coverage while executing the test case, but in my case my test.exe is executed and NCover console.exe start monitor my test.exe for calculating the coverage
i tried by adding
C:\Program Files\NCover\NCover.Console.exe" infront of "D:\Set Up\Nuint\NUnit-2.6.2\bin\nunit-console.exe" "E:\Myapp\test.sln" /xml="E:\Newfolder\TestResult.xml , build succeded. and in console o/p found some coverage data like
Execution Time: 92.4688 s
Symbol Coverage: 43.72%
Branch Coverage: 22.70%
and a coverage.nccov file is created . but i need to create/show a coverage report in html format.

You can use the NCover plug-in or a post-build-task to start the calculation.
For second variant we mostly use a simple batch-file to start the action (in your case the ncover calculation). This batch-file will be called by the jenkins post-build-task.
Edit:
To get HTML you can do it via (look here):
NCover.Reporting Coverage.xml //or FullCoverageReport:Html //op "C:\Coverage Report"