I am working on a Windows Phone 8.1 app. This app has it's Unit Tests implemented using the MSTestFramework. To run the tests we need to use vstest.console.exe and also generate an .appx file for the unit test project. Now I need to use OpenCover for analyzing the tests and get a coverage report.
I am following this tutorial but so far I can't get it working.
As per the tutorial, I have created a batch file which contains the following line:
vstest.console.exe myApp_1.0.0.0_x86_Debug.appx /Settings:C:\Test\Test.runsettings /logger:trx
I then call OpenCover using the following command:
OpenCover.Console.exe -target:C:\Test\myBat.bat -register -output:out.xml
but this results in the missing PDBs exception. The above command actually kicks off all the tests and I can see that vstest.console has created a trx file and all tests passing but no report is generated by OpenCover.
I have tried to use the following command as well:
OpenCover.Console.exe -target:C:\Test\myBat.bat -register -output.xml -targetdir:<TargetDir>
In the TargetDir field I have tried giving the path of myProject\obj\x86\Debug - as this contains PDB files. After this did not work I tried giving TargetDir the path of myProject\AppPackages\myProject_x86_Debug_Test - this contains both appx and appxsym files. Finally, I tried copying all files from the Debug folder into the app packages folder and that did not work as well.
I am guessing that OpenCover isn't yet ready for providing coverage for windows phone apps. If OpenCover supports Windows Phone Apps then I would like to know how and if there is anything wrong in my approach.
I'm currently have the same problem so I can't provide the answer yet. But have you tried -register:user instead of -register?
openCover.Console.exe -target:C:\Test\myBat.bat -register:user -output:out.xml
Related
It's a very basic problem and should have a simple solution. I want to have all the code coverage results for my unit tests project in a solution to output in a single directory. Currently, all the coverage output files are being written in the individual test project folder. I tried to look into the coverlet github issues where in they mentioned that they have a workaround (https://github.com/tonerdo/coverlet/pull/220) for this to merge all the coverage files into a single one but I couldn't place them into my solution directory
dotnet test .\MySolution.sln /p:CollectCoverage=true /p:CoverletOutput=/results/coverage /p:MergeWith=\results\coverage.json /p:CoverletOutputFormat='json%2copencover'
What I am struggling is to specify the solution directory in the /p:CoverletOutput parameter. Is there a special paramter in dotnet cli to point to the solution directory similar to what we have in msbuild $(MSBuildProjectDirectory)?
Just so anyone interested in the answer, the following script works well in windows machine
dotnet test MySolution.sln /p:CollectCoverage=true /p:CoverletOutput=..\results\coverage
/p:MergeWith=..\results\coverage.json /p:CoverletOutputFormat=\"opencover,json\"
Created a simple framework with Scenario Outline tests, it can be found here. Copied allureConfig.json file into bin/debug directory. Nuget versions:
Nunit 3.9.0;
SpecFlow 2.3.0;
SpecFlow.Allure 2.2.0.11;
Net framework 4.6.2.
After tests execution, using search, I found test-results directory in the next location:
C:\Users\Valentins\AppData\Local\Temp\allure-results
And it contains just .json files, like:
52bb84abaec342e1bf8e7fbc3e46a181-container
and
9ab91b6fc6e84eafbda735c49ccfcb1c-result
Can anyone help to understand, how to make this folder to appear in the tests and how to normally open Test results to see it in Html file?
Thanks in advance,
Valentins.
The problem was because I was running tests, using Visual Studio. When I started to run, using CI server (TeamCity), then allure-results folder appeared in the Project, but for creating a report, allure-commandline-tools were needed.
I want to run tests from a console like this (being in any directory, the DLL file can be for a different .NET version):
$ nunit3-console test.dll
I googled a lot, but can't find how to set up this.
The official tutorial has nothing useful and is complete zero. Following it gets me nowhere: https://github.com/nunit/docs/wiki/Installation
It is hard to find, because there is a lot of outdated documentation, either for NUnit2 or NUnit3.
Steps:
Official NUnit3 console installers are here: https://github.com/nunit/nunit-console/releases (path is different than in docs)
Download NUnit.Console-*.msi package and install
Add to system PATH variable this: C:\Program Files (x86)\NUnit.org\nunit-console
Open command line
Type:
$ nunit3-console test.dll
// For running multiple test assemblies in parallel see: https://stackoverflow.com/a/45486444/1453525
I am using NUnit3-console.exe with Selenium WebDriver to run my automation, all of which is written in C#. I have multiple environments set up under discreet logins of Windows Server 2012.
NUnit-Console doesn't have to be "installed", although the .msi is readily available. Instead, I use the .zip and extract the files to a directory, C:\Nunit, rather than allowing the invocation to resolve from the PATH. All invocations are from a Windows Forms scheduler in the form -
C:\Nunit\NUnit3-Console.exe -work:C:\Users\xxxx\Automation\TestResults\ -out:TestResult.xml --where "name =~ 'yyyy'" --p environment=qa;browser=Firefox;browserSizeX=1200;browserSizeY=800 "C:\QA_Libraries3\zzzz.dll"
The test parameters are passed on the command line and the NUnit results plus results from the test are extracted from the TestResult.xml which is distinct for each user (environment).
What I do and recommend is to add nuget package NUnit.ConsoleRunner. Note that there are similarly named packages (NUnit.Runners, NUnit.Console) that might work too, but I know that NUnit.ConsoleRunner for sure has the nunit3-console.exe in it ... well at least the version of the package that I'm using (3.4.1) does :) Sadly, nunit versioning and packaging seems to be messy. There are lots of old docs and packages that seems to overlap. And I can't find good/solid up-to-date docs.
Anyway, once you get that package then you can run the exe that's now under your packages directory. For me it's packages\NUnit.ConsoleRunner.3.4.1\tools\nunit3-console.exe. This works well for calling from a build automation script that is in the solution folder or knows how to find the solution folder.
There's another option that although is not a direct answer to your question does get to what I assume is your desire: to run your nunit3 tests from the command line. If you add package NUnit3TestAdapter, then you can use Visual Studio's built in runner, vstest. If you open a Developer Command Prompt (or PowerShell), then it can be run as 'vstest.console' (without path info since the exe is in the path env var). Of course it has its own command syntax to learn.
I realize this thread is a bit dated, but here is how I run a specific SINGLE test.
install nunit-console (https://github.com/nunit/nunit-console/releases/latest)
Open a powershell window and run nunit3-console.exe with "--test" option set to reference the specific test you want to run (including namespace and class). and finally, provide the location of the assembly where the test can be found.
Example (paths need to be adjusted to point to your specific files):
& "C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" --test=MyApp.Mvc.WebTests.CardsControllerTests.TheNameOfYourTestMethod "c:\src\MyApp.Mvc.WebTests\bin\Debug\MyApp.Mvc.WebTests.dll"
Hope this helps someone.
I have a project with classes and methods that I want to test. I have another project with the test methods that will test the methods of my main project.
I run the tests with opencover and I generate the reports with reportgenerator, with this commands that I have in a .bet file:
..\tools\OpenCover.Console.exe -register:user -target:"C:\myDllWithTests.dll" -output:"c:\coverage\opencovertests.xml"
.\ReportGenerator.exe "-reports:c:\coverage\opencovertests.xml" "-targetdir:c:\coverage\opencovertests.xml\reports"
I am using MSTest for testing.
The problem is that in the html report, I see that the code that is covered is the tests methods, not the methods in my test main project.
How I could add the main methods in the result?
Thanks.
In target argument for OpenCover pass the path to MSTest (e.g. "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe") and specify your test assemblies (e.g. "C:\myDllWithTests.dll") in targetargs argument.
To remove test assemblies from code coverage statistics, specify them in filter argument.
Below is OpenCover command that works fine for me. Here code under test is placed in SampleApp.dll and test code is placed in SampleApp.Tests.dll.
.\OpenCover.Console.exe -register:user -mergebyhash -target:"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/testcontainer:\"d:\test\SampleApp\SampleApp.Tests\bin\Debug\SampleApp.Tests.dll\"" -output:UTResults.xml -filter:"+[SampleApp*]* -[SampleApp.Tests]*"
Result report contains only stats for SampleApp.dll assembly, SampleApp.Tests.dll is excluded:
Check this answer for some more details.
There is also a great article by Allen Conway on using OpenCover & ReportGenerator for .Net projects.
This might be quite a late answer here, but I've spent an hour or two playing with this and found the following will fix this. It's worth noting that I had the original bat script from another project that I know works, and just changed the DLL file name, so I know the script was OK.
The additional check to make is to:-
Right click the project that has the source code you want visible in the coverage report (not the unit test project) and click Properties
Select Build > Output > Advanced
Set Debugging information to Full
Rebuild solution and re-run bat file.
Works for me in Visual Studio 2019 with .NET Framework 4.7.2 projects.
I created a C# sample unit test library (windows store apps) project.
Using command line, I tried to build the solution and created .appx file using command line statements. I am able to sign it successfully. However when I run the .appx file using vstest.console.exe, I am getting "Warning: No test discoverer is registered to perform discovery of test cases. Register a test discoverer and try again"
My sample code is available at https://github.com/budgetpreneur/SampleCode/tree/master/demoutest
To reproduce the problem, just get the code into your local machine and run testrun.bat file. This batch file builds the solution, copies the required files to "demoappx" folder for appx file, runs makeappx.exe, creates cert related files and signs the .appx file before running vstest.console.exe.
The goal of this batch file is to run Windows 8 Store Apps Unit Tests automatically without user interaction.
I used Visual Studio 2012 edition for this.
Can anyone point me where I can find more information to resolve it?
Any solution or idea to resolve it?
Thanks in advance.