I want to run tests from a console like this (being in any directory, the DLL file can be for a different .NET version):
$ nunit3-console test.dll
I googled a lot, but can't find how to set up this.
The official tutorial has nothing useful and is complete zero. Following it gets me nowhere: https://github.com/nunit/docs/wiki/Installation
It is hard to find, because there is a lot of outdated documentation, either for NUnit2 or NUnit3.
Steps:
Official NUnit3 console installers are here: https://github.com/nunit/nunit-console/releases (path is different than in docs)
Download NUnit.Console-*.msi package and install
Add to system PATH variable this: C:\Program Files (x86)\NUnit.org\nunit-console
Open command line
Type:
$ nunit3-console test.dll
// For running multiple test assemblies in parallel see: https://stackoverflow.com/a/45486444/1453525
I am using NUnit3-console.exe with Selenium WebDriver to run my automation, all of which is written in C#. I have multiple environments set up under discreet logins of Windows Server 2012.
NUnit-Console doesn't have to be "installed", although the .msi is readily available. Instead, I use the .zip and extract the files to a directory, C:\Nunit, rather than allowing the invocation to resolve from the PATH. All invocations are from a Windows Forms scheduler in the form -
C:\Nunit\NUnit3-Console.exe -work:C:\Users\xxxx\Automation\TestResults\ -out:TestResult.xml --where "name =~ 'yyyy'" --p environment=qa;browser=Firefox;browserSizeX=1200;browserSizeY=800 "C:\QA_Libraries3\zzzz.dll"
The test parameters are passed on the command line and the NUnit results plus results from the test are extracted from the TestResult.xml which is distinct for each user (environment).
What I do and recommend is to add nuget package NUnit.ConsoleRunner. Note that there are similarly named packages (NUnit.Runners, NUnit.Console) that might work too, but I know that NUnit.ConsoleRunner for sure has the nunit3-console.exe in it ... well at least the version of the package that I'm using (3.4.1) does :) Sadly, nunit versioning and packaging seems to be messy. There are lots of old docs and packages that seems to overlap. And I can't find good/solid up-to-date docs.
Anyway, once you get that package then you can run the exe that's now under your packages directory. For me it's packages\NUnit.ConsoleRunner.3.4.1\tools\nunit3-console.exe. This works well for calling from a build automation script that is in the solution folder or knows how to find the solution folder.
There's another option that although is not a direct answer to your question does get to what I assume is your desire: to run your nunit3 tests from the command line. If you add package NUnit3TestAdapter, then you can use Visual Studio's built in runner, vstest. If you open a Developer Command Prompt (or PowerShell), then it can be run as 'vstest.console' (without path info since the exe is in the path env var). Of course it has its own command syntax to learn.
I realize this thread is a bit dated, but here is how I run a specific SINGLE test.
install nunit-console (https://github.com/nunit/nunit-console/releases/latest)
Open a powershell window and run nunit3-console.exe with "--test" option set to reference the specific test you want to run (including namespace and class). and finally, provide the location of the assembly where the test can be found.
Example (paths need to be adjusted to point to your specific files):
& "C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" --test=MyApp.Mvc.WebTests.CardsControllerTests.TheNameOfYourTestMethod "c:\src\MyApp.Mvc.WebTests\bin\Debug\MyApp.Mvc.WebTests.dll"
Hope this helps someone.
Related
I have a solution written on c#, .netcore 2.2 and the testing framework is xunit that looks pretty much like this:
-src
--controllers
--services
-test
--controllers.integrationtest
--services.integratiotest
we are performing some integration test with two distinct dlls (controllers.integrationtest.dll & services.integrationtest) that use the same database (solutionname.test)
now, when I run "dotnet test" on the solution, it seems that the two dlls are trying to access the same resource (database) and one dll gets a deadlock.
The thing is that I want to disable the parallel execution of the dlls when I run "dotnet test" to avoid that deadlock, so I search that up and the documentation says that to disable parallel execution of dlls you have to:
Add a xunit.runner.json in the root of the csproject which I did and works fine. (this is working on visual studio because I tested it with another feature that deletes the "_" character of test names: [methodDisplay])
Configure xunit.runner.json file to copy always or preserve newest in visual studio so that gets copied on bin folder (as any appsetings.json file)
I've read that you can place an assembly attribute in the assemblyInfo.cs file that by the way it seems that was replaced by the plain csproj so I'm a bit confused.
The ultimate goal that I want to achieve is that when devops use dotnet test the integration test doesn't blow up on concurrency problems.
In xUnit the default value for ParallelizeAssemblies is false so I can only assume that a behavior of dotnet test is causing both to be executed together.
Try and run the tests using dotnet vstest. This requires the DLLS to have been built.
dotnet vstest **/*test.dll
I have a project with classes and methods that I want to test. I have another project with the test methods that will test the methods of my main project.
I run the tests with opencover and I generate the reports with reportgenerator, with this commands that I have in a .bet file:
..\tools\OpenCover.Console.exe -register:user -target:"C:\myDllWithTests.dll" -output:"c:\coverage\opencovertests.xml"
.\ReportGenerator.exe "-reports:c:\coverage\opencovertests.xml" "-targetdir:c:\coverage\opencovertests.xml\reports"
I am using MSTest for testing.
The problem is that in the html report, I see that the code that is covered is the tests methods, not the methods in my test main project.
How I could add the main methods in the result?
Thanks.
In target argument for OpenCover pass the path to MSTest (e.g. "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe") and specify your test assemblies (e.g. "C:\myDllWithTests.dll") in targetargs argument.
To remove test assemblies from code coverage statistics, specify them in filter argument.
Below is OpenCover command that works fine for me. Here code under test is placed in SampleApp.dll and test code is placed in SampleApp.Tests.dll.
.\OpenCover.Console.exe -register:user -mergebyhash -target:"c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/testcontainer:\"d:\test\SampleApp\SampleApp.Tests\bin\Debug\SampleApp.Tests.dll\"" -output:UTResults.xml -filter:"+[SampleApp*]* -[SampleApp.Tests]*"
Result report contains only stats for SampleApp.dll assembly, SampleApp.Tests.dll is excluded:
Check this answer for some more details.
There is also a great article by Allen Conway on using OpenCover & ReportGenerator for .Net projects.
This might be quite a late answer here, but I've spent an hour or two playing with this and found the following will fix this. It's worth noting that I had the original bat script from another project that I know works, and just changed the DLL file name, so I know the script was OK.
The additional check to make is to:-
Right click the project that has the source code you want visible in the coverage report (not the unit test project) and click Properties
Select Build > Output > Advanced
Set Debugging information to Full
Rebuild solution and re-run bat file.
Works for me in Visual Studio 2019 with .NET Framework 4.7.2 projects.
I am getting the following error message when trying to run unit tests in Visual Studio:
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
I am using
Visual Studio Community 2013
NUnit Adapter 3.4.0.0
NUnit 3.4.1
The weird thing is, that I have another project which is set up the same way as this one and it works just fine.
I also downloaded NUnit 3.4.1 and installed it. When I run
nunit3-console.exe Trading.Tools.Test.dll
everything works just fine.
Any ideas what I can do?
Many thanks
Konstantin
Edit #1
Here is the full console output from Visual Studio when trying to run all test.
Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run:
Trading.Tools.Test.dll, Trading.Tools.dll are built for Framework Framework45 and Platform X64.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
NUnit Adapter 3.4.0.0: Test discovery starting
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
Assembly contains no NUnit 3.0 tests: w:\Repos\trading.tools\Trading.Tools\bin\x64\Debug\Trading.Tools.dll
NUnit Adapter 3.4.0.0: Test discovery complete
As you can see it is very obvious that NUnit expects a x86 build, but I build for a x64 platform. And again, my x64 build works just fine if I execute it using nunit3-console.exe.
What I see in the csproj file is this:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\packages\NUnit.3.4.1\lib\net45\nunit.framework.dll</HintPath>
</Reference>
The weird thing here is that it specifies using Version=2.6.4.14350 but referencing a 3.4.1 dll.
So the next question from this point is how can I make NUnit execute my x64 build? Any ideas?
I had a similar issue, the key is the fact that it is the Test Runner in Visual Studio that is stating that only x86 assemblies will be tested. I am assuming from this that it then forces the use of the x86 NUnit runner. To change this (in VS2015 and VS2017 at least), go to Test > Test Settings > Default Processor Architecture > X64.
You can also set the execution target in the runsettings file. You then have to select that file. This should make the solution more stable.
A runsettings file which only set this can look like:
To enable it, do as shown in the figure below:
When you select it from the test menu (1), it will be added as the selected one in the menu (2), and a Rebuild will then make the test appear in the Test Explorer (3)
There is an extra bonus by using a runsettings file, and that is that it will then run properly on the TFS Build system, if you use that. I have written a blog post on that issue, see http://hermit.no/how-to-control-the-selection-of-test-runner-in-tfsvsts-making-it-work-with-x86x64-selected-targets/
I couldn't execute my tests and found that to be one of the issues. It turns out that my TestFixture was internal. Just switching it to public solved my case.
After unsuccessfully trying all other responses above, the following worked for me:
In my case the .NET project and solution is on a mounted drive (I use a MacBook and Parallels for .NET development). The mount also contains the /bin/debug and /bin/release locations where NUnit was attempting to read the "test" DLL from.
The fix was to move the solution/project files to the C: drive of my Windows image. The tests were discovered immediately.
Apparently the shared/mounted location was not to its liking. I don't know why, since the mount is permanent and read/writable to all users on the Windows image. I suspect file permissions problems or maybe somehow the entire mount is not accessible to the user/process running the NUnit discovery logic.
I happened to get this error when writing the unit test method. And noticed the root cause that one of the dependent dll was missing to load. This error("NUnit failed to load the .dll") was shown in the Output("Test") window after modifying the test method code and trying to run it. After updating the nuget package for the dependent dll, nunit started picking the test project dll and test cases got executed.
Today I was running into this issue as well (on a .NET Framework 4.8 based NUnit project). The solution for me was to also install the Microsoft.NET.Test.Sdk package.
To get to the root cause, it may help to attempt to run your tests using the NUnit CLI.
In my case, the CLI logged a bind failure I didn't see in Visual Studio. After I had fixed that, my tests ran correctly via CLI and VS.
I got this error in a .Net 6.0 Asp.Net solution and here's how I solved it.
It was only occurring in one Test project whose tests wouldn't run, the other Test projects were running fine in Test Explorer and in Debug.
The tests that were failing to be detected had "Test Not Run":
In the Output is the error:
NUnit Adapter 4.3.1.0: Test discovery starting NUnit failed to load [dll path]
No test is available in [dll path]
What I did was comment out each class and bring them back one by one until the Tests would STOP to RUN. Then with the failing class put a break point on a [Test] method.
If you can't hit the break point then its failing in this classes [SetUp]. Put a BreakPoint in the [SetUp] and use F11 to step off the edge of the method, ie F11 off the final curly brace..
AND THEN you get a prompt saying which DLL it can't load.
In my case it was “couldn’t load a DLL Microsoft.AspNetCore.Mvc.Core”
Referencing the DLL via Package Manager resolved the problem.
Edit: if this happens in a Unit Test you may want to reference the Microsoft.AspNetCore.App FrameworkReference rather than the Microsoft.AspNetCore.Mvc.Core package reference:
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>
I am working on a Windows Phone 8.1 app. This app has it's Unit Tests implemented using the MSTestFramework. To run the tests we need to use vstest.console.exe and also generate an .appx file for the unit test project. Now I need to use OpenCover for analyzing the tests and get a coverage report.
I am following this tutorial but so far I can't get it working.
As per the tutorial, I have created a batch file which contains the following line:
vstest.console.exe myApp_1.0.0.0_x86_Debug.appx /Settings:C:\Test\Test.runsettings /logger:trx
I then call OpenCover using the following command:
OpenCover.Console.exe -target:C:\Test\myBat.bat -register -output:out.xml
but this results in the missing PDBs exception. The above command actually kicks off all the tests and I can see that vstest.console has created a trx file and all tests passing but no report is generated by OpenCover.
I have tried to use the following command as well:
OpenCover.Console.exe -target:C:\Test\myBat.bat -register -output.xml -targetdir:<TargetDir>
In the TargetDir field I have tried giving the path of myProject\obj\x86\Debug - as this contains PDB files. After this did not work I tried giving TargetDir the path of myProject\AppPackages\myProject_x86_Debug_Test - this contains both appx and appxsym files. Finally, I tried copying all files from the Debug folder into the app packages folder and that did not work as well.
I am guessing that OpenCover isn't yet ready for providing coverage for windows phone apps. If OpenCover supports Windows Phone Apps then I would like to know how and if there is anything wrong in my approach.
I'm currently have the same problem so I can't provide the answer yet. But have you tried -register:user instead of -register?
openCover.Console.exe -target:C:\Test\myBat.bat -register:user -output:out.xml
I have .net framework 4.0 (and older versions, i.e. multiple versions) installed, want to use the command line compiler for C#
C:\>csc.exe
Now I found across few forums using the direct path I can use the compiler, also using batch file one can do that,
I need to set environment variables LINK but that is part of visual studio(if VS is installed then only it works) I want to use the compiler from the redistributable .net framework(using command line only) which is free to use and distribute.
Do not want to install Visual Studio in the system where I run the code.
my Current CSC.exe file location PATH is
C:\Windows\Microsoft.NET\Framework\v4.0.30319
This path and compiler file works fine but every time I have written like this and some times I have seen an error related to Library.
C:\Windows\Microsoft.NET\Framework\v4.0.30319\csc.exe sample.cs
How to avoid writing long path each time while testing the sample codes?
Please provide best available alternate(Batch file or something)
Edit 1: Setting Path:
Paul's answer is works fine with simple program, But the issue with that is if there are any Library files used/added, were not found in this case.
Note: Version is not a constraint actually, we may use any .net redistributable version. Just CSC.exe should accessible from any path without any issue(example library files used).
if you are using a batch file add to the start
set PATH=C:\Windows\Microsoft.NET\Framework\v4.0.30319\;%PATH%
This will add the .net framework to the start of your path for the current run of the batch file only.
now your batch file can just use
csc.exe sample.cs
Got the working answer from one of the forums, for the complete solution to build with reference added.
We need to use /r option with compilation.
First add path using command below.
Example:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\csc.exe /t:exe /out:sample.exe sample.cs /r:ReferenceName.dll
After adding Path:
C:\test>csc /t:exe /out:sample.exe sample.cs /r:ReferenceName.dll
This solved my problem with reference.
Note: if reference is not in same directory as CS file it need to be added with its path.