I have a .NET 4.8 project with Specflow tests. These are not found and displayed by VS 2019, so I can't execute them. I looked into the log files and found out that VS is trying to execute the tests with testhost.x86.exe although I have set everywhere AnyCPU and Auto as processor architecture (which is enough for other projects). I then set the processor architecture to x64 and now VS is using testhost.exe but still not finding anything. There are no errors in the logs except failing to load Microsoft.VisualStudio.Coverage.Interop extension. What could be the reason for not seeing anything in the test explorer? Or where could I eventually find some more information?
I use the same Specflow (3.1) and Specflow Runner (3.2) versions in other projects and there it works, so it shouldn't be the source of the problem. Also in the project file and elsewhere in the project I couldn't find anything special. All feature files have been generated without problems.
Not sure if this will fix your issue, but check if you have your Test Explorer set to the same processor architecture as your project.
I had some issues with this in the past.
Full disclosure: I am one of the developers of SpecFlow and SpecFlow+
I switched to an older Specflow version (3.0) and it worked. No idea what the problem was but I had it with multiple old projects migrated from Specflow 2 to 3.
I've been looking at similar questions on different websites in the hope I could find an answer for it but I've not come across something which has worked for me.
I have a Visual Studio 2019 solution (.Net Framework) which contains Specflow related tests. I can see all my tests in the Test Explorer but when I try to run any of them, I am returned with the error 'Unit test provider already specified'. My test runner was initially Specflow.SpecRun but I have since removed all references to this and am now using Specflow.MsTest - this is to allow me to associate my test cases to one in Azure DevOps. Only after making this change, I am getting these errors.
I have searched across my solution and there are no longer any references to SpecRun; there are no build errors either. The 'Enable SpecflowSingleFileGenerator' setting is set to False, I do have an app.config file but do not have a specflow.json file - I read that that is only needed for .NET Core projects.
Has anyone else come across the above or have any ideas to resolving it as I am a bit stuck.
I was having this problem when I moved from nUnit to MsTest. I want to add few more steps to comments mentioned
Check your feature.cs files and make sure it has onlyMicrosoft.VisualStudio.TestTools.UnitTesting attributes and not nUnit or specrun
Check whether you are using any local.testsettings file in visual studio
Remove nUnit from tools and extensions if you are not using it.
Restart VS, clear temp folder, clean and rebuild
So this may not be the most ideal solution but I created a new project within the same solution, installed the same nuget packages and copied all the files/folders across to the new project. I removed the previous project, cleaned solution and re-built. Tests appear in the Test Explorer and also run successfully too.
My problem is very similar to NUnit Unit tests not showing in Test Explorer with Test Adapter installed, in that I couldn't get the NUnit tests to appear in the Test Explorer (despite multiple rebuilds, setting the Test architecture to x64, etc). I rebuilt and rebuilt, and despite having NUnit 3 and the NUnit 3 Test Runner installed into my Solution (via Tools -> NuGet Package Manager -> Manage NuGet Packages For Solution...) I couldn't get the tests to appear.
On the advice of another response to the above question, I installed the NUnit 3 Test Runner as an Extension (which modified all of Visual Studio, not just my project).
After the extension is installed (and I do Yet Another Rebuild :) ) all my tests are listed in the Test Explorer.
Question: Why do are the tests shown in the Test Explorer for the Visual Studio-wide NUnit 3 Test Runner Extension, but NOT in the Solution-specific NUnit 3 Test Runner NuGet package?
What's even weirder is that the exact same project, with the per-Solution NuGet packages for NUnit, work just fine on my personal computer (WITHOUT the NUnit Extensions installed), using Visual Studio 2017 Community edition. It's only when I .ZIP the project and email it to another computer (which runs Visual Studio 2017 Enterprise) where I have this problem.
I'd prefer to install NUnit into the solution so that I can hand out copies to my students and have the project be completely self-contained.
Does anyone have any advice on how to further dig into the problem with the per-Solution NuGet package? I'm kinda stymied since the only obvious thing to try is "rebuild it again and see if that helps" :), so any thoughts on stuff to try / strategies to use would be great.
At this point it looks like the Virtual Machine that I'm running the tests in thinks that the project is being run from a network drive. .Net has different rules for running code remotely and by default doesn't want to execute remote code (which seems reasonable :) ).
It's worth pointing out that NUnit should (and does!) work just fine in a VM where the project is saved onto a virtual disk - it's just the 'network drive'/file share that's causing the problem.
I'm using Selenium with C# to create my automated tests. Now i have to use some sort of reporting tools to save the test cases whether they passed,failed...etc.
I've seen a lot of tools like Allure, and Jenkins. But they require an XML file. and i can't find it when i run my tests using Visual 2013. Why is that ?
how to obtain these XML files in details if i'm missing something ?
Is there any way i could achieve this with minimum effort ?
EDIT:
How can i run my tests using Nunit console runner ? where can i find it? i downloaded the .zip for nunit but i couldn't find the runner?
The NUnit Visual Studio Adapter does not currently produce XML results, although there is an issue on GitHub to add the ability. Your best bet is to run your tests using the NUnit Console runner when you want to create reports. It always produces an XML result file.
You can install the NUnit Console by adding the NUnit.Console NuGet package to your test project. The console will be in a directory like packages\NUnit.ConsoleRunner.3.4.0\tools in your solution root.
Another option is to install the MSI for the console runner. It will then be under C:\Program Files (x86)\NUnit.org\nunit-console
For creating reports, ReportUnit creates excellent HTML reports from your test results.
To run the tests, I would recommend setting up a simple command line build that builds your solution, runs the tests and then produces the report. For day to day development, just running your tests in Visual Studio or on the command line will likely be enough. Most developers find the Visual Studio Test Explorer Window to be a poor UI, but usable for seeing passed and failed tests and running/debugging them.
If you want to setup a command line build, one good tool is Cake Build. It will take a bit of time to setup, but it is an excellent way to run your build tasks as your project gets larger.
NUnit testing can be run with the nunit-console.exe application which is installed with nunit under {Project_root}/lib/nunit/nunit-console.exe. It downloads with NuGet when NUnit installs.
It can be passed a list of testing binaries, or testing project files, or an nunit project (listing multiple if needed).
{PathToProject}\lib\nunit\nunit-console.exe {PathToTestDll}\Project1.Tests.dll {PathToTestDll}\Project2.Tests.dll /xml=nunit-result.xml
or create an NUnit Project with the NUnit Project Editor if you want to group all your test projects into a single config file.
{PathToProject}\lib\nunit\nunit-console.exe {PathToNUnitProject}\Project.Tests.nunit /xml=nunit-result.xml
Allure doesn't require xml. At lease now :)
https://github.com/unickq/allure-nunit
You just use [AllureNUnit] attribure
I have a simple solution in visual studio 2013 that is composed by one web project, one library project and one unit test project. When I open the solution and try to run the unit tests they are not discover by visual studio. To run the tests I try to go to the menu and choose Test -> Run -> Run all tests or by opening the test explorer window. By those to methods visual studio doesn’t discover any tests in the solution.
Creating first a simple unit tests project and try to run the test, visual studio know discover the test and I can run it. Then, if I open my previous solution visual studio now discovers all the tests. I try to save my solution but closing it and reopening, without creating a unit test project first, the visual studio doesn’t find the tests again. This is a very strange behave that I don’t know why this is happening.
I used to working alone in this project that was using the source control git integrated with the visual studio team foundation. The problem of visual studio not discover the unit tests start when a new element came to the project and when I need to recreate the solution through the source control online. Before this, all tests always been discovered by visual studio.
For creation the unit tests I use the dll Microsoft.VisualStudio.QualityTools.UnitTestFramework. My version of visual studio is: Microsoft Visual Studio Express 2013 for Web Version 12.0.30723.00 Update 3. My version of .net framework is 4.5.50938.
All of my tests are like this:
[TestClass]
public class Service1Test
{
[TestMethod]
public void Test1()
{
Assert.IsTrue(True);
}
}
Some things I've noticed I have to do from time to time to get tests to show up properly.
If your solution is in a protected drive that you need administrator access to read/write, sometimes only a portion of the tests come up. Definitely run VS as administrator in that case.
If your solution is 64 bit, make sure that Test > Test Settings > Default Processor Architecture is set to x64. Sometimes it gets set to x86. Set it to x64, then rebuild.
Sometimes just restarting Visual Studio does the trick because the test explorer will start up again.
Don't forget to actually build the test project/solution. (If you want it to get built with the rest of the projects, right-click on your solution > Properties > Configuration Properties > Configuration > check the "Build" box for your test project)
Ensure the tests are in a public section of your test class
If you using NUnit, make sure to download NUnit Adapter first.
Go to Tools → Extensions and Updates… → Online → search for "NUnit Test Adapter".
Make sure your test class is public so it can be found. And if you're referencing another class, make sure of the same.
Also, sometimes if you have no Asserts or you're not decorating the test with a [TestMethod], a test might not be recognized.
2 more things:
1) Async unit tests act funny at best, and none at all at worst. Have a look at this article by Stephen Cleary and keep from there if it interests you.
2) If you use NUnit and you run into the same issues, keep in mind it's [TestCase] for Nunit, instead of [TestMethod]
Having said the above, here's an article I've posted on the code project, with both MSTest & NUnit, in case you want to give it a spin and make sure you're not missing anything.
I had the same issue but none of the other solutions worked. Turns out that I was using the NUnit 3 framework with the 2 adapter.
If you're using NUnit 3, go to Extensions and Updates and install the NUnit3 Test Adapter.
I'm having this issue from time to time. What works for me is to shutdown Visual Studio and go to folder:
%LocalAppData%\Microsoft\VisualStudio\12.0\ComponentModelCache
and delete it content.
Once you open Visual Studio and load your project again Test Explorer should contain back your tests
XUnit users may notice Test Explorer window no longer lists any tests. To make tests discoverable again try this important tip, highlighted below.
If you're having problems discovering or running tests, you may be a victim of a corrupted runner cache inside Visual Studio. To clear this cache, shut down all instances of Visual Studio, then delete the folder %TEMP%\VisualStudioTestExplorerExtensions. Also make sure your project is only linked against a single version of the Visual Studio runner NuGet package (xunit.runner.visualstudio).
For future googlers I had a rare scenario that caused this.
On my base test class I had a property named TestContext. This interfered with MSTest's reserved TestContext property causing all my tests to be hidden from VS/Resharper except one (which did not inherit from the base).
for me it was changing 'solution configurations' to Debug (instead of Release).
My problem was because my unit test method wasn't void, and it was receiving parameters.
I have found that unit test methods marked as async void aren't discovered by the VS Test Explorer. This seems to be because VS would not have any way to wait for a test to finish and decide if it succeeded or not. If you absolutely need to have a test method to run asynchronously then get it to return a Task instead like async Task.
I found that this fixed the issue for me.
Try building all projects as MSIL (Any CPU) instead of x86/x64. Worked for me strangely
While AndyG's solution works, a more lasting solution could be to set the PreferredToolArchitecture environment variable to "x64", either by:
How to make Visual Studio use the native amd64 toolchain
or by:
Control Panel | System and Security | System | Advanced System Settings|Environment Variables
PreferredToolArchitecture = x64
DefaultToolArchitecture = Native64Bit
PROCESSOR_ARCHITECTURE = x64
ProcessorArchitecture = x64
I was facing the same problem and I've remembered, again (this situation happened before), that selecting "Mixed Platform" on the solutions platform menu works, as well as the other answers.
I'd managed to add mine as
public static void TestMethod1(){}
started working once I removed static....
Go to Nuget package manager and download Nunit Adapter as follow.
go to the project menu > Configuration Manager check your test project platform matches the rest of the project and is check to build then rebuild.
Just ran into this as well as I didn't see a similar case that was similar to mine.
In the .csproj file of my test project, NUnit reference privacy was set to False:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<HintPath>..\packages\NUnit.2.6.4\lib\nunit.framework.dll</HintPath>
<Private>False</Private>
</Reference>
After I set <Private> to True it worked.
You just need to install this package only:
NUnit TestAdapter
To get tests to show in the Test Explorer Window I had to install NUnit3 Test Adapter 3.0 which was not available in Package Manager.
Downloaded from https://visualstudiogallery.msdn.microsoft.com/0da0f6bd-9bb6-4ae3-87a8-537788622f2d
I had the exact same problem.
It was due to incompatible version of NUnit I'd added to my project (3.2.0) and the Test Adapter I had installed (2.0.0).
To fix, use "Tools > Extensions and Updates" and search for NUnit3 Test Adapter, it discovered my tests after that.
Cheers
Let's just say for argument's sake that you need to use X64 architecture on your test project in order for the dependencies to build properly (as in my case). You may need to modify your Default Processor Architecture under the Test--Test Settings menu. Setting this to X64 allowed my test explorer to find my tests using Microsoft.VisualStudio.TestTools.UnitTesting.
Sorry for adding to the long list, but I had a completely a different problem.
First, I would like to mention I discovered my issue when clicking 'Run All' in the Test Explorer and then watching the build output window in Visual Studio. You have to actively watch it, as afterwards the message disappears.
As for the issue, it looks like during the scanning of the tests, the DLL gets loaded and its test types are enumerated. This causes the references to be loaded and if any failure occurs during this process, the tests will not be shown in the explorer. I had two issues preventing the test DLL to be successfully loaded:
There was still a binding redirect left in the config file (redirecting to a version lower version NHiberate than what was referenced in the test project).
A conflicting assembly reference (2nd level references not being able to load). AsmSpy is btw a great tool to hunt for these.
If you load a Visual Studio (VS 2015 Community in my case) solution from a network share or My Documents directory that is part of a share, you will get into this trouble. I solved it by moving the solution and its underlying projects to a local folder.
After spending 2 days... none of the above worked for me. The only "solution" was: Go to project properties -> Build Tab. Then click Advanced button on the right bottom corner of the pane. Change "Debug Info:" to "full" and click OK.
Here are the screen shots:
I came across the same issue.
And investigated and found that the dll's were not build,
and put in the right folder. as soon as I changed my configuration
they appeared.
- the Projects build options, what folder should be used?
- the build menu entry build configuration, they should be checked.
that fixed it for me.
For the Visual Studio 2013.5, clearing the \TestResults directory in the solution helped. Visual Studio corrupted the mdf file in which it stores the discovered tests, thus preventing the discovery of unit tests.
Be sure your all projects are runing with the same configuration.
Under your project's Properties => Debug => Platform in drop down list choose the appropriate platform (for me it was "Any CPU") as determined at your other projects.
I know that the unit tests are not found if the solution is not built, so that is something to try (Build the solution), but that solution is like the help desk asking if your computer is plugged in...
After a clean rebuild did not fix the problem for me, running a full batch build did fix it.
Had the same issue; tests suddenly stopped being discovered.
Nunit Test Adapter had become disabled somehow. Clicking enable in the extension manager fixed it for me.
I had the same problem until I realized I made a cut/paste error and left off [Test Method] before the test.