I've been looking at similar questions on different websites in the hope I could find an answer for it but I've not come across something which has worked for me.
I have a Visual Studio 2019 solution (.Net Framework) which contains Specflow related tests. I can see all my tests in the Test Explorer but when I try to run any of them, I am returned with the error 'Unit test provider already specified'. My test runner was initially Specflow.SpecRun but I have since removed all references to this and am now using Specflow.MsTest - this is to allow me to associate my test cases to one in Azure DevOps. Only after making this change, I am getting these errors.
I have searched across my solution and there are no longer any references to SpecRun; there are no build errors either. The 'Enable SpecflowSingleFileGenerator' setting is set to False, I do have an app.config file but do not have a specflow.json file - I read that that is only needed for .NET Core projects.
Has anyone else come across the above or have any ideas to resolving it as I am a bit stuck.
I was having this problem when I moved from nUnit to MsTest. I want to add few more steps to comments mentioned
Check your feature.cs files and make sure it has onlyMicrosoft.VisualStudio.TestTools.UnitTesting attributes and not nUnit or specrun
Check whether you are using any local.testsettings file in visual studio
Remove nUnit from tools and extensions if you are not using it.
Restart VS, clear temp folder, clean and rebuild
So this may not be the most ideal solution but I created a new project within the same solution, installed the same nuget packages and copied all the files/folders across to the new project. I removed the previous project, cleaned solution and re-built. Tests appear in the Test Explorer and also run successfully too.
My new workplace has a very large C# solution in Visual Studio 2017 with over 100 projects, of which only one is directly relevant to my job (web acceptance tests via specflow & selenium).
We have intermittent problems with CPU usage & tests not being found in the test explorer, occurring whenever Specflow decides it needs to search for and regenerate test bindings - as it takes some hours to complete, during which Visual Studio is not useful.
From experimentation, I know that if I open only the one project relevant to my work, then specflow finds all the tests in seconds and is immediately useable.
But sadly we need the whole solution to be loaded at least when building new code to be tested.
Is there a way to limit which projects (or paths) Specflow and/or test explorer is searching in for tests, & subsequently generating step bindings from?
I’m hoping for a local settings file (references to .runsettings), but haven’t been able to find any instructions for how to configure this.
There isn't such an option available.
This looks to me we have to improve the VS Extension too handle that many projects.
Could you please open an issue here: https://github.com/techtalk/SpecFlow
There we can discuss, what we can do, to make SpecFlow for you usable.
I am trying out the new Visual Studio 2015 CTP 6 and the first thing I notice was that for new ASP.NET 5 project there isn't an option to create a Test project (it's disable)... Now, VS2015 introduced Smart Unit Tests but it is not suitable for TDD, the idea is to generate unit tests for already existing code.
Probably exists a nuget-package to implement TDD but what I find weird, I believe, it's not longer come out of the box.
So, how implement TDD in the new Visual Studio 2015 CTP6?
Update 3/22/2015
Here some links related to this unresolved issue:
Getting Started with xUnit.net and ASP.NET 5
How can I run xUnit Unit Tests with VS2015 Preview?
Great question. Here is what I settled on. I wouldn't call it ideal and maybe there is a better way that I am not aware of.
1) Use xUnit - if you look at the aspnet repository on github, you will see most if not all the tests written with xUnit.
2) Visual Studio IDE support for xUnit is not yet available for ASPNET5 based tests. You must run them from the command prompt. This link discusses how to set up and run unit tests for ASPNET5 http://xunit.github.io/docs/getting-started-aspnet.html
3) Complicating matters, the just released version of xUnit does not support ASPNET5, but RC3 does and it is still available on Nuget. Per this tweet https://twitter.com/xunit/status/574692021123731456 ASPNET5 support will return in xUnit 2.1, a version of which should be released soon.
4) Even after following all the instructions, I had a problem that I believe was related to storing my project on a UNC path instead of a path with a drive letter, so don't let that bite you. https://github.com/xunit/xunit/issues/295
When trying to run NUnit tests with ReSharper, the following message appears in a dialog box and the tests are not run:
The build could not be started. There are multiple projects in the solution at... <name of the csproj file>
I have this problem with Resharper 8.2.1
I have several projects in my solution. Most of the projects are class libraries. One of the projects is a Web Site (not project exactly). Another one is a Web Application. The Web Site and the Web Application projects are located in the same directory. The Web Site is set to be built only in the Release build.
I have this problem only in The Web Application project, when I try to run some embedded tests in this project. I do not have this problem in other (class library) projects.
The problem was in JetBrains' bug tracker, but was closed: https://youtrack.jetbrains.com/issue/RSRP-328627
Perhaps there some workaround?
Update:
When I "Project -> Unload Project" on the Web Site, the problem disappears
I had the same problem (also with 8.2), at least the same error when running my tests through the menu Resharper > Unit Tests > Run All Tests from Solution. But I found the following work around :
Right click on your unit testing project and select Run Unit Tests. this will start running all the tests from the project (from the resharper interface)
Alternatively you can set up you unit testing solution as start up project and from the Resharper menu choose > Unit Tests > Run Unit Tests
Of course you have to do it for each project that has unit tests, but I guess this is acceptable.
Remark: it seems to be a known bug of version 8.2 https://youtrack.jetbrains.com/issue/RSRP-411631
And that rolling back to 8.1 can fix the problem
Hope this helps.
In my case this happened after running performance analysis (i.e. profiling) in Visual Studio.
As documented here, the solution was to remove these lines from the .sln file:
GlobalSection(Performance) = preSolution
HasPerformanceSessions = true
EndGlobalSection}
Check whether you have any changes to your solution file. I encountered this problem after a source control branch change, when Visual Studio held on to a bunch of changes from the previous branch. Not sure exactly what the cause was, but reverting the changes to my sln file fixed the issue.
I have a simple solution in visual studio 2013 that is composed by one web project, one library project and one unit test project. When I open the solution and try to run the unit tests they are not discover by visual studio. To run the tests I try to go to the menu and choose Test -> Run -> Run all tests or by opening the test explorer window. By those to methods visual studio doesn’t discover any tests in the solution.
Creating first a simple unit tests project and try to run the test, visual studio know discover the test and I can run it. Then, if I open my previous solution visual studio now discovers all the tests. I try to save my solution but closing it and reopening, without creating a unit test project first, the visual studio doesn’t find the tests again. This is a very strange behave that I don’t know why this is happening.
I used to working alone in this project that was using the source control git integrated with the visual studio team foundation. The problem of visual studio not discover the unit tests start when a new element came to the project and when I need to recreate the solution through the source control online. Before this, all tests always been discovered by visual studio.
For creation the unit tests I use the dll Microsoft.VisualStudio.QualityTools.UnitTestFramework. My version of visual studio is: Microsoft Visual Studio Express 2013 for Web Version 12.0.30723.00 Update 3. My version of .net framework is 4.5.50938.
All of my tests are like this:
[TestClass]
public class Service1Test
{
[TestMethod]
public void Test1()
{
Assert.IsTrue(True);
}
}
Some things I've noticed I have to do from time to time to get tests to show up properly.
If your solution is in a protected drive that you need administrator access to read/write, sometimes only a portion of the tests come up. Definitely run VS as administrator in that case.
If your solution is 64 bit, make sure that Test > Test Settings > Default Processor Architecture is set to x64. Sometimes it gets set to x86. Set it to x64, then rebuild.
Sometimes just restarting Visual Studio does the trick because the test explorer will start up again.
Don't forget to actually build the test project/solution. (If you want it to get built with the rest of the projects, right-click on your solution > Properties > Configuration Properties > Configuration > check the "Build" box for your test project)
Ensure the tests are in a public section of your test class
If you using NUnit, make sure to download NUnit Adapter first.
Go to Tools → Extensions and Updates… → Online → search for "NUnit Test Adapter".
Make sure your test class is public so it can be found. And if you're referencing another class, make sure of the same.
Also, sometimes if you have no Asserts or you're not decorating the test with a [TestMethod], a test might not be recognized.
2 more things:
1) Async unit tests act funny at best, and none at all at worst. Have a look at this article by Stephen Cleary and keep from there if it interests you.
2) If you use NUnit and you run into the same issues, keep in mind it's [TestCase] for Nunit, instead of [TestMethod]
Having said the above, here's an article I've posted on the code project, with both MSTest & NUnit, in case you want to give it a spin and make sure you're not missing anything.
I had the same issue but none of the other solutions worked. Turns out that I was using the NUnit 3 framework with the 2 adapter.
If you're using NUnit 3, go to Extensions and Updates and install the NUnit3 Test Adapter.
I'm having this issue from time to time. What works for me is to shutdown Visual Studio and go to folder:
%LocalAppData%\Microsoft\VisualStudio\12.0\ComponentModelCache
and delete it content.
Once you open Visual Studio and load your project again Test Explorer should contain back your tests
XUnit users may notice Test Explorer window no longer lists any tests. To make tests discoverable again try this important tip, highlighted below.
If you're having problems discovering or running tests, you may be a victim of a corrupted runner cache inside Visual Studio. To clear this cache, shut down all instances of Visual Studio, then delete the folder %TEMP%\VisualStudioTestExplorerExtensions. Also make sure your project is only linked against a single version of the Visual Studio runner NuGet package (xunit.runner.visualstudio).
For future googlers I had a rare scenario that caused this.
On my base test class I had a property named TestContext. This interfered with MSTest's reserved TestContext property causing all my tests to be hidden from VS/Resharper except one (which did not inherit from the base).
for me it was changing 'solution configurations' to Debug (instead of Release).
My problem was because my unit test method wasn't void, and it was receiving parameters.
I have found that unit test methods marked as async void aren't discovered by the VS Test Explorer. This seems to be because VS would not have any way to wait for a test to finish and decide if it succeeded or not. If you absolutely need to have a test method to run asynchronously then get it to return a Task instead like async Task.
I found that this fixed the issue for me.
Try building all projects as MSIL (Any CPU) instead of x86/x64. Worked for me strangely
While AndyG's solution works, a more lasting solution could be to set the PreferredToolArchitecture environment variable to "x64", either by:
How to make Visual Studio use the native amd64 toolchain
or by:
Control Panel | System and Security | System | Advanced System Settings|Environment Variables
PreferredToolArchitecture = x64
DefaultToolArchitecture = Native64Bit
PROCESSOR_ARCHITECTURE = x64
ProcessorArchitecture = x64
I was facing the same problem and I've remembered, again (this situation happened before), that selecting "Mixed Platform" on the solutions platform menu works, as well as the other answers.
I'd managed to add mine as
public static void TestMethod1(){}
started working once I removed static....
Go to Nuget package manager and download Nunit Adapter as follow.
go to the project menu > Configuration Manager check your test project platform matches the rest of the project and is check to build then rebuild.
Just ran into this as well as I didn't see a similar case that was similar to mine.
In the .csproj file of my test project, NUnit reference privacy was set to False:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<HintPath>..\packages\NUnit.2.6.4\lib\nunit.framework.dll</HintPath>
<Private>False</Private>
</Reference>
After I set <Private> to True it worked.
You just need to install this package only:
NUnit TestAdapter
To get tests to show in the Test Explorer Window I had to install NUnit3 Test Adapter 3.0 which was not available in Package Manager.
Downloaded from https://visualstudiogallery.msdn.microsoft.com/0da0f6bd-9bb6-4ae3-87a8-537788622f2d
I had the exact same problem.
It was due to incompatible version of NUnit I'd added to my project (3.2.0) and the Test Adapter I had installed (2.0.0).
To fix, use "Tools > Extensions and Updates" and search for NUnit3 Test Adapter, it discovered my tests after that.
Cheers
Let's just say for argument's sake that you need to use X64 architecture on your test project in order for the dependencies to build properly (as in my case). You may need to modify your Default Processor Architecture under the Test--Test Settings menu. Setting this to X64 allowed my test explorer to find my tests using Microsoft.VisualStudio.TestTools.UnitTesting.
Sorry for adding to the long list, but I had a completely a different problem.
First, I would like to mention I discovered my issue when clicking 'Run All' in the Test Explorer and then watching the build output window in Visual Studio. You have to actively watch it, as afterwards the message disappears.
As for the issue, it looks like during the scanning of the tests, the DLL gets loaded and its test types are enumerated. This causes the references to be loaded and if any failure occurs during this process, the tests will not be shown in the explorer. I had two issues preventing the test DLL to be successfully loaded:
There was still a binding redirect left in the config file (redirecting to a version lower version NHiberate than what was referenced in the test project).
A conflicting assembly reference (2nd level references not being able to load). AsmSpy is btw a great tool to hunt for these.
If you load a Visual Studio (VS 2015 Community in my case) solution from a network share or My Documents directory that is part of a share, you will get into this trouble. I solved it by moving the solution and its underlying projects to a local folder.
After spending 2 days... none of the above worked for me. The only "solution" was: Go to project properties -> Build Tab. Then click Advanced button on the right bottom corner of the pane. Change "Debug Info:" to "full" and click OK.
Here are the screen shots:
I came across the same issue.
And investigated and found that the dll's were not build,
and put in the right folder. as soon as I changed my configuration
they appeared.
- the Projects build options, what folder should be used?
- the build menu entry build configuration, they should be checked.
that fixed it for me.
For the Visual Studio 2013.5, clearing the \TestResults directory in the solution helped. Visual Studio corrupted the mdf file in which it stores the discovered tests, thus preventing the discovery of unit tests.
Be sure your all projects are runing with the same configuration.
Under your project's Properties => Debug => Platform in drop down list choose the appropriate platform (for me it was "Any CPU") as determined at your other projects.
I know that the unit tests are not found if the solution is not built, so that is something to try (Build the solution), but that solution is like the help desk asking if your computer is plugged in...
After a clean rebuild did not fix the problem for me, running a full batch build did fix it.
Had the same issue; tests suddenly stopped being discovered.
Nunit Test Adapter had become disabled somehow. Clicking enable in the extension manager fixed it for me.
I had the same problem until I realized I made a cut/paste error and left off [Test Method] before the test.