Specflow:Error:Result collection has not started in test explorer - c#

Working with specflow latest version, When I run my scripts I get result set cannot be started and been at this for a week. the automation script run to completion but it bring up this error and test fails .The specflow version 3.1.97 is and visual studio is 2019 . I have narrowed it down to a simple test and still fails , it seems to have something to do with specflow. I do not want to have to create a new solution and start again and I have wrote a lot of tests but cannot go any further as reports will always fail due to this error. I have checked online . I have tried different solution including https://github.com/SpecFlowOSS/SpecFlow/issues/1851 etc but to no avail.I have gone down version of specflow, specfow.tools.MsBuild.Generation ,nunit but still the same. If I create a new solution and add these dependencies it seems to be fine and do a quick test is works fine.
Feature file:
#Settings About
Scenario: About Screen
Given user is already logged in
Step definition:
[Given(#"user is already logged in")]
public void GivenUserIsAlreadyLoggedIn()
{
Assert.IsTrue(true);
}

I received this error message after moving my feature file into a shared assembly. The reason for the error was that the static initialization methods generated by SpecFlow.MSBuild was not being called at startup. I fixed by chaining the call from the corresponding initialization method in the top level test assembly.
Although the reason for your problem is probably different, I suspect you can diagnose the problem using the same process.
In the obj folder of your test project there should be a file called MsTest.AssemblyHooks.cs. Move this file into the main project folder and include in the project.
Add the following xml into the csproj project file. (This turns off automatic hooks file generation).
<PropertyGroup>
<GenerateSpecFlowAssemblyHooksFile>false</GenerateSpecFlowAssemblyHooksFile>
</PropertyGroup>
You can then put a break point on the initialization code and debug. (You may be getting exceptions, or the initialization may not be being called.)
Note - you may be able to debug the initialization code in place without moving. In my case I needed to modify the code, hence a permanent non-generated version.
For anyone else trying to split the feature file into a shared project, you also need to update app.config with
<configSections>
<section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow">
<configSections>
<specFlow>
<stepAssemblies>
<stepAssembly assembly="[TODO - SharedAssemblyName]" />
<stepAssemblies>
</specFlow>

I had the same problem. However after a pure uninstall all Specflow related packages, then reinstall everything, it works fine? Go figure~ I'm testing 3.1.97 now. Once stable I'll try to 3.3.
Get rid of everything in your solution that uses or requires Specflow, then reinstall each package from scratch.

Related

How can I ignore obsolete type errors when running Live Unit Testing in VS 2022 enterprise?

I have a solution with about 58 projects, including 10 test projects. There's only one I'm interested in running, that relates to my team's work.
I can run the test project manually from the Test Explorer, and it builds with all tests passing fine.
However, when I set up Live Unit Testing, the build fails due to obsolete types being used in some another projects. I don't want to replace all the type references since that code is managed by other teams.
Here are my VS2022 LUT settings.
I have only added the test project I'm interested in to the LUT playlist. I also tried replacing some of the obsolete types (I only had 3 errors initially) but the compiler then found dozens more...
I increased the logging level on LUT to verbose, but that didn't give anything useful. There aren't any other settings I can see for LUT to ignore certain projects or errors.
I have looked up the MS documentation on the [obsolete] attribute, but I'm not sure why I get the error on LUT and not on regular test or debug builds.
The .csproj of a project that fails has this line: <TreatWarningsAsErrors>True</TreatWarningsAsErrors> and setting it to False allows LUT to build it. But I don't want to have to change dozens of .csprojs just for running some tests.
Is there any other way to mark errors as warnings for live unit testing?
Instead of setting <TreatWarningsAsErrors>false</TreatWarningsAsErrors> you can add <NoWarn>CS0618</NoWarn> to the project files. That's probably going to help everybody, because others will have the same problem as well.
You can also add a section such as the following to your Directory.Build.props file (should be in the root of your project structure):
<PropertyGroup Condition="$(MSBuildProjectName.EndsWith('UnitTest')) AND '$(MSBuildProjectExtension)' == '.csproj'">
<NoWarn>$(NoWarn);CS0618</NoWarn>
</PropertyGroup>
This will disable the warning for all projects whose name ends with "UnitTest".

How to disable parallelization of xunit assemblies

I have a solution written on c#, .netcore 2.2 and the testing framework is xunit that looks pretty much like this:
-src
--controllers
--services
-test
--controllers.integrationtest
--services.integratiotest
we are performing some integration test with two distinct dlls (controllers.integrationtest.dll & services.integrationtest) that use the same database (solutionname.test)
now, when I run "dotnet test" on the solution, it seems that the two dlls are trying to access the same resource (database) and one dll gets a deadlock.
The thing is that I want to disable the parallel execution of the dlls when I run "dotnet test" to avoid that deadlock, so I search that up and the documentation says that to disable parallel execution of dlls you have to:
Add a xunit.runner.json in the root of the csproject which I did and works fine. (this is working on visual studio because I tested it with another feature that deletes the "_" character of test names: [methodDisplay])
Configure xunit.runner.json file to copy always or preserve newest in visual studio so that gets copied on bin folder (as any appsetings.json file)
I've read that you can place an assembly attribute in the assemblyInfo.cs file that by the way it seems that was replaced by the plain csproj so I'm a bit confused.
The ultimate goal that I want to achieve is that when devops use dotnet test the integration test doesn't blow up on concurrency problems.
In xUnit the default value for ParallelizeAssemblies is false so I can only assume that a behavior of dotnet test is causing both to be executed together.
Try and run the tests using dotnet vstest. This requires the DLLS to have been built.
dotnet vstest **/*test.dll

How do I build code out of a tutorial repo that has many examples (in Visual Studio/C#)

How do I build sample code, split into folders in a repo, from a class or tutorial, in Visual Studio?
So - I'm pretty much a noob at C#, I've gone through a lot of tutorials and browsed through some large C# projects from work and built them, and done some other minor things. I'm going through a course on writing testable code on Pluralsight. He has a public Github repo for the code examples, writing-testable-code. I connected to the repo and downloaded it okay into a local Git repo. I was able to download all the packages from NuGet and they are all showing as the version he used (a few have updates, but I figured updating might break things).
I can't figure out how to run this code, build it, or run the tests in it.
What I tried so far
My issue is - I open the solution, and there are a bunch of files and folders - each module/chapter is split into folders (i.e. Module1/Easy, Module1/Hard, Module2/Easy, etc.). I want to build the Module1/Easy folder, including unit test examples, and run the tests.
When reviewing Module1/Easy, it has 3 files that should build okay - the program.cs has a main() and looks like a console app, the Calculator.cs has a simple class, and the CalculatorTests.cs has unit tests built for Nunit. The solution has NUnit, Castle.core, and things from later modules (Moq, AutoMoq, Unity, Ninject, etc.). It didn't seem to have a VS runner, so I added Nunit3TestAdapter - the guy in the course has resharper installed, which I don't, and he was using the Resharper test runner, which would explain why he didn't include it.
I tried setting the "Module1/Easy/Project.cs" file the "Set as Startup Item", since it has a main and looks setup as a console app. However, running it (the "Start" button turned into a "Program.cs" button), it fails saying it can't run a dll. The tests aren't showing up in the Test Explorer like some other small projects I've built from examples.
What's the right way to do this?
I'm not sure where to go from here. On the Build menu is only a "Build Solution" and one about Code Analysis - I'm used to a lot more options here. It feels like I have to turn this folder into a project, maybe? I can always reinstall the packages - but what is the best solution here?
I've run into this before on other book, tutorial, or class repos, but finally decided to figure out how to get this one working. I appreciate any help!
Notes
I'm running Visual Studio Community 2017 at the moment.
I can post some of the files, but the repo is publically available, and not sure exactly what to post to help.
Progress from comments and answers
Per Biker-Dude's answer, I switched the project to build a console app rather than a dll, and now I get a compile-time error for having multiple entry points (i.e. every module and sub folder has its own Main() function and should probably be a separate project).
After #1, I removed all folders but one from the solution, it will then compile, run the tests, etc. - but I eventually want to be able to at least separately compile every sub-folder - what's the best way?
The problem must be that the project must have the output type set to class libraries. Browse through the solution tree and:
Select your class's project> right click > Properties > Application >
Output Type > Console Application/ Windows Application.
This should fix it, if the other things are set up properly.
With the help of BikerDude's answer and stijn's comments, I was eventually able to play around with this and get some things working.
First of all, don't try to exclude any folders in this situation, that will just make things worse! They will still be in your underlying folder/repo, just won't be showing up in your solution anywhere, and you won't be able to create a new folder with the same name (weird decision...). And you'll have to add them back in as individual files - I think.
The best solution (so far)
The best solution seems to be:
Create a new project for each buildable set of files in the solution (in my case, at least one project per "module" folder). I used the ".Net framework console app" project type (right click on the solution, use Add/New Project) to get things to work, but this would depend on the particular course or tutorial repo you downloaded.
Move the folder or sub-folder that has the files you want to build out of the main solution and into the new project - you can click and drag to move it.
Visual Studio will make an empty, pre-formatted file in your project that you likely have to delete - for .Net apps, this is the "Program.cs" file in C#. For one of my folders this file already existed, and I had to delete the new one in order to build. Another folder from a different module was setup more like a library and couldn't build standalone, but this procedure did get me to being able to build the files and that allowed the unit tests to show up in the test explorer and run the tests successfully (which was the main point of that module).
Go to the solution and right-click and choose "Manage Nuget Packages for Solution". As long as all the packages are installed for the main solution, then they will all show up in the list of Installed Packages (you might need to click on the "Installed" tab). You can click on each package in turn, then on the right you can checkmark the new project, and the "Install" button should be available - click it. Repeat for all the packages to install them all. Note that you can cut out some repetition here if you create all the projects you need first, then you can install all of them at the same time in this step (i.e. checkmark all the new projects at once instead of reopening the package manager each time).
You might have to fix the NameSpace - it should be consistent within the files/folders you transferred from the original solution, but if you add any new files to play with things, the Namespace for it will likely not match, and to see classes, etc. in the original files, you'll have to update your Namespace on the new files.
Per BikerDude's answer - After transferring everything to new projects, if you keep anything in the original project that came with the solution, it might not be trying to build the right type of item. You may be able to fix that by right-clicking the project, selecting properties, and adjusting the "output type", but it may not have the options you need. If it doesn't, just create a new project with the right type and transfer the files as above.
After following the above steps, I was able to build each new project I created, using the original files from folders that I moved. Mainly I just needed to build, which enabled all the unit tests that this tutorial/class was focused on, but this allowed me to build the console apps as well, when present.
Thanks for the help from all in pointing me in the right direction!

NUnit failed to load DLL

I am getting the following error message when trying to run unit tests in Visual Studio:
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
I am using
Visual Studio Community 2013
NUnit Adapter 3.4.0.0
NUnit 3.4.1
The weird thing is, that I have another project which is set up the same way as this one and it works just fine.
I also downloaded NUnit 3.4.1 and installed it. When I run
nunit3-console.exe Trading.Tools.Test.dll
everything works just fine.
Any ideas what I can do?
Many thanks
Konstantin
Edit #1
Here is the full console output from Visual Studio when trying to run all test.
Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run:
Trading.Tools.Test.dll, Trading.Tools.dll are built for Framework Framework45 and Platform X64.
Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
NUnit Adapter 3.4.0.0: Test discovery starting
NUnit failed to load w:\Repos\trading.tools\Trading.Tools.Test\bin\x64\Debug\Trading.Tools.Test.dll
Assembly contains no NUnit 3.0 tests: w:\Repos\trading.tools\Trading.Tools\bin\x64\Debug\Trading.Tools.dll
NUnit Adapter 3.4.0.0: Test discovery complete
As you can see it is very obvious that NUnit expects a x86 build, but I build for a x64 platform. And again, my x64 build works just fine if I execute it using nunit3-console.exe.
What I see in the csproj file is this:
<Reference Include="nunit.framework, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\packages\NUnit.3.4.1\lib\net45\nunit.framework.dll</HintPath>
</Reference>
The weird thing here is that it specifies using Version=2.6.4.14350 but referencing a 3.4.1 dll.
So the next question from this point is how can I make NUnit execute my x64 build? Any ideas?
I had a similar issue, the key is the fact that it is the Test Runner in Visual Studio that is stating that only x86 assemblies will be tested. I am assuming from this that it then forces the use of the x86 NUnit runner. To change this (in VS2015 and VS2017 at least), go to Test > Test Settings > Default Processor Architecture > X64.
You can also set the execution target in the runsettings file. You then have to select that file. This should make the solution more stable.
A runsettings file which only set this can look like:
To enable it, do as shown in the figure below:
When you select it from the test menu (1), it will be added as the selected one in the menu (2), and a Rebuild will then make the test appear in the Test Explorer (3)
There is an extra bonus by using a runsettings file, and that is that it will then run properly on the TFS Build system, if you use that. I have written a blog post on that issue, see http://hermit.no/how-to-control-the-selection-of-test-runner-in-tfsvsts-making-it-work-with-x86x64-selected-targets/
I couldn't execute my tests and found that to be one of the issues. It turns out that my TestFixture was internal. Just switching it to public solved my case.
After unsuccessfully trying all other responses above, the following worked for me:
In my case the .NET project and solution is on a mounted drive (I use a MacBook and Parallels for .NET development). The mount also contains the /bin/debug and /bin/release locations where NUnit was attempting to read the "test" DLL from.
The fix was to move the solution/project files to the C: drive of my Windows image. The tests were discovered immediately.
Apparently the shared/mounted location was not to its liking. I don't know why, since the mount is permanent and read/writable to all users on the Windows image. I suspect file permissions problems or maybe somehow the entire mount is not accessible to the user/process running the NUnit discovery logic.
I happened to get this error when writing the unit test method. And noticed the root cause that one of the dependent dll was missing to load. This error("NUnit failed to load the .dll") was shown in the Output("Test") window after modifying the test method code and trying to run it. After updating the nuget package for the dependent dll, nunit started picking the test project dll and test cases got executed.
Today I was running into this issue as well (on a .NET Framework 4.8 based NUnit project). The solution for me was to also install the Microsoft.NET.Test.Sdk package.
To get to the root cause, it may help to attempt to run your tests using the NUnit CLI.
In my case, the CLI logged a bind failure I didn't see in Visual Studio. After I had fixed that, my tests ran correctly via CLI and VS.
I got this error in a .Net 6.0 Asp.Net solution and here's how I solved it.
It was only occurring in one Test project whose tests wouldn't run, the other Test projects were running fine in Test Explorer and in Debug.
The tests that were failing to be detected had "Test Not Run":
In the Output is the error:
NUnit Adapter 4.3.1.0: Test discovery starting NUnit failed to load [dll path]
No test is available in [dll path]
What I did was comment out each class and bring them back one by one until the Tests would STOP to RUN. Then with the failing class put a break point on a [Test] method.
If you can't hit the break point then its failing in this classes [SetUp]. Put a BreakPoint in the [SetUp] and use F11 to step off the edge of the method, ie F11 off the final curly brace..
AND THEN you get a prompt saying which DLL it can't load.
In my case it was “couldn’t load a DLL Microsoft.AspNetCore.Mvc.Core”
Referencing the DLL via Package Manager resolved the problem.
Edit: if this happens in a Unit Test you may want to reference the Microsoft.AspNetCore.App FrameworkReference rather than the Microsoft.AspNetCore.Mvc.Core package reference:
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>

Using xUnit test with UWP app on VS2015

This is a follow-up to this question. I followed the steps as described here and the sample tests work as expected. This is the first time I got to this working sample, but wait for the real working setup which is where I am having trouble.
As a next step to testing my app, I added my UWP app project using "Add Reference..." to the xUnit Test project. Now, after I referencing my project, when I run the test (Run All in Test Explorer pane VS2015) I get the following error:
Error Payload contains two or more files with the same destination path 'Assets\SplashScreen.scale-200.png'. Source files:
...\Projects\Sample\SampleUnitTest\Assets\SplashScreen.scale-200.png
...\Projects\Sample\Sample\Assets\SplashScreen.scale-200.png SampleUnitTest
There are two more errors, exactly as above, but referring to Square150x150Logo.scale-200.png and Square44x44Logo.targetsize-24_altform-unplated.png image files.
I can understand what these errors mean; the app being tested and the test project both generate visual resources (splash-screen image, logo, taskbar icon, etc.) destined for the same output but these are required to register the app(s) and run (on a local machine in my case). I've never come across such a contentious issue of two projects outputting the same visual resources and hopefully someone knows how to solve this. The unit test doesn't work if I change the project to Class project, so that is not an option.
How do you deal with the contentious situation (wrt visual resources) between an xUnit test project and a project being tested?
Okay, I figured out how to resolve the conflict between the visual resources residing in the Assets folder, but this leads to a new kind of issue for the xUnit (which will be my follow-up question).
For the xUnit project, rename the Assets folder to, say, Images (or whatever you think is a better alternative name).
Point to the Package.appxmanifest file under xUnit project and open it as a code file. To do this, select the Package.appxmanifest file and press F7, or right-click the file and select the View Code context menu command.
In the code file, replace the folder name Assets with Images or whatever you chose to rename the Assets folder with. Save it.
Now the test project will compile and run with no visual resources output conflicts. With regard to xUnit however, we'll hit another problem as described in my next question.

Categories

Resources