I'm using NUnit (but have also tried this with VS testing) and I'm having a problem getting TestDriven.Net to recognize and run different test fixtures in a single .cs file.
I'm trying to do a little BDD style testing. So what I have in one file is something like this:
[TestFixture]
public class when_view_is_ready : AAA
{
// setup, tests
}
[TestFixture]
public class when_something_happens : AAA
{
// setup, tests
}
When I run this in the NUnit GUI runner it sees the different test fixtures just fine and runs all the tests. When I run it via TestDriven.NET context menu and watch the Output window, it only runs the first fixture's tests. Is there a reason for this? Can this be fixed?
I think I'm running TestDriven.Net 2.0. I can't be certain; not exactly sure how to check my version.
Appreciate any help!
You can check the version of TestDriven in Help -> About in Visual Studio, or in Tools -> Addin Manager.
I can confirm that this is an issue in 3.0.2556, when running tests it runs only the first TestFixture in the file, when you right-click the file, and select "Run Tests". When you right-click on the containing directory or project, and "Run Tests", it runs both fixtures.
When using other test runners, like Resharper's testrunner, even running it on the file runs both fixtures.
Best thing to do, is probably to report a bug with TestDriven.net
Related
I am starting to go through a tutorial on Unit Testing. I am using Visual Studio 2019 and I am trying to run the projects/ tests but keep getting these errors,
CS5001 Program does not contain a static 'Main' method suitable for an entry point
CS006 Metadata file 'C:(filepath)... .exe' could not be found
I know neither class has a main static method but the tutorial I am following along with doesn't either so I am not sure what the issue is. I have searched Stack Overflow and Google but can only find answers going over my head, any idea what how to fix this?
It is pretty simple code as I am just at the beginning so I think the problem is something else but am not sure (obviously)
UPDATE:
Here is a screen shot of the structure of my solution:
structure of solution
The original project was a Console application, the I added a Test project to the solution.
Test project are not applications and are normally run using a test runner.
Run you tests from VS's Test Explorer
Option B. Run the test directly from its code.
It sounds like you've written your unit tests inside a Console Application template, removed the Program class's Main() method and hit F5 to run your unit tests.
You don't F5 unit tests, you run them from a test runner. Right-click any of your test methods and choose something like "Run Unit Tests" to show that test runner. Or click Test -> Test Explorer to see all discovered tests. See also Run unit tests in "Get started with unit testing".
If that works, create a new project that's actually a Unit Test Project so you at least get a fancy project icon, and move your test code into that project.
This is a common question, it's really quite astounding to me how difficult such a simple task is turning out to be ...
I have a Visual Studio (formerly Xamarin) C# cross-platform library. It has some code in the shared "abstract" library, and a few machine-specific implementation classes for Android, iOS, and UWP libraries. It also has a few automated (non-UI) unit tests that use the Microsoft.VisualStudio.TestTools.UnitTesting package. This is a great start, when Visual Studio 2017 is feeling generous it runs my unit tests and life is good.
Now I want to run the unit tests on simulators of the actual devices, from a command line (so I can run them from Jenkins and capture the output). So, for example, I want to run my unit tests on a simulator of an iPad. From various web sites I get the feeling that I can't use the Microsoft package for this, so I need to switch to either Xunit or NUnit. I gave up on Xunit earlier today, here's how NUnit went.
Go into the Unit Test project and remove the NuGet packages for Microsoft.NET.Test.Sdk, MSTest.TestFramework, and MSTest.TestAdapter.
Add NUnit instead (version 3.7.0).
Go into each CS file, change to "using NUnit.Framework", change [TestMethod] to [Test], and [TestClass] to [TestFixture].
Project:Manage NuGet Packages, add the NUnit3TestAdapter.
Test Explorer: Run All
At this point I get this weird error for the UWP machine-specific library:
The "XamlCTask" task could not be initialized with its input parameters
The "DebugType" parameter is not supported by the "XamlCTask" task. Verify the parameter exists on the task, and it is a settable public instance property.
That's a really strange message, and if I Rebuild the UWP library, it rebuilds without errors. Not sure why it's building the UWP library as part of the Unit Tests, the only thing listed under "Projects" is the abstract (non-machine-specific) library.
Also strange that building the Unit Test library works, it's only when I do the Test Explorer: Run All that I get these errors.
OK fine, it's a known error.
Close VS2017, delete bin and obj folders, reopen, no better. Close VS2017, hack the Xamarin.form.targets as listed. Same error. Realize error is in the global Xamarin.Forms.targets, not the one in my solution. Hack it. Close & reopen solution, rebuild. No error this time!
Test Explorer: Run all. Nothing. Output:Build shows no failures, Output:Tests is completely empty. Close & reopen solution. Get new errors:
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_5937a670-0e60-4077-877b-f7221da3dda1.cs'.
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_E7A71F73-0F8D-4B9B-B56E-8E70B10BC5D3.cs'.
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_036C0B5B-1481-4323-8D20-8F5ADCB23D92.cs'.
Test Explorer: Run all. This time get more interesting errors:
------ Discover test started ------
NUnit Adapter 3.7.0.0: Test discovery starting
Assembly contains no NUnit 3.0 tests:
C:\Workspace\PopComm.iOS\bin\iPhone\Debug\PopComm.dll
NUnit Adapter 3.7.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.0530024) ==========
Of course that assembly doesn't contain any unit tests! It's not my unit test project!
Sigh.
OK, create a new project of type "NUnit 3 Unit Test Project". Move my test classes into this new project and add a dependency to my non-machine-specific library.
Test Explorer: Run all - Tests! There are tests! And they even run successfully.
Close VS2017, open again, Test Explorer: Run all. No tests. Output:Tests is empty.
Close project, delete all bin and obj folders, Rebuild Solution, Test Explorer: Run All. No tests.
Push code to git, pull on the Mac.
Open solution in Visual Studio for the Mac.
Use the mostly-hidden View:Test feature to bring up the Unit Tests window, run the tests. Tests found and run. Yippee!
Now I want to run my unit tests on an iPad simulator, not inside VSMac.
Add a new project to my solution, type iOS:Tests:Unified API:Unit Test App.
Wizard asks what is the minimum iOS version I want to support. I figure I should match the iOS version that my machine-specific library targets, so I go looking in the VSMac options for the project. The iOS target version isn't shown anywhere. Use the default (10.3) for the Unit Test App.
Do I have to copy all of my unit test CS files into this new app now? Or can I somehow reference my cross-platform unit test project in this Unit Test App?
Pushing everything and pulling on the Windows machine, I try adding a reference to my unit test library but get an error:
The project 'NUnit.Tests' cannot be referenced. The referenced project is targeted to a different framework family (.NETFramwork)
Hum. I wonder what framework the new NUnit.Tests.iOS application is targeting. Check its options - it doesn't say anywhere. Remove the reference from the Unit Test App back to the unit test files, and just copy the unit test source files over (non-optimal).
Run the application in the iPad simulator, it finds & runs the tests, but I don't know how to (1) run it from the command line, and (2) capture the unit test statuses in a file.
So after all this, I still don't know how to run my unit tests (1) from the command line, (2) on an iPad simulator, and (3) capturing the output in a text file.
I installed SpecFlow and SpecRun recently on top of NUnit. I had a bit of trouble with references and Nuget packages but finally my tests run again. But this time whenever I run test (SpecFlow feature) my TestBase [SetUp] method once it reaches end runs again, resulting with opening browser window again. Test runs to the end with second attempt. Any one had similar problem?
I was checking solutions that point to PDB files as I see this popping up in Debug window but non seemed to work. Also, in Immediate Window I see this: Step into: Stepping over non-user code
I'm running test under recent version of SpecFlow v2.1.0 and NUnit3.21 against WebDriver v2.53.
For future reference. NUnit and SpecFlow hooks are mutually exclusive. Make sure you run your tests with attributes specific for providers you want to run test with.
I have a test suite, comprised of both unit tests and integration tests, in a project using C# on .NET 4.0 with Visual Studio 2010. The test suite uses MSTest. When I run all tests in solution (either by hitting the button in the testing toolbar or using the Ctrl-R A shortcut chord) all of the tests, integration and unit, pass successfully.
When I either attempt to run the same tests from the command line with mstest (explicitly using the only .testsettings file present) or attempt to run them from the Test List Editor or using the .vsmdi file the integration tests fail.
The integration tests test the UI and so have dependencies on deployment items and such, whereas the unit tests do not. However, I cannot seem to pin down what is actually different between these two methods of running the tests.
When I inspect the appropriate Out directories from the test run, not all of the files are present.
What would cause some of the files that deploy correctly in one situation from Visual Studio to not deploy correctly in another?
The static content started being copied shortly after I wrote the comments above. The other major issue I ran into was that the integration test project referenced libraries that were dependencies of the system-under-test (with copy-local set to true) in order to ensure that the DLLs would be present when they were needed. For some reason, these stubbornly refused to copy when the tests were run through Test List or mstest.
What I eventually did to work around it was include [DeploymentItem] attributes for the DLLs that I needed. This got things working no matter how the tests were run. What I am still unclear on, that may have answered the underlying solution, or provided a better solution, is how Test View/mstest differ from the regular test runner (assuming that the correct .settings file was passed to mstest.). I'm putting these notes/workarounds in an answer, but I'll leave the question open in case anyone can address the underlying cause for how the different test execution paths differ.
How can I ignore NUnit tests of a certain category while doing a Jenkins build?
I know how to do this in Team City. Our production environment uses Jenkins whilst our developers use Team City. I need some tests (database integration) to be ignored while doing our production build on Jenkins. Any links, tips, or code are certainly appreciated.
[Test, Category("DBExclude")]
public void WidgetRepository_FindById_ReturnsWidget()
{
//I should not run on Jenkins
}
Thanks All!
If you are using a Windows Batch command to execute the NUnit tests as part of the Jenkins job, you may simply use the command-line option "/exclude" on the category you've assigned to the test. For the example you've given, the command would be something akin to...
nunit-console /exclude:DBExclude nunit.tests.dll
And, just for the record, you can do it the other way and select to run tests WITH a certain category as opposed to only those WITHOUT by using the following (again, making use of your example):
nunit-console /include:DBExclude nunit.tests.dll
NUnit has documentation that details these options and more here: NUnit Documentation