I am trying out the new Visual Studio 2015 CTP 6 and the first thing I notice was that for new ASP.NET 5 project there isn't an option to create a Test project (it's disable)... Now, VS2015 introduced Smart Unit Tests but it is not suitable for TDD, the idea is to generate unit tests for already existing code.
Probably exists a nuget-package to implement TDD but what I find weird, I believe, it's not longer come out of the box.
So, how implement TDD in the new Visual Studio 2015 CTP6?
Update 3/22/2015
Here some links related to this unresolved issue:
Getting Started with xUnit.net and ASP.NET 5
How can I run xUnit Unit Tests with VS2015 Preview?
Great question. Here is what I settled on. I wouldn't call it ideal and maybe there is a better way that I am not aware of.
1) Use xUnit - if you look at the aspnet repository on github, you will see most if not all the tests written with xUnit.
2) Visual Studio IDE support for xUnit is not yet available for ASPNET5 based tests. You must run them from the command prompt. This link discusses how to set up and run unit tests for ASPNET5 http://xunit.github.io/docs/getting-started-aspnet.html
3) Complicating matters, the just released version of xUnit does not support ASPNET5, but RC3 does and it is still available on Nuget. Per this tweet https://twitter.com/xunit/status/574692021123731456 ASPNET5 support will return in xUnit 2.1, a version of which should be released soon.
4) Even after following all the instructions, I had a problem that I believe was related to storing my project on a UNC path instead of a path with a drive letter, so don't let that bite you. https://github.com/xunit/xunit/issues/295
Related
I have a .NET 4.8 project with Specflow tests. These are not found and displayed by VS 2019, so I can't execute them. I looked into the log files and found out that VS is trying to execute the tests with testhost.x86.exe although I have set everywhere AnyCPU and Auto as processor architecture (which is enough for other projects). I then set the processor architecture to x64 and now VS is using testhost.exe but still not finding anything. There are no errors in the logs except failing to load Microsoft.VisualStudio.Coverage.Interop extension. What could be the reason for not seeing anything in the test explorer? Or where could I eventually find some more information?
I use the same Specflow (3.1) and Specflow Runner (3.2) versions in other projects and there it works, so it shouldn't be the source of the problem. Also in the project file and elsewhere in the project I couldn't find anything special. All feature files have been generated without problems.
Not sure if this will fix your issue, but check if you have your Test Explorer set to the same processor architecture as your project.
I had some issues with this in the past.
Full disclosure: I am one of the developers of SpecFlow and SpecFlow+
I switched to an older Specflow version (3.0) and it worked. No idea what the problem was but I had it with multiple old projects migrated from Specflow 2 to 3.
Background
My team is using Visual Studio 2017, though I suspect multiple versions are affected. Basically, the post Unit Tests not discovered in Visual Studio 2017, is the first problem I ran into where multiple test projects in the same solution were added by different team members. The problem for us turned out to be because of different versions were being added and somehow that confused VS and it would not find some of our unit tests. In our case, my teammate had added a v1.1.11, and mine was 1.1.18. That caused my new tests on the latter version not to be discovered. Just now I created a new .net unit test project (full .net) and it created them with a 1.2 reference. This leads me to this question...
The Question
Why does Visual Studio Unit Test Framework version keep changing in a single solution as new test projects are added? It apparently pulls in whatever the latest version is from NuGet, regardless of the pre-existing versions already in use within a solution. Then the follow up question would be, is there a way to control Visual Studio. Certainly you can manually either:
a. rollback to a prior version to have all test projects in your solution match or
b. roll everything forward
c. just ignore and assume that versions will be compatible in the future and not run into issues noted above.
Note that either a. or b. requires manual intervention in what would originally (back in the day before NuGet and MSTest was just included in your VS install) just be handled automatically via a VS patch etc.
Also, given we are starting to see side-by-side versions of MS, I'm curious about best practices (yes I know that is not in question format, so just consider blogging about it somewhere else... I just wanted to plant the seed).
Ref.
https://github.com/microsoft/testfx
https://www.nuget.org/packages/MSTest.TestFramework/
General VS Unit Test discover-ability issues, see: Visual Studio 2015 does not discover unit tests
Can Nunit be used with C#? Which Exception Asserts methods should be used?
I was trying to make use of Assert.DoesNotThrow but this is not recognized by Microsoft.VisualStudio.TestTools.UnitTesting.
Which Exception Assert has to be used to test a case that should not throw an exception?
NUnit is actually a .NET library, so it was created to be used with C#, but I'm guessing this was not actually your question.
I'm guessing you created a unit test project in Visual Studio, which would have created a project using Microsoft's own unit test framework (Microsoft.VisualStudio.TestTools.UnitTesting). You then googled "how to assert that a method throws" and found results around NUnit.
NUnit is also a unit test framework, so I would advise not using both at the same time. In order to use just NUnit, just create a normal C# project and add an NUnit reference.
In order to just add NUnit in a Visual Studio project, you should use NuGet unless you have a good reason not to. To do that, after creating your new normal-C#-project-meant-to-be-a-NUnit-test-project, right click on the project in the Solution Explorer, and click Manage NuGet packages.... Find NUnit and install it. This will automatically add a reference to it (and allow you updating it when you want).
Also, to solve your initial issue, the Microsoft Visual Studio framework does have a method to assert exceptions. See here.
That being said, you could use NUnit's assertions when writing Microsoft unit tests, all you have to do is reference NUnit in your existing Microsoft test project. Again, really not recommended, as you would be using two frameworks at the same time.
You could also look for other assertions framework that could complement Microsoft's own one (Fluent Assertions being one for instance).
Or you could create your own assertion method, something along the lines of:
private static void AssertThrows<T>(Action action) where T: Exception
{
try
{
action();
}
catch (T)
{
return;
}
Assert.Fail("Failed to fail");
}
I would really advise against this though (as much fun as writing your own framework is), because the less test framework your write yourself the better (Who whatches the watchmen?), and you could never make it as feature-rich and bug-poor as something that has been written by a dedicated community (NUnit) or company (VS test framework).
It seems you're confusing NUnit (which is a separate .NET library that needs to be installed) and the Visual Studio Unit Testing Framework which comes with Visual Studio.
They follow the similar XUnit patterns, so you can write tests in a similar fashion, using different methods.
You can read more about the Assert statements included in Visual Studio Unit testing framework in the Unit Testing Framework Asserts documentation.
I have a suite of SpecFlow tests written in C# using MSTest as the framework. I understand its possible to upload the DLL to MTM and run them through there.
However I was wondering if it was possible for me to execute these tests outside of Visual Studio. For example via a dashboard. I wouldn't want the results uploaded to MTM or TFS as we have a few projects that are not connected to them.
My framework uploads results to a database so thats good enough for me in terms of logging.
I gather I would have to do something with the DLL thats generated from building the project that contains SpecFlow, but I want to avoid using reflection if at all possible.
Does anyone have any ideas on how I could go about this?
Tests written using Specflow are simply tests in selected testing framework (MSTest in your case) so you can run them using test runner for that framework.
In case of MSTest you can use MSTest.exe (located in Visual Studio directory, probably also part of TFS installation). Take a look here http://msdn.microsoft.com/en-US/library/ms182489.aspx on how to run tests (easiest way is to use /testcontainer option).
Please note that MSTest is not the best test framework and running outside of Visual Studio is not the same as running them inside of it. There are problems with config file and separate files needed by tests. If you hit these problems, you can switch to other test framework (I use NUnit).
Is there a tool for automated test discovery for .NET. I am using the Visual Studio unit testing stuff and wanted functionality similar to Python Nose. I need to have a tool automatically discover all the unit test available and run for example the ones "marked" as unit and in different scenarios run the tests "marked" as Integration and so on. I have found an individual that has created his own implementation of the MSBuild test task and an considering creating my own with annotation attributes to do what Nose does but wanted to see if anyone was aware of an existing tool that could work.
Thanks
Visual Studio integrated test framework does exactly that when running from the IDE.
If you need a command line tool that does exactly the same functionality (finds all the tests in a specific directory/solution) I guess you have to write something.
Because MSTest command line needs at least the assembly to be specified. I suggest you write a short script that iterates all the assemblies and find if they have tests in them and then run each assembly using MSTest.exe
Update:
I've just published a new CodePlex project called #Nose that does exactly what you need. Currently it only supports NUnit but I plan on adding VSTest as well.
Try ReSharper from JetBrains. It does what you want with unit tests plus a whole lot more.