Develop Reference

Nunit TestCaseSource with setName also shows original test in testexplorer

I'm attempting to use TestCaseSource to re-use the test with different data. Here I'm trying to give my tests their own name with the 'setName' property which works fine. My issue is that the original test also shows up in the testexplorer. It doesn't seem possible to run. How can I get rid of it in the test explorer?
Simple reproduction:
[TestFixture]
public class Tests
{
[TestCaseSource(nameof(MyTestData))]
[Category(name: "MyCategory")]
public void OriginalTest(string first, string second)
{
Assert.IsTrue(true);
}
private static IEnumerable<TestCaseData> MyTestData
{
get
{
yield return new TestCaseData("firstString", "secondString").SetName("FirstTest");
yield return new TestCaseData("firstString", "secondString").SetName("SecondTest");
}
}
}
My test explorer looks like this

This seemed to be a problem with the adapter.
I was having this same issue, using SetArgDisplayNames instead, which, while not providing the expected visual result, was the best fit for this kind of usage until the problem was fixed.
Updating NUnit3TestAdapter to v3.16.0 the problem no longer occurred:

C# AssemblyInitialize not running

I'm trying to do unit testing on my C# ASP.NET project but need to run some initialization code for all tests:
[TestClass()]
public class Init
{
[AssemblyInitialize]
public static void initialize()
{
ContextProvider.setContext(new TestContext());
}
}
That's all I need to run before my tests, but it is not running. I tried to debug all my tests and put a breakpoint in that line but it is not hit. If I comment out the [AssemblyInitialize] and run one particular test that does not require this initialization, it runs successfully. However, with this line in, no tests run (and no breakpoints at all are hit)
What am I missing here?

Phil1970's helpful comment assisted in resolving the issue.
The problem with the initialize method is that it must receive TestContext (Microsoft.VisualStudio.TestTools.UnitTesting.TestContext). I couldn't find any guide/Microsoft documentation that explicitly states this, but the examples in this msdn page do have TestContext as input for the method.
After I added it, the tests ran successfully. On a side note, I had created a class to mock some data I needed and called it TestContext, which turned out to be a very poor name selection and made it more difficult to understand my mistake. I renamed it to APITestContext, here's my fixed initialization class.
[TestClass()]
public class Init
{
[AssemblyInitialize()]
public static void initialize(TestContext testContext)
{
ContextProvider.setContext(new APITestContext());
}
}

Here is (I think) the full list of things to check:
You should be using Microsoft.VisualStudio.TestTools.UnitTesting;
The class must be public
The class CANNOT be static or abstract
The class must be decorated with [TestClass]
The method must be public
The method MUST be static
The method must be decorated with [AssemblyInitialize]
The method must have the signature: void InitName(TestContext tc)

Don't know if it can be related but in the documentation it says
This attribute should not be used on ASP.NET unit tests, that is, any test with [HostType("ASP.NET")] attribute. Because of the stateless nature of IIS and ASP.NET, a method decorated with this attribute might be called more than once per test run.
Are you using that HostType?

Prevent NUnit tests to run depending on environment

I'm struggle with this one quite a long time now. Some background: I created my automated test framework using Selenium. With one project being pure NUnit tests and second one that does all the work for the tests. Now, in test project I keep the directories to all environments I run my tests against. So far many of my tests were read-only and didn't bother much about if tests did not run on environment that they supposed to. This has changed as I started to run some 'write' tests.
I need to prevent this 'Write' tests to run on any other environment then localhost. So far I tried to use method attributes and getting test method names on run time and do work then but this is not quite efficient. Can you guys point me a good solution? Thanks!

I would tag the tests to exclude with a particular category name then define a SetUp function which will stop the tests from running if they are tagged with that name and if you are on a particular environment such as Production. Place the SetUp function in a BaseClass and have all your test fixtures inherit it. The SetUp function will run before every test and prevent it from running if it needs to.
Something like this:
public class BaseSetup
{
protected const string CategoryToExclude = "Write";
[SetUp]
public void Init()
{
string env = ConfigurationManager.GetEnvironment();
if ( env == Constants.Environments.PROD && (TestContext.CurrentContext.Test.Properties["Categories"].Contains(CategoryToExclude)))
{
Assert.Inconclusive(String.Format("Cannot run this test on environment: {0}", env));
}
}
}
[TestFixture]
public class UnitTests : BaseSetup
{
[Test]
[Category(CategoryToExclude)]
public void TestMethod1()
{
Console.WriteLine("TestMethod1");
}
[Test]
public void TestMethod2()
{
Console.WriteLine("TestMethod2");
}
}
Hope this helps!

NUnit have category attribute.
You can group tests by categories and run only wanted categories.

Impossible to run test with TestCaseData generated from RhinoMocks stub

I am working with C# in Visual Studio 2015 Community, with NUnit3 and Rhino Mocks and attempting to write a test for a component of my system (it is NOT a unit test).
I have encountered a problem when trying to use a generated stub as the argument to a TestCaseData that is provided to a TestCaseSource. I get the following errors in the output window:
Test adapter sent back a result for an unknown test case. Ignoring result for 'MyTest(Castle.Proxies.IMyInterfaceProxy4c6c716794ef48818f41fd5012345ead)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'MyTest(Castle.Proxies.IMyInterfaceProxy4c6c716794ef48818f41fd5012345ead)'.
The test name appears in the VS2015 integrated test-runner when I rebuild the test project, but as soon I try to run it it becomes greyed out.
Here there is some sample code based on my test code:
using NUnit.Framework;
using Rhino.Mocks;
using System.Collections;
namespace FenixLib.Core.Tests
{
public interface IMyInterface
{
int Property { get; }
}
[TestFixture]
class TestMocks
{
[Test, TestCaseSource( "TestCases" )]
public void MyTest(IMyInterface what)
{
// Do stuff
}
public static IEnumerable TestCases()
{
yield return new TestCaseData ( CreateFake ( 2 ) );
yield return new TestCaseData ( CreateFake ( 4 ) );
}
public static IMyInterface CreateFake ( int something )
{
var fake = MockRepository.GenerateStub<IMyInterface> ();
fake.Stub ( x => x.Property ).Return ( something );
return fake;
}
}
}
I have been able to overcome the problem if I create a decorator class that wraps the generated stub:
public class Decorator : IMyInterface
{
IMyInterface decorated;
public Decorator ( IMyInterface decorated )
{
this.decorated = decorated;
}
public int Property
{
get
{
return decorated.Property;
}
}
}
And change the prior return fake; by return new Decorator ( fake );. Everything works fine then.
However this is a bit of a pain in my real scenario because my interface does not only have a single property as in this example but is more complex (and yes I know that VS2015 can generate the code for implementing through the decorated field, but that's not the point). Besides, it feels pointless to use RinhoMocks if I am going to end up creating an implementation of the interface I am wishing to not fully implement for my test purposes.
Anyway, what annoys me is that I do not understand why it does not work and therefore I ask:
Can anyone help me to understand why the code without the decorator did not work?

At discovery time, TestExplorer creates a process and asks our adapter to find tests. Your TestCaseSource method runs and generates the cases. It has a name generated by Moq.
Much later in the lifetime of the program, your test is executed by Visual studio, which creates a different process and asks the adapter to run the tests. Since we are running in a new process, the adapter has to discover (that is generate) the cases all over. Most likely, Moq generates them with a different name. The originally created cases are never run and NUnit runs these new cases, which, from the point of view of TestExplorer, were never discovered in the first place.
I haven't seen this symptom before, but we have a similar problem with random arguments being regenerated at execution time. It's essentially a problem with the architecture and could only be resolved if the Adapter could somehow keep the originally loaded test assembly around to be found by the execution process.

Clean Up after Canceling tests

I'm currently running tests through visual studio. Before all the tests are run, I automatically create a set number of users with know credentials, and at the end of the run I delete those users. However, sometimes I need to cancel my tests midway. In these cases the test never gets the chance to clean up, this means that there is left over fake user info from the test run and may causes the next test run to crash (when it attempts to add user info into the DB). Is there anyway to force visual studio/mstest to run a clean up method even if the test is canceled?
I know one option is to have the test check and make sure that the user info doesn't already exist, and if it does remove it before creating the new users. But this still wouldn't solve the issue of the canceled test run leaving unwanted test data.
Update:
Sorry for the miscommunication, however cleaning up the data at the start of the test is not an option. I'm giving a very simplistic view of the issue, but put simply, I have no easy way of making sure that no test data exists at the start of the test. All clean up must occur at the end of the test.

That is impossible. You better find an alternative solution like using separate database for testing and clean all data before each test run, using fixed set of test users or mark test data with some flag. Check Isolating database data in integration tests article by Jimmy Bogard.
There is no built-in way to change MSTest default behavior. In theory you can write MSTest extension that utilizes TestExecution.OnTestStopping event, but that is not an easy process and it requires registry change. Moreover, a lot of people complain that it is not working.
There also MSTest V2, a new version of MSTest with new extensibility points. But it looks like you can't alter cancel behavior with this points, only write attribute decorators. See Extending MSTest V2.
You can't use AppDomain.CurrentDomain.ProcessExit and Process.GetCurrentProcess().Exited events because cancel seems to kill test run process.
NUnit also doesn't support this at the moment. See related NUnit test adapter Run TearDowns on VS Cancel Test Run issue.

Instead of calling the cleanup function at the end of the test, I call mine at the beginning of each test in order to address this exact problem.

Perform the clean up before creating the data as well, this will ensure that you have no leftover data whatever happens. Of course this is only possible if you can identify any leftover data before running the setup.

The idea is that a transaction is initialized before the test begins. In order for the data to be saved in the database, the transaction must be commited, but it is not commited never. It works in case when test a stop, in the case of a successful or unsuccessful completion of the test.
In integration tests we use somthing like this (with NUnit)(It real production code)
public class RollbackAttribute : TestAttribute, ITestAction
{
private TransactionScope _transaction;
public void BeforeTest(ITest test)
{
_transaction = new TransactionScope();
}
public void AfterTest(ITest test)
{
_transaction.Dispose();
}
public ActionTargets Targets => ActionTargets.Test;
}
[TestFixture]
public class SomeTestClass
{
[Rollback] //No need [Test] because Rollback is inherit it.
public void SomeTestMethod()
{
}
}
On MsTest you can make somthing similar, but in this case you should inherit from base class, I hope it works. For example:
public class RollbackTestBase
{
private TransactionScope _transaction;
[TestInitialize]
public virtual void Setup()
{
_transaction = new TransactionScope();
}
[TestCleanup]
public virtual void TearDown()
{
_transaction.Dispose();
}
}
[TestClass]
public class IntegrationTest : RollbackTestBase
{
[TestMethod]
public void TestDataBase()
{
Assert.IsTrue(true);
}
[TestInitialize]
public virtual void Init()
{
}
[TestCleanup]
public virtual void CleanUp()
{
}
}

There are 2 cases which we need to consider while allocating resources in ATPs (Resources might be Creating users, Connection with database). They are
Creation and deletion of resources after each test.
Creation and deletion of resources after set of tests.
Creation and deletion of resources after each test:
If we want to create instance of particular object before execution of a test and want to clean up memory allocated to that object after execution of that test, then we use Test SetUp and Test TearDown attributes of NUnit. In your case object is creation of number of Users.
[SetUp] : Function which is decorated with Test SetUp attribute contains piece of code that executes before the execution of any test.
[TearDown] : Function which is decorated with Test TearDown attributes contains piece of code that executes after execution of any test
Implementation:
[TestClass]
public class UnitTest1
{
[SetUp]
public void SetUP()
{
// Creating Users with proper credentials
}
[TestMethod]
public void TestMethod1()
{
//Write your ATP
}
[TearDown]
public void TearDown()
{
//Clean up
}
}
Creation and deletion of resources after set of tests:
Now If we want to create instance of an object for set of tests and want to clean up memory after execution of all tests then [TestFixtureSetUp] and [TestFixureTearDown] to initialize an object and to clean up memory respectively. Again In your case object can be creation of set of users.
[TestFixtureSetUp] : Function decorated with TestFixtureSetUp will executes once before the execution of group of tests.
[TestFixtureTearDown] : Function decorated with TestFixtureTearDown will executes once after the execution of group of tests.
Implementation
[TestFixture]
public class Tests
{
[TestFixtureSetUp]
public void Setup()
{
//Create users with credentials
}
[Test]
public void _Test1()
{
//Test_1
}
[Test]
public void _Test2()
{
//Test2
}
[TestFixtureTearDown]
public void CleanUp()
{
//Cleanup; Here you need to add code to Delete all users
}
}
Note: I will suggest you, if you are trying to create and delete users for particular ATP then go with SetUp and TearDown. If you are trying same for bunch of ATPs, I would recommend you to go with TestFixtureSetUp and TestFixtureTearDown.
"If your test get pass or fail, SetUp and TearDown functions will execute"
References:
#Shuvra's Answer.
Nunit: SetUp, TearDown, SetUpFixture, TearDownFixture

I think you should open a transaction before you test, create the data and finish test test. But do not commit the transaction. That will ensure that the test can't affect your DB at all.
Update:
The easier approach is use docker container.
You can run a container from your image and remove that container after the test is done. This should definitely reduce the complexity of your test.

Visual studio uses NUNIT therefore, you can use TearDownAttribute. It should run after the test, even if the test is canceled. You can write a function to clean your data.
Please read the reference documentation here: http://nunit.org/docs/2.2/teardown.html

Just to clear more about the NUNIT standrad. Please follow the steps in the Test class:
[TestFixture]
public class _TestClass
{
[TestFixtureSetUp]
public void Setup()
{
//Clearup can be here before start of the tests. But not Recommended
}
[Test]
public void _Test1()
{
}
[Test]
public void _Test2()
{
}
[TestFixtureTearDown]
public void CleanUp()
{
//I will recommend to clean up after all the tests complete
}
}
Reference: http://nunit.org/docs/2.5/fixtureTeardown.html

A better solution to the problem to use what is called "database mocking". In this case you would have your tests run with a different database (or a fake, virtual one).
This article explains how to implement it in C#
https://msdn.microsoft.com/en-us/library/ff650441.aspx

You should begin a transaction and not commit your records to the DB. Thus, all your changes will be automatically rollbacked when the session is over.