Develop Reference

Is it possible to pass data from fixture to ClassData in xunit? [duplicate]

I have the following DatabaseFixture which has worked well for all tests I have created up to this point. I use this fixture for integration tests so I can make real assertions on database schema structures.
public class DatabaseFixture : IDisposable
{
public IDbConnection Connection => _connection.Value;
private readonly Lazy<IDbConnection> _connection;
public DatabaseFixture()
{
var environment = Environment.GetEnvironmentVariable("ASPNET_ENVIRONMENT") ?? "Development";
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("AppSettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"AppSettings.{environment}.json", optional: true, reloadOnChange: true)
.Build();
_connection = new Lazy<IDbConnection>(() =>
{
var connection = new MySqlConnection(configuration["ConnectionStrings:MyDatabase"]);
connection.Open();
return connection;
});
}
public void Dispose()
{
Connection?.Dispose();
}
}
[CollectionDefinition("Database Connection Required")]
public class DatabaseConnectionFixtureCollection : ICollectionFixture<DatabaseFixture>
{
}
The problem I am facing is I now need to invoke a test method like MyDataIsAccurate(...) with each record from a table in the database. xUnit offers the [MemberData] attribute which is exactly what I need but it requires a static enumerable set of data. Does xUnit offer a clean way of sharing my DatabaseFixture connection instance statically or do I just need to suck it up and expose a static variable of the same connection instance?
[Collection("Database Connection Required")]
public class MyTests
{
protected DatabaseFixture Database { get; }
// ERROR: Can't access instance of DatabaseFixture from static context...
public static IEnumerable<object[]> MyData => Database.Connection.Query("SELECT * FROM table).ToList();
public MyTests(DatabaseFixture databaseFixture)
{
Database = databaseFixture;
}
[Theory]
[IntegrationTest]
[MemberData(nameof(MyData))]
public void MyDataIsAccurate(int value1, string value2, string value3)
{
// Assert some stuff about the data...
}
}

You cannot access the fixture from the code that provides the test cases (whether that is a MemberData property or a ClassData implementation or a custom DataAttribute subclass.
Reason
Xunit creates an AppDomain containing all the data for the test cases. It builds up this AppDomain with all of those data at the time of test discovery. That is, the IEnumerable<object[]>s are sitting in memory in the Xunit process after the test assembly is built, and they are sitting there just waiting for the tests to be run. This is what enables different test cases to show up as different tests in test explorer in visual studio. Even if it's a MemberData-based Theory, those separate test cases show up as separate tests, because it's already run that code, and the AppDomain is standing by waiting for the tests to be run. On the other hand, fixtures (whether class fixtures or collection fixtures) are not created until the test RUN has started (you can verify this by setting a breakpoint in the constructor of your fixture and seeing when it is hit). This is because they are meant to hold things like database connections that shouldn't be left alive in memory for long periods of time when they don't need to be. Therefore, you cannot access the fixture at the time the test case data is created, because the fixture has not been created.
If I were to speculate, I would guess that the designers of Xunit did this intentionally and would have made it this way even if the test-discovery-loads-the-test-cases-and-therefore-must-come-first thing was not an issue. The goal of Xunit is not to be a convenient testing tool. It is to promote TDD, and a TDD-based approach would allow anyone to pick up the solution with only their local dev tools and run and pass the same set of tests that everyone else is running, without needing certain records containing test case data to be pre-loaded in a local database.
Note that I'm not trying to say that you shouldn't do what you're trying, only that I think the designers of Xunit would tell you that your test cases and fixtures should populate the database, not the other way around. I think it's at least worth considering whether that approach would work for you.
Workaround #1
Your static database connection may work, but it may have unintended consequences. That is, if the data in your database changes after the test discovery is done (read: after Xunit has built up the test cases) but before the test itself is run, your tests will still be run with the old data. In some cases, even building the project again is not enough--it must be cleaned or rebuilt in order for test discovery to be run again and the test cases be updated.
Furthermore, this would kind of defeat the point of using an Xunit fixture in the first place. When Xunit disposes the fixture, you are left with the choice to either: dispose the static database connection (but then it will be gone when you run the tests again, because Xunit won't necessarily build up a new AppDomain for the next run), or do nothing, in which case it might as well be a static singleton on some service locator class in your test assembly.
Workaround #2
You could parameterize the test with data that allows it to go to the fixture and retrieve the test data. This has the disadvantage that you don't get the separate test cases listed as separate tests in either test explorer or your output as you would hope for with a Theory, but it does load the data at the time of the tests instead of at setup and therefore defeats the "old data" problem as well as the connection lifetime problem.
Summary
I don't think such a thing exists in Xunit. As far as I know, your options are: have the test data populate the database instead of the other way around, or use a never-disposed static singleton database connection, or pull the data in your test itself. None of these are the "clean" solution you were hoping for, but I doubt you'll be able to get much better than one of these.

There is a way of achieving what you want, using delegates. This extremely simple example explains it quite well:
using System;
using System.Collections.Generic;
using Xunit;
namespace YourNamespace
{
public class XUnitDeferredMemberDataFixture
{
private static string testCase1;
private static string testCase2;
public XUnitDeferredMemberDataFixture()
{
// You would populate these from somewhere that's possible only at test-run time, such as a db
testCase1 = "Test case 1";
testCase2 = "Test case 2";
}
public static IEnumerable<object[]> TestCases
{
get
{
// For each test case, return a human-readable string, which is immediately available
// and a delegate that will return the value when the test case is run.
yield return new object[] { "Test case 1", new Func<string>(() => testCase1) };
yield return new object[] { "Test case 2", new Func<string>(() => testCase2) };
}
}
[Theory]
[MemberData(nameof(TestCases))]
public void Can_do_the_expected_thing(
string ignoredTestCaseName, // Not used; useful as this shows up in your test runner as human-readable text
Func<string> testCase) // Your test runner will show this as "Func`1 { Method = System.String.... }"
{
Assert.NotNull(testCase);
// Do the rest of your test with "testCase" string.
}
}
}
In the OP's case, you could access the database in the XUnitDeferredMemberDataFixture constructor.

How to identify when my code is testing in C#?

I am having troubles when testing a controller, because there are some lines at my Startup that are null when testing, I want to add a condition for run this lines only if it's not testing.
// Desired method that retrieves if testing
if (!this.isTesting())
{
SwaggerConfig.ConfigureServices(services, this.AuthConfiguration, this.ApiMetadata.Version);
}

The correct answer (although of no help): It should not be able to tell so. The application should to everything it does unaware if it is in productino or test.
However to test the application in a simpler setting, you can use fake modules or mock-up modules that are loaded instead of the heavy-weight production modules.
But in order to use that, you have to refactor your solution and use injection for instance.
Some links I found:
Designing with interfaces
Mock Objects
Some more on Mock objects

It really depends on which framework you use for testing. It can be MSTest, NUnit or whatever.
Rule of thumb, is that your application should not know whether it is tested. It means everything should be configured before actual testing through injection of interfaces. Simple example of how tests should be done:
//this service in need of tests. You must test it's methods.
public class ProductionService: IProductionService
{
private readonly IImSomeDependency _dep;
public ImTested(IImSomeDependency dep){ _dep = dep; }
public void PrintStr(string str)
{
Console.WriteLine(_dep.Format(str));
}
}
//this is stub dependency. It contains anything you need for particular test. Be it some data, some request, or just return NULL.
public class TestDependency : IImSomeDependency
{
public string Format(string str)
{
return "TEST:"+str;
}
}
//this is production, here you send SMS, Nuclear missle and everything else which cost you money and resources.
public class ProductionDependency : IImSomeDependency
{
public string Format(string str)
{
return "PROD:"+str;
}
}
When you run tests you configure system like so:
var service = new ProductionService(new TestDependency());
service.PrintStr("Hello world!");
When you run your production code you configure it like so:
var service = new ProductionService(new ProductionDependency());
service.PrintStr("Hello world!");
This way ProductionService is just doing his work, not knowing about what is inside it's dependencies and don't need "is it testing case №431" flag.
Please, do not use test environment flags inside code if possible.
UPDATE:
See #Mario_The_Spoon explanation for better understanding of dependency management.

Impossible to run test with TestCaseData generated from RhinoMocks stub

I am working with C# in Visual Studio 2015 Community, with NUnit3 and Rhino Mocks and attempting to write a test for a component of my system (it is NOT a unit test).
I have encountered a problem when trying to use a generated stub as the argument to a TestCaseData that is provided to a TestCaseSource. I get the following errors in the output window:
Test adapter sent back a result for an unknown test case. Ignoring result for 'MyTest(Castle.Proxies.IMyInterfaceProxy4c6c716794ef48818f41fd5012345ead)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'MyTest(Castle.Proxies.IMyInterfaceProxy4c6c716794ef48818f41fd5012345ead)'.
The test name appears in the VS2015 integrated test-runner when I rebuild the test project, but as soon I try to run it it becomes greyed out.
Here there is some sample code based on my test code:
using NUnit.Framework;
using Rhino.Mocks;
using System.Collections;
namespace FenixLib.Core.Tests
{
public interface IMyInterface
{
int Property { get; }
}
[TestFixture]
class TestMocks
{
[Test, TestCaseSource( "TestCases" )]
public void MyTest(IMyInterface what)
{
// Do stuff
}
public static IEnumerable TestCases()
{
yield return new TestCaseData ( CreateFake ( 2 ) );
yield return new TestCaseData ( CreateFake ( 4 ) );
}
public static IMyInterface CreateFake ( int something )
{
var fake = MockRepository.GenerateStub<IMyInterface> ();
fake.Stub ( x => x.Property ).Return ( something );
return fake;
}
}
}
I have been able to overcome the problem if I create a decorator class that wraps the generated stub:
public class Decorator : IMyInterface
{
IMyInterface decorated;
public Decorator ( IMyInterface decorated )
{
this.decorated = decorated;
}
public int Property
{
get
{
return decorated.Property;
}
}
}
And change the prior return fake; by return new Decorator ( fake );. Everything works fine then.
However this is a bit of a pain in my real scenario because my interface does not only have a single property as in this example but is more complex (and yes I know that VS2015 can generate the code for implementing through the decorated field, but that's not the point). Besides, it feels pointless to use RinhoMocks if I am going to end up creating an implementation of the interface I am wishing to not fully implement for my test purposes.
Anyway, what annoys me is that I do not understand why it does not work and therefore I ask:
Can anyone help me to understand why the code without the decorator did not work?

At discovery time, TestExplorer creates a process and asks our adapter to find tests. Your TestCaseSource method runs and generates the cases. It has a name generated by Moq.
Much later in the lifetime of the program, your test is executed by Visual studio, which creates a different process and asks the adapter to run the tests. Since we are running in a new process, the adapter has to discover (that is generate) the cases all over. Most likely, Moq generates them with a different name. The originally created cases are never run and NUnit runs these new cases, which, from the point of view of TestExplorer, were never discovered in the first place.
I haven't seen this symptom before, but we have a similar problem with random arguments being regenerated at execution time. It's essentially a problem with the architecture and could only be resolved if the Adapter could somehow keep the originally loaded test assembly around to be found by the execution process.

Clean Up after Canceling tests

I'm currently running tests through visual studio. Before all the tests are run, I automatically create a set number of users with know credentials, and at the end of the run I delete those users. However, sometimes I need to cancel my tests midway. In these cases the test never gets the chance to clean up, this means that there is left over fake user info from the test run and may causes the next test run to crash (when it attempts to add user info into the DB). Is there anyway to force visual studio/mstest to run a clean up method even if the test is canceled?
I know one option is to have the test check and make sure that the user info doesn't already exist, and if it does remove it before creating the new users. But this still wouldn't solve the issue of the canceled test run leaving unwanted test data.
Update:
Sorry for the miscommunication, however cleaning up the data at the start of the test is not an option. I'm giving a very simplistic view of the issue, but put simply, I have no easy way of making sure that no test data exists at the start of the test. All clean up must occur at the end of the test.

That is impossible. You better find an alternative solution like using separate database for testing and clean all data before each test run, using fixed set of test users or mark test data with some flag. Check Isolating database data in integration tests article by Jimmy Bogard.
There is no built-in way to change MSTest default behavior. In theory you can write MSTest extension that utilizes TestExecution.OnTestStopping event, but that is not an easy process and it requires registry change. Moreover, a lot of people complain that it is not working.
There also MSTest V2, a new version of MSTest with new extensibility points. But it looks like you can't alter cancel behavior with this points, only write attribute decorators. See Extending MSTest V2.
You can't use AppDomain.CurrentDomain.ProcessExit and Process.GetCurrentProcess().Exited events because cancel seems to kill test run process.
NUnit also doesn't support this at the moment. See related NUnit test adapter Run TearDowns on VS Cancel Test Run issue.

Instead of calling the cleanup function at the end of the test, I call mine at the beginning of each test in order to address this exact problem.

Perform the clean up before creating the data as well, this will ensure that you have no leftover data whatever happens. Of course this is only possible if you can identify any leftover data before running the setup.

The idea is that a transaction is initialized before the test begins. In order for the data to be saved in the database, the transaction must be commited, but it is not commited never. It works in case when test a stop, in the case of a successful or unsuccessful completion of the test.
In integration tests we use somthing like this (with NUnit)(It real production code)
public class RollbackAttribute : TestAttribute, ITestAction
{
private TransactionScope _transaction;
public void BeforeTest(ITest test)
{
_transaction = new TransactionScope();
}
public void AfterTest(ITest test)
{
_transaction.Dispose();
}
public ActionTargets Targets => ActionTargets.Test;
}
[TestFixture]
public class SomeTestClass
{
[Rollback] //No need [Test] because Rollback is inherit it.
public void SomeTestMethod()
{
}
}
On MsTest you can make somthing similar, but in this case you should inherit from base class, I hope it works. For example:
public class RollbackTestBase
{
private TransactionScope _transaction;
[TestInitialize]
public virtual void Setup()
{
_transaction = new TransactionScope();
}
[TestCleanup]
public virtual void TearDown()
{
_transaction.Dispose();
}
}
[TestClass]
public class IntegrationTest : RollbackTestBase
{
[TestMethod]
public void TestDataBase()
{
Assert.IsTrue(true);
}
[TestInitialize]
public virtual void Init()
{
}
[TestCleanup]
public virtual void CleanUp()
{
}
}

There are 2 cases which we need to consider while allocating resources in ATPs (Resources might be Creating users, Connection with database). They are
Creation and deletion of resources after each test.
Creation and deletion of resources after set of tests.
Creation and deletion of resources after each test:
If we want to create instance of particular object before execution of a test and want to clean up memory allocated to that object after execution of that test, then we use Test SetUp and Test TearDown attributes of NUnit. In your case object is creation of number of Users.
[SetUp] : Function which is decorated with Test SetUp attribute contains piece of code that executes before the execution of any test.
[TearDown] : Function which is decorated with Test TearDown attributes contains piece of code that executes after execution of any test
Implementation:
[TestClass]
public class UnitTest1
{
[SetUp]
public void SetUP()
{
// Creating Users with proper credentials
}
[TestMethod]
public void TestMethod1()
{
//Write your ATP
}
[TearDown]
public void TearDown()
{
//Clean up
}
}
Creation and deletion of resources after set of tests:
Now If we want to create instance of an object for set of tests and want to clean up memory after execution of all tests then [TestFixtureSetUp] and [TestFixureTearDown] to initialize an object and to clean up memory respectively. Again In your case object can be creation of set of users.
[TestFixtureSetUp] : Function decorated with TestFixtureSetUp will executes once before the execution of group of tests.
[TestFixtureTearDown] : Function decorated with TestFixtureTearDown will executes once after the execution of group of tests.
Implementation
[TestFixture]
public class Tests
{
[TestFixtureSetUp]
public void Setup()
{
//Create users with credentials
}
[Test]
public void _Test1()
{
//Test_1
}
[Test]
public void _Test2()
{
//Test2
}
[TestFixtureTearDown]
public void CleanUp()
{
//Cleanup; Here you need to add code to Delete all users
}
}
Note: I will suggest you, if you are trying to create and delete users for particular ATP then go with SetUp and TearDown. If you are trying same for bunch of ATPs, I would recommend you to go with TestFixtureSetUp and TestFixtureTearDown.
"If your test get pass or fail, SetUp and TearDown functions will execute"
References:
#Shuvra's Answer.
Nunit: SetUp, TearDown, SetUpFixture, TearDownFixture

I think you should open a transaction before you test, create the data and finish test test. But do not commit the transaction. That will ensure that the test can't affect your DB at all.
Update:
The easier approach is use docker container.
You can run a container from your image and remove that container after the test is done. This should definitely reduce the complexity of your test.

Visual studio uses NUNIT therefore, you can use TearDownAttribute. It should run after the test, even if the test is canceled. You can write a function to clean your data.
Please read the reference documentation here: http://nunit.org/docs/2.2/teardown.html

Just to clear more about the NUNIT standrad. Please follow the steps in the Test class:
[TestFixture]
public class _TestClass
{
[TestFixtureSetUp]
public void Setup()
{
//Clearup can be here before start of the tests. But not Recommended
}
[Test]
public void _Test1()
{
}
[Test]
public void _Test2()
{
}
[TestFixtureTearDown]
public void CleanUp()
{
//I will recommend to clean up after all the tests complete
}
}
Reference: http://nunit.org/docs/2.5/fixtureTeardown.html

A better solution to the problem to use what is called "database mocking". In this case you would have your tests run with a different database (or a fake, virtual one).
This article explains how to implement it in C#
https://msdn.microsoft.com/en-us/library/ff650441.aspx

You should begin a transaction and not commit your records to the DB. Thus, all your changes will be automatically rollbacked when the session is over.

How to do unit testing on a static method that uses database access?

Here's the sample static method, say
public static void UpdateSchedule(int selectedScheduleId)
{
using (var dc = new MyDataContext())
{
var selectedSchedule = dc.Schedules.SingleOrDefault(p => p.ScheduleId == selectedScheduleId)
if selectedSchedule != null)
{
selectedSchedule.Name = name;
//and update other properties...
}
dc.SubmitChanges();
}
}
So what would be the correct approach to test on methods like this? Is there a way to avoid calling new MyDataContext() as it might increase the execution time of the unit test.
Also, I am using MsTest test framework in VS2012.

Static functions interfere with testing primarily because they:
Are difficult (sometimes impossible) to substitute from within a consumer
Tend to have "hidden" dependencies
Since you want to test the function itself, number 1 isn't an issue. However, number 2 is problematic because the static function is tightly coupled to the MyDataContext class.
If you want to test the static function without MyDataContext (that is, in isolation) you need to introduce a code seam. This requires some refactoring work, but we can do it fairly painlessly.
Consider if you had the following additional method:
public static void UpdateScheduleWithContext(int selectedScheduleId, IDataContext dc)
{
var selectedSchedule = dc.Schedules.SingleOrDefault(p => p.ScheduleId == selectedScheduleId)
if selectedSchedule != null)
{
selectedSchedule.Name = name;
//and update other properties...
}
dc.SubmitChanges();
}
This new function gives the consumer (i.e., the test) the ability to supply a test double for the data context. This is the seam.
Obviously, though, you don't want all consumers to explicitly provide a data context (which is why you had the original static function to begin with). So you can keep that function around, and just modify it:
public static void UpdateSchedule(int selectedScheduleId)
{
using (var dc = new MyDataContext())
{
UpdateScheduleWithDataContext(selectedScheduleId, dc);
}
}
If you don't have a seam like this, it will be impossible to test the function in isolation. As a result, an integration test would be the best you could hope for.

As Simon Whitehead said, it is impossible to do effective unit testing on static methods and objects. However, using the unit test framework in Visual Studio, you can create a 'unit test' function that is effectively a integration test. Tell Visual Studio to generate a Unit Test for UpdateSchedule, and then modify the function generated to set up the environment/state of the program such that the UpdateSchedule method can execute (create a database connection, instantiate classes, etc.). This includes ensuring that your database has records to update.
Once that is done, you can execute your integration test in the same manner that you would a unit test.