I am currently writing Unit tests fo ASP Net MVC application, and one of the things I have to mock is an HTTP Request.
For this I use HttpSimulator library (Subtext.TestLibrary).
I call SimulateRequest() method, and one of things it does is it creates a HostingEnvironment object.
_httpSimulator = new HttpSimulator().SimulateRequest(new Uri());
Before simulating HTTP request, I have to use the value of HostingEnvironment.IsHosted property in my code (in condition statement).
All works fine for the first test, because I get the proper value for HostingEnvironment.IsHosted (which is FALSE).
However, when the SimulateRequest() creates a HostingEnvironment object, HostingEnvironment.IsHosted becomes TRUE, so when the second test runs I receive a TRUE value, which leads to wrong results (or exception).
HostingEnvironment environment = new HostingEnvironment();
Therefore I am trying to find a way to reset HostingEnvironment.IsHosted back to FALSE. This property, however, is read-only.
I also cannot see any method, which would reset this property (I expect either Terminate, Finalize or whatever else it could be).
It sounds like your tests are too tightly coupled with multiple tests using the same instance of HostingEnvironment().
Each test should typically be executed independently of one another so that the results of e.g. Test 1 do not have any effect on the results of e.g. Test 2.
Just use separate instances of HostingEnvironment():
[Test]
public void InitializeContext()
{
using (HttpSimulator simulator = new HttpSimulator())
{
//Test #1...
}
}
[Test]
public void Simulator_Assigns_CurrentContext()
{
using (HttpSimulator simulator = new HttpSimulator())
{
//Test #2...
}
}
Related
I have the following DatabaseFixture which has worked well for all tests I have created up to this point. I use this fixture for integration tests so I can make real assertions on database schema structures.
public class DatabaseFixture : IDisposable
{
public IDbConnection Connection => _connection.Value;
private readonly Lazy<IDbConnection> _connection;
public DatabaseFixture()
{
var environment = Environment.GetEnvironmentVariable("ASPNET_ENVIRONMENT") ?? "Development";
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("AppSettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"AppSettings.{environment}.json", optional: true, reloadOnChange: true)
.Build();
_connection = new Lazy<IDbConnection>(() =>
{
var connection = new MySqlConnection(configuration["ConnectionStrings:MyDatabase"]);
connection.Open();
return connection;
});
}
public void Dispose()
{
Connection?.Dispose();
}
}
[CollectionDefinition("Database Connection Required")]
public class DatabaseConnectionFixtureCollection : ICollectionFixture<DatabaseFixture>
{
}
The problem I am facing is I now need to invoke a test method like MyDataIsAccurate(...) with each record from a table in the database. xUnit offers the [MemberData] attribute which is exactly what I need but it requires a static enumerable set of data. Does xUnit offer a clean way of sharing my DatabaseFixture connection instance statically or do I just need to suck it up and expose a static variable of the same connection instance?
[Collection("Database Connection Required")]
public class MyTests
{
protected DatabaseFixture Database { get; }
// ERROR: Can't access instance of DatabaseFixture from static context...
public static IEnumerable<object[]> MyData => Database.Connection.Query("SELECT * FROM table).ToList();
public MyTests(DatabaseFixture databaseFixture)
{
Database = databaseFixture;
}
[Theory]
[IntegrationTest]
[MemberData(nameof(MyData))]
public void MyDataIsAccurate(int value1, string value2, string value3)
{
// Assert some stuff about the data...
}
}
You cannot access the fixture from the code that provides the test cases (whether that is a MemberData property or a ClassData implementation or a custom DataAttribute subclass.
Reason
Xunit creates an AppDomain containing all the data for the test cases. It builds up this AppDomain with all of those data at the time of test discovery. That is, the IEnumerable<object[]>s are sitting in memory in the Xunit process after the test assembly is built, and they are sitting there just waiting for the tests to be run. This is what enables different test cases to show up as different tests in test explorer in visual studio. Even if it's a MemberData-based Theory, those separate test cases show up as separate tests, because it's already run that code, and the AppDomain is standing by waiting for the tests to be run. On the other hand, fixtures (whether class fixtures or collection fixtures) are not created until the test RUN has started (you can verify this by setting a breakpoint in the constructor of your fixture and seeing when it is hit). This is because they are meant to hold things like database connections that shouldn't be left alive in memory for long periods of time when they don't need to be. Therefore, you cannot access the fixture at the time the test case data is created, because the fixture has not been created.
If I were to speculate, I would guess that the designers of Xunit did this intentionally and would have made it this way even if the test-discovery-loads-the-test-cases-and-therefore-must-come-first thing was not an issue. The goal of Xunit is not to be a convenient testing tool. It is to promote TDD, and a TDD-based approach would allow anyone to pick up the solution with only their local dev tools and run and pass the same set of tests that everyone else is running, without needing certain records containing test case data to be pre-loaded in a local database.
Note that I'm not trying to say that you shouldn't do what you're trying, only that I think the designers of Xunit would tell you that your test cases and fixtures should populate the database, not the other way around. I think it's at least worth considering whether that approach would work for you.
Workaround #1
Your static database connection may work, but it may have unintended consequences. That is, if the data in your database changes after the test discovery is done (read: after Xunit has built up the test cases) but before the test itself is run, your tests will still be run with the old data. In some cases, even building the project again is not enough--it must be cleaned or rebuilt in order for test discovery to be run again and the test cases be updated.
Furthermore, this would kind of defeat the point of using an Xunit fixture in the first place. When Xunit disposes the fixture, you are left with the choice to either: dispose the static database connection (but then it will be gone when you run the tests again, because Xunit won't necessarily build up a new AppDomain for the next run), or do nothing, in which case it might as well be a static singleton on some service locator class in your test assembly.
Workaround #2
You could parameterize the test with data that allows it to go to the fixture and retrieve the test data. This has the disadvantage that you don't get the separate test cases listed as separate tests in either test explorer or your output as you would hope for with a Theory, but it does load the data at the time of the tests instead of at setup and therefore defeats the "old data" problem as well as the connection lifetime problem.
Summary
I don't think such a thing exists in Xunit. As far as I know, your options are: have the test data populate the database instead of the other way around, or use a never-disposed static singleton database connection, or pull the data in your test itself. None of these are the "clean" solution you were hoping for, but I doubt you'll be able to get much better than one of these.
There is a way of achieving what you want, using delegates. This extremely simple example explains it quite well:
using System;
using System.Collections.Generic;
using Xunit;
namespace YourNamespace
{
public class XUnitDeferredMemberDataFixture
{
private static string testCase1;
private static string testCase2;
public XUnitDeferredMemberDataFixture()
{
// You would populate these from somewhere that's possible only at test-run time, such as a db
testCase1 = "Test case 1";
testCase2 = "Test case 2";
}
public static IEnumerable<object[]> TestCases
{
get
{
// For each test case, return a human-readable string, which is immediately available
// and a delegate that will return the value when the test case is run.
yield return new object[] { "Test case 1", new Func<string>(() => testCase1) };
yield return new object[] { "Test case 2", new Func<string>(() => testCase2) };
}
}
[Theory]
[MemberData(nameof(TestCases))]
public void Can_do_the_expected_thing(
string ignoredTestCaseName, // Not used; useful as this shows up in your test runner as human-readable text
Func<string> testCase) // Your test runner will show this as "Func`1 { Method = System.String.... }"
{
Assert.NotNull(testCase);
// Do the rest of your test with "testCase" string.
}
}
}
In the OP's case, you could access the database in the XUnitDeferredMemberDataFixture constructor.
I have the following DatabaseFixture which has worked well for all tests I have created up to this point. I use this fixture for integration tests so I can make real assertions on database schema structures.
public class DatabaseFixture : IDisposable
{
public IDbConnection Connection => _connection.Value;
private readonly Lazy<IDbConnection> _connection;
public DatabaseFixture()
{
var environment = Environment.GetEnvironmentVariable("ASPNET_ENVIRONMENT") ?? "Development";
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("AppSettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"AppSettings.{environment}.json", optional: true, reloadOnChange: true)
.Build();
_connection = new Lazy<IDbConnection>(() =>
{
var connection = new MySqlConnection(configuration["ConnectionStrings:MyDatabase"]);
connection.Open();
return connection;
});
}
public void Dispose()
{
Connection?.Dispose();
}
}
[CollectionDefinition("Database Connection Required")]
public class DatabaseConnectionFixtureCollection : ICollectionFixture<DatabaseFixture>
{
}
The problem I am facing is I now need to invoke a test method like MyDataIsAccurate(...) with each record from a table in the database. xUnit offers the [MemberData] attribute which is exactly what I need but it requires a static enumerable set of data. Does xUnit offer a clean way of sharing my DatabaseFixture connection instance statically or do I just need to suck it up and expose a static variable of the same connection instance?
[Collection("Database Connection Required")]
public class MyTests
{
protected DatabaseFixture Database { get; }
// ERROR: Can't access instance of DatabaseFixture from static context...
public static IEnumerable<object[]> MyData => Database.Connection.Query("SELECT * FROM table).ToList();
public MyTests(DatabaseFixture databaseFixture)
{
Database = databaseFixture;
}
[Theory]
[IntegrationTest]
[MemberData(nameof(MyData))]
public void MyDataIsAccurate(int value1, string value2, string value3)
{
// Assert some stuff about the data...
}
}
You cannot access the fixture from the code that provides the test cases (whether that is a MemberData property or a ClassData implementation or a custom DataAttribute subclass.
Reason
Xunit creates an AppDomain containing all the data for the test cases. It builds up this AppDomain with all of those data at the time of test discovery. That is, the IEnumerable<object[]>s are sitting in memory in the Xunit process after the test assembly is built, and they are sitting there just waiting for the tests to be run. This is what enables different test cases to show up as different tests in test explorer in visual studio. Even if it's a MemberData-based Theory, those separate test cases show up as separate tests, because it's already run that code, and the AppDomain is standing by waiting for the tests to be run. On the other hand, fixtures (whether class fixtures or collection fixtures) are not created until the test RUN has started (you can verify this by setting a breakpoint in the constructor of your fixture and seeing when it is hit). This is because they are meant to hold things like database connections that shouldn't be left alive in memory for long periods of time when they don't need to be. Therefore, you cannot access the fixture at the time the test case data is created, because the fixture has not been created.
If I were to speculate, I would guess that the designers of Xunit did this intentionally and would have made it this way even if the test-discovery-loads-the-test-cases-and-therefore-must-come-first thing was not an issue. The goal of Xunit is not to be a convenient testing tool. It is to promote TDD, and a TDD-based approach would allow anyone to pick up the solution with only their local dev tools and run and pass the same set of tests that everyone else is running, without needing certain records containing test case data to be pre-loaded in a local database.
Note that I'm not trying to say that you shouldn't do what you're trying, only that I think the designers of Xunit would tell you that your test cases and fixtures should populate the database, not the other way around. I think it's at least worth considering whether that approach would work for you.
Workaround #1
Your static database connection may work, but it may have unintended consequences. That is, if the data in your database changes after the test discovery is done (read: after Xunit has built up the test cases) but before the test itself is run, your tests will still be run with the old data. In some cases, even building the project again is not enough--it must be cleaned or rebuilt in order for test discovery to be run again and the test cases be updated.
Furthermore, this would kind of defeat the point of using an Xunit fixture in the first place. When Xunit disposes the fixture, you are left with the choice to either: dispose the static database connection (but then it will be gone when you run the tests again, because Xunit won't necessarily build up a new AppDomain for the next run), or do nothing, in which case it might as well be a static singleton on some service locator class in your test assembly.
Workaround #2
You could parameterize the test with data that allows it to go to the fixture and retrieve the test data. This has the disadvantage that you don't get the separate test cases listed as separate tests in either test explorer or your output as you would hope for with a Theory, but it does load the data at the time of the tests instead of at setup and therefore defeats the "old data" problem as well as the connection lifetime problem.
Summary
I don't think such a thing exists in Xunit. As far as I know, your options are: have the test data populate the database instead of the other way around, or use a never-disposed static singleton database connection, or pull the data in your test itself. None of these are the "clean" solution you were hoping for, but I doubt you'll be able to get much better than one of these.
There is a way of achieving what you want, using delegates. This extremely simple example explains it quite well:
using System;
using System.Collections.Generic;
using Xunit;
namespace YourNamespace
{
public class XUnitDeferredMemberDataFixture
{
private static string testCase1;
private static string testCase2;
public XUnitDeferredMemberDataFixture()
{
// You would populate these from somewhere that's possible only at test-run time, such as a db
testCase1 = "Test case 1";
testCase2 = "Test case 2";
}
public static IEnumerable<object[]> TestCases
{
get
{
// For each test case, return a human-readable string, which is immediately available
// and a delegate that will return the value when the test case is run.
yield return new object[] { "Test case 1", new Func<string>(() => testCase1) };
yield return new object[] { "Test case 2", new Func<string>(() => testCase2) };
}
}
[Theory]
[MemberData(nameof(TestCases))]
public void Can_do_the_expected_thing(
string ignoredTestCaseName, // Not used; useful as this shows up in your test runner as human-readable text
Func<string> testCase) // Your test runner will show this as "Func`1 { Method = System.String.... }"
{
Assert.NotNull(testCase);
// Do the rest of your test with "testCase" string.
}
}
}
In the OP's case, you could access the database in the XUnitDeferredMemberDataFixture constructor.
I'm working on a project built on Asp.Net Boilerplate, and now I have to unit test the services using the real repositories with a real database connection (no mocking). I've been using the last post of BringerOd in https://gist.github.com/hikalkan/1e5d0f0142484da994e0 as a guide for setting up my UnitOfWorkScope instance. So, my code currently looks something like this:
IDisposableDependencyObjectWrapper<IUnitOfWork> _unitOfWork;
[TestInitialize]
public void SetUpService()
{
//initialize service
_unitOfWork = IocManager.Instance.ResolveAsDisposable<IUnitOfWork>();
UnitOfWorkScope.Current = _unitOfWork.Object;
UnitOfWorkScope.Current.Initialize(true);
UnitOfWorkScope.Current.Begin();
}
[TestCleanup]
public void CleanUpService()
{
UnitOfWorkScope.Current.Cancel();
_unitOfWork.Dispose();
UnitOfWorkScope.Current = null;
}
This works like a charm for the first unit test, but when I try to make a repository call in a second test, I get: "The operation cannot be completed because the DbContext has been disposed."
My guess is that when the TestInitialize method runs again, the unit of work scope is getting assigned with the same (disposed) DbContext, rather than a new one. I suppose, inside my actual test methods, I could set up my UnitOfWorkScope inside a using block with the IUnitOfWork. However, I really don't want to repeat that logic inside inside every single test. Does anyone know how to manually get the effect of a using block so that I get a brand new DbContext each time?
Check: http://aspnetboilerplate.com/Pages/Documents/Repositories
You must mark the calling method with [UnitOfWork] attribute.
The reason for this, as explained in the linked document is
When you call GetAll() out of a repository method, there must be an open database connection. This is because of deferred execution of IQueryable<T>. It does not perform database query unless you call ToList() method or use the IQueryable<T> in a foreach loop (or somehow access to queried items). So, when you call ToList() method, database connection must be alive. This can be achieved by marking caller method with the [UnitOfWork] attribute of ASP.NET Boilerplate. Note that Application Service methods are already using [UnitOfWork] as default, so, GetAll() will work without adding the [UnitOfWork] attribute for application service methods.
I'm running ncrunch, in a new MVC 4 solution in VS2012 using nunit and ninject.
When I first open the solution all 50 or so test run and pass successfully.
After I make any code change (even just a added empty space) ncrunch reports that most of my unit test fail. The same thing happens if I press the 'run all tests' in the ncrunch window.
But if you hit the 'Run all tests visible here' button all 50 test pass again and ncrunch reports everything is fine.
Also when you run each test individually they pass every time.
When they do fail they seem to be failing in my ninject setup code
Error: TestFixtureSetUp failed in ControllerTestSetup
public class ControllerTestSetup
{
[SetUp]
public void InitIntegrationTest()
{
var context = IntegrationTestContext.Instance;
context.Init();
context.NinjectKernel.Load<MediGapWebTestModule>();
}
[TearDown]
public void DisposeIntegrationTest()
{
IntegrationTestContext.Instance.Dispose();
}
}
public class IntegrationTestContext : IDisposable
{
private static IntegrationTestContext _instance = null;
private static readonly object _monitor = new object();
private IntegrationTestContext() { }
public static IntegrationTestContext Instance
{
get
{
if (_instance == null)
{
lock (_monitor)
{
if (_instance == null)
{
_instance = new IntegrationTestContext();
}
}
}
return _instance;
}
}
}
All the test also run in the resharper test runner without problems every time.
Does anyone know what could be causing this?
I guessing its something to do with the singleton lock code inside the Instance property but I'm not sure.
==============================================================================
Progress:
I was able to track this down to a error in the ninject setup method above by wrapping it in a try catch statement, and writing the error to the output window.
The exception was caused by trying to load a module more than once, even tho I definitely haven't and I don't use any type of automatic module loading.
This happen on the lines
LocalSessionFactoryModule.SetMappingAssemblies(() => new[] { typeof(ProviderMap).Assembly });
_kernel.Load<LocalSessionFactoryModule>();
_sessionFactory = _kernel.Get<ISessionFactory>();
where LocalSessionFactoryModule is the ninject module class derived for NinjectModule class.
Why is this only happening with ncrunch and what can I do to solve this issue? Is there a way to check if a module has already been loaded?
NCrunch will never execute tests concurrency within the same process, so unless you have multi-threaded behaviour inside your test logic, then it should be safe to say that this isn't an issue caused by the locking or threading over the singleton.
As you've already tried disabling parallel execution and this hasn't made a difference, I'm assuming that the problem wouldn't be caused by concurrent use of resources outside the test runner process (i.e. files on disk).
This means that the problem is almost certainly related to the sequence in which the tests are being executed. Almost all manual test runners (including Resharper) will run tests in a defined sequence from start to finish. This is good for consistency, but it can mask problems that may surface when the tests are run in an inconsistent/random order. NCrunch will execute tests in order of priority and can also reuse test processes between test runs, which can make the runtime behaviour of your tests different if they haven't been designed with this in mind.
A useful way to surface (and thus debug) sequence related issues is to try running your tests in a manually defined order by using NCrunch. If you right-click a test inside the NCrunch Tests Window, under the 'Advanced' menu you'll find an option to run a test using an existing task runner process. Try this action against several of your tests to see if you can reproduce the sequence that surfaces the problem. When it happens, you should easily be able to get a debugger onto the test and find out why it is failing.
Most sequence related problems are caused by uncleared static members, so make sure each of your tests is written in the assumption that existing state may be left behind by another test that has been run within the process. Another option is to ensure all state is fully cleared by tests on tear down (although in my opinion, this is often a less pragmatic approach).
I'm currently building a class using TDD. The class is responsible for waiting for a specific window to become active, and then firing some method.
I'm using the AutoIt COM library (for more information about AutoIt look here) since the behavior I want is actually a single method in AutoIt.
The code is pretty much as the following:
public class WindowMonitor
{
private readonly IAutoItX3 _autoItLib;
public WindowMonitor(IAutoItX3 autoItLib)
{
_autoItLib = autoItLib;
}
public void Run() // indefinitely
{
while(true)
{
_autoItLib.WinWaitActive("Open File", "", 0);
// Do stuff now that the window named "Open File" is finally active.
}
}
}
As you can see the AutoIt COM library implements an interface wich I can mock (Using NUnit and Rhino Mocks):
[TestFixture]
public class When_running_the_monitor
{
WindowMonitor subject;
IAutoItX3 mockAutoItLibrary;
AutoResetEvent continueWinWaitActive;
AutoResetEvent winWaitActiveIsCalled;
[SetUp]
public void Setup()
{
// Arrange
mockAutoItLibrary = MockRepository.GenerateStub<IAutoItX3>();
mockAutoItLib.Stub(m => m.WinWaitActive("", "", 0))
.IgnoreArguments()
.Do((Func<string, string, int, int>) ((a, b, c) =>
{
winWaitActiveIsCalled.Set();
continueWinWaitActive.WaitOne();
return 1;
}));
subject = new Subject(mockAutoItLibrary)
// Act
new Thread(new ThreadStart(subject.Run)).Start();
winWaitActiveIsCalled.WaitOne();
}
// Assert
[Test]
[Timeout(1000)]
public void should_call_winWaitActive()
{
mockAutoItLib.AssertWasCalled(m => m.WinWaitActive("Bestand selecteren", "", 0));
}
[Test]
[Timeout(1000)]
public void ensure_that_nothing_is_done_while_window_is_not_active_yet()
{
// When you do an "AssertWasCalled" for the actions when the window becomes active, put an equivalent "AssertWasNotCalled" here.
}
}
The problem is, the first test keeps timing out. I have already found out that when the stub "WinWaitActive" is called, it blocks (as intended, on the seperate thread), and when the "AssertWasCalled" is called after that, execution never returns.
I'm at a loss how to proceed, and I couldn't find any examples of mocking out a blocking call.
So in conclusion:
Is there a way to mock a blocking call without making the tests timeout?
(P.S. I'm less interested in changing the design (i.e. "Don't use a blocking call") since it may be possible to do that here, but I'm sure there are cases where it's a lot harder to change the design, and I'm interested in the more general solution. But if it's simply impossible to mock blocking calls, suggestions like that are more that welcome!)
Not sure if I understand the problem.
Your code is just calling a method on the mock (WinWaitActive). Of course, it can't proceed before the call returns. This is in the nature of the programming language and nothing you need to test.
So if you test that WinWaitActive gets called, your test is done. You could test if WinWaitActive gets called before anything else, but this requires ordered expectations, which requires the old style rhino mocks syntax and is usually not worth to do.
mockAutoItLibrary = MockRepository.GenerateStub<IAutoItX3>();
subject = new Subject(mockAutoItLibrary)
subject.Run()
mockAutoItLib.AssertWasCalled(m => m.WinWaitActive("Open File", "", 0));
You don't do anything else then calling a method ... so there isn't anything else to test.
Edit: exit the infinite loop
You could make it exit the infinite loop by throwing an exception from the mocks. This is not very nice, but it avoids having all this multi-threading stuff in the unit test.
mockAutoItLibrary = MockRepository.GenerateStub<IAutoItX3>();
// make loop throw an exception on second call
// to exit the infinite loop
mockAutoItLib
.Stub(m => m.WinWaitActive(
Arg<string>.Is.Anything,
Arg<string>.Is.Anything,
Arg<int>.Is.Anything));
.Repeat.Once();
mockAutoItLib
.Stub(m => m.WinWaitActive(
Arg<string>.Is.Anything,
Arg<string>.Is.Anything,
Arg<int>.Is.Anything));
.Throw(new StopInfiniteLoopException());
subject = new Subject(mockAutoItLibrary)
try
{
subject.Run()
}
catch(StopInfiniteLoopException)
{} // expected exception thrown by mock
mockAutoItLib.AssertWasCalled(m => m.WinWaitActive("Open File", "", 0));
Your Test only contains a call to the mocked method. Therefore it tests only your mock instead of any real code, which is an odd thing to do. We might need a bit more context to understand the problem.
Use Thread.Sleep() instead of AutoResetEvents: Since you are mocking the COM object that does the blocking window-active check, you can just wait for some time to mimick the behavior, and then make sure that the window is indeed active by making it active programmatically. How you block should not be important in the test, only that you block for some significant time.
Although from your code it is not clear how winWaitActiveIsCancelled and continueWinWaitActive contribute, I suspect they should be left out of the WinWaitActive mock. Replace them with a Thread.Sleep(500).