Develop Reference

Run a C# unittest only if another one passes

I'm trying to run a UnitTest in C#, but only if another UnitTest passes? Can't quite get it to work, does anybody know how I can do this?
Edit: It's in NUnit

I encountered this problem a while back. A solution I came up with may not be the most elegant but it works. What I did to get around the ordering issue is I created my own unit testing framework. It was propriety to the company I worked for so I can't share it with you.
In the testing framework, there existed
A template for each type of test & a generic template to aggregate all the tests
A utility to execute each template type
For example, if I was doing an integration test, I would have a "http utility" and the template would contain the endpoint & payload.
The tests I wanted to run would need to be stored into an intermediate data structure such as json. This allowed me to serialize the tests into templates.
Now this is where it gets tricky... Using some fancy T4 templating, I would get the json data & serialize it to a list of templates. Then I would order the tests by execution order and dependency (one test could be dependent on another for chaining integration tests). I would then generate a unit test for every template. The generated unit tests would then run on build
For your question about canceling test execution if one fails, you can build that into your templating using some fancy logic
static List<ITestTemplate> requiredTests = new List<ITestTemplate>();
...
if(requiredTests.Any(t => t.Failed))
Assert.IsTrue(false) //fail subsequent tests

You could accomplish this, though it may not be the cleanest solution, by using the [Order()] attribute. (docs)
This will allow you to run the dependency-test as [Order(1)] and the test relying on the first test as [Order(2)]. You can share your driver across the tests, and if the first test fails, close the driver, causing the other tests relying on the first test passing to fail.

How to run a SpecFlow test through a test harness?

Good afternoon/morning/evening folks,
I was wondering is it possible for me to "execute" a SpecFlow test via some sort of test harness (not NUnit)?
Previously my test harness I built ran MS Unit tests by calling methods from within the DLL that was created when I compiled the tests.
I'm assuming the same is possible in theory since a DLL is created, but im wondering how it will get all of the arguments etc.
So in short, is this possible if so is there a straight forward way to do this or am I barking up the wrong tree?

It's possible, but I'm not clear why you would want to.
Specflow is basically just a clever way of generating tests. Normally these are nUnit tests, but they can also be switched to use mstest. When you save your edits to the .feature file then VS runs a Custom Tool that converts your plaintext into a .feature.cs file that contains a code version of what you wrote with nUnit attributes applied to the methods.
Later, an nUnit runner (nUnit, resharper, gallio, teamcity etc) loads the dll and looks for all public methods marked with [Test] inside public classes marked with [TestFixture]. These methods get called.
There is nothing to stop you writing your own runner, however I'm not sure why you would do that. nUnit provides extensive reporting of the success of your test run in xml format, so its probably faster just to write something to parse that.

So I decided to invest some time on this and figured that using reflection was the way to do this task.
Here is some of my code:
TestRunner.TestDLLString = getDLL(project);
var TestDLL = Assembly.LoadFrom(TestDLLString);
Type myClassType = TestDLL.GetType("SeleniumDPS." + testname);
var instance = Activator.CreateInstance(myClassType);
MethodInfo myInitMethod = myClassType.GetMethod("Initialize");
try
{
myInitMethod.Invoke(instance, null);
}
catch (Exception ex)
{
//Error logging etc
}
I then repeat that for the "[TestMethod]" etc. I understand some people dislike reflection but in this instance the performance isnt critical so it works quite well for us.
So essentially what im doing is reading on the name of the test from an XML file then searching the DLL for that test method, then executing the Intitialize method, and later on executing the test method itself. After the test is run I then execute the cleanup method.
It might seem a bit hacky and NUnit might seem the logical choice for some, but as I mentioned earlier I needed a customizable approach. Thanks for all the suggestions though.

NUnit - Loads ALL TestCaseSources even if they're not required by current test

I recently starting using NUnit to do integration testing for my project. It's a great tool, but I've found one drawback that I cannot seem to get the answer to. All my integration tests use the TestCaseSource attribute and specify a test case source name for each test. Now the problem is that preparing these test case sources takes quite some time (~1 min.) and if I'm running a single test, NUnit always loads EVERY SINGLE test case source, even if it's not a test case source for the test that I'm running.
Can this behavior be changed so that only the test case source(s) for the test I'm running load? I want to avoid creating new assemblies every time I want to create a new test (seems rather superfluous and cumbersome, not to mention, hard to maintain), since I've read that tests in different assemblies are loaded separately, but I don't know about the test case sources. It's worth mentioning that I'm using Resharper as the test runner.
TL;DR: Need to tell NUnit to only load the TestCaseSources that are needed for the tests running in the current session. Current behavior is that ALL TestCaseSources are loaded for any test that is run.

Could you do this by moving your sources instantiation to a helper method and call them in the setup methods for each set of tests?
I often have a set of helper methods in my integration test suite that set up shared data for different tests.
I call just the helper methods that I need for the current suite in the [Setup]

Dependent Tests

Alright, so I'm using the Microsoft Test Framework for testing and I need to somehow build dependent tests. Why you might say? Because one of the tests ensures that I can load data from the database and the others need to operate against that set -- making the round trips is something we don't want to do to keep the automated nature of the tests efficient.
I have been searching and just can't seem to find a way to do a couple of things:
Decorate the test methods so that they are seen as dependent.
Persist the set between tests.
What have I tried?
Well, regarding the decoration for dependent tests, not much because I can't seem to find an attribute that's built for that.
Regarding the persistence of the set, I've tried a private class variable and adding to the Properties of the test context. Both of those have resulted in the loss of the set between tests.
I hope that you can help me out here!

Create your test separately and then use the Ordered Test to run them in the order you want.
If any of the tests fails then the whole test is aborted and it considered failed:

I think, what you need is an orderer test list. You can create this in you test project under 'New Item...'. The ordered test list is a list with tests in a specified order that are executed in the same context.
By the way: Unit tests should test only the smallest unit in an application, not a huge set of operations. Because if one unit is not working correctly you can find easy wich one.

Moq - Tests run ok when executed independently, but fail when executed in a row

We are using Moq to perform some unit tests, and there's some strange behaviour, perhaps it's some configuration issue that i'm missing.
basically, i have 2 tests (which call a windows workflow, which calls custom activities that call a method using Invoke. i don't know if this helps but i want to give as much info as i can). The tests run OK when executed alone, but if I execute them in a same run, the first one passes and the second fails (doesn't matter if i change the order of them, the 2nd one always fails)
The mock is recreated every time, loaded using Unity. ex :
MockProcessConfigurator = MockFactory.Create<IProcessConfigurator>();
MockProcessConfigurator.Setup(x => x.MyMethod(It.IsAny<Order>()));
[...]
InversionOfControl.Instance.Register<IProcessConfigurator>(MockProcessConfigurator .Object)
The invoked call (WF custom activity) is
var invoker = new WorkflowInvoker(new MyWorkflow());
invoker.Invoke(inputParameter);
The call (Invoked call) is
MyModuleService.ProcessConfigurator.MyMethod(inputOrder);
when debugging, i see that the ProcessConfigurator is always mocked.
The call that fails in the test is something as simple as this :
MockEnvironment.MockProcessConfigurator.Verify(x => x.MyMethod(It.IsAny<Order>()), Times.Exactly(1));
When debugging, the method is actually called everytime, so i suspect that there's something worng with the mock instance. I'm a bit lost here, because things seem to be implemented correctly, but for some reason when they're run one after the other, there's a problem

This type of error commonly occurs when the two tests share something.
For example, you set up your mock with an expectation that a method will be called 1 time in your test setup, and then two tests each call that method 1 time - your expectation will fail because it has now been called 2 times.
This suggests that you should be moving the set up of expectations into each test.

A generic troubleshooting for this type of problem is to try to isolate the dependency between the two test.
Move any setup-code to inside the tests.
Move any tear-down code to inside the tests.
Move any field initializers to inside the tests. (Those are only run once per fixture!)
This should make both your test pass when run together. When you got the green-lights, you can start moving out duplicated stuff again to initializers/setup one piece at the time, running the tests after each change you make.
You should be able to learn what is causing this coupling between the tests. Good luck!

Thought I'd add an additional situation and solution I just came across:
If you are running tests from two separate projects at the same time and each project is using a different version of Moq, this same problem can happen.
For me, I had TestProjectA using Moq 4.2.14 and TestProjectB using Moq 4.2.15 on accident (courtesy of Nuget). When running a test from A and a test from B simultaneously, the first test succeeded and the second silently failed.
Adjusting both projects to use the same version solved the issue.

To expand on the answer Sohnee gave, I would check my setup/teardown methods to make sure you're tidying everything up properly. I've had similar issues when running tests in bulk and not having tidied up my mock objects.

I don't know if this is relevant, but MockFactory.Create seems odd. I normally create mocks as follows:
var mockProcessConfigurator = new Mock<IProcessConfigurator>();
When using a MockFactory (which I never have needed), you would normally create an instance of it. From the moq QuickStart:
var factory = new MockFactory(MockBehavior.Strict) { DefaultValue = DefaultValue.Mock };
Calling a static method on MockFactory seems to defeat the purpose. If you have a nonstandard naming convention where MockFactory is actually a variable of type MockFactory, that's probably not your issue (but will be a constant source of confusion). If MockFactory is a property of your test class, insure that it is recreated on SetUp.
If possible I would eliminate the factory, as it is a potential source of shared state.
EDIT: As an aside, WorkflowInvoker.Invoke takes an Activity as a parameter. Rather than creating an entire workflow to test a custom activity, you can just pass an instance of the custom activity. If that's what you want to test, it keeps your unit test more focused.