Develop Reference

"Run a method only if a condition is satisfied" pattern

I have a method
using Microsoft.VisualStudio.TestTools.UnitTesting; // using visual studio's test framework
[TestMethod]
public void ATestMethod()
{
// stuff
}
from a public class ATestClass. This test class runs two types of tests :
tests requiring that a certain software is installed on the machine running the test
tests that can run free
To handle this, I added a public class BaseTestClass from which I made ATestClass derive, and in ATestClass I added a :
public bool isTheSoftwareInstalledOnTheMachine()
{
// stuff
}
and I "decorated" all internal scopes of tests from ATestClass as follows :
[TestMethod]
public void ATestMethod()
{
if (isTheSoftwareInstalledOnTheMachine())
{
// stuff
}
}
I find this horrible. I would rather like to be able to write something like :
[TestMethod]
[RunIfTheSoftwareInstalledOnTheMachine]
public void ATestMethod()
{
// stuff
}
but I don't know if one is allowed to define "custom" [characterizer]'s. (I don't even know the right word for them.) If it is, would that be the best design ? (I heard about the decorator pattern, but I don't know if I could make it generic enough in my context, because I would potentially need to use the condition for many other test classes.) Anyway, how would I proceed with characterizer's ?

I know you're using VS test framework but if you can change to NUnit you can accomplish what you want.
Test case:
using NUnit.Framework;
[TestFixture]
public class MyAppTests
{
[Test]
[RunIfTheSoftwareInstalledOnTheMachine]
public void ATestMethod()
{
// Executes if custom attribute is true, otherwise test case is ignored
}
}
Custom attribute:
using NUnit.Framework;
using NUnit.Framework.Interfaces;
public class TestHelper
{
public static bool IsTheSoftwareInstalledOnTheMachine()
{
// Return state of software
return true;
}
}
public class RunIfTheSoftwareInstalledOnTheMachineAttribute : Attribute, ITestAction
{
public ActionTargets Targets { get; private set; }
public void AfterTest(ITest test) {}
public void BeforeTest(ITest test)
{
if (!TestHelper.IsTheSoftwareInstalledOnTheMachine())
{
Assert.Ignore("Omitting {0}. Software is not installed on machine.", test.Name);
}
}
}

If you define your own attribute you surerly have to check for its existance on your own. You can´t expect your framework to guess what the attribute is for.
But I suppose you don´t even need an attribute to do this. You can simply ignore the test by putting the logic inside the test-method anyway:
[Test]
public void MyTest()
{
if(!RunIfTheSoftwareInstalledOnTheMachine)
Assert.Ignore("Test not run because no software was installed");
// your actual test-code
}
Another approach is to use the CategoryAttribute provided by NUnit, with which you can run only those tests that fall within your provided category:
[Test]
[Category("SoftwareInstalled")]
public void MyTest() { /* ... */ }
EDIT: You could also use the TestCaseAttribute with a specific method that returns a TestCase when the condition is met:
[TestCaseSource("ProvideTestcases")]
public void MyTest() { /* ... */ }
private static IEnumerable<TestCaseData> ProvideTestcases()
{
if(RunIfTheSoftwareInstalledOnTheMachine)
yield return new TestCaseData();
}
If the codition is not met no testcase is generated at all.

If the software being installed on the machine is a requirement for any of the tests to pass and any one test failing means the whole suite fails then why bother checking in multiple tests if the software is installed? Just write a single test to fail if the software is not installed and throw a useful exception. Something like:
[Test]
public void EnsureImportantSoftwareIsInstalled()
{
if(!importantSoftwareIsInstalled)
{
Assert.Fail($"Software X must be installed for the tests in {nameof(MyTestClass)} to run, please install it");
}
}

For Nunit 2.6, a slight variation of the HimBromBeere's answer works well for me. The test case is displayed as ignored.
[TestCaseSource("ProvideTestcases")]
public void MyTest() { /* ... */ }
private static IEnumerable<TestCaseData> ProvideTestcases()
{
if(RunIfTheSoftwareInstalledOnTheMachine)
yield return new TestCaseData().Ignore();
}

NUnit - Fails with TestFixtureSetUp method not allowed on a SetUpFixture

I am trying to re-organise some Integration Tests we have so that they use a common class for creating a Database and the Data required in the Database to Test against in other classes in the same assembly using [SetUpFixture] NUnit attribute.
I have :
namespace Tests;
public class TestBaseClass : SolutionBaseClass
{
public void Setup()
{
base.CreateDatabase();
base.CreateData();
}
public void Teardown()
{
base.DestroyDatabase();
}
}
[SetUpFixture]
public class Setup : TestBaseClass
{
[SetUp]
public void Setup()
{
base.Setup();
}
[TearDown]
public void Teardown()
{
base.Teardown();
}
}
then individual test fixture classes:
namespace Tests.Services;
[TestFixture]
public class LibraryTest : TestBaseClass
{
[TestFixtureSetUp]
public void SetupTests()
{
// I know am calling the same Setup twice once from SetUpFixture and TestFixture,
// I have handled it so that only one Database/Data gets set up once (for safety mostly!)
base.SetUp();
// Other class initialisations.
}
}
Any ideas what I am doing wrong, I figure it is a problem with the inheritance model being used, as you can tell I am inheriting this from someone else!!
Thanks.

In NUnit 3 one should use OneTimeSetUpAttribute and OneTimeTearDownAttribute on the static methods of the [SetUpFixture] class. Source: http://bartwullems.blogspot.nl/2015/12/upgrading-to-nunit-30-onetimesetup.html
In NUnit 2.0
[SetUpFixture]
class TestHost
{
[SetUp]
public static void AssemblyInitalize()
{
//Global initialization logic here
}
}
In NUnit 3.0
[SetUpFixture]
class TestHost
{
[OneTimeSetUp]
public static void AssemblyInitalize()
{
//Global initialization logic here
}
}

In NUnit 3.0 TestFixtureSetUp and TestFixtureTearDown has been renamed to OneTimeSetUp and OneTimeTearDown.
Here is the documentation link for above changes:
SetUp and TearDown Changes

Handle any error in NUnit

I have got a standard NUnit unit test.
[TestFixture]
public class MyTest
{
[SetUp]
public virtual void Setup()
{
}
[TearDown]
public void CleanUp()
{
}
[Test]
public void Test01()
{
Assert.AreEqual(10, 10);
}
[Test]
public void Test02()
{
Assert.AreEqual(4, 1);
}
[Test]
public void Test03()
{
Assert.AreEqual(1, 51);
}
//......a huge amount of Tests
[Test]
public void TestN()
{
Assert.AreEqual(1, 1);
}
}
As you see, there are a lot of tests here. What I need to do is to handle any error in each test and to get the information about the error without modifying the body of a test method.
So in JUnit there is an interface called TestListener which allows to handle any test error, see the error message, see the name of the test, etc.
I didn't find something similar in NUnit. Does it exist? Or is there is a way to do it?

It sounds like you probably want EventListeners:
EventListeners are able to respond to events that occur in the course of a test run, usually by recording information of some kind. Note that EventListeners called asynchronously with respect to test execution and are not able to affect the actual execution of the test.
It sounds like you'll be interested in this method:
void TestFinished(TestResult result);
(As an aside, it seems odd to me that this doesn't use normal .NET events or provide a no-op interface implementation in an abstract class, but there we go...)

NUnit: SetUp and TearDown for each test in a test fixture across multiple Fixtures

I would like to have a generic SetUp and TearDown that is run with every test across multiple fixtures but all within a common namespace. This would be something similar to the [SetUpFixture] attribute but would be run with every test.
I've tried using a base class with [SetUp] but resharper resolves this as inconclusive which isn't really ideal.

I don't see any problem in using a base class with the generic code.
public class BaseTest
{
[SetUp]
public void SetUp()
{
//Do generic Stuff
}
[TearDown]
public void TearDown()
{
// Do generic stuff
}
[TestFixture]
public class TestClass : BaseTest
{
[SetUp]
public void SetUp()
{
//Do Stuff
}
[TearDown]
public void TearDown()
{
// Do stuff
}

used in multiple [TestFixture]s

I'm in the process of setting up tests in NUnit and have a newbie question.
Is it possible to have a Test/s that could be used in multiple [TestFixture]s?
So
[Test]ValidateString(string bob)
Could be called in a series of different [TestFixture]?

That doesn't sound like a test to me. Tests are typically parameterless (unless you're using [TestCase]s) and running it within a context of a single fixture would typically be enough -- it either passes once and that's good or it doesn't and it's a broken test.
If you just have a method that does some validation on a string, you could set it up as a static method on some class (e.g. TestHelpers) and call it from whatever tests (in multiple test fixtures) need it.
Here's another idea: inheritance. You can have a base fixture that has all your tests, and then fixtures that inherit from it that set up whatever variables you need. The tests will run for each fixture. I'm not familiar with Selenium RC, but you should be able to adapt the code below to set up whatever variables you need in various fixtures.
[TestFixture]
public class BaseFixtureTests
{
protected IMyClass _myClass;
[TestFixtureSetUp]
public void FixtureSetup()
{
_myClass = ConfigureMyClass();
}
protected virtual IMyClass ConfigureMyClass()
{
// fixtures that inherit from this will set up _myClass here as they see fit.
}
[Test]
public void MyClassTest1()
{
// test something about _myClass;
}
}
[TestFixture]
public class MySpecificFixture1 : BaseFixtureTests
{
protected override IMyClass ConfigureMyClass()
{
return new MySpecificMyClassImplementation();
}
}
public class MySpecificMyClassImplementation : IMyClass
{
//some implementation
}
You can also add extra tests in each fixture as well that don't test common functionality and don't need to be reused across fixtures.

The newer version of NUnit supports generics. This is a great fit if what you are testing doesn’t need to be configured (only created) from your test code. Here is an example copied from http://nunit.net/blogs/:
[TestFixture(typeof(ArrayList))]
[TestFixture(typeof(List<int>))]
public class IList_Tests<TList> where TList : IList, new()
{
private IList list;
[SetUp]
public void CreateList()
{
this.list = new TList();
}
[Test]
public void CanAddToList()
{
list.Add(1); list.Add(2); list.Add(3);
Assert.AreEqual(3, list.Count);
}
}
I’ve also used Anna’s approach of inheritance. One possible refinement to her example (depending on personal preference): Don’t mark the base class as a TestFixture, only the child classes. Each class that you mark as a TestFixture will be displayed as a set of tests in the NUnit client. You will probably never want to run the base class methods directly because the child is providing all of the setup code. If you remove TestFixture from the base class, running invalid tests won’t be an option in the UI. This allows you to run all the tests and see all green… always a nice feeling.

You might be able to achieve what you want with inheritance.
using NUnit.Framework;
namespace ClassLibrary1
{
[TestFixture]
public class TestFixtureBase
{
[SetUp]
public virtual void Setup()
{
// setup code here
}
[Test]
public void CommonTest1()
{
Assert.True(true);
}
[Test]
public void CommonTest2()
{
Assert.False(false);
}
}
public class MyClassTests : TestFixtureBase
{
[SetUp]
public override void Setup()
{
base.Setup();
// additional setup code
}
[Test]
public void MyClassTest1()
{
Assert.True(true);
}
}
}

You can write a method to be called from multiple [Test] methods. But I don't think there is a way to have the same [Test] included in multiple [TestFixture]s.
[TestFixture]
public class TestsOne{
[Test] public void TryOne(){
Helpers.ValidateString("Work?");
}
}
[TestFixture]
public class TestsTwo{
[Test] public void TryTwo(){
Helpers.ValidateString("Work?");
}
}
public static class Helpers{
public static void ValidateString(string s){
Assert.IsNotNull(s);
}
}