I want to do some integration tests on my project. Now I am looking for a mechanism that allow me execute some code before all tests begin and execute some code after all tests ends.
Note that I can have set up methods and tear down method for each tests but I need the same functionality for all tests as a whole.
Note that I using Visual Studio, C# and NUnit.
Annotate your test class with the NUnit.Framework.TestFixture attribute, and annotate you all tests setup and all tests teardown methods with NUnit.Framework.TestFixtureSetUp and NUnit.Framework.TestFixtureTearDown.
These attributes function like SetUp and TearDown but only run their methods once per fixture rather than before and after every test.
EDIT in response to comments:
In order to have a method run after ALL tests have finished, consider the following (not the cleanest but I'm not sure of a better way to do it):
internal static class ListOfIntegrationTests {
// finds all integration tests
public static readonly IList<Type> IntegrationTestTypes = typeof(MyBaseIntegrationTest).Assembly
.GetTypes()
.Where(t => !t.IsAbstract && t.IsSubclassOf(typeof(MyBaseIntegrationTest)))
.ToList()
.AsReadOnly();
// keeps all tests that haven't run yet
public static readonly IList<Type> TestsThatHaventRunYet = IntegrationTestTypes.ToList();
}
// all relevant tests extend this class
[TestFixture]
public abstract class MyBaseIntegrationTest {
[TestFixtureSetUp]
public void TestFixtureSetUp() { }
[TestFixtureTearDown]
public void TestFixtureTearDown() {
ListOfIntegrationTests.TestsThatHaventRunYet.Remove(this.GetType());
if (ListOfIntegrationTests.TestsThatHaventRunYet.Count == 0) {
// do your cleanup logic here
}
}
}
Related
I have two tests: BooUnitTest and BooIntegrationTest.
Within the same testing project I'm holding a method with the AssemblyInitialize attribute decorator:
AssemblyTestsHandler.cs
using Microsoft.VisualStudio.TestTools.UnitTesting;
[TestClass]
public class AssemblyTestsHandler
{
[AssemblyInitialize]
public static async Task Bootstrap()
{
//Do complex stuff...
}
}
Is it possible to make the Bootstrap method that works only for BooIntegrationTest and not for FooUnitTest?
For example
FooTests.cs:
[TestClass]
public class FooTests
{
[TestMethod]
public async Task FooUnitTest()
{
//Skip Bootstrap()!!
}
}
BooTests.cs
[TestClass]
public class BooTests
{
[TestMethod]
public async Task BooIntegrationTest()
{
//Do Not Skip Bootstrap()!!
}
}
Here is the project structure:
TestingProject
-AssemblyTestsHandler.cs
-BooTests.cs
-FooTests.cs
No, you can't. AssemblyInitialize will be called once per assembly and it will be called before all other methods (AssemblyInitializeAttribute Class):
The method marked with this attribute will be run before methods
marked with the ClassInitializeAttribute, TestInitializeAttribute, and
TestMethodAttribute attributes. Only one method in an assembly may be
decorated with this attribute.
Actually unit tests should be executed in random order:
FooUnitTest1
BooIntegrationTest2
FooUnitTest2
BooIntegrationTest1
in this case any static initialization will affect all other unit-tests.
I think, there are two possibilities:
You can use TestInitialize and TestCleanup attributes for FooTests. But this will affect performance
You can use ClassInitialize for BooTests, but in this case, you have to trigger integration tests separately from unit tests. Integration- and unit tests can be distinguished by TestCathegory attribute.
I haved created a class with the SetupFixture attribute to have a one time setup as needed for my integration test assembly.
[SetUpFixture]
public static class IntegrationTestsBase
{
public static IKernel Kernel;
[SetUp]
public static void RunBeforeAnyTests()
{
Kernel = new StandardKernel();
if (Kernel == null)
throw new Exception("Ninject failure on test base startup!");
Kernel.Load(new ConfigModule());
Kernel.Load(new RepositoryModule());
}
[TearDown]
public static void RunAfterAnyTests()
{
Kernel.Dispose();
}
}
Resharpers Unit Test Session window has a grouping set to: Projects and Namespaces. However if I use this instance class, Resharpers Unit Test Session says that:
Ignored: Test should be run explicitly
Even tried running these tests with MsTest runner:
Result Message: IntegrationTestsBase is an abstract class.
I have tried to wrap this class to a namespace but nothing changed. If I run individual tests one-by-one it gets runned, however I cannot run them all from the GUI.
How can I fix this issue to be able to run all the tests included in this assembly?
Using NUnit 2.6.4, Resharper 2015.2 and VS2015 update 1.
Your Testclass doesn't need to be static as it gets instantiated by the Testframework and static classes typically can't be instantiated.
The quickest fix is to remove the static keyword except from your Kernel property.
[SetUpFixture]
public class IntegrationTestsBase
{
public static IKernel Kernel;
[SetUp]
public void RunBeforeAnyTests()
{
Kernel = new StandardKernel();
if (Kernel == null)
throw new Exception("Ninject failure on test base startup!");
Kernel.Load(new ConfigModule());
Kernel.Load(new RepositoryModule());
}
[TearDown]
public void RunAfterAnyTests()
{
Kernel.Dispose();
}
}
Keep in mind that whatever you put in Kernel is now shared so if this test is run with multiple threads, the class in Kernel is not isolated to a single test. Which is something you should either be aware of or compensate for.
I know, that in Coded UI there are two methods (MyTestInitialize and MyTestCleanup) which can perform action before and after each tests. I need add some action which launch before and after all tests. For example, if you familiar with rspec there are two methods before() and after(), which take parameter :each (will call before/after each tests) or :all (will call before/after all test).
Create your methods with the [ClassInitialize] and [ClassCleanup] attributes as necessary. This should be within your Test Class. Example:
[CodedUITest]
public class MyTestClass
{
[ClassInitialize]
public void DoSomethingFirst()
{
// your code here that will run at the beginning of each test run.
}
[TestInitialize]
public void RunBeforeEachTest()
{
// your test initialization here
}
[TestMethod]
public void MyTestMethod()
{
}
}
And you would do the same for your [TestCleanup] and [ClassCleanup].
More on this attribute can be found here.
What i want to do is this :
Create a bunch of Unit Tests.
Create a variety of different permutations/combinations of initialization of mocks, input variables etc.
Run each given unit test with a against a set of such initializations based on some parameters.
How would i go about doing something like this?
Is there already any framework to handle this (I.e. run a given test multiple times while changing initialization)? Can you suggest any design or ideas with which i could make something to do this?
I am aware of unit testing frame works. i use NUnit and Rhino mocks myself.
Shown below is an example of what i need.
[Test Initialize]
Setup( <-possible parameter-> )
[Test Method]
TestA()
now i want TestA() to be run multiple times. Each time the Test initialize would pick another initialization combination.
More clarification
Lets suppose a test would require variables A, B, C. Each of them are very complex objects with the end result that the a large number of combinations can be formed. So i'm hoping that somehow i could create a test initialize that could possible iterate through a list of such combinations, so it would initialize them, run the TESTA, go back to next initialization in the list, run TESTA again and so on until the list runs out. Next it picks another list for TESTB and once again follows this process.
At the least im hoping for some ability to be able to run a given TEST function n times. The rest i know i can build once this is possible
In nUnit you can use the [TestCase] attribute for simple types:
[Test]
[TestCase("a", "b")]
[TestCase("c", "b")]
[TestCase("a", "d")]
public void TestMethod(string param1, string param2){
// run your test with those parameters
}
Or you can use a TestCaseSource method for complex types:
[Test]
[TestCaseSource("GetTestCases")]
public void TestMethod(MyObject1 param1, MyObject2 param2){
// run your test with those parameters
}
private IEnumerable GetTestCases(){
yield return new TestCaseData( new MyObject1("first test args"),
new MyObject2("first test args"))
.SetName("SomeMeaningfulNameForThisTestCase" );
yield return new TestCaseData( new MyObject1("2nd test args"),
new MyObject2("2nd test args"))
.SetName("SomeMeaningfulNameForThisTestCase2" );
}
You can do something similar in MS-Test using a DataSource:
http://codeclimber.net.nz/archive/2008/01/18/How-to-simulate-RowTest-with-MS-Test.aspx
You might be able to do this without needing any framework-specific addons by creating an abstract base class that contains all your test functions, then inheriting that base class with multiple classes, each with their own setup function.
public abstract class MyTests
{
[Test]
public void TestOne()
{
...
}
[Test]
public void TestTwo()
{
...
}
}
[TestFixture]
public class FirstSetup : MyTests
{
[Setup]
public void Setup()
{
...
}
}
[TestFixture]
public class SecondSetup : MyTests
{
[Setup]
public void Setup()
{
...
}
}
I have done this in other languages, but not sure how the various C# frameworks will handle it.
I'm trying to setup my tests using Xunit. I have a requirement to delete all images in a folder start of the tests, and then each method does some image resizing and saves a copy of it's output to the folder. The folder should only be emptied once, and then each method will save their own image into the folder.
When I use IUseFixture<T>, the ClearVisualTestResultFolder function is still being called before every test, so I only end up with one image in the folder.
public class Fixture
{
public void Setup()
{
ImageHelperTest.ClearVisualTestResultFolder();
}
}
public class ImageHelperTest : IUseFixture<EngDev.Test.Fixture>
{
public void SetFixture(EngDev.Test.Fixture data)
{
data.Setup();
}
public static void ClearVisualTestResultFolder()
{
// Logic to clear folder
}
}
If I put the ClearVisualTestResultFolder in the constructor, it is also being called once for every test method. I need this just run once before all test methods are executed, how can I achieve this?
If it matters, I use the ReSharper test runner.
Following the guidance in this xUnit discussion topic, it looks like you need to implement a constructor on the Fixture and also implement IDisposable. Here's a complete sample that behaves the way you want:
using System;
using System.Diagnostics;
using Xunit;
using Xunit.Sdk;
namespace xUnitSample
{
public class SomeFixture : IDisposable
{
public SomeFixture()
{
Console.WriteLine("SomeFixture ctor: This should only be run once");
}
public void SomeMethod()
{
Console.WriteLine("SomeFixture::SomeMethod()");
}
public void Dispose()
{
Console.WriteLine("SomeFixture: Disposing SomeFixture");
}
}
public class TestSample : IUseFixture<SomeFixture>, IDisposable
{
public void SetFixture(SomeFixture data)
{
Console.WriteLine("TestSample::SetFixture(): Calling SomeMethod");
data.SomeMethod();
}
public TestSample()
{
Console.WriteLine("This should be run once before every test " + DateTime.Now.Ticks);
}
[Fact]
public void Test1()
{
Console.WriteLine("This is test one.");
}
[Fact]
public void Test2()
{
Console.WriteLine("This is test two.");
}
public void Dispose()
{
Console.WriteLine("Disposing");
}
}
}
When running this from the console runner, you'll see the following output:
D:\xUnit>xunit.console.clr4.exe test.dll /html foo.htm xUnit.net
console test runner (64-bit .NET 4.0.30319.17929) Copyright (C)
2007-11 Microsoft Corporation.
xunit.dll: Version 1.9.1.1600 Test assembly: test.dll
SomeFixture ctor: This should only be run once
Tests complete: 2 of 2
SomeFixture: Disposing SomeFixture
2 total, 0 failed, 0 skipped, took 0.686 seconds
Then, when you crack open the test output file foo.htm, you'll see the other test output.
The old IUseFixture<T> interface in xUnit.net v1.x has been replaced
by two new interfaces: IClassFixture<T> and ICollectionFixture<T>. In
addition, the mechanism for injecting fixture values into your tests
has changed from property setters to constructor arguments. Class
fixtures are created once and shared amongst all tests in the same
class (much like the old IUseFixture). Collection fixtures work the
same way, except that the single instance is shared amongst all tests
in the same test collection.
NOTE this applies only to xUnit.net v1 - see the accepted answer unless that's your case
IUseFixture<T>.SetFixture gets called once for each test. The Fixture itself is only created once.
In other words, you shouldnt be doing anything in your SetFixture method, but you should be instead be triggering it in the Fixture constructor.
For one-time cleanup, implement an IDisposable.Dispose on the Fixture (it's not required though)
Note that it's a bad idea to be (even potentially) sharing state between tests. Best to use a TemporaryDirectoryFixture like this one.