I have got a standard NUnit unit test.
[TestFixture]
public class MyTest
{
[SetUp]
public virtual void Setup()
{
}
[TearDown]
public void CleanUp()
{
}
[Test]
public void Test01()
{
Assert.AreEqual(10, 10);
}
[Test]
public void Test02()
{
Assert.AreEqual(4, 1);
}
[Test]
public void Test03()
{
Assert.AreEqual(1, 51);
}
//......a huge amount of Tests
[Test]
public void TestN()
{
Assert.AreEqual(1, 1);
}
}
As you see, there are a lot of tests here. What I need to do is to handle any error in each test and to get the information about the error without modifying the body of a test method.
So in JUnit there is an interface called TestListener which allows to handle any test error, see the error message, see the name of the test, etc.
I didn't find something similar in NUnit. Does it exist? Or is there is a way to do it?
It sounds like you probably want EventListeners:
EventListeners are able to respond to events that occur in the course of a test run, usually by recording information of some kind. Note that EventListeners called asynchronously with respect to test execution and are not able to affect the actual execution of the test.
It sounds like you'll be interested in this method:
void TestFinished(TestResult result);
(As an aside, it seems odd to me that this doesn't use normal .NET events or provide a no-op interface implementation in an abstract class, but there we go...)
Related
I'm tasked with writing some tests to one of our codebases.
Now I have this class:
```
public class A
{
public void method_1()
{
this.method_2();
// Do something
}
public void method_2()
{
Environment.CurrentDirectory = Path.GetDirectoryName(Assembly.GetEntryAssembly().Location);
// Do something
}
}
```
Now I'm trying to write a test for method_1 without invoking method_2 and getting NullReferenceException.
[TestClass]
public class MyTestClass
{
[TestMethod]
public void MyTestMethod()
{
// Ignore the call: Assembly.GetEntryAssembly() because it returns null>
new A().method_1();
}
Any suggestion?
EDIT:
I'm not allowed to change the code.
Without the possibility to override/mock the logic inside method_2 there is not much you can do.
If it is your own codebase I suggest to move the call to Assembly.GetEntryAssembly().Location to some helper method which you can override for the tests.
Assuming you cannot change anything to the code, you can use MS Fakes to mock out static methods like DateTime.Now. Should also work on Assembly.GetEntryAssembly().
I'm not a big fan of MS Fakes though, had a few too many problems with it in the past. If you can make a small change to the code, you can make the method_2 virtual and do something like this:
public class MyTestA : A
{
public override void method_2()
{
//Do nothing
}
}
You can then write your tests against MyTestA.
Of course, putting the code in method 2 in a new class/interface and mock that as a dummy interface is an even cleaner solution.
I have a method
using Microsoft.VisualStudio.TestTools.UnitTesting; // using visual studio's test framework
[TestMethod]
public void ATestMethod()
{
// stuff
}
from a public class ATestClass. This test class runs two types of tests :
tests requiring that a certain software is installed on the machine running the test
tests that can run free
To handle this, I added a public class BaseTestClass from which I made ATestClass derive, and in ATestClass I added a :
public bool isTheSoftwareInstalledOnTheMachine()
{
// stuff
}
and I "decorated" all internal scopes of tests from ATestClass as follows :
[TestMethod]
public void ATestMethod()
{
if (isTheSoftwareInstalledOnTheMachine())
{
// stuff
}
}
I find this horrible. I would rather like to be able to write something like :
[TestMethod]
[RunIfTheSoftwareInstalledOnTheMachine]
public void ATestMethod()
{
// stuff
}
but I don't know if one is allowed to define "custom" [characterizer]'s. (I don't even know the right word for them.) If it is, would that be the best design ? (I heard about the decorator pattern, but I don't know if I could make it generic enough in my context, because I would potentially need to use the condition for many other test classes.) Anyway, how would I proceed with characterizer's ?
I know you're using VS test framework but if you can change to NUnit you can accomplish what you want.
Test case:
using NUnit.Framework;
[TestFixture]
public class MyAppTests
{
[Test]
[RunIfTheSoftwareInstalledOnTheMachine]
public void ATestMethod()
{
// Executes if custom attribute is true, otherwise test case is ignored
}
}
Custom attribute:
using NUnit.Framework;
using NUnit.Framework.Interfaces;
public class TestHelper
{
public static bool IsTheSoftwareInstalledOnTheMachine()
{
// Return state of software
return true;
}
}
public class RunIfTheSoftwareInstalledOnTheMachineAttribute : Attribute, ITestAction
{
public ActionTargets Targets { get; private set; }
public void AfterTest(ITest test) {}
public void BeforeTest(ITest test)
{
if (!TestHelper.IsTheSoftwareInstalledOnTheMachine())
{
Assert.Ignore("Omitting {0}. Software is not installed on machine.", test.Name);
}
}
}
If you define your own attribute you surerly have to check for its existance on your own. You can´t expect your framework to guess what the attribute is for.
But I suppose you don´t even need an attribute to do this. You can simply ignore the test by putting the logic inside the test-method anyway:
[Test]
public void MyTest()
{
if(!RunIfTheSoftwareInstalledOnTheMachine)
Assert.Ignore("Test not run because no software was installed");
// your actual test-code
}
Another approach is to use the CategoryAttribute provided by NUnit, with which you can run only those tests that fall within your provided category:
[Test]
[Category("SoftwareInstalled")]
public void MyTest() { /* ... */ }
EDIT: You could also use the TestCaseAttribute with a specific method that returns a TestCase when the condition is met:
[TestCaseSource("ProvideTestcases")]
public void MyTest() { /* ... */ }
private static IEnumerable<TestCaseData> ProvideTestcases()
{
if(RunIfTheSoftwareInstalledOnTheMachine)
yield return new TestCaseData();
}
If the codition is not met no testcase is generated at all.
If the software being installed on the machine is a requirement for any of the tests to pass and any one test failing means the whole suite fails then why bother checking in multiple tests if the software is installed? Just write a single test to fail if the software is not installed and throw a useful exception. Something like:
[Test]
public void EnsureImportantSoftwareIsInstalled()
{
if(!importantSoftwareIsInstalled)
{
Assert.Fail($"Software X must be installed for the tests in {nameof(MyTestClass)} to run, please install it");
}
}
For Nunit 2.6, a slight variation of the HimBromBeere's answer works well for me. The test case is displayed as ignored.
[TestCaseSource("ProvideTestcases")]
public void MyTest() { /* ... */ }
private static IEnumerable<TestCaseData> ProvideTestcases()
{
if(RunIfTheSoftwareInstalledOnTheMachine)
yield return new TestCaseData().Ignore();
}
I googled for this issue but could not find the answer.
My test runs appear to run in parallel and cause each other to fail. They do all pass when run individually. I tried to add thread in the test and put them to sleep but no luck.
Is there a way to run these tests in sequence one after another?
My environment:
Visual Studio 2010
Resharper Jet brains 6.1
I would suggest you have unit tests that are deterministic. That is they don't depend on the order they are run or that other tests be run before or after. Not doing this is a recipe for failure. Most test runners are based on the fact that test methods are completely independent.
This fact is inherently obvious in the way the methods of a test class are invoked. e.g. with MS Test you can have Assembly, Class and Test initialize methods. All of these are invoked for each TestMethod being invoked. For example, with the following class:
[TestClass()]
public class DivideClassTest
{
[AssemblyInitialize()]
public static void AssemblyInit(TestContext context)
{
Console.WriteLine("Assembly Init");
}
[ClassInitialize()]
public static void ClassInit(TestContext context)
{
Console.WriteLine("ClassInit");
}
[TestInitialize()]
public void Initialize()
{
Console.WriteLine("TestMethodInit");
}
[TestCleanup()]
public void Cleanup()
{
Console.WriteLine("TestMethodCleanup");
}
[ClassCleanup()]
public static void ClassCleanup()
{
Console.WriteLine("ClassCleanup");
}
[AssemblyCleanup()]
public static void AssemblyCleanup()
{
Console.WriteLine("AssemblyCleanup");
}
[TestMethod()]
public void Test1()
{
Console.WriteLine("Test1");
}
[TestMethod()]
public void Test2()
{
Console.WriteLine("Test2");
}
}
You'll see output like
Assembly Init
ClassInit
TestMethodInit
Test1
TestMethodCleanup
TestMethodInit
Test2
TestMethodCleanup
ClassCleanup
AssemblyCleanup
Although there is a "Test" class, the TestMethod itself is considered the test. A "test" class can effectively have many tests.
I have some code that I need to see when I am running a test, and
TestContext would have to be part of my Test class I need the
debug.writelines of the class being tested displayed. I considered
just passing a TestContext to my MessageStatus static method and may
have to but that will be a PITA because the UnitTest class would have
to pass the TestContext to the object it is testing. Too tightly
coupled for my taste.
In basic terms
[TestMethod]
public void TestA()
{
//....
Assert.IsTrue(Blah.Blah()));
}
public void Blah()
{
Debug.WriteLine("Hello");
}
never shows up when I run the unit tests!
I could change it to:
TestContext t;
[TestMethod]
public void TestA()
{
//....
Assert.IsTrue(Blah.Blah(t)));
}
public void Blah(TestContext p1)
{
p1.WriteLine("Hello");
}
but that is insane it means changing all my signatures and tight coupling. I read the thread at How to write output from a unit test? it does not help :(
If you need to see lines produced by Debug.WriteLine or deal with assertions produced by Debug.Assert you can create your own System.Diagnostic.TraceListener to redirect output to TestContext - see my answer What do I have to do so that assertions won't block automated tests anymore?.
public class MyListenerThatDoesNotShowDialogOnFail:
System.Diagnostics.TraceListener
{
// This is to avoid message box on Debug.Assert(false);
public override void Fail(string message, string detailMessage)
{// do something UnitTest friendly here like Assert.Fail(false)
}
// This (+Write) is to redirect Debug.WriteLine("Some trace text");
public override void WriteLine(string message)
{// do something UnitTest friendly here like TestContext.Write(message)
}
}
Somewhere in the test setup listener. Note that sample below is simplified, you probably want to save current listeners and restore at the end of test.
Debug.Listeners.Clear();
Debug.Listeners.Add(new MyListenerThatDoesNotShowDialogOnFail());
You need to use Console.WriteLine for the output to show up in the test runner. However, this is a bad thing. I recommend using a logging framework like nlog.
I have an issue with NUnit - wondering if anyone has any ideas.
We're using NUnit 2.5.3.9345 and C# 3.5.
Take the following code:
public class UnitTestBase
{
[TestFixtureSetUp]
public void SetUpTestFixture()
{
//Do something in base
}
}
[TestFixture]
public class SomeTestClass : UnitTestBase
{
[TestFixtureSetUp]
public void FixtureSetUp()
{
//Do something in test class
}
[Test]
public void SomeTest()
{
//Some assertion
}
}
According to the documentation, if I run SomeTestClass.SomeTest(), UnitTestBase.SetUpTestFixture() should be called before SomeTestClass.FixtureSetUp().
This isn't the case - the base method will only be called if I don't provide a [TestFixtureSetUp] method in the derived class.
Any ideas please? Has me really puzzled!
Thanks.
I am not having the problem. I tested the outcome with the following:
Derived Test
[TestFixture]
public class DerivedTest : TestBase
{
[TestFixtureSetUp]
public void FixtureSetup()
{
File.AppendAllText("Out.txt", string.Format("TestFixtureSetUp From DerivedTest{0}", Environment.NewLine));
}
[TestFixtureTearDown]
public void FixtureTearDown()
{
File.AppendAllText("Out.txt", string.Format("TestFixtureTearDown Down From DerivedTest{0}", Environment.NewLine));
}
[SetUp]
public void Setup()
{
File.AppendAllText("Out.txt", string.Format("Setup From DerivedTest{0}", Environment.NewLine));
}
[TearDown]
public void Down()
{
File.AppendAllText("Out.txt", string.Format("TearDown From DerivedTest{0}", Environment.NewLine));
}
[Test]
public void DoATest()
{
File.AppendAllText("Out.txt", string.Format("Did a Test{0}", Environment.NewLine));
}
}
TestBase
public class TestBase
{
[TestFixtureSetUp]
public void BaseTestFixtureSetUp()
{
File.AppendAllText("Out.txt", string.Format("TestFixtureSetUp From TestBase{0}", Environment.NewLine));
}
[TestFixtureTearDown]
public void BaseTestFixtureTearDown()
{
File.AppendAllText("Out.txt", string.Format("TestFixtureTearDown From TestBase{0}", Environment.NewLine));
}
[SetUp]
public void BaseSetup()
{
File.AppendAllText("Out.txt", string.Format("Setup From TestBase{0}", Environment.NewLine));
}
[TearDown]
public void TearDown()
{
File.AppendAllText("Out.txt", string.Format("TearDown From TestBase{0}", Environment.NewLine));
}
}
This produces the following output:
TestFixtureSetUp From TestBase
TestFixtureSetUp From DerivedTest
Setup From TestBase
Setup From DerivedTest
Did a Test
TearDown From DerivedTest
TearDown From TestBase
TestFixtureTearDown Down From DerivedTest
TestFixtureTearDown From TestBase
I am was able to test the output with ReSharper 5 beta and the Nunit GUI v 2.5.3.9345 (32-bit)
Edit
While at work the test runner in ReSharper 4.5 did not work properly, however running the built test project in x86 and x64 with the corresponding NUnit.exe/NUnit-86.exe produced valid output.
A workaround / different way of doing it:
Instead of relying on behaviour that is not immediately clear, do something like this instead using the template method pattern to make the ordering explicit using normal language features:
public class UnitTestBase
{
protected abstract void PerFixtureSetUp();
[TestFixtureSetUp]
public void SetUpTestFixture()
{
PerFixtureSetUp();
}
}
[TestFixture]
public class SomeTestClass : UnitTestBase
{
protected override void PerFixtureSetUp()
{
}
[Test]
public void SomeTest()
{
//Some assertion
}
}
Any time I have had reason to use inherited fixtures or test contexts, this way has worked well enough. :)
My problem with relying on the attributes is that since these types are created and invoked via reflection in the runner with no relation between the methods, (no polymorphism) it's harder to reason about the order in which they're called. Using standard language features helps simplify this a little.
Have you tried giving the base class the [TestFixture] attribute? I don't know that that will fix it, but it seems like it might be worth a try... the idea is that NUnit may be ignoring the attributes on the base class if it's not a TestFixture.
Yep, I've been playing around with this for past half hour and it's definitely a bug. I've tried adding TestFixture to all classes as well as having different combinations. I've also tried static and instance methods. It just doesn't seem to want to play nicely! :-(
Anyway, the best workaround I could find is to put the TestFixtureSetUp code in the constructors of your test class and base class. At least then you can be confident of the inheritance and it's more clear to other readers of your code who maybe don't know the inner workings of NUnit :-)
What are you running your tests with? The behavior you are experiencing is not related to NUnit (framework), rather the runner you are using. Are you using Resharper integrated testrunner?