Develop Reference

xunit skip tests if other tests failed

The problem: I have a bunch of xunit tests that connect to an external server. If that server is up - they fly through very quickly. If that server is down, everything goes into timeouts and retries and the tests all take a long time. The thing is, if I have 100 tests - if the first one fails, I still want all the rest to fail, but it's completely pointless running them... What I'd like is to be able to make my own fact attribute something like:
[FactInFailureGroup( "ConectsToBlah", typeof(ConnectionException)]
public void TestOne()
{
...
}
[FactInFailureGroup( "ConnectsToBlah", typeof(ConnectionException)]
public void TestTwo()
{
...
}
I've looked at before/after attributes, but they don't seem to be able to kill the test or see the result - and I've looked at creating my own fact attribute, which looks like it can prevent the test from running, but not put in a trap for the result. I really want to somehow make code along these lines:
class FactInFailureGroupAttribute : FactAttribute
{
private static _failedGroups = new HashSet<String>();
...
void BeforeTest( TestManipulator manipulator )
{
if (_failedGroups.contains( _thisGroup ))
manipulator.SkipWithMessage( f"all tests in group {_thisGroup} failed because of {_exceptionType}");
}
void AfterTest( TestResult test )
{
if (test.Failed && test.Exception.GetType() == _exceptionType)
_failedGroups.add( _thisGroup );
}
}

You can abuse constructors and statics to accomplish your task.
The constructor of your test class will get called at the beginning of every test but your static should exist for your entire test run.
Try something like this:
private static object Lock = new object();
private static bool? IsServerUp;
public FactAssertionTests()
{
lock (Lock)
{
if (!IsServerUp.HasValue)
{
// check server connection and set IsServerUp
}
else if (!IsServerUp.Value)
{
throw new Exception("Failed to connect to server");
}
}
}
That way whatever test gets into the Lock first will check the server's up-ness. If it fails, every test coming in next will throw the exception.
This isn't quite a decorator attribute and requires you to put all the tests into one class, but it's simple.
Edit:
I tested the following approach, it relies on the fact that xUnit runs all tests in a single class in serial. If you've got tests spanning multiple classes this won't work:
public class ConnectionTests
{
[Fact]
public void Test1()
{
RunServerTest(() =>
{
var svc = new ServiceConnection();
svc.Connect();
});
}
[Fact]
public void Test2()
{
RunServerTest(() =>
{
var svc = new ServiceConnection();
svc.Connect();
});
}
private static bool? ServerIsDown;
public void RunServerTest(Action testAction)
{
var exception = new Exception("Server is down, test will not run.");
if (ServerIsDown.GetValueOrDefault())
{
throw exception;
}
try
{
testAction.Invoke();
}
catch (ServerMissingException ex)
{
ServerIsDown = true;
throw;
}
}
}

You can accomplish what you want by calling your first test explicitly in the constructor of the class. If it fails due to server missing it should throw an exception. This will prevent the class from being instantiated, and none of the other tests in the class will run. They will be marked as failed in the test explorer.
public class SO74607887Tests
{
public SO74607887Tests()
{
TestOne();
}
[Fact]
public void TestOne()
{
// Arrange
// Act
// Oops server is down
throw new Exception("Server is down");
}
[Fact]
public void TestTwo()
{
// Will not run
}
}
If your tests are distributed over a lot of different classes, you can follow #JeremyLakeman 's suggestion to use a fixture. Proceed as follows:
using System;
using Xunit;
namespace SO74607887_XunitCancelTestsIfOneFails
{
// This class will never actually be instantiated, it is only present to provide information to Xunit.
[CollectionDefinition("SO74607887")]
public class SO74607887CollectionDefinition : ICollectionFixture<SO74607887Base>
{
}
// This class creates a single object injected into the constructors of all the other classes in the collection.
public class SO74607887Base : IDisposable
{
public bool serverOK;
public SO74607887Base()
{
// Check server, if it is missing, set the flag
serverOK = false;
}
public void Dispose()
{
// Clean up
}
}
[Collection("SO74607887")]
public class SO74607887Tests
{
public SO74607887Tests(SO74607887Base basis)
{
if (!basis.serverOK)
{
throw new Exception($"Server is down, tests in {nameof(SO74607887Tests)} will not run.");
}
}
[Fact]
public void TestOne()
{
// Arrange
// Act
// Assert
}
[Fact]
public void TestTwo()
{
// Arrange
// Act
// Assert
}
}
}
The check on the server is only done once in the fixture. All the other classes only need to check if it is available. The fixture constructor is guaranteed to run before the test classes are instantiated.
Instead of a boolean to check, you could also simply make a method checkServer(string forClass) in the fixture class to call which itself throws the exception, then you'd only have to call
basis.checkServer(nameof(SO74607887Tests));
in the test classes instead of throwing in each test class.
All the tests in the classes will be marked as failed in the Test Excplorer window:

Getting property name from within property type

I couldn't figure out how to formulate the title any better. I see quite a few posts with similar titles but which discuss entirely different stuff.
So here we go. The actual on-ground situation is complex, but I'll try to post an absolute minimalistic example to describe it.
Let's say we have a class named Animal:
class Animal
{
public void Run()
{
try
{
//try running
}
catch(Exception e)
{
MessageBox.Show(this.SomeCleverWayOfGettingPropertyName() + " failed to run");
}
}
}
Now I define several properties of Animal type in another class:
class Zoo
{
public Animal Zebra {get; set;}
public Animal Lion {get; set;}
public Animal Rhino {get; set;}
public void RunAll()
{
Zebra.Run();
Lion.Run();
Rhino.Run();
}
}
What do I write in place of SomeCleverWayOfGettingPropertyName() to let it show name of the animal (that is name of the declared property), like "Zebra failed to run".
As I said, the actual situation is more complex, so kindly avoid answers like, "why don't you redesign your entire code base and instead try X". My hope is to find something in System.Reflection to find out the calling member's name, but I haven't found anything like that yet.

Ideally you would rethink your problem, and possibly catch outside of the run
Depending on your exact needs, an expression might work.. However it really is a terrible solution, if you went to all the effort you might as well catch outside, or just pass the member name in.
Given
public class Animal
{
public void Run()
{
Console.WriteLine("Running");
}
}
public static class MemberInfoGetting
{
public static void Run<T>(this Expression<Func<T>> memberExpression) where T : Animal
{
var expressionBody = (MemberExpression)memberExpression.Body;
try
{
var animal = Expression.Lambda<Func<Animal>>(expressionBody).Compile()();
animal.Run();
throw new Exception("bob");
}
catch
{
Console.WriteLine($"{expressionBody.Member.Name} : failed to run");
}
}
}
Usage
public static Animal Rhino { get; set; } = new Animal();
public static void Main()
{
MemberInfoGetting.Run(() => Rhino);
}
Output
Running
Rhino : failed to run

This is basically not possible with this approach. What happens when you call Zebra.Run():
Runtime calls the auto-generated get_Zebra() method, putting the Zebra's Animal instance pointer on the stack.
Runtime calls the Animal.Run() instance method.
All variable/property info about where that instance came from is pretty much gone at that point.
Now Animal.Run() doesn't know it's being called on an instance that came from a property, and there's no guarantee it will be. It could as well be a local, a method parameter or a new()ed instance, one from a factory or a collection element. You'll have to pass this info yourself.
Alternatively, if it's for error handling, it may be easier than you think without having to resolve to compiler magic or expensive expression refactoring:
In your exception handler, log the relevant properties that identify the Animal instance. Combined with the stack trace, this should give you enough information.

you can try this:
class Animal
{
public void Run([CallerMemberName] string caller = null)
{
try
{
//try running
}
catch(Exception e)
{
MessageBox.Show(caller + " failed to run");
}
}
}

The only way to reasonable do this is change RunAll() such that it monitors each call, to the now modified run
class Animal
{
static readonly Random rng = new Random();
public bool Run()
{
if (rng.NextDouble() < 0.5)
{
return false;
}
return true;
}
}
class Zoo
{
...
public void RunAll()
{
try
{
if (!Zebra.Run())
{
throw new Exception(nameof(Zebra));
}
if (!Lion.Run())
{
throw new Exception(nameof(Lion));
}
if (!Rhino.Run())
{
throw new Exception(nameof(Rhino));
}
}
catch (Exception ex)
{
Debug.WriteLine($"{ex.Message} failed to run.");
}
}
}

How to test virtual methods using Moles?

How can I test the IsHappy function using Moles?
class SomeClass
{
protected virtual bool IsHappy(string mood)
{
return (mood == "Happy");
}
}
I tried to test if by using Stub:
SSomeClass stub = new SSomeClass();
stub.CallBase = true;
Assert.IsTrue(stub.IsHappyString("Happy"));
... but the IsHappyString method returns null thus throwing a NullReference exception.
So, how can I test the default implementation of IsHappy method?

I'd forget about stubs here. Stubs/mocks are for when you want to fake the behavior of a dependency. You'd stub your SomeClass if had SomeClassClient that you wanted to test and it used SomeClass:
public class Foo
{
public virtual int GetFoosInt()
{
return 12;
}
}
public class FooClient
{
private Foo _foo;
public FooClient(Foo foo)
{
_foo = foo;
}
public int AddOneToFoosInt()
{
return _foo.GetFoosInt() + 1;
}
}
In this example, when testing FooClient, what you want to test is that it returns one more than "GetFoosInt()". You don't actually care what FoosInt is for testing the FooClient. So, you create a Foo stub where you can setup GetFoosInt to return whatever you want.
In your case, testing a protected virtual member, I'd go with this:
[TestClass]
public class SomeClassTest
{
private class DummySomeClass : SomeClass
{
public bool IsHappyWrapper(string mood)
{
return IsHappy(mood);
}
}
[TestMethod]
public void SomeTest()
{
var myClass = new DummySomeClass();
Assert.IsTrue(myClass.IsHappyWrapper("Happy"));
}
}
This gives you 'direct' access to the protected virtual to test default behavior. Only word of caution is that if you start defining abstract members and adding to SomeClass in general, you'll have to add them to this dummy inheritor as well, adding to testing maintenance overhead.
The purist in me says that you should leave protected members alone and only test them through the public interface. But, that may or may not be practical in your situation, and I don't really see any harm in this approach.

Stubs and Moles are for isolating a class from any dependencies it has, either environmental dependencies or class dependencies. This class has no dependencies whatsoever, so why are you trying to mole or stub it?
If you want to make sure this base class works properly when people override it, then you'll need to create a test implementation. In that case this is more or less what your test cases should look like:
public SomeClassTestAdapter : SomeClass
{
public bool GetIsHappy(string mood)
{
return IsHappy(mood);
}
}
[Test]
public void ShouldReturnTrueWhenPassedHappy()
{
var classUnderTest = new SomeClassTestAdapter();
bool result = classUnderTest.IsHappy("Happy");
Assert.IsTrue(result, "Expected result to be true");
}
[Test]
public void ShouldReturnFalseWhenPassedLowerCaseHappy()
{
var classUnderTest = new SomeClassTestAdapter();
bool result = classUnderTest.IsHappy("happy");
Assert.IsFalse(result, "Expected result to be false");
}
[Test]
public void ShouldReturnFalseWhenPassedNull()
{
var classUnderTest = new SomeClassTestAdapter();
bool result = classUnderTest.IsHappy(null);
Assert.IsFalse(result, "Expected result to be false");
}
Etc.
There is no place in this code that stubs or moles should be squeezed in.
If you don't want to create an adapter class for this case, you can use built-in .Net features rather than a big, paid dependency like Moles. Reflections and dynamic let you get access to protected or private members. See this example:
http://igoro.com/archive/use-c-dynamic-typing-to-conveniently-access-internals-of-an-object/

Handle base class exception

I have a following C# scenario-
I have to handle an exception in base class that actually occurs in the derived class.
My base class looks like this:
public interface A
{
void RunA();
}
public class Base
{
public static void RunBase(A a)
{
try
{
a.RunA();
}
catch { }
}
}
The derived class is as follows:
public class B: A
{
public void RunA()
{
try
{
//statement: exception may occur here
}
catch{}
}
}
I want to handle the exception, lets say exception C, that occurs in B(at //statement above).
The exception handling part should be written in base class catch inside RunBase. How can this be done?

public class Base
{
public static void RunBase(A a)
{
try
{
a.RunA();
}
catch(SomeSpecialTypeOfException ex)
{
// Do exception handling here
}
}
}
public class B: A
{
public void RunA()
{
//statement: exception may occur here
...
// Don't use a try-catch block here. The exception
// will automatically "bubble up" to RunBase (or any other
// method that is calling RunA).
}
}

How can this be done?
What do you mean? Just remove the try-catch block from RunA.
Having said that, you need to make sure Class A knows how to handle the exception, this includes streamlining it to UI, logging, ... This is in fact rare for a base class. Handling exception normally happen at the UI level.

public class B: A
{
public void RunA()
{
try
{
// statement: exception may occur here
}
catch(Exception ex)
{
// Do whatever you want to do here if you have to do specific stuff
// when an exception occurs here
...
// Then rethrow it with additional info : it will be processed by the Base class
throw new ApplicationException("My info", ex);
}
}
}
You also might want to throw the exception as it is (use throw alone).
If you dont need to process anything here, dont put try{} catch{}, let the exception bubble up by itself and be processed by the Base class.

Just remove the try catch from class B, if the exception occurs it will propergate up the call chain until it is handled. In this case you can handle the exception in RunBase using your existing try catch block.
Though in your example B isn't derived from your base class Base. If you really want to handle a situation where an exception is thrown in a derived class in its parent you could try something like:
public class A
{
//Public version used by calling code.
public void SomeMethod()
{
try
{
protectedMethod();
}
catch (SomeException exc)
{
//handle the exception.
}
}
//Derived classes can override this version, any exception thrown can be handled in SomeMethod.
protected virtual void protectedMethod()
{
}
}
public class B : A
{
protected override void protectedMethod()
{
//Throw your exception here.
}
}

How to convert an exception to another one using PostSharp?

I would like to automagically add the following code around the body of some methods:
try
{
// method body
}
catch (Exception e)
{
throw new MyException("Some appropriate message", e);
}
I am working with PostSharp 1.0 and this is what I've done at the moment:
public override void OnException(MethodExecutionEventArgs eventArgs)
{
throw new MyException("Some appropriate message", eventArgs.Exception);
}
My problem is that I can see the PostSharp OnException call in the stack.
What would be the good practice to avoid this and get the same call stack as implementing by hand the exception handler?

There is no way to hide "OnException" from the call stack.

Two things working in tandem will allow you to do this:
The fact that Exception.StackTrace is virtual
The use of the skipFrames parameter to the StackFrame constructor. This is not required, but makes things easier
The below example demonstrates how to customize the stack trace. Note that I know of no way to customize the Exception.TargetSite property, which still gives the details of the method from which the exception originated.
using System;
using System.Diagnostics;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
// exception is reported at method A, even though it is thrown by method B
MethodA();
}
private static void MethodA()
{
MethodB();
}
private static void MethodB()
{
throw new MyException();
}
}
public class MyException : Exception
{
private readonly string _stackTrace;
public MyException()
{
// skip the top two frames, which would be this constructor and whoever called us
_stackTrace = new StackTrace(2).ToString();
}
public override string StackTrace
{
get { return _stackTrace; }
}
}
}