I'm working on unit-tests for an application which has a constructor that takes three values as arguments. The numbers shall be 0 or higher, and now I'm writing on an unit-test for the constructor that throws an exception if this is not the case.
What I can't figure out is how I what to write after "Assert" to determine this so that the test passes if illegal numbers are passed to the constructor. Thanks in advance.
EDIT: I'm using MSTest framework
public void uniqueSidesTest2()
{
try {
Triangle_Accessor target = new Triangle_Accessor(0, 10, 10);
}
catch (){
Assert // true (pass the test)
return;
}
Assert. // false (test fails)
}
// From the code...
public Triangle(double a, double b, double c) {
if ((a <= 0) || (b <= 0) || (c <= 0)){
throw new ArgumentException("The numbers must higher than 0.");
}
sides = new double[] { a, b, c };
}
First of all, you should throw an ArgumentOutOfRangeException rather than just an ArgumentException.
Second, your unit test should expect an Exception to be thrown, like so:
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public static void MyUnitTestForArgumentA()
{
...
}
So, you need to create separate unit tests -- one for each argument -- that test whether the method throws a correct exception when the argument is out of range.
No need to use a try catch block. Using NUnit or the MSTest framework you can use an attribute on your test method declaration to specify that you expect an exception.
MSTest
[TestMethod]
[ExpectedException(typeof(ArgumentException))]
public void uniqueSidesTest2()
It may not be the best solution, but if I'm testing to make sure an Exception is thrown, I will do something like the following:
public void uniqueSidesTest2()
{
try {
Triangle_Accessor target = new Triangle_Accessor(0, 10, 10);
Assert.Fail("An exception was not thrown for an invalid argument.");
}
catch (ArgumentException ex){
//Do nothing, test passes if Assert.Fail() was not called
}
}
Since your constructor call should throw an error, if it ever gets to the second line (The Assert.Fail() line) then you know it didn't properly throw the exception.
If you don't have nunit (or other framework that has this support built in you can use the following type of helper method
public static void ThrowsExceptionOfType<T>(Action action) where T: Exception
{
try
{
action();
}
catch (T)
{
return;
}
catch (Exception exp)
{
throw new Exception(string.Format("Assert failed. Expecting exception of type {0} but got {1}.", typeof(T).Name, exp.GetType().Name));
}
throw new Exception(string.Format("Assert failed. Expecting exception of type {0} but no exception was thrown.", typeof(T).Name));
}
Your test would look like this
AssertHelper.ThrowsExceptionOfType<ArgumentException>(
() =>
{
new Triangle_Accessor(0, 10, 10);
});
You will not need an Assert in the catch (but you might want to catch a more specific exception, like ArgumentException).
To always fail, there is an Assert.Fail.
You don't mention what framework you are using for unit testing, but I think what you're looking for is something like what is shown here:
http://www.nunit.org/index.php?p=exception&r=2.4
Related
A little confusion as to the behaviour of Moq with MsTest.
Edit: This is not a question of "How do I test?" or "How do I assert?", this is a scratch pad to see how MoQ works so don't focus on the exception type etc.
I think a better question may be => "Does Moq Throws<> behave similar to MsTest ExpectedExceptionAttribute?" That is, they're expecting an exception in the test or the SUT?
I'd like to know just how MoQ "Throws" works when used with MsTest. Is it better to not use the MsTest expected exception attribute? Is it better to perform a try..catch within the test? I have a few more questions surrounding this.
I am Mocking a database call and when an error occurs I would like to return zero (0).
The TestMethod is straight forward with the MsTest exception attribute, and the throws exception with Moq. It only works when I throw an exception within the SaveCart method and not when I return zero.
I would like to understand the underlying behaviour because it feels as though I shouldn't, nor want to throw an exception within the SaveCart method.
Here is the Test under question:
[TestMethod]
[ExpectedException(typeof(ApplicationException))]
public void CartRepoSaveCartExceptionShouldReturnZero()
{
_cartDatabaseMock.Setup(c => c.SaveCart(_cart))
.Throws<ApplicationException>();
var result = _cartRepository.SaveCart(_cart);
Assert.AreEqual(result, _cartSaveExceptionValue);
}
Here is the basic SaveCart which does NOT throw an exception causing the test to fail:
public long SaveCart(Cart cart )
{
long returnValue;
try
{
returnValue = _cartDatabase.SaveCart(cart);
}
catch (Exception)
{
return 0;
}
return returnValue;
}
Here is a basic SaveCart where the test works because it's throwing an exception:
public long SaveCart(Cart cart )
{
long returnValue;
try
{
returnValue = _cartDatabase.SaveCart(cart);
}
catch (Exception)
{
throw new ApplicationException();
}
return returnValue;
}
Feel free to suggest a better title for the question if it doesn't quite explain it clearly.
You should use ExpectedExceptionAttribute when the unit under test throws an exception.
In your first example the method didn't throw any exception therefore the test failed.
Since your method under test doesn't throw any exception you don't need to use this attribute at all...(just verify the return value in this scenario)
When you want to verify that exception was thrown and you want to verify that some additional operations occurred, use the following pattern:
[TestMethod]
[ExpectedException(typeof(<The specific exception>))]
public void FooTest()
{
//arrange
try
{
// act
}
catch(<the specific exception>)
{
// some asserts
throw;
}
}
The above snippet will failed if:
wrong exception raised
exception was not raised
one of your asserts failed.
BTW, since your catch in the method is no Exception instead of ApplicationException, I offer you to change the setup to:
_cartDatabaseMock.Setup(c => c.SaveCart(_cart)).Throws<Exception>();
You are right - the second test "SaveCart" works because it's throwing an exception and the the first test fail because you are turning 0. From your response to previous answers, I am sure you already know all of this. If you are asking for the behavior how it failed your first test... it goes like this:
SaveCart is called
It returns an exception (result of your moq setup)
Your try catch caught the exception (you did this on purpose to alter the result)
Your try catch returns 0 (result is now 0 as you intended to alter it)
Assert checks your result against _cartSaveExceptionValue
You get a fail test stating something similar to this "Message: Assert.AreEqual failed. Expected. Actual<0 (System.Int32)>."
If you want to double check this... you can try the following test
comment out the [ExpectedException(typeof())]
change the Assert.AreEqual(result, _cartSaveExceptionValue) to Assert.AreEqual(result, 0);
the test should pass because you are comparing "result" (aka 0) to 0
I hope this answer your question.
catch (Exception)
{
return 0;
}
you are not throwing the exception, rather swallowing the exception, so why would you expect exception? It has nothing to do with MOQ. Your test and code are not in sync.
This is a bad practice btw, to swallow exception.
catch (Exception)
{
throw new ApplicationException();
}
That's also a code smell. You are catching all kinds of exception and then throwing a different type.
In the test below, if it enters the catch block I want to indicate that the test has passed. If the catch block is bypassed I want the test to fail.
Is there a way to do this, or am I missing the point with how tests should be structured?
[TestMethod]
public void CommandExecutionWillThrowExceptionIfUserDoesNotHaveEnoughEminence()
{
IUserCommand cmd = CreateDummyCommand("TEST", 10, 10);
IUser user = new User("chris", 40);
try
{
cmd.Execute(user);
}
catch(UserCannotExecuteCommandException e)
{
//Test Passed
}
// Test Failed
}
I tend to use this pattern when I have a similar situation:
// ...
catch (UserCannotExecuteCommandException e)
{
return; // Test Passed
}
Assert.Fail(); // Test Failed -- expected exception not thrown
Declare the test to throw UserCannotExecuteCommandException, when that happens the test will succeed
[ExpectedException( typeof( UserCannotExecuteCommandException) )]
I would suggest using Assert.Throws() method:
Assert.Throws<UserCannotExecuteCommandException>() => cmd.Execute(user));
Id does all what you need. It expect that an exception of type UserCannotExecuteCommandException would be thrown whilst execution of the cmd.Execute() method, otherwise automatially marks a test as failed.
Since [ExpectedException(...)] is concidered as a bad practice you might use:
MSTest: Assert.ThrowsException<UserCannotExecuteCommandException >( () => cmd.Execute(user) );
NUnit: Assert.Throws<UserCannotExecuteCommandException >( () => cmd.Execute(user) );
then you are able to point which line exactly should throw an exception.
I have some cases where I don't care what exception is thrown (as long as some exception is thrown). Unfortunately,
Assert.Throws<Exception>(someDelegate);
doesn't pass unless exactly an instance of Exception (so not an instance of a derived class) is thrown. I know I can obtain the behavior I want with
Exception exception = Record.Exception(someDelegate);
Assert.NotNull(exception);
but it doesn't read right. Am I missing something in xUnit that has the behavior I want? Here are two tests that indicate what I mean:
[Fact]
public void Throws_exception_and_passes() {
Exception exception = Record.Exception(
() => { throw new InvalidOperationException(); }
);
Assert.NotNull(exception);
}
[Fact]
public void Throws_exception_and_fails() {
Assert.Throws<Exception>(
() => { throw new InvalidOperationException(); }
);
}
Per the documentation here:
http://xunit.codeplex.com/wikipage?title=HowToUse&referringTitle=Home
You have to specify the type of exception you want to be thrown. In general, this is good practice. You should be able to predict what scenarios a test would throw what type of exception. You should be able to design both you method and your test in a way that will allow you to predict this.
There are ways around this, like doing a try catch yourself, but you should look into changing your design a bit.
It didn't exist at the time of this question, but now one can use Assert.ThrowsAny<Exception> to test for any exception derived from Exception (and hence any exception at all), along with variants such as Assert.ThrowsAny<ArgumentException> which would test for any exception derived from ArgumentException and so on.
As you've identified if Assert.Throws<T> doesn't fit the bill, the only OOTB thing in xUnit you're left with is using Record.Exception.
As you've identified, the main way of doing a 'Assert throws anything` is to do
Assert.NotNull( Record.Exception( lambda ))
Look at it - not pretty. This is likely by design; there are very few things in xUnit.net that are by accident (as opposed to carefully considered opinionated design).
Record.Exception returns a result for a reason (and if you were using F#, you'd have to |> ignore to chuck away the value). You should always be able to Assert something about the nature of the Exception that's happening so that an actual problem in your code doesn't get ignored by chance as you change your code over time, which is the reason for all this testing stuff in the first place. Perhaps that might take the form of
var exception = Record.Exception( sut.Something );
Assert.True( typeof(SomeException).IsAssignableFrom( exception ) );
Looking at that, it's safer that an Assert.NotNull(), but still doesn't feel right. It's time to, as discussed in GOOS, listen to your tests (and in the case of an opinionated test framework, your test framework).
The biggest problem in your question is however that in a real example from a real test, there is always a way to make your interface clearer or express your expectation in another way, so the real answer is Mu.
xUnit won't stand in your way if you want to do your own Custom Assertion, something like:
public static bool Throws<T>(this Action action, bool discardExceptions = false)
where T : Exception
{
try
{
action.Invoke();
}
catch (T)
{
return true;
}
catch (Exception)
{
if (discardExceptions)
{
return false;
}
throw;
}
return false;
}
Or:
public static bool Throws(this Action action)
{
try
{
action.Invoke();
}
catch (Exception)
{
return true;
}
return false;
}
I was just looking in the xUnit.net source and here is the culprit:
private static Exception Throws(Type exceptionType, Exception exception)
{
Guard.ArgumentNotNull("exceptionType", exceptionType);
if (exception == null)
throw new ThrowsException(exceptionType);
if (!exceptionType.Equals(exception.GetType()))
throw new ThrowsException(exceptionType, exception);
return exception;
}
What would solve your problem is if this change were applied:
if(!exceptionType.Equals(exception.GetType()))
to:
if(!exception.GetType().IsAssignableTo(exceptionType))
You could possibly offer to submit a patch?
public static void SuppressException<TSut>(this TSut value, Action<TSut> action) where TSut : class
{
try
{
action.Invoke(value);
}
catch (Exception)
{
//do nothing
}
}
So, I have a list containing a custom class, MyClass
MyClass has properties, which can be null (but aren't meant to be).
When this class is sorted, using a custom sorter, where the sorter accesses this null property and throws an exception, the exception is considered unhandled, even though there is a try-catch block around the sort method.
Now for some reason the exception still gets written to the console, which is what the exception handler is doing.
I have a real application with this same issue, causing my unit tests to fail, even though the exception is handled correctly and I cannot explain this.
So I have attached some sample code to explain myself better, run this from VS.
Updated Code
Results:
System.InvalidOperationException
Failed to compare two elements in the array.
Done!
So it seems to be handling my custom exception, and throwing its own?
using System;
using System.Collections.Generic;
using System.Data;
namespace TestSortException
{
class Program
{
static void Main()
{
try
{
var list = new List<MyClass>
{
new MyClass("1"),
new MyClass(null),
new MyClass("fdsfsdf")
};
list.Sort(new MyClassSorter());
}
catch(Exception e)
{
Console.WriteLine(e.GetType());
Console.WriteLine(e.Message);
}
Console.WriteLine("Done!");
Console.ReadLine();
}
}
class MyClassSorter : IComparer<MyClass>
{
public int Compare(MyClass x, MyClass y)
{
// try
// {
if (x.MyString == y.MyString)
return 0;
// Unhandled??? Exception here
if (x.MyString.Length > y.MyString.Length)
return 1;
return -1;
// }
// catch (Exception)
// {
// return -1;
// }
}
}
class MyClass
{
private string _myString;
public string MyString
{
get
{
if (_myString == null) throw new DataException("MyString is Null");
return _myString;
}
}
public MyClass(string myString)
{
_myString = myString;
}
}
}
There's a try/catch block round the Sort method, yes - and that catch block catches the exception. In other words, Sort throws an exception and your catch block catches it. It doesn't propagate out beyond Main - so "Done!" is printed.
This is exactly what I'd expect. In what way is it "unhandled" in your experience? Were you expecting Sort not to throw the exception? It needs to do something to indicate the failure to compare two elements, and this seems to be the most appropriate course of action.
In what way are your unit tests failing? Are you deliberately giving them invalid data? How do you want your comparison code to react to invalid data? If it should ignore it (and return a comparison based on another property), then you should actively check the property rather than letting an exception propagate. In most cases I'd rather allow the exception if this indicates that there's a bug earlier on though.
EDIT: Based on your other comments, it sounds like you're doing the appropriate thing, letting the exception bubble up - but it's not clear in what way you're seeing the exception not be handled.
If you're running in the debugger, it may be breaking on the exception being thrown, but that doesn't mean it won't be handled. Try either changing your exception settings or running without the debugger.
EDIT: Yes, Sort will catch the exception and throw an InvalidOperationException instead - but you can use the InnerException property of that exception to get hold of the original one. It's unfortunate that the documentation doesn't specify this :(
For example, when it checks that string "1" isn't equal to null. But it wants then to compare lengths of "1" string and null => which is impossible.
I assume you work with .Net Framework 4.0. The new thing there is that a NullRefenrenceException can not be caught any more (similar to OutOfMemory exception).
I'm completely new at C# and NUnit.
In Boost.Test there is a family of BOOST_*_THROW macros. In Python's test module there is TestCase.assertRaises method.
As far as I understand it, in C# with NUnit (2.4.8) the only method of doing exception test is to use ExpectedExceptionAttribute.
Why should I prefer ExpectedExceptionAttribute over - let's say - Boost.Test's approach? What reasoning can stand behind this design decision? Why is that better in case of C# and NUnit?
Finally, if I decide to use ExpectedExceptionAttribute, how can I do some additional tests after exception was raised and catched? Let's say that I want to test requirement saying that object has to be valid after some setter raised System.IndexOutOfRangeException. How would you fix following code to compile and work as expected?
[Test]
public void TestSetterException()
{
Sth.SomeClass obj = new SomeClass();
// Following statement won't compile.
Assert.Raises( "System.IndexOutOfRangeException",
obj.SetValueAt( -1, "foo" ) );
Assert.IsTrue( obj.IsValid() );
}
Edit: Thanks for your answers. Today, I've found an It's the Tests blog entry where all three methods described by you are mentioned (and one more minor variation). It's shame that I couldn't find it before :-(.
I'm surprised I haven't seen this pattern mentioned yet. David Arno's is very similar, but I prefer the simplicity of this:
try
{
obj.SetValueAt(-1, "foo");
Assert.Fail("Expected exception");
}
catch (IndexOutOfRangeException)
{
// Expected
}
Assert.IsTrue(obj.IsValid());
If you can use NUnit 2.5 there's some nice helpers there.
Assert.That( delegate { ... }, Throws.Exception<ArgumentException>())
The MbUnit syntax is
Assert.Throws<IndexOutOfRangeException>(delegate {
int[] nums = new int[] { 0, 1, 2 };
nums[3] = 3;
});
Your preferred syntax:
Assert.Raises( "System.IndexOutOfRangeException",
obj.SetValueAt( -1, "foo" ) );
woiuldn't work with C# anyway - the obj.SetValueAt would be evaluated and the result passed to Assert.Raises. If SetValue throws an exception, then you'd never get into Assert.Raises.
You could write a helper method to do it:
void Raises<T>(Action action) where T:Exception {
try {
action();
throw new ExpectedException(typeof(T));
catch (Exception ex) {
if (ex.GetType() != typeof(T)) {
throw;
}
}
}
Which allows the similar syntax of:
Assert.Raises<System.IndexOutOfRangeException>(() =>
obj.SetValueAt(-1, "foo")
;
I've always adopted the following approach:
bool success = true;
try {
obj.SetValueAt(-1, "foo");
} catch () {
success = false;
}
assert.IsFalse(success);
...