I've looked at other post concerning unit testing but none that I saw actually test what's in the exception when thrown. The main goal is to raise an exception and examine the details of the stack trace by sending a bad parameter to a helper class.
Since the original code wasn't throwing an exception I decided to do some online research on NUnit testing and come across a very nice piece of code that is lot shorter then what I wrote but failes to examine the error object. I need to be able to Assert a presence of certain verbiage in the stack trace.
Originally this is what the code looked like, but I must admit it is not very pretty:
[Test]
public void TestExceptionHandling()
{
try
{
DoExceptionScenario(new SomeCustomRadioButtonControl(), FieldManager.GetField("access_mode"));
}
catch (Exception ex)
{
Assert.IsInstanceOf(typeof(CustomException), ex);
string details = Util.GetExceptionDetails((CustomException)ex);
Assert.IsTrue(details.Contains("Detail Name=\"ControlName\" Description=\"SomeCustomRadioButtonControl\""));
}
}
The problem as you may see is a possiblity of bunch of false positives.
The other way I modified the test is like this:
[Test]
public void TestExceptionHandling()
{
Assert.That(() => DoExceptionScenario(new SomeCustomRadioButtonControl(), FieldManager.GetField("access_mode")),
Throws.TypeOf<CustomException>());
}
This will fail if there is no exception. But if there is an exception how do I capture that and examine its contents? Something along lines of (the if statement would work in this case):
[Test]
public void ShouldControlNameBeListedInStackTrace()
{
bool exceptionStatus = Assert.That(() => DoExceptionScenario(new SomeCustomRadioButtonControl(), FieldManager.GetField("access_mode")),
Throws.TypeOf<CustomException>());
if (exceptionStatus != true)
{
string details = Util.GetExceptionDetails((CustomException)ex);
Assert.IsTrue(details.Contains("detail name=\"controlname\" description=\"SomeCustomRadioButtonControl\""));
}
}
Assume a CustomException class that looks like this. It doesn't do much of anything... just overrides the "Message" property from the base Exception class:
public class CustomException : Exception
{
private string message;
public override string Message
{
get { return string.Format("{0}: {1}", DateTime.Now, message); }
}
public CustomException(string message)
{
this.message = message;
}
}
And assume you have a method that throws the exception, such as this:
public class ProductionClass
{
public void SomeMethod()
{
throw new CustomException("Oh noz!");
}
}
Here's a few sample tests you can use in nUnit. You want the last one.
[TestFixture]
public class MyTests
{
private ProductionClass p;
[SetUp]
public void Setup()
{
p = new ProductionClass();
}
// Use the ExpectedException attribute to make sure it threw.
[Test]
[ExpectedException(typeof(CustomException)]
public void Test1()
{
p.SomeMethod();
}
// Set the ExpectedMessage param to test for a specific message.
[Test]
[ExpectedException(typeof(CustomException), ExpectedMessage = "Oh nozzzz!")]
public void Test2()
{
p.SomeMethod();
}
// For even more detail, like inspecting the Stack Trace, use Assert.Throws<T>.
[Test]
public void Test3()
{
var ex = Assert.Throws<CustomException>(() => p.SomeMethod());
Assert.IsTrue(ex.StackTrace.Contains("Some expected text"));
}
}
The Assert.Throws<T> method works with any Exception. It executes the delegate in the parentheses and detects whether or not it threw the exception.
In the above test, if it did throw, it then tests the stack trace for the specified contents too. If both steps pass, the test passes.
Related
In this test code, why does the test fail with an ArgumentNullException, despite that it is handled?
[Test]
public void ExceptionBehaviorTest()
{
// This works. An instance is returned
var testInstance = (TestClass)Activator.CreateInstance(typeof(TestClass), "Hello World");
Assert.NotNull(testInstance);
// This passes. Exception is generated and caught
Assert.Throws<ArgumentNullException>(() => new TestClass(null));
try
{
// This throws ArgumentNullException but the catch handler is not invoked. This fails the test
testInstance = (TestClass)Activator.CreateInstance(typeof(TestClass), (string)null);
Assert.Fail("Should not get here");
}
catch (ArgumentNullException)
{
}
}
private sealed class TestClass
{
public TestClass(string arg)
{
Argument = arg ?? throw new ArgumentNullException(nameof(arg));
}
public string Argument
{
get;
}
}
If I run the code in the debugger, it stops in the TestClass ctor, saying that the exception is not handled. But the calling function is visible in the stack, so the problem is not related to some part being executed on a different thread.
[Background: In my real-world code, I'm iterating over a bunch of classes and testing whether they have a ctor with a specific argument. This is to prevent runtime errors later, as the classes are constructed using dependency injection.]
This is in the documentation
TargetInvocationException
The constructor being called throws an
exception.
So you need to catch TargetInvocationException in this situation, also if you like you can use when though I am not sure it helps with your test much
catch (TargetInvocationException ex) when (ex.InnerException is ArgumentNullException)
{
Console.WriteLine("Caught");
}
How can I pass types into my unit tests?
public void MethodUnderTest()
{
try
{
var businessService = _businessService.DoWork();
}
catch (SomeException exception)
{
//do some stuff
}
catch (SomeOtherException exception)
{
//do other stuff
}
}
My unit test should be something like this:
[TestCase(typeof(SomeException))]
[TestCase(typeof(SomeOtherException))]
public void UnitTest(Exception exception)
{
_businessService.Setup(x=>x.DoWork).Throws.InstanceOf<exception>();
//verify that when we called DoWork, that the logic inside of one of the catches was executed
}
One way this can be implemented would be, by utilizing inferred generics ..
you will have a test case source in your test fixture.
public static List< Exception > Exceptions => new List< Exception >
{
new InvalidOperationException(),
new OverflowException()
};
and then modify your unit test method as :
[Test]
[TestCaseSource(nameof( Exceptions ))]
public void UnitTest<T>( T exception ) where T : Exception, new(){
_businessService.Setup(x=>x.DoWork).Throws.InstanceOf<T>();
//verify that when we called DoWork, that the logic inside of one of the catches was executed
}
Then the code would infer the types from the instance of the exceptions in the test case source. I can't think of a way to do this with out instantiating the exceptions ...
You haven't said what you have tried that didn't work. On the surface, it appears easy enough, but perhaps I don't understand what you want to do.
The example unit test you show is incorrect. It takes an Exception as an argument, but you are giving it a Type. Based on your title, the test method should accept a type. Then do something like...
[TestCase(typeof(SomeException))]
[TestCase(typeof(SomeOtherException))]
public void UnitTest(Type exceptionType)
{
Assert.That(()=>_businessService.Setup(x=>x.DoWork),
Throws.InstanceOf(exceptionType));
}
```
Did I misunderstand the problem?
I'm trying to run nunit test cases from the console application using reflection. I get an exception which is not handled by my catch block. Could you give an advice how to handle all the exceptions from the invoked test methods?
static void Main(string[] args)
{
// Take all classes of the current assebly to which TestFixture attribute is applied
var testClasses = Assembly.GetExecutingAssembly().GetTypes().Where(c =>
{
var attributes = c.GetCustomAttributes(typeof(TestFixtureAttribute));
return attributes.Any();
});
foreach (var testClass in testClasses)
{
var testMethods = testClass.GetMethods().Where(m =>
{
var attributes = m.GetCustomAttributes(typeof (TestAttribute));
return attributes.Any();
});
var instance = Activator.CreateInstance(testClass);
foreach (var method in testMethods)
{
try
{
Action action = (Action) Delegate.CreateDelegate(typeof (Action),
instance, method);
action();
}
catch (AggregateException ae)
{
Console.WriteLine(ae.Message);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
}
}
It's really unclear why you're trying to do this, since there's already nunit-console that can run unit tests from a console application. It's also unclear what exception you don't think is being caught, but I suspect it's not the type you think it is. I took your code and put it into a fresh console application, along with some very basic tests:
[TestFixture]
public class SomeFailingTests
{
[Test]
public void Fails()
{
Assert.AreEqual(1, 0);
}
[Test]
[ExpectedException(typeof(ArgumentException))]
public void TestExceptionExpected()
{
}
[Test]
public void TestThrows()
{
throw new InvalidOperationException();
}
[Test]
[ExpectedException(typeof(InvalidOperationException))]
public void TestThrowsExpected()
{
throw new InvalidOperationException();
}
}
All of the tests that threw exceptions were caught by the line:
catch (Exception e)
This makes sense, since none of them threw an AggregateException. I suspect whichever test you are running is also not throwing one and is also being caught by your outer catch. A good start might be to rewrite this block as:
catch (Exception e)
{
Console.WriteLine(string.Format("{0}: {1}", e.GetType().Name, e.Message));
}
So that you can see any exception types that you aren't handling. At a very basic level, you may have to account for AssertionException for example.
If you want to support a similar feature set to other nunit runners you're also going to need to pay attention to the ExpectedException attribute on any methods you run and check to see if that exception is thrown when you invoke the method. You'll also need to check for the Ignored attribute...
As has been mentioned in my answer to this question, you may also need to pay attention to other test attributes such as TestCase and TestCaseSource if you want to capture all of the tests in the assembly.
Unless you're writing this as a learning exercise, you might want to reconsider your approach.
I wonder whether there is a short and simple way for checking whether variable/property values match some condition?
Currently one of the most popular line in my code is similar to this one:
if (string.IsNullOrWhiteSpace(someFileName))
{
throw new NullReferenceException("'someFileName' must not be null.");
}
then the exception gets logged in the catch part and the execution continues and so on.
I don't like writing this line all over the place and just changing the variable name. It would be great if one could write something like this:
Assert.IsNotNullOrWhiteSpace(someFileName);
and it threw an exception saying that "{my variable} must not be null" with maybe some additional information like the parent class etc. that would help you to debug the code if you only have the logs available.
The problem with writing such a utility class that I encountered was that the thrown exception had of course the wrong stack trace like it happened in the utility method and not inside the method that called the assertion function.
This kind of value checking is required to especially work at runtime because I most of the time check user input like settings, paths, inputs etc.
EDIT:
I think I should have given an example of what I try to achieve:
public class FileExtractor {
public Form MainForm { get; set; }
public void ExtractFile(string fileName) {
Assert.IsNotNullOrWhiteSpace(fileName);
Assert.IsNotNull(MainForm);
// ...
}
}
and the let's call it Assert library should do this:
public static Assert {
public static void IsNotNullOrWhiteSpace(this string value) {
if (string.IsNullOrWhiteSpace(value)) {
// throw an exception like it occured in the ExtractFile
// the message should contain a hint like: "fileName must not be null"
}
}
public static void IsNotNull(this object value) {
if (value == null) {
// throw an excaption like it occured in the ExtractFile,
// the messagge should contain a hint like: "FileExtractor.MainForm must not be null."
}
}
EDIT-2
#CodeCaster - unfortunately I cannot not use C# 6 yet.
After some research and inspirated by two other questions here on stackoverflow
How to get Property Value from MemberExpression without .Compile()?
and
get name of a variable or parameter
I came up with this so far:
namespace ExceptionTest
{
class Program
{
static void Main(string[] args)
{
object test = null;
Assert.IsNotNull(() => test);
}
}
static class Assert
{
public static void IsNotNull<T>(Expression<Func<T>> expression)
{
MemberExpression memberExpr = expression.Body as MemberExpression;
var constExpr = memberExpr.Expression as ConstantExpression;
var value = (memberExpr.Member as FieldInfo).GetValue(constExpr.Value);
if (value == null)
{
throw new ArgumentNullException(memberExpr.Member.Name);
}
}
}
}
It almost does what I need. The last thing is to modify the stack trace so that it points to the Main method and not to the IsNotNull
You could use Debug Methods (http://msdn.microsoft.com/en-us/library/System.Diagnostics.Debug_methods%28v=vs.110%29.aspx), which however only work when compiling in debug mode.
Maybe Debug.WriteLineIf(Boolean, String) does what you need?
http://msdn.microsoft.com/en-us/library/y94y4370%28v=vs.110%29.aspx
How about applying attributes to the properties
http://msdn.microsoft.com/en-us/library/dd901590(VS.95).aspx
I think that you should try with Fody library library. For null-guards there is a package that you can find here. All libs are available via Nuget.
Fody is some kind AOP library that uses "weaving" technique to manipulate IL of an assembly and inject additional code.
So NullReferenceExcpetion (or maybe NullArgumentException) will be thrown exactly from your method.
Example from GitHub:
Your code
public void SomeMethod(string arg)
{
// throws ArgumentNullException if arg is null.
}
public void AnotherMethod([AllowNull] string arg)
{
// arg may be null here
}
What gets complied
public void SomeMethod(string arg)
{
if (arg == null)
{
throw new ArgumentNullException("arg");
}
}
public void AnotherMethod(string arg)
{
}
I have this:
Expect.Once.On( someObj ).Method( "SomeMethod" )
.With(1) // correct value is 2, I want this to fail
.Will( Throw.Exception( new Exception() ) );
An exception is thrown by nmock when it detects that I put 1 instead of 2. However, the test is failing (red) instead of passing. How to make this test pass, even though I'm expecting an exception?
If you're using NUnit then you can do:
Assert.Throws<Exception>(() => { someObj.SomeMethod(1); });
You can also decorate the test with an ExpectedException attribute, although that will cause the test to pass if any Exception is thrown, rather than just the statement you want to test.
EDIT: If you're using MSTest, as far as I know, you can only use attributes to expect exceptions i.e.
[ExpectedException(typeof(Exception)]
public void TestMethod() { ... }
You should consider throwing a more specific exception type from your mock and expecting that type instead of a plain Exception.
You could also define your own method to replicate the NUnit functionality:
public static class ExceptionAssert
{
public static void Throws<T>(Action act) where T : Exception
{
try
{
act();
}
catch (T ex)
{
return;
}
catch (Exception ex)
{
Assert.Fail(string.Format("Unexpected exception of type {0} thrown", ex.GetType().Name));
}
Assert.Fail(string.Format("Expected exception of type {0}", typeof(T).Name));
}
}
[ExpectedException (typeof(Exception))]
Edit: thanks, don't have the studio right now and was not 100% sure about the syntax.