I'm trying to run nunit test cases from the console application using reflection. I get an exception which is not handled by my catch block. Could you give an advice how to handle all the exceptions from the invoked test methods?
static void Main(string[] args)
{
// Take all classes of the current assebly to which TestFixture attribute is applied
var testClasses = Assembly.GetExecutingAssembly().GetTypes().Where(c =>
{
var attributes = c.GetCustomAttributes(typeof(TestFixtureAttribute));
return attributes.Any();
});
foreach (var testClass in testClasses)
{
var testMethods = testClass.GetMethods().Where(m =>
{
var attributes = m.GetCustomAttributes(typeof (TestAttribute));
return attributes.Any();
});
var instance = Activator.CreateInstance(testClass);
foreach (var method in testMethods)
{
try
{
Action action = (Action) Delegate.CreateDelegate(typeof (Action),
instance, method);
action();
}
catch (AggregateException ae)
{
Console.WriteLine(ae.Message);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
}
}
It's really unclear why you're trying to do this, since there's already nunit-console that can run unit tests from a console application. It's also unclear what exception you don't think is being caught, but I suspect it's not the type you think it is. I took your code and put it into a fresh console application, along with some very basic tests:
[TestFixture]
public class SomeFailingTests
{
[Test]
public void Fails()
{
Assert.AreEqual(1, 0);
}
[Test]
[ExpectedException(typeof(ArgumentException))]
public void TestExceptionExpected()
{
}
[Test]
public void TestThrows()
{
throw new InvalidOperationException();
}
[Test]
[ExpectedException(typeof(InvalidOperationException))]
public void TestThrowsExpected()
{
throw new InvalidOperationException();
}
}
All of the tests that threw exceptions were caught by the line:
catch (Exception e)
This makes sense, since none of them threw an AggregateException. I suspect whichever test you are running is also not throwing one and is also being caught by your outer catch. A good start might be to rewrite this block as:
catch (Exception e)
{
Console.WriteLine(string.Format("{0}: {1}", e.GetType().Name, e.Message));
}
So that you can see any exception types that you aren't handling. At a very basic level, you may have to account for AssertionException for example.
If you want to support a similar feature set to other nunit runners you're also going to need to pay attention to the ExpectedException attribute on any methods you run and check to see if that exception is thrown when you invoke the method. You'll also need to check for the Ignored attribute...
As has been mentioned in my answer to this question, you may also need to pay attention to other test attributes such as TestCase and TestCaseSource if you want to capture all of the tests in the assembly.
Unless you're writing this as a learning exercise, you might want to reconsider your approach.
Related
In this test code, why does the test fail with an ArgumentNullException, despite that it is handled?
[Test]
public void ExceptionBehaviorTest()
{
// This works. An instance is returned
var testInstance = (TestClass)Activator.CreateInstance(typeof(TestClass), "Hello World");
Assert.NotNull(testInstance);
// This passes. Exception is generated and caught
Assert.Throws<ArgumentNullException>(() => new TestClass(null));
try
{
// This throws ArgumentNullException but the catch handler is not invoked. This fails the test
testInstance = (TestClass)Activator.CreateInstance(typeof(TestClass), (string)null);
Assert.Fail("Should not get here");
}
catch (ArgumentNullException)
{
}
}
private sealed class TestClass
{
public TestClass(string arg)
{
Argument = arg ?? throw new ArgumentNullException(nameof(arg));
}
public string Argument
{
get;
}
}
If I run the code in the debugger, it stops in the TestClass ctor, saying that the exception is not handled. But the calling function is visible in the stack, so the problem is not related to some part being executed on a different thread.
[Background: In my real-world code, I'm iterating over a bunch of classes and testing whether they have a ctor with a specific argument. This is to prevent runtime errors later, as the classes are constructed using dependency injection.]
This is in the documentation
TargetInvocationException
The constructor being called throws an
exception.
So you need to catch TargetInvocationException in this situation, also if you like you can use when though I am not sure it helps with your test much
catch (TargetInvocationException ex) when (ex.InnerException is ArgumentNullException)
{
Console.WriteLine("Caught");
}
How can I pass types into my unit tests?
public void MethodUnderTest()
{
try
{
var businessService = _businessService.DoWork();
}
catch (SomeException exception)
{
//do some stuff
}
catch (SomeOtherException exception)
{
//do other stuff
}
}
My unit test should be something like this:
[TestCase(typeof(SomeException))]
[TestCase(typeof(SomeOtherException))]
public void UnitTest(Exception exception)
{
_businessService.Setup(x=>x.DoWork).Throws.InstanceOf<exception>();
//verify that when we called DoWork, that the logic inside of one of the catches was executed
}
One way this can be implemented would be, by utilizing inferred generics ..
you will have a test case source in your test fixture.
public static List< Exception > Exceptions => new List< Exception >
{
new InvalidOperationException(),
new OverflowException()
};
and then modify your unit test method as :
[Test]
[TestCaseSource(nameof( Exceptions ))]
public void UnitTest<T>( T exception ) where T : Exception, new(){
_businessService.Setup(x=>x.DoWork).Throws.InstanceOf<T>();
//verify that when we called DoWork, that the logic inside of one of the catches was executed
}
Then the code would infer the types from the instance of the exceptions in the test case source. I can't think of a way to do this with out instantiating the exceptions ...
You haven't said what you have tried that didn't work. On the surface, it appears easy enough, but perhaps I don't understand what you want to do.
The example unit test you show is incorrect. It takes an Exception as an argument, but you are giving it a Type. Based on your title, the test method should accept a type. Then do something like...
[TestCase(typeof(SomeException))]
[TestCase(typeof(SomeOtherException))]
public void UnitTest(Type exceptionType)
{
Assert.That(()=>_businessService.Setup(x=>x.DoWork),
Throws.InstanceOf(exceptionType));
}
```
Did I misunderstand the problem?
I have this:
Expect.Once.On( someObj ).Method( "SomeMethod" )
.With(1) // correct value is 2, I want this to fail
.Will( Throw.Exception( new Exception() ) );
An exception is thrown by nmock when it detects that I put 1 instead of 2. However, the test is failing (red) instead of passing. How to make this test pass, even though I'm expecting an exception?
If you're using NUnit then you can do:
Assert.Throws<Exception>(() => { someObj.SomeMethod(1); });
You can also decorate the test with an ExpectedException attribute, although that will cause the test to pass if any Exception is thrown, rather than just the statement you want to test.
EDIT: If you're using MSTest, as far as I know, you can only use attributes to expect exceptions i.e.
[ExpectedException(typeof(Exception)]
public void TestMethod() { ... }
You should consider throwing a more specific exception type from your mock and expecting that type instead of a plain Exception.
You could also define your own method to replicate the NUnit functionality:
public static class ExceptionAssert
{
public static void Throws<T>(Action act) where T : Exception
{
try
{
act();
}
catch (T ex)
{
return;
}
catch (Exception ex)
{
Assert.Fail(string.Format("Unexpected exception of type {0} thrown", ex.GetType().Name));
}
Assert.Fail(string.Format("Expected exception of type {0}", typeof(T).Name));
}
}
[ExpectedException (typeof(Exception))]
Edit: thanks, don't have the studio right now and was not 100% sure about the syntax.
I'm using NUnit and Rhino Mocks. I use the AAA-syntax and I do the Arrange and Act in the setup method, and every Test method is an Assert.
[TestFixture]
public class When_subSystem_throws_exception
{
SomeClass subject; // System under test
[SetUp]
public void Setup()
{
// Arrange
IDependency dependency = MockRepository.GenerateStub<IDependency>();
dependency.Stub(m => m.DoStuff()).Throw(new Exception()); // This method is called from within SomeMethod()
subject = new SomeClass(dependency);
// Act
subject.SomeMethod("Invalid Input");
}
// Assert
[Test]
public void should_log_an_exception_to_the_logger()
{
// Do stuff to verify that an exception has been logged
}
// More tests
}
As you might expect, the code in SomeMethod() throws an exception (as expected), wich makes every test fail (unwanted). I workaround this by doing
try
{
// Act
subject.SomeMethod("Invalid Input");
}
catch(Exception ex)
{
// Swallow, this exception is expected.
}
But that is just ugly.
What I would like to be able to do is
[SetUp]
[ExpectedException] // <-- this works for Test methods, but not for SetUp methods
public void Setup()
{
// etc...
}
but I can't find anything like it.
Do you know of anything?
I don't think using an attribute like ExpectedException is a good idea.
SetUp is to prepare something for the test methods, it shouldn't throw exception.
If it must throw, and you want to limit the code line number. Then put them into one line like below:
try { subject.SomeMethod("Invalid Input"); }catch { }
It doesn't work in Setup for a reason, not because of NUnit's bug.
It's a very bad practice for a unit-test to have exception throwing inside the SetUp method. If you are testing a particular scenario where a exception is the expected result, it should be done inside a [Test] method. You should rearrange your code subsequently.
Your "act" step should be in the test method not the setup.
The setup is for setting up pre-requisite conditions and common objects for the test(s) - i.e. common or repeated "arrange" steps.
Each test method should "act" and "assert" individually (and may also need additional "arrange" steps specific to the test).
If I want to test that a method throws an exception of a particular type, NUnit's ExpectedException attribute doesn't care about the actual type; if I throw a generic Exception before the method call, the test passes:
[Test, ExpectedException(typeof(TestCustomException))]
public void FirstOnEmptyEnumerable()
{
throw new Exception(); // with this, the test should fail, but it doesn't
this.emptyEnumerable.First(new TestCustomException());
}
If I want to check that the test throws the exact exception type, I have to do something manual like this:
[Test]
public void FirstOnEmptyEnumerable()
{
try
{
throw new Exception(); // now the test fails correctly.
this.emptyEnumerable.First(new TestCustomException());
}
catch (TestCustomException)
{
return;
}
Assert.Fail("Exception not thrown.");
}
Am I missing something?
I've never used ExpectedException, so I don't have any experience to share on this. An option is to Assert that it Throws directly inside the test. Something like this:
[Test]
public void FirstOnEmptyEnumerable()
{
Assert.Throws<TestCustomException>(() => this.emptyEnumerable.First(new TestCustomException()));
}
I find this approach more readable as you test for the exception exactly where you expect it to happen instead of saying "somewhere inside this function I except an exception to be thrown".
I always test for the string representation of the exception e.g.:
[Test, ExpectedException("Your.Namespace.TestCustomException")]
public void FirstOnEmptyEnumerable()
{
throw new Exception(); // with this, the test should fail, but it doesn't
this.emptyEnumerable.First(new TestCustomException());
}
Which seems to work fine for me.
If you want to use the ExpectedException(string) signature, the best practice would be to use typeof(Exception).Name and typeof(Exception).Namespace