Is there any way how to test contracts set using Code contracts? - c#

I would like to write unit test that verify that my method does not accept invalid arguments. Validity of arguments is checked using Code Contract's Contract.Requires call. Why would I want to test contracts? I consider my tests to be kind of method specification (that is actually idea taken from TDD) so by testing that the method fails for some arguments I specify that such arguments should not be used.
The problem is that since I started to use Code contracts I cannot test method contracts because I cannot access the exception that is thrown by Contract.Requires. I can catch generic Exception but that just is not nice... Is there recommended/supported way how to test contract set using Code Contracts?
Seems to me that Code Contracts does not really support unit testing...
EDIT: My test example (I am forced to catch generic exception)
[ExpectedException(typeof(Exception), AllowDerivedTypes = true)]
public void Compute_Throws_ForNullArgument()
{
new ComputingService().Compute(null);
}

You can hook into the Contract.ContractFailed event.
http://msdn.microsoft.com/en-us/library/system.diagnostics.contracts.contract.contractfailed(v=vs.110).aspx
This will be raised before the exception is thrown. You can combine this with catching Exception to be pretty certain that it represented a contract failure
public void VerifyContract(Action action) {
bool failed = false;
bool thrown = false;
EventHandler e = (sender, e) => { failed = true; }
Contract.ContractFailed += e;
try {
action();
} catch (Execption) {
Assert.True(failed);
thrown = true;
} finally {
Contract.ContractFailed -= e;
}
Assert.True(thrown);
}

You can't explicitly catch the right exception type, but you could catch Exception and then check that it's a ContractException using reflection (rethrowing otherwise).
That would be ugly to do everywhere, but you just need to do it once:
public static void AssertContractFailure(Action action)
{
try
{
action();
Assert.Fail("Expected contract violation");
}
catch (Exception e)
{
if (...) // I can't remember offhand what you'd need to check
{
throw;
}
}
}
Then:
AssertContractFailure(() => SomeContractViolation(...));
Currently if you have that in a helper class you'd need to qualify the call everywhere, but as of C# 6 you'll hopefully be able to import it easily :)

If I recall correctly, contracts throw exceptions if the Requires construct fails, and if the Return construct fails. Therefore surely for unit-testing purposes you simply need to catch these exceptions and you'll know if the contract was violated or not.

Related

How to be explicit about NOT throwing an exception?

This might be a broad question, but recently I ahve wondered about the following: In our C# backend we have many places that wrap some code in a try/catch block, specifically calls to external WcF services. Some of these calls are crucial for the application so in the catch block we log the error and rethrow, like:
catch(Exception ex)
{
_logger.Error("Some good error message");
throw ex;
}
On the other hand there are services we allow to fail, but we still want to log the error, so they look like:
catch(Exception ex)
{
_logger.Error("Some good error message");
}
Now reading the code of team members I can not be sure if they forgot to throw or if this is the intended behaviour.
Q: Is there a way, resp. what is the default way, to explicitly NOT rethrow (without including a comment in the code).
I have considered something like this:
catch(Exception ex)
{
_logger.Error("Some good error message");
NotThrowingHereOnPurpose();
}
// ...
// and further below a private method
// ...
private void NotThrowingHereOnPurpose(){}
One approach that may be useful here is to change the way of invoking the code that you explicitly allow to fail in such a way that it does not look like a try/catch block at all.
For example, you could write a helper method that does error reporting, and call it with actions expressed as lambdas:
void InvokeFailSafe(Action action, Action<Exception> onFailure = null) {
try {
action();
} catch (Exception e) {
if (onFailure != null) {
onFailure(e);
}
}
}
Now instead of try/catch you would write this:
InvokeFailSafe(
() => {
... The code that may fail
}
, exception => _logger.Error("Some good error message: {0}", exception)
);
or like this, if you don't want anything logged:
InvokeFailSafe(
() => {
... The code that may fail
}
);
If you code things this way, there would be no doubts about a missing throw statement.
It's an opposite solution to dasblinkenlight's answer. Instead of notifying others that the exception mustn't be rethrown it would say that it must be.
If you only want to log it then use the Error method as usual. Otherwise, you can write an extension method for your logger to log and throw exceptions.
The method would take the catched exception and rethrow it using the ExceptionDispatchInfo class. The ExceptionDispatchInfo is used to rethrow the exception with the original stack trace information and Watson information. It behaves like throw; (without the specified exception).
public static void ErrorAndThrow(this ILogger logger, string message, Exception exception)
{
var exceptionInfo = ExceptionDispatchInfo.Capture(exception);
logger.Error(message);
exceptionInfo.Throw();
}
And use it this way:
try
{
}
catch (Exception ex)
{
// ex would be rethrown here
_logger.ErrorAndThrow("Some good error message", ex);
}
Q: Is there a way, resp. what is the default way, to explicitly NOT
rethrow (without including a comment in the code).
Ideal way would be not to catch a generic exception. Now, to throw or not that entirely depends on your case. You need to understand that Exception handling is used when you know what to do in case an exception occurs. So, only specific exceptions should be handled. Catching exceptions without knowing what you are catching will change the behavior of your application.
Now reading the code of team members I can not be sure if they forgot
to throw or if this is the intended behaviour.
This is something the author of the code can explain to you. But here is a learning to take from this. Your code should be self explanatory. In specific cases where you are unable to express yourself with the code, add a meaningful comment.
You can check this link for better understanding.
I actually found another way that kind of includes what other have suggested here, but uses a built in feature: exception filters. I was free to modify the example given in here to illustrate this:
public void MethodThatFailsSometimes()
{
try {
PerformFailingOperation();
}
catch (Exception e) when (e.LogAndBeCaught())
{
}
}
and then one could have two extension methods on Exception, say LogAndBeCaught and LogAndEscape like so:
public static bool LogAndBeCaught(this Exception e)
{
_logger.Error(#"Following exception was thrown: {e}");
return true;
}
public static bool LogAndEscape(this Exception e)
{
_logger.Error(#"Following exception was thrown: {e}");
return false;
}

NUnit constraint for delegate that does not throw a particular exception

I want to test that a delegate does not throw FooException, but I don't care if it throws anything else. Therefore I can't use the Nothing constraint.
The constraints model doesn't have something like this:
Assert.That(del, Throws.Not.InstanceOf<FooException>()); //can't use Not like this
Is that possible somehow?
This is slightly awkward but should do it
Assert.That(Assert.Catch(del), Is.Null.Or.Not.TypeOf<FooException>());
Personally, I prefer the two-line version
var ex = Assert.Catch(del);
Assert.That(ex, Is.Null.Or.Not.TypeOf<FooException>());
or the even clearer three-liner
var ex = Assert.Catch(del);
if (ex != null)
Assert.That(ex, Is.Not.TypeOf<FooException>());
This works because not asserting at all is the same as succeeding.
The lack of a more direct way to test this in the syntax reflects an opinion of the developers - at least at the time - that you should always know what exception you are expecting.
Looks like nunit does not provide it out of the box.
But you have some workarounds:
You may use additional assertion framework, like FluentAssertions,
which allows you to do next assertion:
del.ShouldNotThrow<FooException>();
Also you can write your own custom-constraints, like ThrowsNothingConstraint
Or you can just write custom metod
public void AssertException<T>(Action act) where T : Exception
{
try
{
act();
}
catch (T e)
{
Assert.Fail();
}
catch (Exception e)
{
Assert.Pass();
}
}

Assert a handled exception was thrown

I am handling several exceptions and not re-throwing them once handled.
How can I assert the exception was invoked in my unit test.
What I would do is create a custom Exception for your legacy code and place the exceptions thrown by your legacy stuff into it's inner exception. Then you can always swallow your custom exception to ignore them in your main app but then they'll still be thrown for your unit testing.
Example:
try
{
//rubbish legacy code which will throw all kinds of exceptions
}
catch(Exception ex)
{
throw new CustomException(message: "something bad happened", innerException: ex);
}
Then normally you can do the following:
try
{
MethodWhichCallsRubbishLegacyStuffAndWillProbablyThrowException();
}
catch (CustomException c)
{
//do nothing or better yet - logging!
}
catch (Exception ex)
{
//handle potential exceptions caused elsewhere
}
Now in your unit test you can assert against the CustomException or indeed the specific InnerException which was thrown.
Based on your comment in your question:
I am sending an email to first line support if one of my messages fails due to a legacy application interface which could throw many different exceptions for which I am handling. It would be nice for my test to assert the exception was thrown and handled.
The cleanest way to handle this is to make sure that the code that is handling the exceptions and then passing them on as an email receives the emailer as an Interface on your constructor.
You can then mock the email handler, pass that to your code under test, and Assert that it was given the proper type of exception.
Something like this:
public interface IExceptionEmailer {
void HandleGenericException( Exception e );
void HandleYourExceptionTypeA ( ExceptionTypeA e );
// ... continue with your specific exceptions
}
public class YourClassThatCatchesExceptions( ){
private IExceptionEmailer emailer;
public void TheMethodThatCatches ( ) {
try {
// actions
} catch ( ExceptionTypeA e ) {
this.emailer.HandleYourExceptionTypeA( e );
} catch ( Exception e ) {
this.emailer.HandleGenericException( e );
}
}
public YourClassThatCatchesExceptions( IExceptionEmailer emailer ) {
this.emailer = emailer;
}
}
Then your test class (assuming Moq and Xunit) would be:
public class GivenAnExceptionEmailer ( ) {
[Fact]
public void WhenYourSpecificActionHappens ( ) {
var emailer = new Mock<IExceptionEmailer>();
// ARRANGE the rest of your system here
var target = new YourClassThatCatchesExceptions( emailer.Object );
// do whatever ACTions needed here to make it throw
target.Whatever( );
// then ASSERT that the emailer was given correct type
// this will fail if the exception wasn't thrown or wasn't
// properly caught and handled.
emailer.Verify ( e =>
e.HandleYourExceptionTypeA ( It.IsAny<ExceptionTypeA>( )),
Times.Once( )
);
}
}
I haven't tested that so you may find syntax issues, but, that isolates your system so that you can verify that the exact behavior you expect in YourClassThatCatchesExceptions fires (and your admins will thank you for not spamming them with a bunch of test emails!)
I have done something like this, not sure its good practice or not...
First:
[TestMethod]
public void MethodName_TestErrorMessage_When_SomeException()
{
// Arrange
const string ExpectedMessgae= "Error in Application ";
this.MockedInterface.Setup(x=>x.MethodCall()).Throws<SomeException>();
// Act
var result=this.Controller.Action() as JsonResult;
// Assert
Assert.AreEqual(ExpectedMessage, result.Data.ToString());
}
This is just an example, but typically if you are not re-throwing exception and it has been handled in code, then we can verify that the message is correct or not. But this also implies at least you have not lost the stack trace in your code and returning it. I will appreciate if someone helps me improving this.
One other way is ExcpectedException Attribute, exception should not be handled for that.
Why care if nobody outside your code's gonna see it? I wouldn't unit test such functionality which is not exposed to the callers.

Ignoring Exceptions in xUnit.net

I have some cases where I don't care what exception is thrown (as long as some exception is thrown). Unfortunately,
Assert.Throws<Exception>(someDelegate);
doesn't pass unless exactly an instance of Exception (so not an instance of a derived class) is thrown. I know I can obtain the behavior I want with
Exception exception = Record.Exception(someDelegate);
Assert.NotNull(exception);
but it doesn't read right. Am I missing something in xUnit that has the behavior I want? Here are two tests that indicate what I mean:
[Fact]
public void Throws_exception_and_passes() {
Exception exception = Record.Exception(
() => { throw new InvalidOperationException(); }
);
Assert.NotNull(exception);
}
[Fact]
public void Throws_exception_and_fails() {
Assert.Throws<Exception>(
() => { throw new InvalidOperationException(); }
);
}
Per the documentation here:
http://xunit.codeplex.com/wikipage?title=HowToUse&referringTitle=Home
You have to specify the type of exception you want to be thrown. In general, this is good practice. You should be able to predict what scenarios a test would throw what type of exception. You should be able to design both you method and your test in a way that will allow you to predict this.
There are ways around this, like doing a try catch yourself, but you should look into changing your design a bit.
It didn't exist at the time of this question, but now one can use Assert.ThrowsAny<Exception> to test for any exception derived from Exception (and hence any exception at all), along with variants such as Assert.ThrowsAny<ArgumentException> which would test for any exception derived from ArgumentException and so on.
As you've identified if Assert.Throws<T> doesn't fit the bill, the only OOTB thing in xUnit you're left with is using Record.Exception.
As you've identified, the main way of doing a 'Assert throws anything` is to do
Assert.NotNull( Record.Exception( lambda ))
Look at it - not pretty. This is likely by design; there are very few things in xUnit.net that are by accident (as opposed to carefully considered opinionated design).
Record.Exception returns a result for a reason (and if you were using F#, you'd have to |> ignore to chuck away the value). You should always be able to Assert something about the nature of the Exception that's happening so that an actual problem in your code doesn't get ignored by chance as you change your code over time, which is the reason for all this testing stuff in the first place. Perhaps that might take the form of
var exception = Record.Exception( sut.Something );
Assert.True( typeof(SomeException).IsAssignableFrom( exception ) );
Looking at that, it's safer that an Assert.NotNull(), but still doesn't feel right. It's time to, as discussed in GOOS, listen to your tests (and in the case of an opinionated test framework, your test framework).
The biggest problem in your question is however that in a real example from a real test, there is always a way to make your interface clearer or express your expectation in another way, so the real answer is Mu.
xUnit won't stand in your way if you want to do your own Custom Assertion, something like:
public static bool Throws<T>(this Action action, bool discardExceptions = false)
where T : Exception
{
try
{
action.Invoke();
}
catch (T)
{
return true;
}
catch (Exception)
{
if (discardExceptions)
{
return false;
}
throw;
}
return false;
}
Or:
public static bool Throws(this Action action)
{
try
{
action.Invoke();
}
catch (Exception)
{
return true;
}
return false;
}
I was just looking in the xUnit.net source and here is the culprit:
private static Exception Throws(Type exceptionType, Exception exception)
{
Guard.ArgumentNotNull("exceptionType", exceptionType);
if (exception == null)
throw new ThrowsException(exceptionType);
if (!exceptionType.Equals(exception.GetType()))
throw new ThrowsException(exceptionType, exception);
return exception;
}
What would solve your problem is if this change were applied:
if(!exceptionType.Equals(exception.GetType()))
to:
if(!exception.GetType().IsAssignableTo(exceptionType))
You could possibly offer to submit a patch?
public static void SuppressException<TSut>(this TSut value, Action<TSut> action) where TSut : class
{
try
{
action.Invoke(value);
}
catch (Exception)
{
//do nothing
}
}

Additional try statement in catch statement - code smell?

Situation:
My application need to process the first step in the business rules (the initial try-catch statement). If an certain error occurs when the process calls the helper method during the step, I need to switch to a second process in the catch statement. The back up process uses the same helper method. If an same error occurs during the second process, I need to stop the entire process and throw the exception.
Implementation:
I was going to insert another try-catch statement into the catch statement of the first try-catch statement.
//run initial process
try
{
//initial information used in helper method
string s1 = "value 1";
//call helper method
HelperMethod(s1);
}
catch(Exception e1)
{
//backup information if first process generates an exception in the helper method
string s2 = "value 2";
//try catch statement for second process.
try
{
HelperMethod(s2);
}
catch(Exception e2)
{
throw e2;
}
}
What would be the correct design pattern to avoid code smells in this implementation?
I caused some confusion and left out that when the first process fails and switches to the second process, it will send different information to the helper method. I have updated the scenario to reflect the entire process.
If the HelperMethod needs a second try, there is nothing directly wrong with this, but your code in the catch tries to do way too much, and it destroys the stacktrace from e2.
You only need:
try
{
//call helper method
HelperMethod();
}
catch(Exception e1)
{
// maybe log e1, it is getting lost here
HelperMethod();
}
I wouldn't say it is bad, although I'd almost certainly refactor the second block of code into a second method, so keep it comprehensible. And probably catch something more specific than Exception. A second try is sometimes necessary, especially for things like Dispose() implementations that might themselves throw (WCF, I'm looking at you).
The general idea putting a try-catch inside the catch of a parent try-catch doesn't seem like a code-smell to me. I can think of other legitimate reasons for doing this - for instance, when cleaning up an operation that failed where you do not want to ever throw another error (such as if the clean-up operation also fails). Your implementation, however, raises two questions for me: 1) Wim's comment, and 2) do you really want to entirely disregard why the operation originally failed (the e1 Exception)? Whether the second process succeeds or fails, your code does nothing with the original exception.
Generally speaking, this isn't a problem, and it isn't a code smell that I know of.
With that said, you may want to look at handling the error within your first helper method instead of just throwing it (and, thus, handling the call to the second helper method in there). That's only if it makes sense, but it is a possible change.
Yes, a more general pattern is have the basic method include an overload that accepts an int attempt parameter, and then conditionally call itself recursively.
private void MyMethod (parameterList)
{ MyMethod(ParameterList, 0)l }
private void MyMethod(ParameterList, int attempt)
{
try { HelperMethod(); }
catch(SomeSpecificException)
{
if (attempt < MAXATTEMPTS)
MyMethod(ParameterList, ++attempt);
else throw;
}
}
It shouldn't be that bad. Just document clearly why you're doing it, and most DEFINITELY try catching a more specific Exception type.
If you need some retry mechanism, which it looks like, you may want to explore different techniques, looping with delays etc.
It would be a little clearer if you called a different function in the catch so that a reader doesn't think you're just retrying the same function, as is, over again. If there's state happening that's not being shown in your example, you should document it carefully, at a minimum.
You also shouldn't throw e2; like that: you should simply throw; if you're going to work with the exception you caught at all. If not, you shouldn't try/catch.
Where you do not reference e1, you should simply catch (Exception) or better still catch (YourSpecificException)
If you're doing this to try and recover from some sort of transient error, then you need to be careful about how you implement this.
For example, in an environment where you're using SQL Server Mirroring, it's possible that the server you're connected to may stop being the master mid-connection.
In that scenario, it may be valid for your application to try and reconnect, and re-execute any statements on the new master - rather than sending an error back to the caller immediately.
You need to be careful to ensure that the methods you're calling don't have their own automatic retry mechanism, and that your callers are aware there is an automatic retry built into your method. Failing to ensure this can result in scenarios where you cause a flood of retry attempts, overloading shared resources (such as Database servers).
You should also ensure you're catching exceptions specific to the transient error you're trying to retry. So, in the example I gave, SqlException, and then examining to see if the error was that the SQL connection failed because the host was no longer the master.
If you need to retry more than once, consider placing an 'automatic backoff' retry delay - the first failure is retried immediately, the second after a delay of (say) 1 second, then doubled up to a maximum of (say) 90 seconds. This should help prevent overloading resources.
I would also suggest restructuring your method so that you don't have an inner-try/catch.
For example:
bool helper_success = false;
bool automatic_retry = false;
//run initial process
try
{
//call helper method
HelperMethod();
helper_success = true;
}
catch(Exception e)
{
// check if e is a transient exception. If so, set automatic_retry = true
}
if (automatic_retry)
{ //try catch statement for second process.
try
{
HelperMethod();
}
catch(Exception e)
{
throw;
}
}
Here's another pattern:
// set up state for first attempt
if(!HelperMethod(false)) {
// set up state for second attempt
HelperMethod(true);
// no need to try catch since you're just throwing anyway
}
Here, HelperMethod is
bool HelperMethod(bool throwOnFailure)
and the return value indicates whether or not success occurred (i.e., false indicates failure and true indicates success). You could also do:
// could wrap in try/catch
HelperMethod(2, stateChanger);
where HelperMethod is
void HelperMethod(int numberOfTries, StateChanger[] stateChanger)
where numberOfTries indicates the number of times to try before throwing an exception and StateChanger[] is an array of delegates that will change the state for you between calls (i.e., stateChanger[0] is called before the first attempt, stateChanger[1] is called before the second attempt, etc.)
This last option indicates that you might have a smelly setup though. It looks like the class that is encapsulating this process is responsible for both keeping track of state (which employee to look up) as well as looking up the employee (HelperMethod). By SRP, these should be separate.
Of course, you need to a catch a more specific exception than you currently are (don't catch the base class Exception!) and you should just throw instead of throw e if you need to rethrow the exception after logging, cleanup, etc.
You could emulate C#'s TryParse method signatures:
class Program
{
static void Main(string[] args)
{
Exception ex;
Console.WriteLine("trying 'ex'");
if (TryHelper("ex", out ex))
{
Console.WriteLine("'ex' worked");
}
else
{
Console.WriteLine("'ex' failed: " + ex.Message);
Console.WriteLine("trying 'test'");
if (TryHelper("test", out ex))
{
Console.WriteLine("'test' worked");
}
else
{
Console.WriteLine("'test' failed: " + ex.Message);
throw ex;
}
}
}
private static bool TryHelper(string s, out Exception result)
{
try
{
HelperMethod(s);
result = null;
return true;
}
catch (Exception ex)
{
// log here to preserve stack trace
result = ex;
return false;
}
}
private static void HelperMethod(string s)
{
if (s.Equals("ex"))
{
throw new Exception("s can be anything except 'ex'");
}
}
}
Another way is to flatten the try/catch blocks, useful if you're using some exception-happy API:
public void Foo()
{
try
{
HelperMethod("value 1");
return; // finished
}
catch (Exception e)
{
// possibly log exception
}
try
{
HelperMethod("value 2");
return; // finished
}
catch (Exception e)
{
// possibly log exception
}
// ... more here if needed
}
An option for retry (that most people will probably flame) would be to use a goto. C# doesn't have filtered exceptions but this could be used in a similar manner.
const int MAX_RETRY = 3;
public static void DoWork()
{
//Do Something
}
public static void DoWorkWithRetry()
{
var #try = 0;
retry:
try
{
DoWork();
}
catch (Exception)
{
#try++;
if (#try < MAX_RETRY)
goto retry;
throw;
}
}
In this case you know this "exception" probably will happen so I would prefer a simple approach an leave exceptions for the unknown events.
//run initial process
try
{
//initial information used in helper method
string s1 = "value 1";
//call helper method
if(!HelperMethod(s1))
{
//backup information if first process generates an exception in the helper method
string s2 = "value 2";
if(!HelperMethod(s2))
{
return ErrorOfSomeKind;
}
}
return Ok;
}
catch(ApplicationException ex)
{
throw;
}
I know that I've done the above nested try catch recently to handle decoding data where two third party libraries throw exceptions on failure to decode (Try json decode, then try base64 decode), but my preference is to have functions return a value which can be checked.
I generally only use the throwing of exceptions to exit early and notify something up the chain about the error if it's fatal to the process.
If a function is unable to provide a meaningful response, that is not typically a fatal problem (Unlike bad input data).
It seems like the main risk in nested try catch is that you also end up catching all the other (maybe important) exceptions that might occur.

Categories

Resources