Develop Reference

C# recovery techniques from a timeout

If I have a "Timeout Expired" exception, what can be a good ways to recover from it and continue doing what you are doing by trying again calling the same routing?
Thank you

while (true)
{
try
{
/////put your code here
break; // Exit the loop. Could return from the method, depending
// on what it does...
}
catch
{
// Log, I suspect...
}
}

C# try catch continue execution

I have a question that might seem fairly simple (of course if you know the answer).
A certain function I have calls another function but I want to continue execution from the caller even though the callee has thrown an exception. Let me give you an example:
something function1()
{
try
{
//some code
int idNumber = function2();
//other code that need to execute even if function2 fails
return something;
}
catch(Exception e)
{//... perhaps something here}
}
EDIT: function1 also has a return statement so nothing can in fact crash on the way
In function2 I need to do stuff but I only need to log if anything fails, example:
int function2()
{
try
{
//dostuff
}
catch(Exception e)
{
//Log stuff to db
}
}
ok, now my question is, what should I do if I wanted to continue execution in function1 even if function 2 throws an error?
Sometimes I mix up if I should do throw; or throw e; or throw nothing at all (leave catch block empty)

Leaving the catch block empty should do the trick. This is almost always a bad idea, though. On one hand, there's a performance penalty, and on the other (and this is more important), you always want to know when there's an error.
I would guess that the "callee" function failing, in your case, is actually not necessarily an "error," so to speak. That is, it is expected for it to fail sometimes. If this is the case, there is almost always a better way to handle it than using exceptions.
There are, if you'll pardon the pun, exceptions to the "rule", though. For example, if function2 were to call a web service whose results aren't really necessary for your page, this kind of pattern might be ok. Although, in almost 100% of cases, you should at least be logging it somewhere. In this scenario I'd log it in a finally block and report whether or not the service returned. Remember that data like that which may not be valuable to you now can become valuable later!
Last edit (probably):
In a comment I suggested you put the try/catch inside function2. Just thought I would elaborate. Function2 would look like this:
public Something? function2()
{
try
{
//all of your function goes here
return anActualObjectOfTypeSomething;
}
catch(Exception ex)
{
//logging goes here
return null;
}
}
That way, since you use a nullable return type, returning null doesn't hurt you.

Why cant you use the finally block?
Like
try {
} catch (Exception e) {
// THIS WILL EXECUTE IF THERE IS AN EXCEPTION IS THROWN IN THE TRY BLOCK
} finally {
// THIS WILL EXECUTE IRRESPECTIVE OF WHETHER AN EXCEPTION IS THROWN WITHIN THE TRY CATCH OR NOT
}
EDIT after question amended:
You can do:
int? returnFromFunction2 = null;
try {
returnFromFunction2 = function2();
return returnFromFunction2.value;
} catch (Exception e) {
// THIS WILL EXECUTE IF THERE IS AN EXCEPTION IS THROWN IN THE TRY BLOCK
} finally {
if (returnFromFunction2.HasValue) { // do something with value }
// THIS WILL EXECUTE IRRESPECTIVE OF WHETHER AN EXCEPTION IS THROWN WITHIN THE TRY CATCH OR NOT
}

Or you can encapsulate the looping logic itself in a try catch e.g.
for(int i = function2(); i < 100 /*where 100 is the end or another function call to get the end*/; i = function2()){
try{
//ToDo
}
catch { continue; }
}
Or...
try{
for(int i = function2(); ; ;) {
try { i = function2(); return; }
finally { /*decide to break or not :P*/continue; } }
} catch { /*failed on first try*/ } finally{ /*afterwardz*/ }

just do this
try
{
//some code
try
{
int idNumber = function2();
}
finally
{
do stuff here....
}
}
catch(Exception e)
{//... perhaps something here}
For all intents and purposes the finally block will always execute. Now there are a couple of exceptions where it won't actually execute: task killing the program, and there is a fast fail security exception which kills the application instantly. Other than that, an exception will be thrown in function 2, the finally block will execute the needed code and then catch the exception in the outer catch block.

Do you mean you want to execute code in function1 regardless of whether function2 threw an exception or not? Have you looked at the finally-block? http://msdn.microsoft.com/en-us/library/zwc8s4fz.aspx

In your second function remove the e variable in the catch block then add throw.
This will carry over the generated exception the the final function and output it.
Its very common when you dont want your business logic code to throw exception but your UI.

Retries in Catch block?

How can I implement the code in the catch block?
try
{
// Call a MS SQL stored procedure (MS SQL 2000)
// Stored Procedure may deadlock
}
catch
{
// if deadlocked Call a MS SQL stored procedure (may deadlock again)
// If deadlocked, keep trying until stored procedure executes
}
finally
{
}

Doing this isn't recommended and could cause serious problems in your program. For example, what if the database was down?
But, here's how to do it in a loop:
for(int attempts = 0; attempts < 5; attempts++)
// if you really want to keep going until it works, use for(;;)
{
try
{
DoWork();
break;
}
catch { }
Thread.Sleep(50); // Possibly a good idea to pause here, explanation below
}
Update: As Mr. Disappointment mentioned in a comment below: The Thread.Sleep method pauses the execution for the specified number of milliseconds. No error is completely random, most that would work simply by trying again only work because something has changed in the time it took between the tries. Pausing the execution of the thread will give a much bigger window of opportunity for this to happen (for example, more time for the database engine to start up).

What about something like this
bool retry = true;
while( retry ){
try{
...
retry = false;
}
catch
{
...
}
finally
{
...
}
}
As long as the last line of the try block gets run ( retry = false ), it will carry on. If some exception occurs, it will run the catch and finally block, and then loop back up and run the try block again.
If you want to only try x times, you can replace the retry with a int with a startvalue of number of tries first. Then check if it equals 0 in the while loop, decrement it in the start of the loop, and set it to 0 as the last line of the try block.
And you should of course do something to that empty catch block so it catches the exceptions you anticipate, and not one that catches everything.

Copied verbatim from a Microsoft Developer Network page on what they call the Retry Pattern:
private int retryCount = 3;
...
public async Task OperationWithBasicRetryAsync()
{
int currentRetry = 0;
for (; ;)
{
try
{
// Calling external service.
await TransientOperationAsync();
// Return or break.
break;
}
catch (Exception ex)
{
Trace.TraceError("Operation Exception");
currentRetry++;
// Check if the exception thrown was a transient exception
// based on the logic in the error detection strategy.
// Determine whether to retry the operation, as well as how
// long to wait, based on the retry strategy.
if (currentRetry > this.retryCount || !IsTransient(ex))
{
// If this is not a transient error
// or we should not retry re-throw the exception.
throw;
}
}
// Wait to retry the operation.
// Consider calculating an exponential delay here and
// using a strategy best suited for the operation and fault.
Await.Task.Delay();
}
}
// Async method that wraps a call to a remote service (details not shown).
private async Task TransientOperationAsync()
{
...
}
They go into more detail, explaining appropriate uses, and non-appropriate uses of this pattern. For example, if you expect the errors you're running into are transient, and that retrying again in a moment will likely succeed, this may be for you. If this is to help you deal with some scaling problems, this is not for you.
You may also be interested in their Circuit Breaker Pattern which they describe as being able to, "Handle faults that may take a variable amount of time to rectify when connecting to a remote service or resource."

Don't implement it in the catch block. Instead write a loop around it that repeats until either it was successful or some limit is reached.
Something like:
bool quit = false;
int loopcount = 0;
while(!quit )
{
try
{
// execute the command, might throw an exception)
quit = true; // no exception if you got here
}
catch(Exception ex)
{
if (ex != deadlock) // doesn't work like this :-(
quit = true;
}
finally
{
// etc.
}
loopcount++;
if (loopcount > 3)
quit = true;
}

It may be as simple as wrapping the whole try/catch in a while loop:
while (!success) {
try
{
// Call a MS SQL stored procedure (MS SQL 2000)
// Stored Procedure may deadlock
success = true;
}
catch
{
// if deadlocked Call a MS SQL stored procedure (may deadlock again)
// If deadlocked, keep trying until stored procedure executes
success = false;
}
}

You really shouldn't just hammer the database until it succeeds in executing your SP, but that's another story.
You could do it like this:
Boolean succeeded = false;
while (!succeeded)
{
try
{
// Call a MS SQL stored procedure (MS SQL 2000)
// Stored Procedure may deadlock
succeeded = true;
}
catch (Exception ex)
{
// Log
}
}

You can implement Timers to check the healthy of your store procedures, and throw answers based on that, inside a loop as the colleagues said.

How to in case of timeout to execute method again and again until it completes successfully?

I have asp.net application. All business logic in business layer.
Here is the example of the method
public void DoSomething()
{
PersonClass pc = new PersonClass();
pc.CreatePerson();
pc.AssignBasicTask();
pc.ChangePersonsStatus();
pc.CreateDefaultSettings();
}
what happens once in a while, one of the sub method can timeout, so as a result the process can be incompleted.
what I think in this case to make sure all steps completed properly is
public void DoSomething()
{
PersonClass pc = new PersonClass();
var error = null;
error = pc.CreatePerson();
if(error != timeout exception)
error = pc.AssignBasicTask();
else
return to step above
if(error != timeout exception)
error = pc.ChangePersonsStatus();
else
return to step above
if(error != timeout exception)
error = pc.CreateDefaultSettings();
else
return to step above
}
but it's just an idea, more then sure it's a proper way how to handle this.

Of course, this can be done more or less elegantly, with different options for timing out or giving up - but an easy way to achieve what you want, would be to define a retry method which keeps retrying an action until it succeeds:
public static class RetryUtility
{
public T RetryUntilSuccess<T>(Func<T> action)
{
while(true)
{
try
{
return action();
}
catch
{
// Swallowing exceptions is BAD, BAD, BAD. You should AT LEAST log it.
}
}
}
public void RetryUntilSuccess(Action action)
{
// Trick to allow a void method being passed in without duplicating the implementation.
RetryUntilSuccess(() => { action(); return true; });
}
}
Then do
RetryUtility.RetryUntilSuccess(() => pc.CreatePerson());
RetryUtility.RetryUntilSuccess(() => pc.AssignBasicTask());
RetryUtility.RetryUntilSuccess(() => pc.ChangePersonsStatus());
RetryUtility.RetryUntilSuccess(() => pc.CreateDefaultSettings());
I must urge you to think about what to do if the method keeps failing, you could be creating an infinite loop - perhaps it should give up after N retries or back off with exponentially raising retry time - you will need to define that, since we cannot know enough about your problem domain to decide that.

You have it pretty close to correct in your psuedo-code, and there a lot of ways to do this, but here is how I would do it:
PersonClass pc = new PersonClass();
while(true)
if(pc.CreatePerson())
break;
while(true)
if(pc.AssignBasicTask())
break;
This assumes that your methods return true to indicate success, false to indicate a timeoiut failure (and probably an exception for any other kind of failure). And while I didn't do it here, I would strongly recommend some sort of try counting to make sure it doesn't just loop forever and ever.

Use a TransactionScope for to make sure everything is executed as a unit. More info here: Implementing an Implicit Transaction using Transaction Scope
You should never retry a timed out operation infinitely, you may end up hanging the server or with an infinite loop or both. There should always be a threshold of how many retries is acceptable to attempt before quitting.
Sample:
using(TransactionScope scope = new TransactionScope())
{
try
{
// Your code here
// If no errors were thrown commit your transaction
scope.Complete();
}
catch
{
// Some error handling
}
}

How can I improve this exception retry scenario?

I have a web service method I am calling which is 3rd party and outside of my domain. For some reason every now and again the web service fails with a gateway timeout. Its intermittent and a call to it directly after a failed attempt can succeed.
Now I am left with a coding dilemma, I have code that should do the trick, but the code looks like amateur hour, as you'll see below.
Is this really bad code, or acceptable given the usage? If its not acceptable, how can I improve it?
Please try hard to keep a straight face while looking at it.
try
{
MDO = OperationsWebService.MessageDownload(MI);
}
catch
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
}
catch
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
}
catch
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
}
catch
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
}
catch (Exception ex)
{
// 5 retries, ok now log and deal with the error.
}
}
}
}
}

You can do it in a loop.
Exception firstEx = null;
for(int i=0; i<5; i++)
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
firstEx = null;
break;
}
catch(Exception ex)
{
if (firstEx == null)
{
firstEx = ex;
}
Thread.Sleep(100 * (i + 1));
}
}
if (firstEx != null)
{
throw new Exception("WebService call failed after 5 retries.", firstEx);
}

Here's another way you might try:
// Easier to change if you decide that 5 retries isn't right for you
Exception exceptionKeeper = null;
for (int i = 0; i < MAX_RETRIES; ++i)
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
break; // correct point from Joe - thanks.
}
catch (Exception ex)
{
exceptionKeeper = ex;
// 5 retries, ok now log and deal with the error.
}
}
I think it documents the intent better. It's less code as well; easier to maintain.

All of the answers so far assume that the reaction to any exception should be to retry the operation. This is a good assumption right up until it's a false assumption. You could easily be retrying an operation that is damaging your system, all because you didn't check the exception type.
You should almost never use a bare "catch", nor "catch (Exception ex). Catch a more-specific exception - one you know you can safely recover from.

Try a loop, with some kind of limit:
int retryCount = 5;
var done = false;
Exception error = null;
while (!done && retryCount > 0)
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
done = true;
}
catch (Exception ex)
{
error = ex;
}
if (done)
break;
retryCount--;
}

You should use recursion (or a loop), and should only retry if you got the error you expected.
For example:
static void TryExecute<TException>(Action method, Func<TException, bool> retryFilter, int maxRetries) where TException : Exception {
try {
method();
} catch(TException ex) {
if (maxRetries > 0 && retryFilter(ex))
TryExecute(method, retryFilter, maxRetries - 1);
else
throw;
}
}
EDIT: With a loop:
static void TryExecute<TException>(Action method, Func<TException, bool> retryFilter, int maxRetries) where TException : Exception {
while (true) {
try {
method();
return;
} catch(TException ex) {
if (maxRetries > 0 && retryFilter(ex))
maxRetries--;
else
throw;
}
}
}
You can try to prevent future errors in retryFilter, perhaps by Thread.Sleep.
If the last retry fails, this will throw the last exception.

Here is some retry logic we are using. We don't do this a lot and I was going to pull it out and document it as our Retry Pattern/Standard. I had to wing it when I first wrote it so I came here to see if I was doing it correctly. Looks like I was. The version below is fully commented. See below that for an uncommented version.
#region Retry logic for SomeWebService.MyMethod
// The following code wraps SomeWebService.MyMethod in retry logic
// in an attempt to account for network failures, timeouts, etc.
// Declare the return object for SomeWebService.MyMethod outside of
// the following for{} and try{} code so that we have it afterwards.
MyMethodResult result = null;
// This logic will attempt to retry the call to SomeWebService.MyMethod
for (int retryAttempt = 1; retryAttempt <= Config.MaxRetryAttempts; retryAttempt++)
{
try
{
result = SomeWebService.MyMethod(myId);
// If we didn't get an exception, then that (most likely) means that the
// call was successful so we can break out of the retry logic.
break;
}
catch (Exception ex)
{
// Ideally we want to only catch and act on specific
// exceptions related to the failure. However, in our
// testing, we found that the exception could be any type
// (service unavailable, timeout, database failure, etc.)
// and attempting to trap every exception that was retryable
// was burdensome. It was easier to just retry everything
// regardless of the cause of the exception. YMMV. Do what is
// appropriate for your scenario.
// Need to check to see if there will be another retry attempt allowed.
if (retryAttempt < Config.MaxRetryAttempts)
{
// Log that we are re-trying
Logger.LogEvent(string.Format("Retry attempt #{0} for SomeWebService.MyMethod({1})", retryAttempt, myId);
// Put the thread to sleep. Rather than using a straight time value for each
// iteration, we are going to multiply the sleep time by how many times we
// have currently tried to call the method. This will allow for an easy way to
// cover a broader range of time without having to use higher retry counts or timeouts.
// For example, if MaxRetryAttempts = 10 and RetrySleepSeconds = 60, the coverage will
// be as follows:
// - Retry #1 - Sleep for 1 minute
// - Retry #2 - Sleep for 2 minutes (covering three minutes total)
// - Retry #10 - Sleep for 10 minutes (and will have covered almost an hour of downtime)
Thread.Sleep(retryAttempt * Config.RetrySleepSeconds * 1000);
}
else
{
// If we made it here, we have tried to call the method several
// times without any luck. Time to give up and move on.
// Moving on could either mean:
// A) Logging the exception and moving on to the next item.
Logger.LogError(string.Format("Max Retry Attempts Exceeded for SomeWebService.MyMethod({0})", MyId), ex);
// B) Throwing the exception for the program to deal with.
throw new Exception(string.Format("Max Retry Attempts Exceeded for SomeWebService.MyMethod({0})", myId), ex);
// Or both. Your code, your call.
}
}
}
#endregion
I like Samuel Neff's example of using an exception variable to see if it completely failed or not. That would have made some of the evaluations in my logic a little simpler. I could go either way. Not sure that either way has a significant advantage over the other. However, at this point in time, I'm not going to change how we do it. The important thing is to document what you are doing and why so that some idiot doesn't come through behind you and muck with everything.
Just for kicks though, to get a better idea if the code is any shorter or cleaner one way or the other, I pulled out all the comments. They came out exactly the same number of lines. I went ahead and compiled the two versions and ran them through Reflector Code Metrics and got the following:
Metric: Inside-Catch / Outside-For
CodeSize: 197 / 185
CyclomaticComplexity: 3 / 3
Instructions: 79 / 80
Locals: 6 / 7
Final exception logic inside the catch (22 lines):
MyMethodResult result = null;
for (int retryAttempt = 1; retryAttempt <= Config.MaxRetryAttempts; retryAttempt++)
{
try
{
result = SomeWebService.MyMethod(myId);
break;
}
catch (Exception ex)
{
if (retryAttempt < Config.MaxRetryAttempts)
{
Logger.LogEvent(string.Format("Retry attempt #{0} for SomeWebService.MyMethod({1})", retryAttempt, myId);
Thread.Sleep(retryAttempt * Config.RetrySleepSeconds * 1000);
}
else
{
Logger.LogError(string.Format("Max Retry Attempts Exceeded for SomeWebService.MyMethod({0})", MyId), ex);
throw new Exception(string.Format("Max Retry Attempts Exceeded for SomeWebService.MyMethod({0})", myId), ex);
}
}
}
Final exception logic after the for-loop (22 lines):
MyMethodResult result = null;
Exception retryException = null;
for (int retryAttempt = 1; retryAttempt <= Config.MaxRetryAttempts; retryAttempt++)
{
try
{
result = SomeWebService.MyMethod(myId);
retryException = null;
break;
}
catch (Exception ex)
{
retryException = ex;
Logger.LogEvent(string.Format("Retry attempt #{0} for SomeWebService.MyMethod({1})", retryAttempt, myId);
Thread.Sleep(retryAttempt * Config.RetrySleepSeconds * 1000);
}
}
if (retryException != null)
{
Logger.LogError(string.Format("Max Retry Attempts Exceeded for SomeWebService.MyMethod({0})", MyId), ex);
throw new Exception(string.Format("Max Retry Attempts Exceeded for SomeWebService.MyMethod({0})", myId), ex);
}

I'm using the following generic method for a retry scenario. I especially want to draw attention to the PreserveStackTrace method which helps to preserve the full call stack trace, because (as I learned the hard way) neither throw or throw ex yields the complete call stack trace information.
public static void RetryBeforeThrow<T>(Action action, int retries, int timeout) where T : Exception
{
int tries = 1;
do
{
try
{
action();
return;
}
catch (T ex)
{
if (retries <= 0)
{
PreserveStackTrace(ex);
throw;
}
Thread.Sleep(timeout);
}
}
while (tries++ < retries);
}
/// <summary>
/// Sets a flag on an <see cref="T:System.Exception"/> so that all the stack trace information is preserved
/// when the exception is re-thrown.
/// </summary>
/// <remarks>This is useful because "throw" removes information, such as the original stack frame.</remarks>
/// <see href="http://weblogs.asp.net/fmarguerie/archive/2008/01/02/rethrowing-exceptions-and-preserving-the-full-call-stack-trace.aspx"/>
public static void PreserveStackTrace(Exception ex)
{
MethodInfo preserveStackTrace = typeof(Exception).GetMethod("InternalPreserveStackTrace", BindingFlags.Instance | BindingFlags.NonPublic);
preserveStackTrace.Invoke(ex, null);
}

As everyone else has pointed out the correct approach is to wrap your try/catch inside some loop with a MAX_RETRY of some sort.
You might also consider adding a timeout between each loop iteration. Otherwise you're likely to burn through your retry counter before the transient issue has had a chance to resolve itself.

It seems you have the answers you need, but I thought I'd post this link, What is an Action Policy?, that I found to provide a much more elegant solution. Lokad has some rather labyrinthine implementations, but the guy's logic is pretty solid, and the end code you'd end up writing is pretty and simple.

int cnt=0;
bool cont = true;
while (cont)
{
try
{
MDO = OperationsWebService.MessageDownload(MI);
cont = false;
}
catch (Exception ex)
{
++cnt;
if (cnt == 5)
{
// 5 retries, ok now log and deal with the error.
cont = false;
}
}
}
UPDATED : Fixed code based on comments.