Azure Service Bus Retry Options Not Working - c#

With no luck, I tried configuring my ServiceBusClient to retry a message with a Fixed Delay of 10 seconds. I also tried Exponential retries configuration. However, the code is always retrying the message within a second or 2 and completely ignoring the configuration. It even ignores the MaxRetries and only retries 10 times, the value configured in Azure Portal for the queue. What am I doing wrong?
I am using The Azure.Messaging.ServiceBus library, NuGet package 7.0.0.
The code:
ServiceBusClient client = new ServiceBusClient(serviceBusConnectionString, new ServiceBusClientOptions()
{
RetryOptions = new ServiceBusRetryOptions()
{
Mode = ServiceBusRetryMode.Fixed,
Delay = TimeSpan.FromSeconds(10),
MaxDelay = TimeSpan.FromMinutes(3),
MaxRetries = 30
}
});
ServiceBusProcessor processor = client.CreateProcessor(queueName, new ServiceBusProcessorOptions());
// throwing an exception in MyMessageHandlerAsync on purpose
// to test out the retries configuration
processor.ProcessMessageAsync += MyMessageHandlerAsync;
// The uncaught exception causes this method to execute.
// Processing is attempted 10 times with
// virtually no delay between each attempt.
// After the 10th attempt, the message goes to deadletter,
// which is expected.
processor.ProcessErrorAsync += MyErrorHandler;
I'm adding more to this question after receiving the 1st response:
Currently, MyMessageHandlerAsync is:
private async Task MyMessageHandlerAsync(EventArgs eventArgs)
{
var args = (ProcessMessageEventArgs)eventArgs;
var body = args.Message.Body.ToString();
// ...
// process body
// ...
await args.CompleteMessageAsync(args.Message);
}
How should I change the method's contents to retry a non-transient ServiceBusException? Please help provide the code where the TODOs are below:
private async Task MyMessageHandlerAsync(EventArgs eventArgs)
{
var args = (ProcessMessageEventArgs)eventArgs;
try
{
var body = args.Message.Body.ToString();
// ...
// process body
// ...
await args.CompleteMessageAsync(args.Message);
}
catch (ServiceBusException sbe)
{
if (sbe.IsTransiet)
{
// TODO: Is it correct that the exponential retry will work
// here? The one defined in the ServiceBusClient.
// So, no code is needed here, just throw.
throw;
}
else
{
// TODO: for non-transient, this is where the
// options in the ServiceBusClient don't apply.
// Is that correct? How do I do an
// exponential retry here?
}
}
catch (Exception e)
{
// TODO: same problem as else in first catch.
}
}

ServiceBusRetryOptions is intended to be used by the ASB client when there are transient errors that are not bubbled up to your code right away, i.e. an internal retry mechanism built into the client to perform retries on your behalf before exception is raised.
Use retry policy to specify to the ASB client how to deal with transient errors prior to giving up, not how many times a message handler throws error:

Related

Issue with MessagingCenter firing multiple times Xamarin

I have looked all over for a solution to an issue. I have noticed that in my android app, every time I fire an event from <button Clicked="GPSToggle_Clicked">, for some reason it increments the number of times my methods get called. So after I compile and load this on my phone, I hit my "GPSToggle_Clicked" button, and then to stop hit that button again. On the first "stop", I'll get a single instance of the below output:
---------------------------------------------------------------Attempting string parsing
---------------------------------------------------------------Sending string to SubmitGeneratedGPX
---------------------------------------------------------------path: /storage/emulated/0/Download/GPX/2022-10-27-02-44-06.gpx
---------------------------------------------------------------GPX File creation success
---------------------------------------------------------------:RawBufferToJsonString: [{"Timestamp":"2022-10-27T18:43:52.849+00:00","Latitude":41.5263818,"Longitude":-81.6507923,"Altitude":153.29998779296875,"Accuracy":20.0,"VerticalAccuracy":1.7990270853042603,"Speed":null,"Course":null,"IsFromMockProvider":false,"AltitudeReferenceSystem":2},{"Timestamp":"2022-10-27T18:43:53.696+00:00","Latitude":41.5263819,"Longitude":-81.6507921,"Altitude":153.29998779296875,"Accuracy":20.0,"VerticalAccuracy":1.7697961330413818,"Speed":null,"Course":null,"IsFromMockProvider":false,"AltitudeReferenceSystem":2},{"Timestamp":"2022-10-27T18:43:54.526+00:00","Latitude":41.5263819,"Longitude":-81.6507921,"Altitude":153.29998779296875,"Accuracy":20.0,"VerticalAccuracy":1.7697961330413818,"Speed":null,"Course":null,"IsFromMockProvider":false,"AltitudeReferenceSystem":2},{"Timestamp":"2022-10-27T18:43:55.374+00:00","Latitude":41.5263819,"Longitude":-81.6507921,"Altitude":153.29998779296875,"Accuracy":20.0,"VerticalAccuracy":1.7697961330413818,"Speed":null,"Course":null,"IsFromMockProvider":false,"AltitudeReferenceSystem":2},{"Timestamp":"2022-10-27T18:43:56.21+00:00","Latitude":41.5263811,"Longitude":-81.650792,"Altitude":153.29998779296875,"Accuracy":20.0,"VerticalAccuracy":1.7160584926605225,"Speed":null,"Course":null,"IsFromMockProvider":false,"AltitudeReferenceSystem":2}]
Every subsequent time I hit start/stop on the app, I get the real-time data in the output multiplied by the number of times I've started/stopped since the last compiling.
the main app page button event thats fired:
private async void GPSToggle_Clicked(object sender, EventArgs e)
{
var LocationPermissionStatus = await Xamarin.Essentials.Permissions.RequestAsync<Xamarin.Essentials.Permissions.LocationAlways>();
var FileReadPermissionStatus = await Xamarin.Essentials.Permissions.RequestAsync<Xamarin.Essentials.Permissions.StorageRead>();
var FileWritePermissionStatus = await Xamarin.Essentials.Permissions.RequestAsync<Xamarin.Essentials.Permissions.StorageWrite>();
if(LocationPermissionStatus == Xamarin.Essentials.PermissionStatus.Denied)
{
// TODO
return;
}
// run if device is android
if(Device.RuntimePlatform == Device.Android)
{
if (!CrossGeolocator.Current.IsGeolocationAvailable || !CrossGeolocator.Current.IsGeolocationEnabled)
{
// gps is not enabled, throw alert
Console.WriteLine("---------------------------------------------------------------GPS is DISABLED");
await DisplayAlert("Error", "GPS is not enabled. You must enable GPS to use this feature", "Ok");
}
else
{
// set our IsTracking = true flag
if (!IsTracking)
{
// start background listening for GPS
await StartListening();
Console.WriteLine("---------------------------------------------------------------Listening: " + CrossGeolocator.Current.IsListening);
StartService();
Console.WriteLine("---------------------------------------------------------------Service initiated");
IsTracking = true;
Console.WriteLine("---------------------------------------------------------------Tracking initiated");
GPSToggle.Text = "Stop Tracking";
}
else
{
//
// verify that the submittal wasn't done in error, before stopping services and submitting data
bool DoneInError = await DisplayAlert("Finish?", "Are you sure you want to stop services and submit?", "No", "Yes");
if (!DoneInError)
{
await StopListening();
Console.WriteLine("---------------------------------------------------------------listening:" + CrossGeolocator.Current.IsListening);
IsTracking = false;
Console.WriteLine("---------------------------------------------------------------Tracking ceased");
// stop the gps service
StopService();
Console.WriteLine("---------------------------------------------------------------Service ceased");
// stop the background listening for gps
Console.WriteLine("---------------------------------------------------------------Attempt GPX parse from buffer obj");
GPSToggle.Text = "Start Tracking";
}
}
}
}
}
Specifically the line:
StartService();
Fires this method off within the same class, specifically the MessagingCenter.Send<>, which initiates my foreground service to handle logging the gps data into a buffer:
private void StartService()
{
var startServiceMessage = new StartServiceMessage();
MessagingCenter.Send(startServiceMessage, "ServiceStarted");
Preferences.Set("LocationServiceRunning", true);
StatusLabel.Text = "Location service has been started";
Console.WriteLine("---------------------------------------------------------------location service has been started. preferences saved");
}
and
StopService();
Fires this method off to stop the services and retrieve the gps buffer data from the foreground to the main thread:
private void StopService()
{
var stopServiceMessage = new StopServiceMessage();
MessagingCenter.Unsubscribe<App, List<Location>>(this, "GPXBufferData");
MessagingCenter.Subscribe<App, List<Location>>(this, "GPXBufferData", (sender, args) =>
{
RawGPXData = args;
Generate_CreateGPX_File(RawGPXData);
RawBufferToJsonString = GPXParse.GenerateJSON_GPXPoints(RawGPXData);
Console.WriteLine("---------------------------------------------------------------:RawBufferToJsonString: " + RawBufferToJsonString);
PromptForSubmission_GPXPoints_API();
});
Console.WriteLine("--------------------------------------------------------------------------");
MessagingCenter.Send(stopServiceMessage, "ServiceStopped");
Preferences.Set("LocationServiceRunning", false);
Console.WriteLine("---------------------------------------------------------------Location service stopped. preferences saved");
}
In the above snippet, this line is subscribed to in the GPSLocationService.cs file:
MessagingCenter.Send(stopServiceMessage, "ServiceStopped");
This is a portion of my GPSLocationService.cs file that is relevant to this:
public async Task Run(CancellationToken token)
{
int ObjCount = 0;
await Task.Run(async () => {
// if the task was stopped
// check the buffer for data, if data, send to GPXGenerator
MessagingCenter.Subscribe<StopServiceMessage>(this, "ServiceStopped",
message =>
{
if (GPSBufferObj != null)
{
Device.BeginInvokeOnMainThread(() =>
{
MessagingCenter.Unsubscribe<App, List<Location>>((App)Xamarin.Forms.Application.Current, "GPXBufferData");
MessagingCenter.Send<App, List<Location>>((App)Xamarin.Forms.Application.Current, "GPXBufferData", GPSBufferObj);
});
}
});
return;
}, token);
}
I believe I have tracked down where the issue is starting. In my StopService() method, I have the following line (just to keep track of where Im at in the buffer) and it is only sent to output once.
Console.WriteLine("--------------------------------------------------------------------------");
BUT if I place that same line within the pasted portion of my GPSLocationService.cs file, I will get the incremented output. I'm leaning towards the nested task being the issue, I wrote this based losely off of this example repro:
https://github.com/jfversluis/XFBackgroundLocationSample
You don't have MessagingCenter.Unsubscribe<StopServiceMessage> anywhere in your code. StopServiceMessage is what you are accumulating subscriptions to.
You need to make sure Unsubscribe is unsubscribing the instance that was previously subscribed to. It sounds to me like there are multiple instances of GPSLocationService. [In which case, this is no longer referring to the original instance. Unsubscribe won't do anything, unless you have the this that was originally Subscribed.]
If so, instead create an instance of GPSLocationService ONCE, and store it in a static variable. Re-use it. start/stop it, but don't discard it.
Alternatively, if you only want a message ONE TIME from each Subscribe, then Unsubscribe as soon as you receive each message:
MessagingCenter.Subscribe<StopServiceMessage>(this, "ServiceStopped",
message =>
{
MessagingCenter.Unsubscribe<StopServiceMessage>(this, "ServiceStopped");
... your code ...
});
Use this same pattern EVERYWHERE you have a Subscribe (unless you Subscribe ONLY ONE TIME at app start, as Jason suggested.)

Set durationOfBreak in Polly CircuitBreaker

I am using Polly Circuit Breaker for an API which throws a user defined Exception if service is down for maintenance.
The Exception will contain a value how long the service will be down.
Is it possible to configure/update the circuit breaker policy after first user exception is handled ?
e.g.
CircuitBreakerPolicy breaker = Policy.Handle<UserException>()
.CircuitBreaker(
exceptionsAllowedBeforeBreaking: 1,
durationOfBreak: TimeSpan.FromMinutes(1));
Example()
{
try
{
string response = await breaker.ExecuteAsync<String>(() =>
{
return client.GetStringAsync("/api/values/");
});
}
catch (UserException ex)
{
var downtime = GetDowntime(ex);
//how to update the duration of break ?
breaker.durationOfBreak = downtime;
}
catch (Exception ex)
{
…
}
}
It is not possible to change the durationOfBreak on an existing CircuitBreakerPolicy instance, after creation.
For general dynamic reconfiguration during running, the Polly team recommends atomically updating a policy instance stored in the PolicyRegistry.
For the specific scenario:
[the] API throws a user defined Exception if service is down for maintenance [which] will contain a value how long the service will be down
you could, more simply, use the circuit-breaker's manual controls:
.Isolate() to manually force the circuit open;
.Reset() to manually reset the circuit.
You could .Isolate() the circuit when the given exception is thrown; and set a Timer to call Reset() on it at the time that the exception indicates the service should be available again.

Web API 2 Asp.Net - setting explicit Timeout value

I have an Asp.NET Web API 2. My client calls a particular action method but I need someway to set the timeout value.
I need the timeout value because I want the client to take appropriate action if the action method hasn't returned anything back in say 40 seconds. (Basically that's an arbitrary limit I've chosen...so if it hasn't completed it's job..i.e. hasn't returned back the valid JSON in 40 seconds, we're going to have to assume that something is taking way too long on Azure and we're going to perform a rollback).
Also, if the timeout has occurred I want someone way to Rollback the transaction.
If it helps, I'm using the UnitOfWorkActionFilter along with NHibernate.
The controller and action method are both asynchronous, and I'm explicitly catching the ExecuteAsync method along with the CancellationToken variable.
However, I'm unaware of a way to cancel this call OR use the CancellationToken variable.
I can post code if necessary.
I did read in a few places that since WebApi2 is asynchronous that I may not be able to cancel this!
Any recommendations on how to go about solving this?
I think setting a timeout on the request is the wrong approach as you will have no visibility of what is going on during the 40 seconds.
Rather make a ajax web request and then subsequent web requests to see if the process has completed.
For example,
Queue the request somehow with the initial request.
Write something to pick up and process the item from the queue. This also means if something goes wrong, you can just roll back at this point. You also need to store the status of the item somewhere.
Write a periodic poll in Javascript that makes another ajax request every 5 seconds to see if the request has been processed or not.
Depending on what kind of method is running on your WebApi service you could try the following:
Start a StopWatch at the start of your action
Check periodically if the elapsed time is greater than your arbitrary limit. When that happens throw an Exception (I called mine
CalculationTimeLimitExceededException)
Catch the exception and perform a rollback (assuming you want to do a rollback on the server)
Return a response (e.g. HTTP 500 with some useful information, e.g. server timeout)
Since the client gets a response within your time limit you can then handle the error on the client side.
Update (added code for PerformanceWatcher):
public class PerformanceWatcher : IDisposable
{
private System.Diagnostics.Stopwatch _sw;
private Timer _timer;
private int _maxSeconds;
public bool TimeLimitExceeded { get; private set; }
public PerformanceWatcher(int maxSeconds)
{
_maxSeconds = maxSeconds;
// start the StopWatch
_sw = System.Diagnostics.Stopwatch.StartNew();
// check every second
_timer = new Timer(1000);
_timer.AutoReset = true;
// set event-handler
_timer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
_timer.Enabled = true;
}
private void OnTimedEvent(object source, ElapsedEventArgs e)
{
// check if time limit was reached
if (this._sw.Elapsed.TotalSeconds > _maxSeconds)
{
this.TimeLimitExceeded = true;
}
}
public void Dispose()
{
this._timer.Dispose();
}
}
You can use this class in your action:
[HttpGet]
public HttpResponseMessage GetResultFromLongRunningMethod()
{
using (PerformanceWatcher watcher = new PerformanceWatcher(10))
{
try
{
// begin long-running operation
for (int i = 0; i < 20; i++)
{
if (watcher.TimeLimitExceeded)
{
throw new TimeLimitExceededException();
}
Thread.Sleep(1000);
}
// end long-running operation
} catch (TimeLimitExceededException e)
{
return this.Request.CreateErrorResponse(HttpStatusCode.InternalServerError, "Time limit exceeded");
}
}
return this.Request.CreateResponse(HttpStatusCode.OK, "everything ok");
}
The above code isn't tested; I just copied some elements from my own classes.
Update 2: Fixed a bug in the code (Exception thrown from event handler wasn't caught)

How to create a generic timeout object for various code blocks?

I have a series of code blocks that are taking too long. I don't need any finesse when it fails. In fact, I want to throw an exception when these blocks take too long, and just fall out through our standard error handling. I would prefer to NOT create methods out of each block (which are the only suggestions I've seen so far), as it would require a major rewrite of the code base.
Here's what I would LIKE to create, if possible.
public void MyMethod( ... )
{
...
using (MyTimeoutObject mto = new MyTimeoutObject(new TimeSpan(0,0,30)))
{
// Everything in here must complete within the timespan
// or mto will throw an exception. When the using block
// disposes of mto, then the timer is disabled and
// disaster is averted.
}
...
}
I've created a simple object to do this using the Timer class. (NOTE for those that like to copy/paste: THIS CODE DOES NOT WORK!!)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Timers;
public class MyTimeoutObject : IDisposable
{
private Timer timer = null;
public MyTimeoutObject (TimeSpan ts)
{
timer = new Timer();
timer.Elapsed += timer_Elapsed;
timer.Interval = ts.TotalMilliseconds;
timer.Start();
}
void timer_Elapsed(object sender, ElapsedEventArgs e)
{
throw new TimeoutException("A code block has timed out.");
}
public void Dispose()
{
if (timer != null)
{
timer.Stop();
}
}
}
It does not work because the System.Timers.Timer class captures, absorbs and ignores any exceptions thrown within, which -- as I've discovered -- defeats my design. Any other way of creating this class/functionality without a total redesign?
This seemed so simple two hours ago, but is causing me much headache.
OK, I've spent some time on this one and I think I have a solution that will work for you without having to change your code all that much.
The following is how you would use the Timebox class that I created.
public void MyMethod( ... ) {
// some stuff
// instead of this
// using(...){ /* your code here */ }
// you can use this
var timebox = new Timebox(TimeSpan.FromSeconds(1));
timebox.Execute(() =>
{
/* your code here */
});
// some more stuff
}
Here's how Timebox works.
A Timebox object is created with a given Timespan
When Execute is called, the Timebox creates a child AppDomain to hold a TimeboxRuntime object reference, and returns a proxy to it
The TimeboxRuntime object in the child AppDomain takes an Action as input to execute within the child domain
Timebox then creates a task to call the TimeboxRuntime proxy
The task is started (and the action execution starts), and the "main" thread waits for for as long as the given TimeSpan
After the given TimeSpan (or when the task completes), the child AppDomain is unloaded whether the Action was completed or not.
A TimeoutException is thrown if action times out, otherwise if action throws an exception, it is caught by the child AppDomain and returned for the calling AppDomain to throw
A downside is that your program will need elevated enough permissions to create an AppDomain.
Here is a sample program which demonstrates how it works (I believe you can copy-paste this, if you include the correct usings). I also created this gist if you are interested.
public class Program
{
public static void Main()
{
try
{
var timebox = new Timebox(TimeSpan.FromSeconds(1));
timebox.Execute(() =>
{
// do your thing
for (var i = 0; i < 1000; i++)
{
Console.WriteLine(i);
}
});
Console.WriteLine("Didn't Time Out");
}
catch (TimeoutException e)
{
Console.WriteLine("Timed Out");
// handle it
}
catch(Exception e)
{
Console.WriteLine("Another exception was thrown in your timeboxed function");
// handle it
}
Console.WriteLine("Program Finished");
Console.ReadLine();
}
}
public class Timebox
{
private readonly TimeSpan _ts;
public Timebox(TimeSpan ts)
{
_ts = ts;
}
public void Execute(Action func)
{
AppDomain childDomain = null;
try
{
// Construct and initialize settings for a second AppDomain. Perhaps some of
// this is unnecessary but perhaps not.
var domainSetup = new AppDomainSetup()
{
ApplicationBase = AppDomain.CurrentDomain.SetupInformation.ApplicationBase,
ConfigurationFile = AppDomain.CurrentDomain.SetupInformation.ConfigurationFile,
ApplicationName = AppDomain.CurrentDomain.SetupInformation.ApplicationName,
LoaderOptimization = LoaderOptimization.MultiDomainHost
};
// Create the child AppDomain
childDomain = AppDomain.CreateDomain("Timebox Domain", null, domainSetup);
// Create an instance of the timebox runtime child AppDomain
var timeboxRuntime = (ITimeboxRuntime)childDomain.CreateInstanceAndUnwrap(
typeof(TimeboxRuntime).Assembly.FullName, typeof(TimeboxRuntime).FullName);
// Start the runtime, by passing it the function we're timboxing
Exception ex = null;
var timeoutOccurred = true;
var task = new Task(() =>
{
ex = timeboxRuntime.Run(func);
timeoutOccurred = false;
});
// start task, and wait for the alloted timespan. If the method doesn't finish
// by then, then we kill the childDomain and throw a TimeoutException
task.Start();
task.Wait(_ts);
// if the timeout occurred then we throw the exception for the caller to handle.
if(timeoutOccurred)
{
throw new TimeoutException("The child domain timed out");
}
// If no timeout occurred, then throw whatever exception was thrown
// by our child AppDomain, so that calling code "sees" the exception
// thrown by the code that it passes in.
if(ex != null)
{
throw ex;
}
}
finally
{
// kill the child domain whether or not the function has completed
if(childDomain != null) AppDomain.Unload(childDomain);
}
}
// don't strictly need this, but I prefer having an interface point to the proxy
private interface ITimeboxRuntime
{
Exception Run(Action action);
}
// Need to derive from MarshalByRefObject... proxy is returned across AppDomain boundary.
private class TimeboxRuntime : MarshalByRefObject, ITimeboxRuntime
{
public Exception Run(Action action)
{
try
{
// Nike: just do it!
action();
}
catch(Exception e)
{
// return the exception to be thrown in the calling AppDomain
return e;
}
return null;
}
}
}
EDIT:
The reason I went with an AppDomain instead of Threads or Tasks only, is because there is no bullet proof way for terminating Threads or Tasks for arbitrary code [1][2][3]. An AppDomain, for your requirements, seemed like the best approach to me.
Here's an async implementation of timeouts:
...
private readonly semaphore = new SemaphoreSlim(1,1);
...
// total time allowed here is 100ms
var tokenSource = new CancellationTokenSource(100);
try{
await WorkMethod(parameters, tokenSource.Token); // work
} catch (OperationCancelledException ocx){
// gracefully handle cancellations:
label.Text = "Operation timed out";
}
...
public async Task WorkMethod(object prm, CancellationToken ct){
try{
await sem.WaitAsync(ct); // equivalent to lock(object){...}
// synchronized work,
// call tokenSource.Token.ThrowIfCancellationRequested() or
// check tokenSource.IsCancellationRequested in long-running blocks
// and pass ct to other tasks, such as async HTTP or stream operations
} finally {
sem.Release();
}
}
NOT that I advise it, but you could pass the tokenSource instead of its Token into WorkMethod and periodically do tokenSource.CancelAfter(200) to add more time if you're certain you're not at a spot that can be dead-locked (waiting on an HTTP call) but I think that would be an esoteric approach to multithreading.
Instead your threads should be as fast as possible (minimum IO) and one thread can serialize the resources (producer) while others process a queue (consumers) if you need to deal with IO multithreading (say file compression, downloads etc) and avoid deadlock possibility altogether.
I really liked the visual idea of a using statement. However, that is not a viable solution. Why? Well, a sub-thread (the object/thread/timer within the using statement) cannot disrupt the main thread and inject an exception, thus causing it to stop what it was doing and jump to the nearest try/catch. That's what it all boils down to. The more I sat and worked with this, the more that came to light.
In short, it can't be done the way I wanted to do it.
However, I've taken Pieter's approach and mangled my code a bit. It does introduce some readability issues, but I've tried to mitigate them with comments and such.
public void MyMethod( ... )
{
...
// Placeholder for thread to kill if the action times out.
Thread threadToKill = null;
Action wrappedAction = () =>
{
// Take note of the action's thread. We may need to kill it later.
threadToKill = Thread.CurrentThread;
...
/* DO STUFF HERE */
...
};
// Now, execute the action. We'll deal with the action timeouts below.
IAsyncResult result = wrappedAction.BeginInvoke(null, null);
// Set the timeout to 10 minutes.
if (result.AsyncWaitHandle.WaitOne(10 * 60 * 1000))
{
// Everything was successful. Just clean up the invoke and get out.
wrappedAction.EndInvoke(result);
}
else
{
// We have timed out. We need to abort the thread!!
// Don't let it continue to try to do work. Something may be stuck.
threadToKill.Abort();
throw new TimeoutException("This code block timed out");
}
...
}
Since I'm doing this in three or four places per major section, this does get harder to read over. However, it works quite well.

How to in case of timeout to execute method again and again until it completes successfully?

I have asp.net application. All business logic in business layer.
Here is the example of the method
public void DoSomething()
{
PersonClass pc = new PersonClass();
pc.CreatePerson();
pc.AssignBasicTask();
pc.ChangePersonsStatus();
pc.CreateDefaultSettings();
}
what happens once in a while, one of the sub method can timeout, so as a result the process can be incompleted.
what I think in this case to make sure all steps completed properly is
public void DoSomething()
{
PersonClass pc = new PersonClass();
var error = null;
error = pc.CreatePerson();
if(error != timeout exception)
error = pc.AssignBasicTask();
else
return to step above
if(error != timeout exception)
error = pc.ChangePersonsStatus();
else
return to step above
if(error != timeout exception)
error = pc.CreateDefaultSettings();
else
return to step above
}
but it's just an idea, more then sure it's a proper way how to handle this.
Of course, this can be done more or less elegantly, with different options for timing out or giving up - but an easy way to achieve what you want, would be to define a retry method which keeps retrying an action until it succeeds:
public static class RetryUtility
{
public T RetryUntilSuccess<T>(Func<T> action)
{
while(true)
{
try
{
return action();
}
catch
{
// Swallowing exceptions is BAD, BAD, BAD. You should AT LEAST log it.
}
}
}
public void RetryUntilSuccess(Action action)
{
// Trick to allow a void method being passed in without duplicating the implementation.
RetryUntilSuccess(() => { action(); return true; });
}
}
Then do
RetryUtility.RetryUntilSuccess(() => pc.CreatePerson());
RetryUtility.RetryUntilSuccess(() => pc.AssignBasicTask());
RetryUtility.RetryUntilSuccess(() => pc.ChangePersonsStatus());
RetryUtility.RetryUntilSuccess(() => pc.CreateDefaultSettings());
I must urge you to think about what to do if the method keeps failing, you could be creating an infinite loop - perhaps it should give up after N retries or back off with exponentially raising retry time - you will need to define that, since we cannot know enough about your problem domain to decide that.
You have it pretty close to correct in your psuedo-code, and there a lot of ways to do this, but here is how I would do it:
PersonClass pc = new PersonClass();
while(true)
if(pc.CreatePerson())
break;
while(true)
if(pc.AssignBasicTask())
break;
This assumes that your methods return true to indicate success, false to indicate a timeoiut failure (and probably an exception for any other kind of failure). And while I didn't do it here, I would strongly recommend some sort of try counting to make sure it doesn't just loop forever and ever.
Use a TransactionScope for to make sure everything is executed as a unit. More info here: Implementing an Implicit Transaction using Transaction Scope
You should never retry a timed out operation infinitely, you may end up hanging the server or with an infinite loop or both. There should always be a threshold of how many retries is acceptable to attempt before quitting.
Sample:
using(TransactionScope scope = new TransactionScope())
{
try
{
// Your code here
// If no errors were thrown commit your transaction
scope.Complete();
}
catch
{
// Some error handling
}
}

Categories

Resources