How to get current retry count while executing using SqlAzureExecutionStrategy - c#

I'm currently using SqlAzureExecutionStrategy to implement retry login in my database transactions.
Here is the custom strategy which is inherited from SqlAzureExecutionStrategy :
using System;
using System.Data.Entity.SqlServer;
using System.Data.SqlClient;
public class DbExecutionStrategy : SqlAzureExecutionStrategy
{
private const int retryCount = 2;
private static readonly TimeSpan maxDelay = TimeSpan.FromMinutes(1);
public DbExecutionStrategy ()
: base(retryCount, maxDelay)
{
}
public DbExecutionStrategy (int maxRetryCount, TimeSpan maxDelay)
: base(maxRetryCount, maxDelay)
{
}
protected override bool ShouldRetryOn(Exception exception)
{
if(exception.InnerException != null && exception.InnerException is SqlException)
{
return true;
}
return false;
}
}
I'm using this custom strategy like :
var executionStrategy = new DbExecutionStrategy();
executionStrategy.Execute(() => { some business logic here });
I want to print the current retry count inside the Execute method. Is it even possible?
This is for the logging purpose. To see which retry yielded best results etc. Please feel free to suggest best retry count and maxDelay before the next retry.
In ShouldRetryOn method, what is the best way to define what are the exceptions you want to retry on? I'm hoping it is switch wth exceptions as cases and retrun true in each case.

Related

IAsyncEnumerable like a Source for akka streams

I want to use IAsyncEnumerable like a Source for akka streams. But I not found, how do it.
No sutiable method in Source class for this code.
using System.Collections.Generic;
using System.Threading.Tasks;
using Akka.Streams.Dsl;
namespace ConsoleApp1
{
class Program
{
static async Task Main(string[] args)
{
Source.From(await AsyncEnumerable())
.Via(/*some action*/)
//.....
}
private static async IAsyncEnumerable<int> AsyncEnumerable()
{
//some async enumerable
}
}
}
How use IAsyncEnumerbale for Source?
This has been done in the past as a part of Akka.NET Streams contrib package, but since I don't see it there anymore, let's go through on how to implement such source. The topic can be quite long, as:
Akka.NET Streams is really about graph processing - we're talking about many-inputs/many-outputs configurations (in Akka.NET they're called inlets and outlets) with support for cycles in graphs.
Akka.NET is not build on top of .NET async/await or even on top of .NET standard thread pool library - they're both pluggable, which means that the lowest barier is basically using callbacks and encoding what C# compiler sometimes does for us.
Akka.NET streams is capable of both pushing and pulling values between stages/operators. IAsyncEnumerable<T> can only pull data while IObservable<T> can only push it, so we get more expressive power here, but this comes at a cost.
The basics of low level API used to implement custom stages can be found in the docs.
The starter boilerplate looks like this:
public static class AsyncEnumerableExtensions {
// Helper method to change IAsyncEnumerable into Akka.NET Source.
public static Source<T, NotUsed> AsSource<T>(this IAsyncEnumerable<T> source) =>
Source.FromGraph(new AsyncEnumerableSource<T>(source));
}
// Source stage is description of a part of the graph that doesn't consume
// any data, only produce it using a single output channel.
public sealed class AsyncEnumerableSource<T> : GraphStage<SourceShape<T>>
{
private readonly IAsyncEnumerable<T> _enumerable;
public AsyncEnumerableSource(IAsyncEnumerable<T> enumerable)
{
_enumerable = enumerable;
Outlet = new Outlet<T>("asyncenumerable.out");
Shape = new SourceShape<T>(Outlet);
}
public Outlet<T> Outlet { get; }
public override SourceShape<T> Shape { get; }
/// Logic if to a graph stage, what enumerator is to enumerable.
protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) =>
new Logic(this);
sealed class Logic: OutGraphStageLogic
{
public override void OnPull()
{
// method called whenever a consumer asks for new data
}
public override void OnDownstreamFinish()
{
// method called whenever a consumer stage finishes,used for disposals
}
}
}
As mentioned, we don't use async/await straight away here: even more, calling Logic methods in asynchronous context is unsafe. To make it safe we need to register out methods that may be called from other threads using GetAsyncCallback<T> and call them via returned wrappers. This will ensure, that not data races will happen when executing asynchronous code.
sealed class Logic : OutGraphStageLogic
{
private readonly Outlet<T> _outlet;
// enumerator we'll call for MoveNextAsync, and eventually dispose
private readonly IAsyncEnumerator<T> _enumerator;
// callback called whenever _enumerator.MoveNextAsync completes asynchronously
private readonly Action<Task<bool>> _onMoveNext;
// callback called whenever _enumerator.DisposeAsync completes asynchronously
private readonly Action<Task> _onDisposed;
// cache used for errors thrown by _enumerator.MoveNextAsync, that
// should be rethrown after _enumerator.DisposeAsync
private Exception? _failReason = null;
public Logic(AsyncEnumerableSource<T> source) : base(source.Shape)
{
_outlet = source.Outlet;
_enumerator = source._enumerable.GetAsyncEnumerator();
_onMoveNext = GetAsyncCallback<Task<bool>>(OnMoveNext);
_onDisposed = GetAsyncCallback<Task>(OnDisposed);
}
// ... other methods
}
The last part to do are methods overriden on `Logic:
OnPull used whenever the downstream stage calls for new data. Here we need to call for next element of async enumerator sequence.
OnDownstreamFinish called whenever the downstream stage has finished and will not ask for any new data. It's the place for us to dispose our enumerator.
Thing is these methods are not async/await, while their enumerator's equivalent are. What we basically need to do there is to:
Call corresponding async methods of underlying enumerator (OnPull → MoveNextAsync and OnDownstreamFinish → DisposeAsync).
See, if we can take their results immediately - it's important part that usually is done for us as part of C# compiler in async/await calls.
If not, and we need to wait for the results - call ContinueWith to register our callback wrappers to be called once async methods are done.
sealed class Logic : OutGraphStageLogic
{
// ... constructor and fields
public override void OnPull()
{
var hasNext = _enumerator.MoveNextAsync();
if (hasNext.IsCompletedSuccessfully)
{
// first try short-path: values is returned immediately
if (hasNext.Result)
// check if there was next value and push it downstream
Push(_outlet, _enumerator.Current);
else
// if there was none, we reached end of async enumerable
// and we can dispose it
DisposeAndComplete();
}
else
// we need to wait for the result
hasNext.AsTask().ContinueWith(_onMoveNext);
}
// This method is called when another stage downstream has been completed
public override void OnDownstreamFinish() =>
// dispose enumerator on downstream finish
DisposeAndComplete();
private void DisposeAndComplete()
{
var disposed = _enumerator.DisposeAsync();
if (disposed.IsCompletedSuccessfully)
{
// enumerator disposal completed immediately
if (_failReason is not null)
// if we close this stream in result of error in MoveNextAsync,
// fail the stage
FailStage(_failReason);
else
// we can close the stage with no issues
CompleteStage();
}
else
// we need to await for enumerator to be disposed
disposed.AsTask().ContinueWith(_onDisposed);
}
private void OnMoveNext(Task<bool> task)
{
// since this is callback, it will always be completed, we just need
// to check for exceptions
if (task.IsCompletedSuccessfully)
{
if (task.Result)
// if task returns true, it means we read a value
Push(_outlet, _enumerator.Current);
else
// otherwise there are no more values to read and we can close the source
DisposeAndComplete();
}
else
{
// task either failed or has been cancelled
_failReason = task.Exception as Exception ?? new TaskCanceledException(task);
FailStage(_failReason);
}
}
private void OnDisposed(Task task)
{
if (task.IsCompletedSuccessfully) CompleteStage();
else {
var reason = task.Exception as Exception
?? _failReason
?? new TaskCanceledException(task);
FailStage(reason);
}
}
}
As of Akka.NET v1.4.30 this is now natively supported inside Akka.Streams via the RunAsAsyncEnumerable method:
var input = Enumerable.Range(1, 6).ToList();
var cts = new CancellationTokenSource();
var token = cts.Token;
var asyncEnumerable = Source.From(input).RunAsAsyncEnumerable(Materializer);
var output = input.ToArray();
bool caught = false;
try
{
await foreach (var a in asyncEnumerable.WithCancellation(token))
{
cts.Cancel();
}
}
catch (OperationCanceledException e)
{
caught = true;
}
caught.ShouldBeTrue();
I copied that sample from the Akka.NET test suite, in case you're wondering.
You can also use an existing primitive for streaming large collections of data. Here is an example of using Source.unfoldAsync to stream pages of data - in this case github repositories using Octokit - until there is no more.
var source = Source.UnfoldAsync<int, RepositoryPage>(startPage, page =>
{
var pageTask = client.GetRepositoriesAsync(page, pageSize);
var next = pageTask.ContinueWith(task =>
{
var page = task.Result;
if (page.PageNumber * pageSize > page.Total) return Option<(int, RepositoryPage)>.None;
else return new Option<(int, RepositoryPage)>((page.PageNumber + 1, page));
});
return next;
});
To run
using var sys = ActorSystem.Create("system");
using var mat = sys.Materializer();
int startPage = 1;
int pageSize = 50;
var client = new GitHubClient(new ProductHeaderValue("github-search-app"));
var source = ...
var sink = Sink.ForEach<RepositoryPage>(Console.WriteLine);
var result = source.RunWith(sink, mat);
await result.ContinueWith(_ => sys.Terminate());
class Page<T>
{
public Page(IReadOnlyList<T> contents, int page, long total)
{
Contents = contents;
PageNumber = page;
Total = total;
}
public IReadOnlyList<T> Contents { get; set; } = new List<T>();
public int PageNumber { get; set; }
public long Total { get; set; }
}
class RepositoryPage : Page<Repository>
{
public RepositoryPage(IReadOnlyList<Repository> contents, int page, long total)
: base(contents, page, total)
{
}
public override string ToString() =>
$"Page {PageNumber}\n{string.Join("", Contents.Select(x => x.Name + "\n"))}";
}
static class GitHubClientExtensions
{
public static async Task<RepositoryPage> GetRepositoriesAsync(this GitHubClient client, int page, int size)
{
// specify a search term here
var request = new SearchRepositoriesRequest("bootstrap")
{
Page = page,
PerPage = size
};
var result = await client.Search.SearchRepo(request);
return new RepositoryPage(result.Items, page, result.TotalCount);
}
}

How to simulate time passing from a unit test?

I have a method running in an infinite loop in a task from the ctor. The message does the equivalent to sending a ping, and upon a certain number of failed pings, prevents other commands from being able to be sent. The method is started from the ctor with Task.Factory.StartNew(()=>InitPingBackgroundTask(pingCts.token));
public async Task InitPingBackgroundTask (CancellationToken token)
{
while(!token.IsCancellationRequested)
{
try
{
Ping();
// handle logic
_pingsFailed = 0;
_canSend = true;
}
catch(TimeoutException)
{
// handle logic
if(++_pingsFailed >= settings.MaxPingLossNoConnection)
_canSend = false;
}
finally
{
await Task.Delay(settings.PingInterval);
}
}
}
I have a second method DoCmd like so:
public void DoCmd()
{
if(_canSend)
{
// handle logic
}
}
In my test, using Moq, I can set the settings to have PingInterval=TimeSpan.FromSeconds(1) and MaxPingLossNoConnection = 2 for the sake of my test.
After setting up the sut and everything else, I want to call DoCmd() and verify that it is not actually sent. (I have mocked its dependencies and can verify that the logic in its method was never called)
One way to achieve this is to add a sleep or delay from within the test, before calling sut.DoCmd(), but this makes the test take longer. Is there a way to simulate time passing (like tick(desiredTime) in angular, or something similar)? To simulate the passing of time without having to actually wait the time?
Any leads would be appreciated :-)
EDIT
Added the test code:
public void NotAllowSendIfTooManyPingsFailed()
{
var settings = new Settings()
{
PingInterval=TimeSpan.FromSeconds(1),
MaxPingLossNoConnection = 0 // ended up changing to 0 to fail immidiately
}
// set up code, that Ping messages should fail
Thread.Sleep(100); // necessary to Ping has a chance to fail before method is called
sut.DoCmd();
// assert logic, verifying that DoCmd did not go through
}
Ideally, I would like to be able to play with MaxPingLossNoConnection and set it to different numbers to test circumstances. That would also require adding sleeps to let the time pass. In this test, I would want to remove the Sleep altogether, as well as in other similar tests where sleep would be longer
In order to deal with time concerns im my tests, I found a little Clock class a while ago and replaced all the calls to DateTime.Now in my systems for Clock.Now. I tweaked it a little bit to have the option to set the time and freeze it or to keep it running during the tests.
Clock.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ComumLib
{
public class Clock : IDisposable
{
private static DateTime? _startTime;
private static DateTime? _nowForTest;
public static DateTime Now
{
get
{
if (_nowForTest == null)
{
return DateTime.Now;
}
else
{
//freeze time
if (_startTime == null)
{
return _nowForTest.GetValueOrDefault();
}
//keep running
else
{
TimeSpan elapsedTime = DateTime.Now.Subtract(_startTime.GetValueOrDefault());
return _nowForTest.GetValueOrDefault().Add(elapsedTime);
}
}
}
}
public static IDisposable NowIs(DateTime dateTime, bool keepTimeRunning = false)
{
_nowForTest = dateTime;
if (keepTimeRunning)
{
_startTime = DateTime.Now;
}
return new Clock();
}
public static IDisposable ResetNowIs()
{
_startTime = null;
_nowForTest = null;
return new Clock();
}
public void Dispose()
{
_startTime = null;
_nowForTest = null;
}
}
}
In my tests, I either call
DateTime dt = new DateTime(2022,01,01,22,12,44,2);
Clock.NowIs(dt,true); //time keeps running during the test
or
Clock.NowIs(dt); //time remains the same during the test

Persist headers when redelivering a RabbitMq message using MassTransit

Purpose: I need to keep track of headers when I redeliver a message.
Configuration:
RabbitMQ 3.7.9
Erlang 21.2
MassTransit 5.1.5
MySql 8.0 for the Quartz database
What I've tried without success:
first attempt:
await context.Redeliver(TimeSpan.FromSeconds(5), (consumeCtx, sendCtx) => {
if (consumeCtx.Headers.TryGetHeader("SenderApp", out object sender))
{
sendCtx.Headers.Set("SenderApp", sender);
}
}).ConfigureAwait(false);
second attempt:
protected Task ScheduleSend(Uri rabbitUri, double delay)
{
return GetBus().ScheduleSend<IProcessOrganisationUpdate>(
rabbitUri,
TimeSpan.FromSeconds(delay),
_Data,
new HeaderPipe(_SenderApp, 0));
}
public class HeaderPipe : IPipe<SendContext>
{
private readonly byte _Priority;
private readonly string _SenderApp;
public HeaderPipe (byte priority)
{
_Priority = priority;
_SenderApp = Assembly.GetEntryAssembly()?.GetName()?.Name ?? "Default";
}
public HeaderPipe (string senderApp, byte priority)
{
_Priority = priority;
_SenderApp = senderApp;
}
public void Probe (ProbeContext context)
{ }
public Task Send (SendContext context)
{
context.Headers.Set("SenderApp", _SenderApp);
context.SetPriority(_Priority);
return Task.CompletedTask;
}
}
Expected: FinQuest.Robot.DBProcess
Result: null
I log in Consume method my SenderApp. The first time it's look like this
Initial trigger checking returns true for FinQuest.Robots.OrganisationLinkedinFeed (id: 001ae487-ad3d-4619-8d34-367881ec91ba, sender: FinQuest.Robot.DBProcess, modif: LinkedIn)
and looks like this after the redelivery
Initial trigger checking returns true for FinQuest.Robots.OrganisationLinkedinFeed (id: 001ae487-ad3d-4619-8d34-367881ec91ba, sender: , modif: LinkedIn)
What I'm doing wrong ? I don't want to use the Retry feature due to its maximum number of retry (I don't want to be limited).
Thanks in advance.
There is a method, used by the redelivery filter, that you might want to use:
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit/SendContextExtensions.cs#L90
public static void TransferConsumeContextHeaders(this SendContext sendContext, ConsumeContext consumeContext)
In your code, you would use it:
await context.Redeliver(TimeSpan.FromSeconds(5), (consumeCtx, sendCtx) => {
sendCtx.TransferConsumeContextHeaders(consumeCtx);
});

Implementing retry logic for deadlock exceptions

I've implemented a generic repository and was wondering if there is a smart way to implement a retry logic in case of a deadlock exception?
The approach should be the same for all repository methods. So is there anyway I can avoid writing 'try/catch - call method again with retry-count', in every single method?
Any suggetsion are welcome.
A bit of my Repository code:
public class GenericRepository : IRepository
{
private ObjectContext _context;
public List<TEntity> ExecuteStoreQuery<TEntity>(string commandText, params object[] parameters) where TEntity : class
{
List<TEntity> myList = new List<TEntity>();
var groupData = _context.ExecuteStoreQuery<TEntity>(commandText, parameters);
return myList;
}
public IQueryable<TEntity> GetQuery<TEntity>() where TEntity : class
{
var entityName = GetEntityName<TEntity>();
return _context.CreateQuery<TEntity>(entityName);
}
public IEnumerable<TEntity> GetAll<TEntity>() where TEntity : class
{
return GetQuery<TEntity>().AsEnumerable();
}
EDIT:
1.Solution:
Modified slightly from chris.house.00's solution
public static T DeadlockRetryHelper<T>(Func<T> repositoryMethod, int maxRetries)
{
var retryCount = 0;
while (retryCount < maxRetries)
{
try
{
return repositoryMethod();
}
catch (System.Data.SqlClient.SqlException ex)
{
if (ex.Number == 1205)// Deadlock
retryCount++;
else
throw;
}
}
return default(T);
}
And you call it like this:
public TEntity FirstOrDefault<TEntity>(Expression<Func<TEntity, bool>> predicate) where TEntity : class
{
return RetryUtility.DeadlockRetryHelper<TEntity>( () =>p_FirstOrDefault<TEntity>(predicate), 3);
}
protected TEntity p_FirstOrDefault<TEntity>(Expression<Func<TEntity, bool>> predicate) where TEntity : class
{
return GetQuery<TEntity>().FirstOrDefault<TEntity>(predicate);
}
How about something like this:
public T DeadlockRetryHelper<T>(Func<T> repositoryMethod, int maxRetries)
{
int retryCount = 0;
while (retryCount < maxRetries)
{
try
{
return repositoryMethod();
}
catch (SqlException e) // This example is for SQL Server, change the exception type/logic if you're using another DBMS
{
if (e.Number == 1205) // SQL Server error code for deadlock
{
retryCount++;
}
else
{
throw; // Not a deadlock so throw the exception
}
// Add some code to do whatever you want with the exception once you've exceeded the max. retries
}
}
}
With the above code, your retry logic is all in this method and you can just pass your repository method in as a delegate.
I know this is an old post but wanted to share an updated answer.
EF 6 now has a built-in solution, you can set the execution strategy which would be a one time implementation. You create a class that inherits from DbExectutionStrategy and overrides the ShouldRetryOn() virtual method. You can create a static class of the exceptions containing constant field valuess that are retry eligible codes and loop through each one to determine if the current sql exception being thrown matches the list of eligible retry codes...
public static class SqlRetryErrorCodes
{
public const int TimeoutExpired = -2;
public const int Deadlock = 1205;
public const int CouldNotOpenConnection = 53;
public const int TransportFail = 121;
}
public class MyCustomExecutionStrategy : DbExecutionStrategy
{
public MyCustomExecutionStrategy(int maxRetryCount, TimeSpan maxDelay) : base(maxRetryCount, maxDelay) { }
private readonly List<int> _errorCodesToRetry = new List<int>
{
SqlRetryErrorCodes.Deadlock,
SqlRetryErrorCodes.TimeoutExpired,
SqlRetryErrorCodes.CouldNotOpenConnection,
SqlRetryErrorCodes.TransportFail
};
protected override bool ShouldRetryOn(Exception exception)
{
var sqlException = exception as SqlException;
if (sqlException != null)
{
foreach (SqlError err in sqlException.Errors)
{
// Enumerate through all errors found in the exception.
if (_errorCodesToRetry.Contains(err.Number))
{
return true;
}
}
}
return false;
}
}
Finally once, you've set up your custom execution strategy, you simply create another class that inherits from DbConfiguration with a public constructor that Sets the execution strategy:
public class MyEfConfigurations : DbConfiguration
{
public MyEfConfigurations()
{
SetExecutionStrategy("System.Data.SqlClient",() => new MyCustomExecutionStrategy(5,TimeSpan.FromSeconds(10)));
}
}
EntityFramework 6 add ExecutionStrategy feature. All that is need is to setup up the strategy properly.
My retry policy:
public class EFRetryPolicy : DbExecutionStrategy
{
public EFRetryPolicy() : base()
{
}
//Keep this constructor public too in case it is needed to change defaults of exponential back off algorithm.
public EFRetryPolicy(int maxRetryCount, TimeSpan maxDelay): base(maxRetryCount, maxDelay)
{
}
protected override bool ShouldRetryOn(Exception ex)
{
bool retry = false;
SqlException sqlException = ex as SqlException;
if (sqlException != null)
{
int[] errorsToRetry =
{
1205, //Deadlock
-2, //Timeout
};
if (sqlException.Errors.Cast<SqlError>().Any(x => errorsToRetry.Contains(x.Number)))
{
retry = true;
}
}
return retry;
}
}
Tell EF to apply my strategy:
public class EFPolicy: DbConfiguration
{
public EFPolicy()
{
SetExecutionStrategy(
"System.Data.SqlClient",
() => new EFRetryPolicy());
}
}
Sources:
Implementing Connection Resiliency with Entity Framework 6
Microsoft documentation
The retry strategy will not work with user initiated transactions (transaction created with TransactionScope) as explained here. If used you will get the Error The configured execution strategy does not support user initiated transactions
The solution works though I prefer not to have to worry about the number of arguments to the Action or Func that will be retired. If you create a single retry method with a generic Action, you can handle all of the variability of the method to be called in a lambda:
public static class RetryHelper
{
public static void DeadlockRetryHelper(Action method, int maxRetries = 3)
{
var retryCount = 0;
while (retryCount < maxRetries)
{
try
{
method();
return;
}
catch (System.Data.SqlClient.SqlException ex)
{
if (ex.Number == 1205)// Deadlock
{
retryCount++;
if (retryCount >= maxRetries)
throw;
// Wait between 1 and 5 seconds
Thread.Sleep(new Random().Next(1000, 5000));
}
else
throw;
}
}
}
}
Then use it like so:
RetryHelper.DeadlockRetryHelper(() => CopyAndInsertFile(fileModel));
Have you considered some form of policy injection? You could use Unity interception, just as an example, to capture all your repository calls. Then you just write the retry logic once, in the interceptor, rather than repeating it many times in each method.
I have used the below solution provided by MiguelSlv in above post and it worked for me as expected. Its simple and easy.
EntityFramework 6 add ExecutionStrategy feature. All that is need is to setup up the strategy properly.
My retry policy:
public class EFRetryPolicy : DbExecutionStrategy
{
public EFRetryPolicy() : base()
{
}
//Keep this constructor public too in case it is needed to change defaults of exponential back off algorithm.
public EFRetryPolicy(int maxRetryCount, TimeSpan maxDelay): base(maxRetryCount, maxDelay)
{
}
protected override bool ShouldRetryOn(Exception ex)
{
bool retry = false;
SqlException sqlException = ex as SqlException;
if (sqlException != null)
{
int[] errorsToRetry =
{
1205, //Deadlock
-2, //Timeout
};
if (sqlException.Errors.Cast<SqlError>().Any(x => errorsToRetry.Contains(x.Number)))
{
retry = true;
}
}
return retry;
}
}
Tell EF to apply this policy
public class EFPolicy: DbConfiguration
{
public EFPolicy()
{
SetExecutionStrategy(
"System.Data.SqlClient",
() => new EFRetryPolicy());
}
}
Sources:
Implementing Connection Resiliency with Entity Framework 6
Microsoft documentation
The retry strategy will not work with user initiated transactions (transaction created with TransactionScope) as explained here. If used you will get the Error The configured execution strategy does not support user initiated transactions

Fody Async MethodDecorator to Handle Exceptions

I am trying to use Fody to wrap all exceptions thrown from a method with a common exception format.
So I have added the required interface declaration and class implementation that looks like this :
using System;
using System.Diagnostics;
using System.Reflection;
using System.Threading.Tasks;
[module: MethodDecorator]
public interface IMethodDecorator
{
void Init(object instance, MethodBase method, object[] args);
void OnEntry();
void OnExit();
void OnException(Exception exception);
void OnTaskContinuation(Task t);
}
[AttributeUsage(
AttributeTargets.Module |
AttributeTargets.Method |
AttributeTargets.Assembly |
AttributeTargets.Constructor, AllowMultiple = true)]
public class MethodDecorator : Attribute, IMethodDecorator
{
public virtual void Init(object instance, MethodBase method, object[] args) { }
public void OnEntry()
{
Debug.WriteLine("base on entry");
}
public virtual void OnException(Exception exception)
{
Debug.WriteLine("base on exception");
}
public void OnExit()
{
Debug.WriteLine("base on exit");
}
public void OnTaskContinuation(Task t)
{
Debug.WriteLine("base on continue");
}
}
And the domain implementation that looks like this
using System;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
using System.Runtime.ExceptionServices;
namespace CC.Spikes.AOP.Fody
{
public class FodyError : MethodDecorator
{
public string TranslationKey { get; set; }
public Type ExceptionType { get; set; }
public override void Init(object instance, MethodBase method, object[] args)
{
SetProperties(method);
}
private void SetProperties(MethodBase method)
{
var attribute = method.CustomAttributes.First(n => n.AttributeType.Name == nameof(FodyError));
var translation = attribute
.NamedArguments
.First(n => n.MemberName == nameof(TranslationKey))
.TypedValue
.Value
as string;
var exceptionType = attribute
.NamedArguments
.First(n => n.MemberName == nameof(ExceptionType))
.TypedValue
.Value
as Type;
TranslationKey = translation;
ExceptionType = exceptionType;
}
public override void OnException(Exception exception)
{
Debug.WriteLine("entering fody error exception");
if (exception.GetType() != ExceptionType)
{
Debug.WriteLine("rethrowing fody error exception");
//rethrow without losing stacktrace
ExceptionDispatchInfo.Capture(exception).Throw();
}
Debug.WriteLine("creating new fody error exception");
throw new FodyDangerException(TranslationKey, exception);
}
}
public class FodyDangerException : Exception
{
public string CallState { get; set; }
public FodyDangerException(string message, Exception error) : base(message, error)
{
}
}
}
This works fine for synchronous code. But for asynchronous code the exception handler is skipped, even though all the other IMethodDecorator are executed (like OnExit, and OnTaskContinuation).
For example, looking at the following test class :
public class FodyTestStub
{
[FodyError(ExceptionType = typeof(NullReferenceException), TranslationKey = "EN_WHATEVER")]
public async Task ShouldGetErrorAsync()
{
await Task.Delay(200);
throw new NullReferenceException();
}
public async Task ShouldGetErrorAsync2()
{
await Task.Delay(200);
throw new NullReferenceException();
}
}
I see that ShouldGetErrorAsync produces the following IL code :
// CC.Spikes.AOP.Fody.FodyTestStub
[FodyError(ExceptionType = typeof(NullReferenceException), TranslationKey = "EN_WHATEVER"), DebuggerStepThrough, AsyncStateMachine(typeof(FodyTestStub.<ShouldGetErrorAsync>d__3))]
public Task ShouldGetErrorAsync()
{
MethodBase methodFromHandle = MethodBase.GetMethodFromHandle(methodof(FodyTestStub.ShouldGetErrorAsync()).MethodHandle, typeof(FodyTestStub).TypeHandle);
FodyError fodyError = (FodyError)Activator.CreateInstance(typeof(FodyError));
object[] args = new object[0];
fodyError.Init(this, methodFromHandle, args);
fodyError.OnEntry();
Task task;
try
{
FodyTestStub.<ShouldGetErrorAsync>d__3 <ShouldGetErrorAsync>d__ = new FodyTestStub.<ShouldGetErrorAsync>d__3();
<ShouldGetErrorAsync>d__.<>4__this = this;
<ShouldGetErrorAsync>d__.<>t__builder = AsyncTaskMethodBuilder.Create();
<ShouldGetErrorAsync>d__.<>1__state = -1;
AsyncTaskMethodBuilder <>t__builder = <ShouldGetErrorAsync>d__.<>t__builder;
<>t__builder.Start<FodyTestStub.<ShouldGetErrorAsync>d__3>(ref <ShouldGetErrorAsync>d__);
task = <ShouldGetErrorAsync>d__.<>t__builder.Task;
fodyError.OnExit();
}
catch (Exception exception)
{
fodyError.OnException(exception);
throw;
}
return task;
}
And ShouldGetErrorAsync2 generates :
// CC.Spikes.AOP.Fody.FodyTestStub
[DebuggerStepThrough, AsyncStateMachine(typeof(FodyTestStub.<ShouldGetErrorAsync2>d__4))]
public Task ShouldGetErrorAsync2()
{
FodyTestStub.<ShouldGetErrorAsync2>d__4 <ShouldGetErrorAsync2>d__ = new FodyTestStub.<ShouldGetErrorAsync2>d__4();
<ShouldGetErrorAsync2>d__.<>4__this = this;
<ShouldGetErrorAsync2>d__.<>t__builder = AsyncTaskMethodBuilder.Create();
<ShouldGetErrorAsync2>d__.<>1__state = -1;
AsyncTaskMethodBuilder <>t__builder = <ShouldGetErrorAsync2>d__.<>t__builder;
<>t__builder.Start<FodyTestStub.<ShouldGetErrorAsync2>d__4>(ref <ShouldGetErrorAsync2>d__);
return <ShouldGetErrorAsync2>d__.<>t__builder.Task;
}
If I call ShouldGetErrorAsync, Fody is intercepting the call, and wrapping the method body in a try catch. But if the method is async, it never hits the catch statement even though the fodyError.OnTaskContinuation(task) and fodyError.OnExit() are still called.
On the other hand, ShouldGetErrorAsync will handle the error just fine, even though there is no error handling block in the IL.
My question is, how should Fody be generating the IL to properly inject the error block and make it so async errors are intercepted?
Here is a repo with tests that reproduces the issue
You are only placing the try-catch around the content of the 'kick-off' method, this will only protect you up to the point where it first needs to reschedule (the 'kick-off' method will end when the async method first needs to reschedule and so will not be on the stack when the async method resumes).
You should look at modifying the method implementing IAsyncStateMachine.MoveNext() on the state machine instead. In particular, look for the call to SetException(Exception) on the async method builder (AsyncVoidMethodBuilder, AsyncTaskMethodBuilder or AsyncTaskMethodBuilder<TResult>) and wrap the exception just before passing it in.
await sure makes asynchronous methods look simple, doesn't it? :) You just found a leak in that abstraction - the method usually returns as soon as the first await is found, and your exception helper has no way to intercept any later exceptions.
What you need to do is implement both the OnException, and handle the return value from the method. When the method returns, and the task isn't completed, you need to wind up an error continuation on the task, which needs to handle exceptions the way you want them to be handled. The Fody guys thought of that - that's what the OnTaskContinuation is for. You need to check the Task.Exception to see if there's an exception lurking in the task, and handle it however you need to.
I think this will only work if you want to rethrow the exception while doing logging or something - it does not allow you to replace the exception with something different. You should test that :)

Categories

Resources