Develop Reference

Persist headers when redelivering a RabbitMq message using MassTransit

Purpose: I need to keep track of headers when I redeliver a message.
Configuration:
RabbitMQ 3.7.9
Erlang 21.2
MassTransit 5.1.5
MySql 8.0 for the Quartz database
What I've tried without success:
first attempt:
await context.Redeliver(TimeSpan.FromSeconds(5), (consumeCtx, sendCtx) => {
if (consumeCtx.Headers.TryGetHeader("SenderApp", out object sender))
{
sendCtx.Headers.Set("SenderApp", sender);
}
}).ConfigureAwait(false);
second attempt:
protected Task ScheduleSend(Uri rabbitUri, double delay)
{
return GetBus().ScheduleSend<IProcessOrganisationUpdate>(
rabbitUri,
TimeSpan.FromSeconds(delay),
_Data,
new HeaderPipe(_SenderApp, 0));
}
public class HeaderPipe : IPipe<SendContext>
{
private readonly byte _Priority;
private readonly string _SenderApp;
public HeaderPipe (byte priority)
{
_Priority = priority;
_SenderApp = Assembly.GetEntryAssembly()?.GetName()?.Name ?? "Default";
}
public HeaderPipe (string senderApp, byte priority)
{
_Priority = priority;
_SenderApp = senderApp;
}
public void Probe (ProbeContext context)
{ }
public Task Send (SendContext context)
{
context.Headers.Set("SenderApp", _SenderApp);
context.SetPriority(_Priority);
return Task.CompletedTask;
}
}
Expected: FinQuest.Robot.DBProcess
Result: null
I log in Consume method my SenderApp. The first time it's look like this
Initial trigger checking returns true for FinQuest.Robots.OrganisationLinkedinFeed (id: 001ae487-ad3d-4619-8d34-367881ec91ba, sender: FinQuest.Robot.DBProcess, modif: LinkedIn)
and looks like this after the redelivery
Initial trigger checking returns true for FinQuest.Robots.OrganisationLinkedinFeed (id: 001ae487-ad3d-4619-8d34-367881ec91ba, sender: , modif: LinkedIn)
What I'm doing wrong ? I don't want to use the Retry feature due to its maximum number of retry (I don't want to be limited).
Thanks in advance.

There is a method, used by the redelivery filter, that you might want to use:
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit/SendContextExtensions.cs#L90
public static void TransferConsumeContextHeaders(this SendContext sendContext, ConsumeContext consumeContext)
In your code, you would use it:
await context.Redeliver(TimeSpan.FromSeconds(5), (consumeCtx, sendCtx) => {
sendCtx.TransferConsumeContextHeaders(consumeCtx);
});

Close task before run again

I working on real-time search. At this moment on property setter which is bounded to edit text, I call a method which calls API and then fills the list with the result it looks like this:
private string searchPhrase;
public string SearchPhrase
{
get => searchPhrase;
set
{
SetProperty(ref searchPhrase, value);
RunOnMainThread(SearchResult.Clear);
isAllFriends = false;
currentPage = 0;
RunInAsync(LoadData);
}
}
private async Task LoadData()
{
var response = await connectionRepository.GetConnections(currentPage,
pageSize, searchPhrase);
foreach (UserConnection uc in response)
{
if (uc.Type != UserConnection.TypeEnum.Awaiting)
{
RunOnMainThread(() =>
SearchResult.Add(new ConnectionUser(uc)));
}
}
}
But this way is totally useless because of it totally mashup list of a result if a text is entering quickly. So to prevent this I want to run this method async in a property but if a property is changed again I want to kill the previous Task and star it again. How can I achieve this?

Some informations from this thread:
create a CancellationTokenSource
var ctc = new CancellationTokenSource();
create a method doing the async work
private static Task ExecuteLongCancellableMethod(CancellationToken token)
{
return Task.Run(() =>
{
token.ThrowIfCancellationRequested();
// more code here
// check again if this task is canceled
token.ThrowIfCancellationRequested();
// more code
}
}
It is important to have this checks for cancel in the code.
Execute the function:
var cancellable = ExecuteLongCancellableMethod(ctc.Token);
To stop the long running execution use
ctc.Cancel();
For further details please consult the linked thread.

This question can be answered in many different ways. However IMO I would look at creating a class that
Delays itself automatically for X (ms) before performing the seach
Has the ability to be cancelled at any time as the search request changes.
Realistically this will change your code design, and should encapsulate the logic for both 1 & 2 in a separate class.
My initial thoughts are (and none of this is tested and mostly pseudo code).
class ConnectionSearch
{
public ConnectionSearch(string phrase, Action<object> addAction)
{
_searchPhrase = phrase;
_addAction = addAction;
_cancelSource = new CancellationTokenSource();
}
readonly string _searchPhrase = null;
readonly Action<object> _addAction;
readonly CancellationTokenSource _cancelSource;
public void Cancel()
{
_cancelSource?.Cancel();
}
public async void PerformSearch()
{
await Task.Delay(300); //await 300ms between keystrokes
if (_cancelSource.IsCancellationRequested)
return;
//continue your code keep checking for
//loop your dataset
//call _addAction?.Invoke(uc);
}
}
This is basic, really just encapsulates the logic for both points 1 & 2, you will need to adapt the code to do the search.
Next you could change your property to cancel a previous running instance, and then start another instance immediatly after something like below.
ConnectionSearch connectionSearch;
string searchPhrase;
public string SearchPhrase
{
get => searchPhrase;
set
{
//do your setter work
if(connectionSearch != null)
{
connectionSearch.Cancel();
}
connectionSearch = new ConnectionSearch(value, addConnectionUser);
connectionSearch.PerformSearch();
}
}
void addConnectionUser(object uc)
{
//pperform your add logic..
}
The code is pretty straight forward, however you will see in the setter is simply cancelling an existing request and then creating a new request. You could put some disposal cleanup logic in place but this should get you started.

You can implement some sort of debouncer which will encapsulate the logics of task result debouncing, i.e. it will assure if you run many tasks, then only the latest task result will be used:
public class TaskDebouncer<TResult>
{
public delegate void TaskDebouncerHandler(TResult result, object sender);
public event TaskDebouncerHandler OnCompleted;
public event TaskDebouncerHandler OnDebounced;
private Task _lastTask;
private object _lock = new object();
public void Run(Task<TResult> task)
{
lock (_lock)
{
_lastTask = task;
}
task.ContinueWith(t =>
{
if (t.IsFaulted)
throw t.Exception;
lock (_lock)
{
if (_lastTask == task)
{
OnCompleted?.Invoke(t.Result, this);
}
else
{
OnDebounced?.Invoke(t.Result, this);
}
}
});
}
public async Task WaitLast()
{
await _lastTask;
}
}
Then, you can just do:
private readonly TaskDebouncer<Connections[]> _connectionsDebouncer = new TaskDebouncer<Connections[]>();
public ClassName()
{
_connectionsDebouncer.OnCompleted += OnConnectionUpdate;
}
public void OnConnectionUpdate(Connections[] connections, object sender)
{
RunOnMainThread(SearchResult.Clear);
isAllFriends = false;
currentPage = 0;
foreach (var conn in connections)
RunOnMainThread(() => SearchResult.Add(new ConnectionUser(conn)));
}
private string searchPhrase;
public string SearchPhrase
{
get => searchPhrase;
set
{
SetProperty(ref searchPhrase, value);
_connectionsDebouncer.Add(RunInAsync(LoadData));
}
}
private async Task<Connection[]> LoadData()
{
return await connectionRepository
.GetConnections(currentPage, pageSize, searchPhrase)
.Where(conn => conn.Type != UserConnection.TypeEnum.Awaiting)
.ToArray();
}
It is not pretty clear what RunInAsync and RunOnMainThread methods are.
I guess, you don't actually need them.

C# await seems not waiting

I've created class for reusing thread. This class has one public method for queueing jobs.
public AwaitableJob<T> PrepareJob(Func<T> job)
{
lock (locker)
{
var aj = new AwaitableJob<T>(job);
taskQueue.Enqueue(aj);
System.Threading.Monitor.PulseAll(locker);
return aj;
}
}
AwaitableJob is custom class which should provide awaiter object.
public class AwaitableJob<T> : INotifyCompletion where T : class
{
public Func<T> Job { get; private set; }
public bool IsCompleted { get; private set; } = false;
private object result;
public AwaitableJob(Func<T> job)
{
this.Job = job;
}
public AwaitableJob<T> GetAwaiter()
{
return this;
}
public void Invoke()
{
result = Job.Invoke();
IsCompleted = true;
}
public object GetResult()
{
return result;
}
public void OnCompleted(Action continuation)
{
continuation.Invoke();
}
}
And I tried to use it this way
public async void Connect()
{
var atm = await Worker.PrepareJob(ConnectHelper) as PresentModel;
if (atm == null) return;
var vm = new SwitchingViewModel(atm);
vm.NavigateTo();
}
But instead of waiting it always continues in execution so atm variable is always null. When I added breakpoints into AwaitableJob it showed that GetResult is called before IsCompleted was setted to true. Anyone knows where could be problem? Thanks for your help.

The problem is here:
public void OnCompleted(Action continuation)
{
continuation.Invoke();
}
The point of OnCompleted is to register a callback for when it has completed, but you are invoking the continuation now. The only time you should do that is for the thread-race condition where somebody checks IsCompleted and gets false, but the status changes between that and registering the callback. Other than that, what you should be doing is storing the callback, and invoking it from the code that actually changes the status to completed (which looks like your Invoke method), again remembering to consider thread-safety around the switchover.
Frankly, if this sounds complex: use TaskCompletionSource<T>

Enabling EAP and async/await on same class

I'm trying create a class that has events AND can be awaitable, but keep coming across stumbling blocks.
First, I tried a TransferJob class that returns a TransferTask object which is already running when it is returned. This would be accomplished through something like this:
public abstract class TransferJob
{
public TransferTask Start()
{
return Start(CancellationToken.None);
}
public TransferTask Start(CancellationToken token)
{
TransferTask task = CreateTransferTask();
task.Start(token);
return task;
}
protected abstract TransferTask CreateTransferTask();
}
public abstract class TransferTask
{
public event EventHandler<TransferStatusChangedEventArgs> StatusChanged;
private Task transferTask;
private TransferStatus status;
public TransferStatus Status
{
get { return this.status; }
protected set
{
TransferStatus oldStatus = this.status;
this.status = value;
OnStatusChanged(new TransferStatusChangedEventArgs(oldStatus, value));
}
}
internal void Start(CancellationToken token)
{
this.transferTask = TransferAsync(cancellationToken);
}
protected abstract Task TransferAsync(CancellationToken cancellationToken);
protected virtual void OnStatusChanged(TransferStatusChangedEventArgs txStatusArgs)
{
if (this.StatusChanged != null)
{
this.StatusChanged(this, txStatusArgs);
}
}
public TaskAwaiter GetAwaiter()
{
return this.transferTask.GetAwaiter();
}
}
The problem with the above is that if the TransferTask finishes very quickly, then users of TransferJob.Start() might not have time to register their event handlers on the returned TransferTask's StatusChanged event before it finishes. So I tried a different approach whereby the user has to call the TransferTask's Start() method themselves. This would give the user time to register their event handlers on the TransferTask in between the transferJob.CreateTask() call and the transferTask.Start() call:
public abstract class TransferJob
{
public abstract TransferTask CreateTask();
}
public abstract class TransferTask
{
public event EventHandler<TransferStatusChangedEventArgs> StatusChanged;
private Task transferTask;
private TransferStatus status;
public TransferStatus Status
{
get { return this.status; }
protected set
{
TransferStatus oldStatus = this.status;
this.status = value;
OnStatusChanged(new TransferStatusChangedEventArgs(oldStatus, value));
}
}
public void Start(CancellationToken token)
{
this.transferTask = TransferAsync(cancellationToken);
}
protected abstract Task TransferAsync(CancellationToken cancellationToken);
protected virtual void OnStatusChanged(TransferStatusChangedEventArgs txStatusArgs)
{
if (this.StatusChanged != null)
{
this.StatusChanged(this, txStatusArgs);
}
}
public TaskAwaiter GetAwaiter()
{
return this.transferTask.GetAwaiter();
}
}
Now, I have a different problem. If a user tries await transferTask; before transferTask.Start(); has been called, then presumably they'll get a NullReferenceException thrown because the task hasn't been started (and therefore assigned to the transferTask field). I'm really struggling for a way to solve this. Is there a way? Or a better pattern to use than the above?

I'm not really convinced this is a good idea. Just expose the TAP pattern. Delete the event as well as transferTask. The caller of Start must hold onto that task and pass it to any code that wants to listen for completion. This results in a very clean API. No mutable state, very simple to understand, supports all use cases.
If you insist, you can create a proxy task that looks like it's the real thing:
public abstract class TransferTask
{
public event EventHandler<TransferStatusChangedEventArgs> StatusChanged;
private TaskCompletionSource<object> transferTask = new ...; //changed
private TransferStatus status;
public TransferStatus Status
{
get { return this.status; }
protected set
{
TransferStatus oldStatus = this.status;
this.status = value;
OnStatusChanged(new TransferStatusChangedEventArgs(oldStatus, value));
}
}
public Task Start(CancellationToken token)
{
await TransferAsync(cancellationToken);
transferTask.SetResult(null); //complete proxy task
}
protected abstract Task TransferAsync(CancellationToken cancellationToken);
protected virtual void OnStatusChanged(TransferStatusChangedEventArgs txStatusArgs)
{
if (this.StatusChanged != null)
{
this.StatusChanged(this, txStatusArgs);
}
}
public TaskAwaiter GetAwaiter()
{
return this.transferTask.Task.GetAwaiter(); //changed
}
}
Now, transferTask.Task is always not null. That task will eventually complete. I quickly hacked this together, I hope the idea is clear.
Probably, you should base the event on transferTask.Task.ContinueWith(...).

The best way I found when trying to mix events and awaitable code in C# is to use the Reactive Extension (Rx) library. From Microsoft:
Reactive Extension (Rx) is a library to compose asynchronous and event-based programs using observable collections and LINQ-style query operators.
You could do something like the following to fix your issue. (I am not sure this is exactly what you want to accomplish, but the goal is just to demonstrate how Rx can be used to combine events with asynchronous code):
public async Task TransferAndWaitStartedAsync()
{
var transferTask = new TransferTask();
// Prepare the observable before executing the transfer to make sure that the observable sequence will receive the event
// You can use Linq operators to filter only specific events. In this case, I only care about events with Status == StatusCode.Started
var whenStatusChanged = Observable.FromEventPattern<TransferStatusChangedEventArgs>(h, transferTask.StatusChanged += h, h => transferTask.StatusChanged -= h)
.Where(e => e.EventArgs.Status == StatusCode.Started)
.FirstAsync();
// Start the transfer asynchronously
await transferTask.TransferAsync();
// Continuation will complete when receiving the first event that matches the predicate in the observable sequence even if the event was triggered too quickly.
await whenStatusChanged;
}
I find that the Rx library has a steep learning curve with all its subtleties, but when you know how to use it, it is a really powerful tool.
Intro to Rx with lot of examples
Design guidelines

ContextBoundObject with Async/Await

I have a system which uses AOP with ContextBoundObject.
This is used to intercept a method call and perform certain operations before and after the function. It all works fine until I make the 'function to be intercepted' async.
I understand that the C# compiler rewrites async methods into a state machine, which returns control to the sink as soon as 'await' is reached
So it continues into the interception and executes the code which is meant to be executed only after the Method.
I can see there is an "AsyncProcessMessage" in IMessageSink, but I can't find a way to invoke it, and I am not sure if it will work in the async/await scenario.
Is there a way to make Async/Await work with the ContextBoundObject? Is using another Aspect Oriented Programming approach the only option here?
The code sample below has the method to be intercepted decorated with the 'Audit' attribute and placed in the AuditFacade which is a ContextBoundObject. The SyncProcessMessage method in the AuditSink has the logic to be executed before and after the method.
[AuditBoundary]
public class AuditFacade : ContextBoundObject
{
[Audit]
public ResponseObject DoSomthing()
{
//Do something
return new ResponseObject();
}
/// <summary>
/// Async Method to be intercepted
/// </summary>
/// <returns></returns>
[Audit]
public async Task<ResponseObject> DoSomthingAysnc()
{
//Do something Async
await Task.Delay(10000);
return new ResponseObject();
}
}
[AttributeUsage(AttributeTargets.Method)]
public class AuditAttribute : Attribute
{
}
[AttributeUsage(AttributeTargets.Class)]
public class AuditBoundaryAttribute : ContextAttribute
{
public AuditBoundaryAttribute()
: base("AuditBoundary" + Guid.NewGuid().ToString())
{
}
public override void GetPropertiesForNewContext(IConstructionCallMessage ctorMsg)
{
ctorMsg.ContextProperties.Add(new AuditProperty());
}
}
public class AuditProperty : IContextProperty, IContributeObjectSink
{
public string Name
{
get { return "AuditProperty"; }
}
public bool IsNewContextOK(Context newCtx)
{
var p = newCtx.GetProperty("AuditProperty") as AuditProperty;
if (p == null)
return false;
return true;
}
public void Freeze(Context newContext)
{
}
public IMessageSink GetObjectSink(MarshalByRefObject obj, IMessageSink nextSink)
{
return new AuditSink(nextSink);
}
}
public class AuditSink : IMessageSink
{
private IMessageSink nextSink;
public AuditSink(IMessageSink nextSink)
{
this.nextSink = nextSink;
}
public IMessage SyncProcessMessage(IMessage msg)
{
var message = msg as IMethodCallMessage;
IMethodReturnMessage returnMessage = null;
ResponseObject response;
//Some Pre Processing happens here
var newMessage = new MethodCallMessageWrapper(message);
//Invoke the Method to be Audited
returnMessage = nextSink.SyncProcessMessage(newMessage) as IMethodReturnMessage;
response = returnMessage.ReturnValue as ResponseObject;
//Some Post Processing happens here with the "response"
return returnMessage;
}
public IMessageSink NextSink
{
get { return this.nextSink; }
}
public IMessageCtrl AsyncProcessMessage(IMessage msg, IMessageSink replySink)
{
return nextSink.AsyncProcessMessage(msg, replySink);
}
}

I don't know anything about ContextBoundObject, but I think that AsyncProcessMessage() has nothing to do with async-await and that the following should work using the normal SyncProcessMessage():
Do your preprocessing step.
Invoke the async method.
Add your postprocessing step as a continuation to the returned Task, using ContinueWith() or await.
Return the continuation Task to the caller.
If you're okay with your postprocessing executing on the thread pool, then ContinueWith() is probably simpler. If you need the postprocessing to execute on the original context, use await.
The await version could look like this:
var responseTask = (Task<ResponseObject>)returnMessage.ReturnValue;
Func<Task<ResponseObject>> postProcessTaskFunc = async () =>
{
var response = await responseTask;
// Some Post Processing happens here with the "response"
return response;
}
return new ReturnMessage(postProcessTaskFunc(), …);