Develop Reference

Calling ServiceBus' IQueueClient.CompleteAsync() from a different thread always fails

I'm receiving from a queue using IQueueClient.RegisterMessageHandler(), and in the receiving method I'm then starting a Hangfire task. But in the Hangfire task, all calls to IQueueClient.CompleteAsync() to mark the message as consumed (because I'm using ReceiveMode.PeekLock mode) fail.
I suspect it's because the Hangfire task re-creates the IQueueClient, that it may be because the call to CompleteAsync() is from a different instance of IQueueClient? Or is something else possibly going wrong? I'm passing Message.SystemProperties.LockToken into the Hangfire task which is what is then being used by CompleteAsync.

I suspect it's because the Hangfire task re-creates the IQueueClient, that it may be because the call to CompleteAsync() is from a different instance of IQueueClient?
That's correct. LockToken is not enough to complete the message. You have to use the same client used to receive the message in order to complete it.

Is there a way to return a response from an API call to ASP.NET while keeping the instance running?

I am writing an API using ASP.NET and I have some potentially long running code from the different end points. The system uses CQRS and Event Sourcing. A Command comes into to an end point and is then published as an event using MediatR. However the Handlers are potentially long running. Since some of the Requests coming in might be sent to multiple Handlers. This process could take longer than the 12s that AWS allows before returning an Error code.
Is there a way to return a response back to the caller to say that the event has been created while still contining with the process? That is to say fire off a separate task that performs the long running piece of code, that also catches and logs errors. Then return a value back to the user saying the Event has been successfully created?
I believe that ASP.NET spins up a new instance each time a call is made, will the old instance die one a value is returned, killing the task?
I could be wrong with a number of points here, this is my knowledge gleaned from the internet but I could have missunderstood articles.
Thanks.

Yes, you should pass the long-running task off to a background process and return to the user. When the task is complete, notifiy the user with whatever mechanism is appropriate for your site.
But do not start a new thread, what you want is to have a background service running for this, and use that to manage your request.

If a new thread is running the long operation it will remain “open/live” until it finishes. Also you can configure the app pool to always be active.
There are a lot of frameworks to work with long running tasks like Hangfire.
And to keep the user updated with the status of the task you can use SignalR to push notifications to the UI

What happens to the task that is initiated if the web api returns the response

I am trying to implement a task in fire and forget manner.
Lets look at the below piece of code.
public IHttpActionResult Update(int id)
{
var updatedResult = _updater.update(id);
// fire and forget a task
sendEmailToUser();
return ok();
}
private async Task sendEmailToUser()
{
var httpclient = new HttpClient();
// assume the client is initiated with required url and other headers
await httpclient.postasync("some url");
}
Given the above code, can i safely assume that whenever Update endpoint is called, sendEmailToUser task is triggered and will be run to completion ?

No. You should almost never start any background threads in web application. HTTP is suppose to be stateless and the web server was designed with that in mind.
The server might be put into sleep state when there is no incoming request for a set period of time. During that time all the background execution will be halt including the one you had. It might and might not get resume when the next request comes in.
Or when IIS decides to recycle your App domain on a scheduled basis your thread will get killed too.
If you really need background tasks then do that using windows service or run it as a separate console application.

Under normal conditions, it's reasonable to expect that the task will run to completion. It will go on independently.
Your biggest concerns, in this case, should be about the web API not being terminated, and the task not throwing an exception.
But if OP needs to be 100% sure, there are other safer ways to code that.

Passing exception or message from thread/task to main thread

We are dealing with processing and uploading of large files in windows service (.net 4.0). Process and upoload steps can take minutes to complete. Admin has the ability to mark job as cancelled, directly in database but to clear in memory queue it requires service restart. Goal is to abandon that job and pick next job in queue without service restart. Here's what I want to do:
In main entry point, start two tasks:
Task processTask = Task.Factory.StartNew(ProcessJob);
Task monitorTask = Task.Factory.StartNew(MonitorDB);
ProcessJob would call multiple long running steps like ProcessFile, UploadFile. We are checking for job status between the steps but job may be stuck in one of these long running steps.
If monitorTask detects job status change in DB, it should communicate that to main thread (through exception or message), so that main thread can quit, removing itself from the processing queue and allow next job in queue to start. Without Wait cannot get to exception but cannot wait because need to run these tasks in parallel.
At this time we are not concerned with the fact that some synch step in ProcessJob may still be going on and it may eventually complete. We would handle that in code.
So far in all of my applications, I have used Task.ContinueWith for success and failure, but never had to communicate back to main thread.

You could communicate between the monitor thread and the main thread through a BlockingCollection<T>.
Here's a simple example I wrote.
BlockingCollection<string> queue =
new BlockingCollection<string>();
// monitor thread.
Task.Factory.StartNew(() =>
{
while (true)
{
Thread.Sleep(1000);
queue.Add("event occured.");
}
});
// main thread.
while (true)
{
// blocks when no messages are in queue.
string message = queue.Take();
// kill process thread here.
}

You can use a CancellationToken for this. If you look at the Task class, there is an overload where you can pass in a CancellationToken. Store a reference to this token in your service and simply have your upload/process routine periodically check if the token has been cancelled like so:
if (yourToken.IsCancellationRequested)
break;

checking if a thread is alive

for a task, potentially taking too long to complete, I'd like a mechanism
to start the task
return back to user interface (its a web page)
periodically/randomly check if the task is complete
cancel the executing task when user wishes so
get notified when the task completes / fails
what are the possible solutions?
Threads?
Start a thread, save its ManagedThreadId, (can you get a thread by its id)
write a windows service,
send the request to service via shared objects/files/db?
keep interacting with the service the same way (objects/files/db,etc)
Services?

Host a WCF Service in a Windows Service that will perform the background tasks by adding/reading from a queue which can be maintained either using MSMQ or in a database.
When you add an item for processing; you should get a task id. You should be able to then log the completion/failed/cancel status of the task in db against the task id.
You can have following methods in your WCF contract
int ProcessItem(ItemDetails details); // returns task id
bool CancelTask(int taskID); // returns true if successfully cancelled; false otherwise
TaskStatus GetTaskStatus(int taskID); // returns Cancelled, Waiting, Failed or Completed

You can do that in single process
Have a look at the Task Paralell Library and / or TPL DataFlow
http://msdn.microsoft.com/en-us/library/dd460717.aspx
http://go.microsoft.com/fwlink/?LinkId=205053