I'm trying to figure out how to cancel a task that is stuck on a class method from the .Net framework.
I have a project that scans a large amount of remote workstations on the network and perform various operations on each. Each machine is its own async task, and each operation being performed within is it's own task that is waited on before going to the next.
Every now and then, maybe 1 in a few thousand machines, for unknown reasons (network connection?, workstation health issues?) a native .Net function, like RegistryKey.OpenRemoteBaseKey, or Enumerable.Count will hang without throwing an exception. In situations like this, I'd like to abort or cancel the task after x seconds, but since the code is essentially frozen, I don't think I can use a cancellation token for that.
The only solution I currently have been able to come up with is to use a task.wait for each operation with a timeout...which does work, except it leaves that thread out there still trying to execute. It's not a true cancellation, it's just abandonment.
Is there an option that I'm missing where I can have task.wait type timeout functionality that actually kills the stuck task?
private static async Task scanRow(string machine)
{
string result;
Task task = null;
foreach (var operationToRun in operations)
{
switch (operationToRun)
{
case "Operation1":
result = await Operation1(machine);
break;
case "Operation2":
task = Task.Run(() => { result = Operation2(machine); });
if (task.Wait(TimeSpan.FromSeconds(10)))
{
//time out logic
//code continues but task is left in limbo
}
else
{
//did not time out logic
}
break;
}
//do stuff with result
}
}
public static Task<string> Operation1(string machine)
{
return Task.Run(() =>
{
//.net method that hangs here
}
}
public static Task<string> Operation2(string machine)
{
return Task.Run(() =>
{
//.net method that hangs here
}
}
Operation1 has the potential to hang forever. Operation2 will time out after 10 seconds...and the code will continue, but never cleans up the abandoned task. I'm hoping for a third option.
Related
I've been working on a project and saw the below code. I am new to the async/await world. As far as I know, only a single task is performing in the method then why it is decorated with async/await. What benefits I am getting by using async/await and what is the drawback if I remove async/await i.e make it synchronous I am a little bit confused so any help will be appreciated.
[Route("UpdatePersonalInformation")]
public async Task<DataTransferObject<bool>> UpdatePersonalInformation([FromBody] UserPersonalInformationRequestModel model)
{
DataTransferObject<bool> transfer = new DataTransferObject<bool>();
try
{
model.UserId = UserIdentity;
transfer = await _userService.UpdateUserPersonalInformation(model);
}
catch (Exception ex)
{
transfer.TransactionStatusCode = 500;
transfer.ErrorMessage = ex.Message;
}
return transfer;
}
Service code
public async Task<DataTransferObject<bool>> UpdateUserPersonalInformation(UserPersonalInformationRequestModel model)
{
DataTransferObject<bool> transfer = new DataTransferObject<bool>();
await Task.Run(() =>
{
try
{
var data = _userProfileRepository.FindBy(x => x.AspNetUserId == model.UserId)?.FirstOrDefault();
if (data != null)
{
var userProfile = mapper.Map<UserProfile>(model);
userProfile.UpdatedBy = model.UserId;
userProfile.UpdateOn = DateTime.UtcNow;
userProfile.CreatedBy = data.CreatedBy;
userProfile.CreatedOn = data.CreatedOn;
userProfile.Id = data.Id;
userProfile.TypeId = data.TypeId;
userProfile.AspNetUserId = data.AspNetUserId;
userProfile.ProfileStatus = data.ProfileStatus;
userProfile.MemberSince = DateTime.UtcNow;
if(userProfile.DOB==DateTime.MinValue)
{
userProfile.DOB = null;
}
_userProfileRepository.Update(userProfile);
transfer.Value = true;
}
else
{
transfer.Value = false;
transfer.Message = "Invalid User";
}
}
catch (Exception ex)
{
transfer.ErrorMessage = ex.Message;
}
});
return transfer;
}
What benefits I am getting by using async/await
Normally, on ASP.NET, the benefit of async is that your server is more scalable - i.e., can handle more requests than it otherwise could. The "Synchronous vs. Asynchronous Request Handling" section of this article goes into more detail, but the short explanation is that async/await frees up a thread so that it can handle other requests while the asynchronous work is being done.
However, in this specific case, that's not actually what's going on. Using async/await in ASP.NET is good and proper, but using Task.Run on ASP.NET is not. Because what happens with Task.Run is that another thread is used to run the delegate within UpdateUserPersonalInformation. So this isn't asynchronous; it's just synchronous code running on a background thread. UpdateUserPersonalInformation will take another thread pool thread to run its synchronous repository call and then yield the request thread by using await. So it's just doing a thread switch for no benefit at all.
A proper implementation would make the repository asynchronous first, and then UpdateUserPersonalInformation can be implemented without Task.Run at all:
public async Task<DataTransferObject<bool>> UpdateUserPersonalInformation(UserPersonalInformationRequestModel model)
{
DataTransferObject<bool> transfer = new DataTransferObject<bool>();
try
{
var data = _userProfileRepository.FindBy(x => x.AspNetUserId == model.UserId)?.FirstOrDefault();
if (data != null)
{
...
await _userProfileRepository.UpdateAsync(userProfile);
transfer.Value = true;
}
else
{
transfer.Value = false;
transfer.Message = "Invalid User";
}
}
catch (Exception ex)
{
transfer.ErrorMessage = ex.Message;
}
return transfer;
}
The await keyword only indicates that the execution of the current function is halted until the Task which is being awaited is completed. This means if you remove the async, the method will continue execution and therefore immediately return the transfer object, even if the UpdateUserPersonalInformation Task is not finished.
Take a look at this example:
private void showInfo()
{
Task.Delay(1000);
MessageBox.Show("Info");
}
private async void showInfoAsync()
{
await Task.Delay(1000);
MessageBox.Show("Info");
}
In the first method, the MessageBox is immediately displayed, since the newly created Task (which only waits a specified amount of time) is not awaited. However, the second method specifies the await keyword, therefore the MessageBox is displayed only after the Task is finished (in the example, after 1000ms elapsed).
But, in both cases the delay Task is ran asynchronously in the background, so the main thread (for example the UI) will not freeze.
The usage of async-await mechanism mainly used
when you have some long calculation process which takes some time and you want it to be on the background
in UI when you don't want to make the main thread stuck which will be reflected on UI performance.
you can read more here:
https://learn.microsoft.com/en-us/dotnet/csharp/async
Time Outs
The main usages of async and await operates preventing TimeOuts by waiting for long operations to complete. However, there is another less known, but very powerful one.
If you don't await long operation, you will get a result back, such as a null, even though the actual request as not completed yet.
Cancellation Tokens
Async requests have a default parameter you can add:
public async Task<DataTransferObject<bool>> UpdatePersonalInformation(
[FromBody] UserPersonalInformationRequestModel model,
CancellationToken cancellationToken){..}
A CancellationToken allows the request to stop when the user changes pages or interrupts the connection. A good example of this is a user has a search box, and every time a letter is typed you filter and search results from your API. Now imagine the user types a very long string with say 15 characters. That means that 15 requests are sent and 15 requests need to be completed. Even if the front end is not awaiting the first 14 results, the API is still doing all the 15 requests.
A cancellation token simply tells the API to drop the unused threads.
I would like to chime in on this because most answers although good, do not point to a definite time when to use and when not.
From my experience, if you are developing anything with a front-end, add async/await to your methods when expecting output from other threads to be input to your UI. This is the best strategy for handling multithread output and Microsoft should be commended to come out with this when they did. Without async/await you would have to add more code to handle thread output to UI (e.g Event, Event Handler, Delegate, Event Subscription, Marshaller).
Don't need it anywhere else except if using strategically for slow peripherals.
In my current project, I have a piece of code that, after simplifying it down to where I'm having issues, looks something like this:
private async Task RunAsync(CancellationToken cancel)
{
bool finished = false;
while (!cancel.IsCancellationRequested && !finished)
finished = await FakeTask();
}
private Task<bool> FakeTask()
{
return Task.FromResult(false);
}
If I use this code without awaiting, I end up blocking anyway:
// example 1
var task = RunAsync(cancel); // Code blocks here...
... // Other code that could run while RunAsync is doing its thing, but is forced to wait
await task;
// example 2
var task = RunAsync(cancelSource.Token); // Code blocks here...
cancelSource.Cancel(); // Never called
In the actual project, I'm not actually using FakeTask, and there usually will be some Task.Delay I'm awaiting in there, so the code most of the time doesn't actually block, or only for a limited amount of iterations.
In unit testing, however, I'm using a mock object that does pretty much do what FakeTask does, so when I want to see if RunAsync responds to its CancellationToken getting cancelled the way I expect it to, I'm stuck.
I have found I can fix this issue by adding for example await Task.Delay(1) at the top of RunAsync, to force it to truly run asynchronous, but this feels a bit hacky. Are there better alternatives?
You have an incorrect mental picture of what await does. The meaning of await is:
Check to see if the awaitable object is complete. If it is, fetch its result and continue executing the coroutine.
If it is not complete, sign up the remainder of the current method as the continuation of the awaitable and suspend the coroutine by returning control to the caller. (Note that this makes it a semicoroutine.)
In your program, the "fake" awaitable is always complete, so there is never a suspension of the coroutine.
Are there better alternatives?
If your control flow logic requires you to suspend the coroutine then use Task.Yield.
Task.FromResult actually runs synchronously, as would await Task.Delay(0). If you want to actually simulate asynchronous code, call Task.Yield(). That creates an awaitable task that asynchronously yields back to the current context when awaited.
As #SLaks said, your code will run synchronously. One thing is running async code, and another thing is running parallel code.
If you need to run your code in parallel you can use Task.Run.
class Program
{
static async Task Main(string[] args)
{
var tcs = new CancellationTokenSource();
var task = Task.Run(() => RunAsync("1", tcs.Token));
var task2 = Task.Run(() => RunAsync("2", tcs.Token));
await Task.Delay(1000);
tcs.Cancel();
Console.ReadLine();
}
private static async Task RunAsync(string source, CancellationToken cancel)
{
bool finished = false;
while (!cancel.IsCancellationRequested && !finished)
finished = await FakeTask(source);
}
private static Task<bool> FakeTask(string source)
{
Console.WriteLine(source);
return Task.FromResult(false);
}
}
C#'s async methods execute synchronously up to the point where they have to wait for a result.
In your example there is no such point where the method has to wait for a result, so the loop keeps running forever and thereby blocking the caller.
Inserting an await Task.Yield() to simulate some real async work should help.
I'm switching from Task.Run to Hangfire. In .NET 4.5+ Task.Run can return Task<TResult> which allows me to run tasks that return other than void. I can normally wait and get the result of my task by accessing the property MyReturnedTask.Result
Example of my old code:
public void MyMainCode()
{
List<string> listStr = new List<string>();
listStr.Add("Bob");
listStr.Add("Kate");
listStr.Add("Yaz");
List<Task<string>> listTasks = new List<Task<string>>();
foreach(string str in listStr)
{
Task<string> returnedTask = Task.Run(() => GetMyString(str));
listTasks.Add(returnedTask);
}
foreach(Task<string> task in listTasks)
{
// using task.Result will cause the code to wait for the task if not yet finished.
// Alternatively, you can use Task.WaitAll(listTasks.ToArray()) to wait for all tasks in the list to finish.
MyTextBox.Text += task.Result + Environment.NewLine;
}
}
private string GetMyString(string str)
{
// long execution in order to calculate the returned string
return str + "_finished";
}
As far as I can see from the Quick Start page of Hangfire, your main guy which is BackgroundJob.Enqueue(() => Console.WriteLine("Fire-and-forget"));
perfectly runs the code as a background job but apparently doesn't support jobs that have a return value (like the code I presented above). Is that right? if not, how can I tweak my code in order to use Hangfire?
P.S. I already looked at HostingEnvironment.QueueBackgroundWorkItem (here) but it apparently lacks the same functionality (background jobs have to be void)
EDIT
As #Dejan figured out, the main reason I want to switch to Hangfire is the same reason the .NET folks added QueueBackgroundWorkItem in .NET 4.5.2. And that reason is well described in Scott Hanselman's great article about Background Tasks in ASP.NET. So I'm gonna quote from the article:
QBWI (QueueBackgroundWorkItem) schedules a task which can run in the background, independent of
any request. This differs from a normal ThreadPool work item in that
ASP.NET automatically keeps track of how many work items registered
through this API are currently running, and the ASP.NET runtime will
try to delay AppDomain shutdown until these work items have finished
executing.
One simple solution would be to poll the monitoring API until the job is finished like this:
public static Task Enqueue(Expression<Action> methodCall)
{
string jobId = BackgroundJob.Enqueue(methodCall);
Task checkJobState = Task.Factory.StartNew(() =>
{
while (true)
{
IMonitoringApi monitoringApi = JobStorage.Current.GetMonitoringApi();
JobDetailsDto jobDetails = monitoringApi.JobDetails(jobId);
string currentState = jobDetails.History[0].StateName;
if (currentState != "Enqueued" && currentState != "Processing")
{
break;
}
Thread.Sleep(100); // adjust to a coarse enough value for your scenario
}
});
return checkJobState;
}
Attention: Of course, in a Web-hosted scenario you cannot rely on continuation of the task (task.ContinueWith()) to do more things after the job has finished as the AppDomain might be shut down - for the same reasons you probably want to use Hangfire in the first place.
I'm writing a Windows Service that will kick off multiple worker threads that will listen to Amazon SQS queues and process messages. There will be about 20 threads listening to 10 queues.
The threads will have to be always running and that's why I'm leaning towards to actually using actual threads for the worker loops rather than threadpool threads.
Here is a top level implementation. Windows service will kick off multiple worker threads and each will listen to it's queue and process messages.
protected override void OnStart(string[] args)
{
for (int i = 0; i < _workers; i++)
{
new Thread(RunWorker).Start();
}
}
Here is the implementation of the work
public async void RunWorker()
{
while(true)
{
// .. get message from amazon sqs sync.. about 20ms
var message = sqsClient.ReceiveMessage();
try
{
await PerformWebRequestAsync(message);
await InsertIntoDbAsync(message);
}
catch(SomeExeception)
{
// ... log
//continue to retry
continue;
}
sqsClient.DeleteMessage();
}
}
I know I can perform the same operation with Task.Run and execute it on the threadpool thread rather than starting individual thread, but I don't see a reason for that since each thread will always be running.
Do you see any problems with this implementation? How reliable would it be to leave threads always running in this fashion and what can I do to make sure that each thread is always running?
One problem with your existing solution is that you call your RunWorker in a fire-and-forget manner, albeit on a new thread (i.e., new Thread(RunWorker).Start()).
RunWorker is an async method, it will return to the caller when the execution point hits the first await (i.e. await PerformWebRequestAsync(message)). If PerformWebRequestAsync returns a pending task, RunWorker returns and the new thread you just started terminates.
I don't think you need a new thread here at all, just use AmazonSQSClient.ReceiveMessageAsync and await its result. Another thing is that you shouldn't be using async void methods unless you really don't care about tracking the state of the asynchronous task. Use async Task instead.
Your code might look like this:
List<Task> _workers = new List<Task>();
CancellationTokenSource _cts = new CancellationTokenSource();
protected override void OnStart(string[] args)
{
for (int i = 0; i < _MAX_WORKERS; i++)
{
_workers.Add(RunWorkerAsync(_cts.Token));
}
}
public async Task RunWorkerAsync(CancellationToken token)
{
while(true)
{
token.ThrowIfCancellationRequested();
// .. get message from amazon sqs sync.. about 20ms
var message = await sqsClient.ReceiveMessageAsync().ConfigureAwait(false);
try
{
await PerformWebRequestAsync(message);
await InsertIntoDbAsync(message);
}
catch(SomeExeception)
{
// ... log
//continue to retry
continue;
}
sqsClient.DeleteMessage();
}
}
Now, to stop all pending workers, you could simple do this (from the main "request dispatcher" thread):
_cts.Cancel();
try
{
Task.WaitAll(_workers.ToArray());
}
catch (AggregateException ex)
{
ex.Handle(inner => inner is OperationCanceledException);
}
Note, ConfigureAwait(false) is optional for Windows Service, because there's no synchronization context on the initial thread, by default. However, I'd keep it that way to make the code independent of the execution environment (for cases where there is synchronization context).
Finally, if for some reason you cannot use ReceiveMessageAsync, or you need to call another blocking API, or simply do a piece of CPU intensive work at the beginning of RunWorkerAsync, just wrap it with Task.Run (as opposed to wrapping the whole RunWorkerAsync):
var message = await Task.Run(
() => sqsClient.ReceiveMessage()).ConfigureAwait(false);
Well, for one I'd use a CancellationTokenSource instantiated in the service and passed down to the workers. Your while statement would become:
while(!cancellationTokenSource.IsCancellationRequested)
{
//rest of the code
}
This way you can cancel all your workers from the OnStop service method.
Additionally, you should watch for:
If you're playing with thread states from outside of the thread, then a ThreadStateException, or ThreadInterruptedException or one of the others might be thrown. So, you want to handle a proper thread restart.
Do the workers need to run without pause in-between iterations? I would throw in a sleep in there (even a few ms's) just so they don't keep the CPU up for nothing.
You need to handle ThreadStartException and restart the worker, if it occurs.
Other than that there's no reason why those 10 treads can't run for as long as the service runs (days, weeks, months at a time).
I'm working on a library which does some lazy idle background work on a UI thread (it has to be that way because of the legacy COM use). The task can be cancelled by the consuming app via cancellation token, or it can be explicitly cancelled as a result of user action (via IUserFeedback.Continue). I'm trying to follow the MSDN pattern for task cancellation.
My question is, should I make the difference between cancellation by user (and return false) and by the calling app (throw), as IdleWorker1 does. Or should I treat both cases equally and just throw, as with IdleWorker2?
I don't have any strict requirements from the designers of the interface (the task is essentially never ending, so they only care about how much work has actually been done so far, and they're receiving the progress via IUserFeedback.Continue).
In a nutshell, IdleWorker1:
interface IUserFeedback
{
bool Continue(int n);
}
class IdleWorker1
{
public async Task<bool> DoIdleWorkAsync(CancellationToken ct, int timeSlice, IUserFeedback feedback)
{
bool more = true;
int n = 0;
while (more)
{
ct.ThrowIfCancellationRequested();
more = feedback.Continue(++n);
await Task.Delay(timeSlice);
}
return more;
}
}
IdleWorker2:
class IdleWorker2
{
public async Task DoIdleWorkAsync(CancellationToken ct, int timeSlice, IUserFeedback feedback)
{
int n = 0;
for (;;)
{
ct.ThrowIfCancellationRequested();
if (!feedback.Continue(++n))
throw new TaskCanceledException();
await Task.Delay(timeSlice);
}
}
}
I think that if you don't need to differentiate between the two types of cancellation, the canceled method shouldn't even know who canceled it.
To do that, you would have one CancellationTokenSource for cancellation by the app and another one for cancellation by user. You would then combine them into one using CreateLinkedSource() and pass the token of that to the method.