Develop Reference

Managing task cancellation and completion on a concurrent thread

[ This question needs to be reimagined. One of my thread queues MUST run on an STA thread, and the code below does not accommodate that. In particular it seems Task<> chooses its own thread and that just is not going to work for me. ]
I have a task queue (BlockingCollection) that I'm running through on a dedicated thread. That queue receives a series of Task<> objects that it runs sequentially within that thread via a while loop.
I need a means of Cancelling that series of tasks, and a means of knowing that the tasks are all complete. I have not been able to figure out how to do this.
Here's a fragment of my queuing class. ProcessQueue is run on a separate thread from main. QueueJob calls occur on the main thread.
using Job = Tuple<Task<bool>, string>;
public class JobProcessor
{
private readonly BlockingCollection<Job> m_queue = new BlockingCollection<Job>();
volatile bool cancel_queue = false;
private bool ProcessQueue()
{
while (true)
{
if (m_queue.IsAddingCompleted)
break;
Job tuple;
if (!m_queue.TryTake(out tuple, Timeout.Infinite))
break;
var task = tuple.Item1;
var taskName = tuple.Item2;
try
{
Console.WriteLine("Task {0}::{1} starting", this.name, taskName);
task.RunSynchronously();
Console.WriteLine("Task {0}::{1} completed", this.name, taskName);
}
catch (Exception e)
{
string message = e.Message;
}
if (cancel_queue) // CANCEL BY ERASING TASKS AND NOT RUNNING.
{
while (m_queue.TryTake(out tuple))
{
}
}
} // while(true)
return true;
}
public Task<bool> QueueJob(Func<bool> input)
{
var task = new Task<bool>(input);
try
{
m_queue.Add(Tuple.Create(task, input.Method.Name));
}
catch (InvalidOperationException)
{
Task<bool> dummy = new Task<bool>(() => false);
dummy.Start();
return dummy;
}
return task;
}
Here are the functions that trouble me:
public void ClearQueue()
{
cancel_queue = true;
// wait for queue to become empty. HOW?
cancel_queue = false;
}
public void WaitForCompletion()
{
// wait for all tasks to be completed.
// not sufficient to wait for empty queue because the last task
// must also execute and finish. HOW?
}
}
Here is some usage:
class SomeClass
{
void Test()
{
JobProcessor jp = new JobProcessor();
// launch Processor loop on separate thread... code not shown.
// send a bunch of jobs via QueueJob... code not show.
// launch dialog... code not shown.
if (dialog_result == Result.Cancel)
jp.ClearQueue();
if (dialog_result == Result.Proceed)
jp.WaitForCompletion();
}
}
The idea is after the work is completed or cancelled, new work may be posted. In general though, new work may come in asynchronously. WaitForCompletion might in fact be "when all work is done, inform the user and then do other stuff", so it doesn't strictly have to be a synchronous function call like above, but I can't figure how to make these happen.
(One further complication, I expect to have several queues that interact. While I am careful to keep things parallelized in a way to prevent deadlocks, I am not confident what happens when cancellation is introduced into the mix, but this is probably beyond scope for this question.)

WaitForCompletion() sounds easy enough. Create a semaphore or event, create a task whose only action is to signal the semaphore, queue up the task, wait on the semaphore.
When the thread finishes the last 'real' task, the semaphore task will be run and so the thread that called WaitForCompletion will become ready/running:)
Would not a similar approach work for cancellation? Have a very high priority thread that you create/signal that drains the queue of all pending jobs, disposing them, queueing up the semaphore task and waiting for the 'last task done' signal?

thread safe processing of a queue

I am putting tasks from the UI-thread into a queue, so that they can be processed in another thread. if there is nothing to do, the thread should wait with an AutoResetEvent - obviously all this should be threadsafe.
i am putting tasks in the queue from the UI-thread like this:
lock (_syncObject)
{
_queue.Enqueue(new FakeTask());
}
_autoResetEvent.Set();
here is how my processing thread-loop looks so far:
while (true)
{
FakeTask task = null;
lock (_syncObject)
{
if (_queue.Count > 0)
{
task = _queue.Dequeue();
}
}
if (task != null)
{
task.Run();
Thread.Sleep(1000); //just here for testing purposes
}
if (_queue.Count == 0)
{
_autoResetEvent.WaitOne();
}
}
i am afraid that the last part where i check if something else is in the queue is not thread safe and would like to know how i can make it so. thanks!

In simple case, try using BlockingCollection which has been specially designed for implementing Producer / Consumer pattern:
private async Task Process() {
using (BlockingCollection<FakeTask> _queue = new BlockingCollection<FakeTask>()) {
Task producer = Task.Run(() => {
while (!completed) {
//TODO: put relevant code here
_queue.Add(new FakeTask());
}
_queue.CompleteAdding();
});
Task consumer = Task.Run(() => {
foreach (FakeTask task in _queue.GetConsumingEnumerable()) {
//TODO: process task - put relevant code here
}
});
await Task.WhenAll(producer, consumer).ConfigureAwait(false);
}
}
Edit: if producer is UI thread itself:
private async Task Process() {
using (BlockingCollection<FakeTask> _queue = new BlockingCollection<FakeTask>()) {
Task consumer = Task.Run(() => {
foreach (FakeTask task in _queue.GetConsumingEnumerable()) {
//TODO: process task - put relevant code here
}
});
// If we produce in UI we don't want any separate Task
while (!completed) {
//TODO: put relevant code here
_queue.Add(new FakeTask());
}
_queue.CompleteAdding();
await consumer.ConfigureAwait(false);
}
}
In case of entangled mesh (e.g. producers #1, #2 genetate tasks for consumers #1, #2, #3 which in turn create tasks for...) try DataFlow

It's not useful to create a thread just so that it can spend basically all of its time sitting around doing nothing waiting for work to do.
All you need to do is use an asynchronous locking mechanism around the UI task's scheduling of background work to be done. SemaphoreSlim provides such an asynchronous synchronization mechanism.
await sempahoreSlim.WaitAsync();
try
{
await Task.Run(() => DoWork());
}
finally
{
sempahoreSlim.Release();
}
Not only is it a lot less code, and has much simpler code that more accurately reflects what the business logic of the application is, but you're consuming quite a lot less system resources.
And of course if different background operations can safely run in parallel, then just use the thread pool, rather than your own message loop, and the code becomes even simpler.

Using Task.WhenAll with a growing list of Tasks

Task.WhenAll(IEnumerable<Task>) waits for all tasks in the IEnumerable are complete --- but only the tasks in the list when it's first called. If any active task adds to the list, they aren't considered. This short example demonstrates:
List<Task> _tasks = new List<Task>();
public async Task QuickExample()
{
for(int n =0; n < 6; ++n)
_tasks.Add(Func1(n));
await Task.WhenAll(_tasks);
Console.WriteLine("Some Tasks complete");
await Task.WhenAll(_tasks);
Console.WriteLine("All Tasks complete");
}
async Task Func1(int n)
{
Console.WriteLine($"Func1-{n} started");
await Task.Delay(2000);
if ((n % 3) == 1)
_tasks.Add(Func2(n));
Console.WriteLine($"Func1-{n} complete");
}
async Task Func2(int n)
{
Console.WriteLine($"Func2-{n} started");
await Task.Delay(2000);
Console.WriteLine($"Func2-{n} complete");
}
This outputs:
Func1-0 started
Func1-1 started
Func1-2 started
Func1-3 started
Func1-4 started
Func1-5 started
Func1-5 complete
Func1-3 complete
Func2-1 started
Func1-1 complete
Func1-0 complete
Func1-2 complete
Func2-4 started
Func1-4 complete
Some Tasks complete
Func2-4 complete
Func2-1 complete
All Tasks complete
Done
The second Task.WhenAll() solves the problem in this case, but that's a rather fragile solution. What's the best way to handle this in the general case?

You are modifying the List<> without locking it... You like to live a dangerous life :-) Save the Count of the _tasks before doing a WaitAll, then after the WaitAll check the Count of _tasks. If it is different, do another round (so you need a while around the WaitAll.
int count = _tasks.Count;
while (true)
{
await Task.WhenAll(_tasks);
lock (_tasks)
{
if (count == _tasks.Count)
{
Console.WriteLine("All Tasks complete");
break;
}
count = _tasks.Count;
Console.WriteLine("Some Tasks complete");
}
}
async Task Func1(int n)
{
Console.WriteLine($"Func1-{n} started");
await Task.Delay(2000);
if ((n % 3) == 1)
{
lock (_tasks)
{
_tasks.Add(Func2(n));
}
}
Console.WriteLine($"Func1-{n} complete");
}
I'll add a second (probably more correct solution), that is different from what you are doing: you could simply await the new Tasks from the Tasks that generated them, without cascading them to the _tasks collection. If A creates B, then A doesn't finish until B finishes. Clearly you don't need to add the new Tasks to the _tasks collection.

Asynchronous function will return to the caller on first await.
So for loop will be complete before you add extra tasks to original tasks list.
Implementation of Task.WhenAll will iterate/copy tasks to local list, so added tasks after Task.WhenAll called will be ignored.
In your particular case moving call to Func1 before await Task.Delay() could be a solution.
async Task Func1(int n)
{
Console.WriteLine($"Func1-{n} started");
if ((n % 3) == 1)
_tasks.Add(Func2(n));
await Task.Delay(2000);
Console.WriteLine($"Func1-{n} complete");
}
But if in real scenario calling of Func2 depend on result of some asynchronous method, then you need some other solution.

Since it seems that additional tasks can be created during the course of executing the original list of tasks, you will need a simple while construct.
while (_tasks.Any( t => !t.IsCompleted )
{
await Task.WhenAll(_tasks);
}
This will check the list for any uncompleted tasks and await them until it catches the list at a moment when there are no tasks left.

Consider this; it sounds like work is being submitted to the "Task List" from another thread. In a sense, the "task submission" thread itself could also be yet another Task for you to wait on.
If you wait for all Tasks to be submitted, then you are guaranteed that your next call to WhenAll will yield a fully-completed payload.
Your waiting function could/should be a two-step process:
Wait for the "Task Submitting" task to complete, signalling all Tasks are submitted
Wait for all the submitted tasks to complete.
Example:
public async Task WaitForAllSubmittedTasks()
{
// Work is being submitted in a background thread;
// Wrap that thread in a Task, and wait for it to complete.
var workScheduler = GetWorkScheduler();
await workScheduler;
// All tasks submitted!
// Now we get the completed list of all submitted tasks.
// It's important to note: the submitted tasks
// have been chugging along all this time.
// By the time we get here, there are probably a number of
// completed tasks already. It does not delay the speed
// or execution of our work items if we grab the List
// after some of the work has been completed.
//
// It's entirely possible that - by the time we call
// this function and wait on it - almost all the
// tasks have already been completed!
var submittedWork = GetAllSubmittedTasks();
await Task.WhenAll(submittedWork);
// Work complete!
}

How to manage a list of tasks using the Task Parallel Library

I'm trying to manage a list of tasks for a Windows Service so when I shut down the Service, I can use the Task.WaitAll() method to stop the Service from shutting down until all of the remaining tasks complete. For example, I have a Run() method that executes until a boolean is updated:
public void Run()
{
while (runFlag)
{
if (MaxTasksAchieved)
{
System.Threading.Thread.Sleep(pollingInterval);
}
else
{
taskList.Add(Task.Factory.StartNew(() =>
{
// do stuff
}));
}
}
}
Then in my Stop() method, I have the following:
public void Stop()
{
runFlag = false;
if (taskList.Count > 0)
{
// wait
Task.WaitAll(taskList.ToArray());
}
else
{
// no wait, great.
}
}
My question is how do I elegantly have the task remove itself from the list after it's done executing? I want the task to remove itself from the list so when Stop() is called, taskList only contains the tasks that are currently in progress.

If you want to remove the task from the list simply add a continuation that does that:
taskList.Add(task.ContinueWith(t = > taskList.Remove(t)));
But you can simply use Task.WaitAll on all the tasks since the completed tasks are already completed.
Task.WaitAll(taskList.ToArray());
You can easily wait only on the running tasks by filtering the list with the tasks status
Task.WaitAll(taskList.Where(t => t.Status != TaskStatus.RanToCompletion).ToArray());

Why not just use
public void Stop()
{
Task.WaitAll(taskList.ToArray());
}
If task list is empty the service will shutdown immediately.

Since WaitAll() is waiting until ALL tasks are complete, you can just reinitialize the taskList....
public void Stop()
{
runFlag = false;
if (taskList.Count > 0)
{
// wait
Task.WaitAll(taskList.ToArray());
taskList = new List<Task>();
}
else
{
// no wait, great.
}
}

As others have stated, it's probably unnecessary to remove the task from the list upon completion. But, if you must, you can use a continuation.
Task myTask = new Task(() => {
// do stuff
});
myTask.ContinueWith(
t => { taskList.Remove(t); }
);

Cancel Long Running task [duplicate]

In a thread, I create some System.Threading.Task and start each task.
When I do a .Abort() to kill the thread, the tasks are not aborted.
How can I transmit the .Abort() to my tasks ?

You can't. Tasks use background threads from the thread pool. Also canceling threads using the Abort method is not recommended. You may take a look at the following blog post which explains a proper way of canceling tasks using cancellation tokens. Here's an example:
class Program
{
static void Main()
{
var ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
Task.Factory.StartNew(() =>
{
while (true)
{
// do some heavy work here
Thread.Sleep(100);
if (ct.IsCancellationRequested)
{
// another thread decided to cancel
Console.WriteLine("task canceled");
break;
}
}
}, ct);
// Simulate waiting 3s for the task to complete
Thread.Sleep(3000);
// Can't wait anymore => cancel this task
ts.Cancel();
Console.ReadLine();
}
}

Like this post suggests, this can be done in the following way:
int Foo(CancellationToken token)
{
Thread t = Thread.CurrentThread;
using (token.Register(t.Abort))
{
// compute-bound work here
}
}
Although it works, it's not recommended to use such approach. If you can control the code that executes in task, you'd better go with proper handling of cancellation.

Aborting a Task is easily possible if you capture the thread in which the task is running in. Here is an example code to demonstrate this:
void Main()
{
Thread thread = null;
Task t = Task.Run(() =>
{
//Capture the thread
thread = Thread.CurrentThread;
//Simulate work (usually from 3rd party code)
Thread.Sleep(1000);
//If you comment out thread.Abort(), then this will be displayed
Console.WriteLine("Task finished!");
});
//This is needed in the example to avoid thread being still NULL
Thread.Sleep(10);
//Cancel the task by aborting the thread
thread.Abort();
}
I used Task.Run() to show the most common use-case for this - using the comfort of Tasks with old single-threaded code, which does not use the CancellationTokenSource class to determine if it should be canceled or not.

This sort of thing is one of the logistical reasons why Abort is deprecated. First and foremost, do not use Thread.Abort() to cancel or stop a thread if at all possible. Abort() should only be used to forcefully kill a thread that is not responding to more peaceful requests to stop in a timely fashion.
That being said, you need to provide a shared cancellation indicator that one thread sets and waits while the other thread periodically checks and gracefully exits. .NET 4 includes a structure designed specifically for this purpose, the CancellationToken.

I use a mixed approach to cancel a task.
Firstly, I'm trying to Cancel it politely with using the Cancellation.
If it's still running (e.g. due to a developer's mistake), then misbehave and kill it using an old-school Abort method.
Checkout an example below:
private CancellationTokenSource taskToken;
private AutoResetEvent awaitReplyOnRequestEvent = new AutoResetEvent(false);
void Main()
{
// Start a task which is doing nothing but sleeps 1s
LaunchTaskAsync();
Thread.Sleep(100);
// Stop the task
StopTask();
}
/// <summary>
/// Launch task in a new thread
/// </summary>
void LaunchTaskAsync()
{
taskToken = new CancellationTokenSource();
Task.Factory.StartNew(() =>
{
try
{ //Capture the thread
runningTaskThread = Thread.CurrentThread;
// Run the task
if (taskToken.IsCancellationRequested || !awaitReplyOnRequestEvent.WaitOne(10000))
return;
Console.WriteLine("Task finished!");
}
catch (Exception exc)
{
// Handle exception
}
}, taskToken.Token);
}
/// <summary>
/// Stop running task
/// </summary>
void StopTask()
{
// Attempt to cancel the task politely
if (taskToken != null)
{
if (taskToken.IsCancellationRequested)
return;
else
taskToken.Cancel();
}
// Notify a waiting thread that an event has occurred
if (awaitReplyOnRequestEvent != null)
awaitReplyOnRequestEvent.Set();
// If 1 sec later the task is still running, kill it cruelly
if (runningTaskThread != null)
{
try
{
runningTaskThread.Join(TimeSpan.FromSeconds(1));
}
catch (Exception ex)
{
runningTaskThread.Abort();
}
}
}

To answer Prerak K's question about how to use CancellationTokens when not using an anonymous method in Task.Factory.StartNew(), you pass the CancellationToken as a parameter into the method you're starting with StartNew(), as shown in the MSDN example here.
e.g.
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
Task.Factory.StartNew( () => DoSomeWork(1, token), token);
static void DoSomeWork(int taskNum, CancellationToken ct)
{
// Do work here, checking and acting on ct.IsCancellationRequested where applicable,
}

You should not try to do this directly. Design your tasks to work with a CancellationToken, and cancel them this way.
In addition, I would recommend changing your main thread to function via a CancellationToken as well. Calling Thread.Abort() is a bad idea - it can lead to various problems that are very difficult to diagnose. Instead, that thread can use the same Cancellation that your tasks use - and the same CancellationTokenSource can be used to trigger the cancellation of all of your tasks and your main thread.
This will lead to a far simpler, and safer, design.

Tasks have first class support for cancellation via cancellation tokens. Create your tasks with cancellation tokens, and cancel the tasks via these explicitly.

You can use a CancellationToken to control whether the task gets cancelled. Are you talking about aborting it before it's started ("nevermind, I already did this"), or actually interrupting it in middle? If the former, the CancellationToken can be helpful; if the latter, you will probably need to implement your own "bail out" mechanism and check at appropriate points in the task execution whether you should fail fast (you can still use the CancellationToken to help you, but it's a little more manual).
MSDN has an article about cancelling Tasks:
http://msdn.microsoft.com/en-us/library/dd997396.aspx

Task are being executed on the ThreadPool (at least, if you are using the default factory), so aborting the thread cannot affect the tasks. For aborting tasks, see Task Cancellation on msdn.

I tried CancellationTokenSource but i can't do this. And i did do this with my own way. And it works.
namespace Blokick.Provider
{
public class SignalRConnectProvider
{
public SignalRConnectProvider()
{
}
public bool IsStopRequested { get; set; } = false; //1-)This is important and default `false`.
public async Task<string> ConnectTab()
{
string messageText = "";
for (int count = 1; count < 20; count++)
{
if (count == 1)
{
//Do stuff.
}
try
{
//Do stuff.
}
catch (Exception ex)
{
//Do stuff.
}
if (IsStopRequested) //3-)This is important. The control of the task stopping request. Must be true and in inside.
{
return messageText = "Task stopped."; //4-) And so return and exit the code and task.
}
if (Connected)
{
//Do stuff.
}
if (count == 19)
{
//Do stuff.
}
}
return messageText;
}
}
}
And another class of the calling the method:
namespace Blokick.Views
{
[XamlCompilation(XamlCompilationOptions.Compile)]
public partial class MessagePerson : ContentPage
{
SignalRConnectProvider signalR = new SignalRConnectProvider();
public MessagePerson()
{
InitializeComponent();
signalR.IsStopRequested = true; // 2-) And this. Make true if running the task and go inside if statement of the IsStopRequested property.
if (signalR.ChatHubProxy != null)
{
signalR.Disconnect();
}
LoadSignalRMessage();
}
}
}

You can abort a task like a thread if you can cause the task to be created on its own thread and call Abort on its Thread object. By default, a task runs on a thread pool thread or the calling thread - neither of which you typically want to abort.
To ensure the task gets its own thread, create a custom scheduler derived from TaskScheduler. In your implementation of QueueTask, create a new thread and use it to execute the task. Later, you can abort the thread, which will cause the task to complete in a faulted state with a ThreadAbortException.
Use this task scheduler:
class SingleThreadTaskScheduler : TaskScheduler
{
public Thread TaskThread { get; private set; }
protected override void QueueTask(Task task)
{
TaskThread = new Thread(() => TryExecuteTask(task));
TaskThread.Start();
}
protected override IEnumerable<Task> GetScheduledTasks() => throw new NotSupportedException(); // Unused
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued) => throw new NotSupportedException(); // Unused
}
Start your task like this:
var scheduler = new SingleThreadTaskScheduler();
var task = Task.Factory.StartNew(action, cancellationToken, TaskCreationOptions.LongRunning, scheduler);
Later, you can abort with:
scheduler.TaskThread.Abort();
Note that the caveat about aborting a thread still applies:
The Thread.Abort method should be used with caution. Particularly when you call it to abort a thread other than the current thread, you do not know what code has executed or failed to execute when the ThreadAbortException is thrown, nor can you be certain of the state of your application or any application and user state that it is responsible for preserving. For example, calling Thread.Abort may prevent static constructors from executing or prevent the release of unmanaged resources.

You can use this class..:
It works for all typs of returned Values..
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace CarNUChargeTester
{
public class TimeOutTaskRunner<T>
{
private Func<T> func;
private int sec;
private T result;
public TimeOutTaskRunner(Func<T> func, int sec)
{
this.func = func;
this.sec = sec;
}
public bool run()
{
var scheduler = new SingleThreadTaskScheduler();
Task<T> task = Task<T>.Factory.StartNew(func, (new CancellationTokenSource()).Token, TaskCreationOptions.LongRunning, scheduler);
if (!task.Wait(TimeSpan.FromSeconds(sec)))
{
scheduler.TaskThread.Abort();
return false;
}
result = task.Result;
return true;
}
public T getResult() { return result; }
}
class SingleThreadTaskScheduler : TaskScheduler
{
public Thread TaskThread { get; private set; }
protected override void QueueTask(Task task)
{
TaskThread = new Thread(() => TryExecuteTask(task));
TaskThread.Start();
}
protected override IEnumerable<Task> GetScheduledTasks() => throw new NotSupportedException();
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued) => throw new NotSupportedException();
}
}
To use it you can write:
TimeOutTaskRunner<string> tr = new TimeOutTaskRunner<string>(f, 10); // 10 sec to run f
if (!tr.run())
errorMsg("TimeOut"); !! My func
tr.getResult() // get the results if it done without timeout..