Develop Reference

Managing task cancellation and completion on a concurrent thread

[ This question needs to be reimagined. One of my thread queues MUST run on an STA thread, and the code below does not accommodate that. In particular it seems Task<> chooses its own thread and that just is not going to work for me. ]
I have a task queue (BlockingCollection) that I'm running through on a dedicated thread. That queue receives a series of Task<> objects that it runs sequentially within that thread via a while loop.
I need a means of Cancelling that series of tasks, and a means of knowing that the tasks are all complete. I have not been able to figure out how to do this.
Here's a fragment of my queuing class. ProcessQueue is run on a separate thread from main. QueueJob calls occur on the main thread.
using Job = Tuple<Task<bool>, string>;
public class JobProcessor
{
private readonly BlockingCollection<Job> m_queue = new BlockingCollection<Job>();
volatile bool cancel_queue = false;
private bool ProcessQueue()
{
while (true)
{
if (m_queue.IsAddingCompleted)
break;
Job tuple;
if (!m_queue.TryTake(out tuple, Timeout.Infinite))
break;
var task = tuple.Item1;
var taskName = tuple.Item2;
try
{
Console.WriteLine("Task {0}::{1} starting", this.name, taskName);
task.RunSynchronously();
Console.WriteLine("Task {0}::{1} completed", this.name, taskName);
}
catch (Exception e)
{
string message = e.Message;
}
if (cancel_queue) // CANCEL BY ERASING TASKS AND NOT RUNNING.
{
while (m_queue.TryTake(out tuple))
{
}
}
} // while(true)
return true;
}
public Task<bool> QueueJob(Func<bool> input)
{
var task = new Task<bool>(input);
try
{
m_queue.Add(Tuple.Create(task, input.Method.Name));
}
catch (InvalidOperationException)
{
Task<bool> dummy = new Task<bool>(() => false);
dummy.Start();
return dummy;
}
return task;
}
Here are the functions that trouble me:
public void ClearQueue()
{
cancel_queue = true;
// wait for queue to become empty. HOW?
cancel_queue = false;
}
public void WaitForCompletion()
{
// wait for all tasks to be completed.
// not sufficient to wait for empty queue because the last task
// must also execute and finish. HOW?
}
}
Here is some usage:
class SomeClass
{
void Test()
{
JobProcessor jp = new JobProcessor();
// launch Processor loop on separate thread... code not shown.
// send a bunch of jobs via QueueJob... code not show.
// launch dialog... code not shown.
if (dialog_result == Result.Cancel)
jp.ClearQueue();
if (dialog_result == Result.Proceed)
jp.WaitForCompletion();
}
}
The idea is after the work is completed or cancelled, new work may be posted. In general though, new work may come in asynchronously. WaitForCompletion might in fact be "when all work is done, inform the user and then do other stuff", so it doesn't strictly have to be a synchronous function call like above, but I can't figure how to make these happen.
(One further complication, I expect to have several queues that interact. While I am careful to keep things parallelized in a way to prevent deadlocks, I am not confident what happens when cancellation is introduced into the mix, but this is probably beyond scope for this question.)

WaitForCompletion() sounds easy enough. Create a semaphore or event, create a task whose only action is to signal the semaphore, queue up the task, wait on the semaphore.
When the thread finishes the last 'real' task, the semaphore task will be run and so the thread that called WaitForCompletion will become ready/running:)
Would not a similar approach work for cancellation? Have a very high priority thread that you create/signal that drains the queue of all pending jobs, disposing them, queueing up the semaphore task and waiting for the 'last task done' signal?

How to wait until method complete in C#?

I have this C# code, it works but it won't wait until the method completed
foreach (var listBoxItem in visualListBox1.Items)
{
lblCursor.Text = "Processing.. " + listBoxItem;
Thread t = new Thread(() => extract_group(listBoxItem.ToString()));
t.IsBackground = false;
t.Name = "Group Scrapper";
t.Start();
}
How to wait until extract_group method is done before moving to the next listBoxItem?
I used t.join() but it made the UI unresponsive.

Using async/await helps you to not block main thread.
public async Task ExtractGroupAsync()
{
... (logic of the method)
... (you should use async methods here as well with await before executing those methods)
}
You execute this "ExtractGroup" task like:
var example = await ExtractGroupAsync();

It makes GUI unresponsive, because you are on GUI thread. Run whole code, in separate thread.
Note: when you want to access GUI elements from another thread, you should use invoke, for example:
t.Invoke(() => t.Name = "Group Scrapper");

If you want to stick with Thread I recommend using a WaitHandle e.g. AsyncManualResetEvent Class. This approach allows to make a thread wait without blocking CPU (e.g. spinlock).
Your provided example would become:
private static AsyncManualResetEvent mre = new AsyncManualResetEvent(false, true);
public async Task DoSomethingAsync(...)
{
foreach (var listBoxItem in visualListBox1.Items)
{
lblCursor.Text = "Processing.. " + listBoxItem;
Thread t = new Thread(() => ExtractGroup(listBoxItem.ToString()));
t.IsBackground = false;
t.Name = "Group Scrapper";
t.Start();
// Wait for signal to proceed without blocking resources
await mre.WaitAsync();
}
}
private void ExtractGroup(string groupName)
{
// Do something ...
// Signal handle to release all waiting threads (makes them continue).
// Subsequent calls to Set() or WaitOne() won't show effects until Rest() was called
mre.Set();
// Reset handle to make future call of WaitOne() wait again.
mre.Reset();
}
Another solution would be to go with the TPL and use Task instead of Thread:
public async Task DoWorkAsync()
{
foreach (var listBoxItem in visualListBox1.Items)
{
lblCursor.Text = "Processing.. " + listBoxItem;
// Wait for signal to proceed without blocking resources
await Task.Run(() => ExtractGroup(listBoxItem.ToString()));
}
}
The issue with your code sample is, that you are currently on the main thread, the UI thread. Calling Thread.Join() does what you think it does: it blocks the waiting thread until the running thread completes. But as mentioned, the waiting thread is the UI thread, so the UI becomes unresponsive and can even deadlock in some scenario. When you use async/await your invocations become asynchronous and hence awaitable without blocking the UI thread.

Run X number of Task<T> at any given time while keeping UI responsive

I have a C# WinForms (.NET 4.5.2) app utilizing the TPL. The tool has a synchronous function which is passed over to a task factory X amount of times (with different input parameters), where X is a number declared by the user before commencing the process. The tasks are started and stored in a List<Task>.
Assuming the user entered 5, we have this in an async button click handler:
for (int i = 0; i < X; i++)
{
var progress = Progress(); // returns a new IProgress<T>
var task = Task<int>.Factory.StartNew(() => MyFunction(progress), TaskCreationOptions.LongRunning);
TaskList.Add(task);
}
Each progress instance updates the UI.
Now, as soon as a task is finished, I want to fire up a new one. Essentially, the process should run indefinitely, having X tasks running at any given time, unless the user cancels via the UI (I'll use cancellation tokens for this). I try to achieve this using the following:
while (TaskList.Count > 0)
{
var completed = await Task.WhenAny(TaskList.ToArray());
if (completed.Exception == null)
{
// report success
}
else
{
// flatten AggregateException, print out, etc
}
// update some labels/textboxes in the UI, and then:
TaskList.Remove(completed);
var task = Task<int>.Factory.StartNew(() => MyFunction(progress), TaskCreationOptions.LongRunning);
TaskList.Add(task);
}
This is bogging down the UI. Is there a better way of achieving this functionality, while keeping the UI responsive?
A suggestion was made in the comments to use TPL Dataflow but due to time constraints and specs, alternative solutions are welcome
Update
I'm not sure whether the progress reporting might be the problem? Here's what it looks like:
private IProgress<string> Progress()
{
return new Progress<string>(msg =>
{
txtMsg.AppendText(msg);
});
}

Now, as soon as a task is finished, I want to fire up a new one. Essentially, the process should run indefinitely, having X tasks running at any given time
It sounds to me like you want an infinite loop inside your task:
for (int i = 0; i < X; i++)
{
var progress = Progress(); // returns a new IProgress<T>
var task = RunIndefinitelyAsync(progress);
TaskList.Add(task);
}
private async Task RunIndefinitelyAsync(IProgress<T> progress)
{
while (true)
{
try
{
await Task.Run(() => MyFunction(progress));
// handle success
}
catch (Exception ex)
{
// handle exceptions
}
// update some labels/textboxes in the UI
}
}
However, I suspect that the "bogging down the UI" is probably in the // handle success and/or // handle exceptions code. If my suspicion is correct, then push as much of the logic into the Task.Run as possible.

As I understand, you simply need a parallel execution with the defined degree of parallelization. There is a lot of ways to implement what you want. I suggest to use blocking collection and parallel class instead of tasks.
So when user clicks button, you need to create a new blocking collection which will be your data source:
BlockingCollection<IProgress> queue = new BlockingCollection<IProgress>();
CancellationTokenSource source = new CancellationTokenSource();
Now you need a runner that will execute your in parallel:
Task.Factory.StartNew(() =>
Parallel.For(0, X, i =>
{
foreach (IProgress p in queue.GetConsumingEnumerable(source.Token))
{
MyFunction(p);
}
}), source.Token);
Or you can choose more correct way with partitioner. So you'll need a partitioner class:
private class BlockingPartitioner<T> : Partitioner<T>
{
private readonly BlockingCollection<T> _Collection;
private readonly CancellationToken _Token;
public BlockingPartitioner(BlockingCollection<T> collection, CancellationToken token)
{
_Collection = collection;
_Token = token;
}
public override IList<IEnumerator<T>> GetPartitions(int partitionCount)
{
throw new NotImplementedException();
}
public override IEnumerable<T> GetDynamicPartitions()
{
return _Collection.GetConsumingEnumerable(_Token);
}
public override bool SupportsDynamicPartitions
{
get { return true; }
}
}
And runner will looks like this:
ParallelOptions Options = new ParallelOptions();
Options.MaxDegreeOfParallelism = X;
Task.Factory.StartNew(
() => Parallel.ForEach(
new BlockingPartitioner<IProgress>(queue, source.Token),
Options,
p => MyFunction(p)));
So all you need right now is to fill queue with necessary data. You can do it whenever you want.
And final touch, when the user cancels operation, you have two options:
first you can break execution with source.Cancel call,
or you can gracefully stop execution by marking collection complete (queue.CompleteAdding), in that case runner will execute all already queued data and finish.
Of course you need additional code to handle exceptions, progress, state and so on. But main idea is here.

Check if at least one thread is completed

First of all I am totally new to threading in C#. I have created multiple threads as shown below.
if (flag)
{
foreach (string empNo in empList)
{
Thread thrd = new Thread(()=>ComputeSalary(empNo));
threadList.Add(thrd);
thrd.Start();
}
}
Before proceeding further I need check if at least one thread is completed its execution so that I can perform additional operations.
I also tried creating the list of type thread and by added it to list, so that I can check if at least one thread has completed its execution. I tried with thrd.IsAlive but it always gives me current thread status.
Is there any other way to check if atleast on thread has completed its execution?

You can use AutoResetEvent.
var reset = new AutoResetEvent(false); // ComputeSalary should have access to reset
.....
....
if (flag)
{
foreach (string empNo in empList)
{
Thread thrd = new Thread(()=>ComputeSalary(empNo));
threadList.Add(thrd);
thrd.Start();
}
reset.WaitOne();
}
.....
.....
void ComputeSalary(int empNo)
{
.....
reset.set()
}
Other options are callback function, event or a flag/counter(this is not advised).

Here is a solution based on the Task Parallel Library:
// Create a list of tasks for each string in empList
List<Task> empTaskList = empList.Select(emp => Task.Run(() => ComputeSalary(emp)))
.ToList();
// Give me the task that finished first.
var firstFinishedTask = await Task.WhenAny(empTaskList);
A couple of things to note:
In order to use await inside your method, you will have to declare it as async Task or or async Task<T> where T is the desired return type
Task.Run is your equivalent of new Thread().Start(). The difference is Task.Run will use the ThreadPool (unless you explicitly tell it not to), and the Thread class will construct an entirely new thread.
Notice the use of await. This tells the compiler to yield control back to the caller until Task.WhenAny returns the first task that finished.
You should read more about async-await here

c# do the equivalent of restarting a Task with some parameter

The main idea here is to fetch some data from somewhere, when it's fetched start writing it, and then prepare the next batch of data to be written, while waiting for the previous write to be complete.
I know that a Task cannot be restarted or reused (nor should it be), although I am trying to find a way to do something like this :
//The "WriteTargetData" method should take the "data" variable
//created in the loop below as a parameter
//WriteData basically do a shedload of mongodb upserts in a separate thread,
//it takes approx. 20-30 secs to run
var task = new Task(() => WriteData(somedata));
//GetData also takes some time.
foreach (var data in queries.Select(GetData))
{
if (task.Status != TaskStatus.Running)
{
//start task with "data" as a parameter
//continue the loop to prepare the next batch of data to be written
}
else
{
//wait for task to be completed
//"restart" task
//continue the loop to prepare the next batch of data to be written
}
}
Any suggestion appreciated ! Thanks. I don't necessarily want to use Task, I just think it might be the way to go.

This may be over simplifying your requirements, but would simply "waiting" for the previous task to complete work for you? You can use Task.WaitAny and Task.WaitAll to wait for previous operations to complete.
pseudo code:
// Method that makes calls to fetch and write data.
public async Task DoStuff()
{
Task currTask = null;
object somedata = await FetchData();
while (somedata != null)
{
// Wait for previous task.
if (currTask != null)
Task.WaitAny(currTask);
currTask = WriteData(somedata);
somedata = await FetchData();
}
}
// Whatever method fetches data.
public Task<object> FetchData()
{
var data = new object();
return Task.FromResult(data);
}
// Whatever method writes data.
public Task WriteData(object somedata)
{
return Task.Factory.StartNew(() => { /* write data */});
}

The Task class is not designed to be restarted. so you Need to create a new task and run the body with the same Parameters. Next i do not see where you start the task with the WriteData function in its body. That will property Eliminate the call of if (task.Status != TaskStatus.Running) There are AFAIK only the class Task and Thread where task is only the abstraction of an action that will be scheduled with the TaskScheduler and executed in different threads ( when we talking about the Common task Scheduler, the one you get when you call TaskFactory.Scheduler ) and the Number of the Threads are equal to the number of Processor Cores.
To you Business App. Why do you wait for the execution of WriteData? Would it be not a lot more easy to gater all data and than submit them into one big Write?

something like ?
public void Do()
{
var task = StartTask(500);
var array = new[] {1000, 2000, 3000};
foreach (var data in array)
{
if (task.IsCompleted)
{
task = StartTask(data);
}
else
{
task.Wait();
task = StartTask(data);
}
}
}
private Task StartTask(int data)
{
var task = new Task(DoSmth, data);
task.Start();
return task;
}
private void DoSmth(object time)
{
Thread.Sleep((int) time);
}

You can use a thread and an AutoResetEvent. I have code like this for several different threads in my program:
These are variable declarations that belong to the main program.
public AutoResetEvent StartTask = new AutoResetEvent(false);
public bool IsStopping = false;
public Thread RepeatingTaskThread;
Somewhere in your initialization code:
RepeatingTaskThread = new Thread( new ThreadStart( RepeatingTaskProcessor ) ) { IsBackground = true; };
RepeatingTaskThread.Start();
Then the method that runs the repeating task would look something like this:
private void RepeatingTaskProcessor() {
// Keep looping until the program is going down.
while (!IsStopping) {
// Wait to receive notification that there's something to process.
StartTask.WaitOne();
// Exit if the program is stopping now.
if (IsStopping) return;
// Execute your task
PerformTask();
}
}
If there are several different tasks you want to run, you can add a variable that would indicate which one to process and modify the logic in PerformTask to pick which one to run.
I know that it doesn't use the Task class, but there's more than one way to skin a cat & this will work.