Join threads from thread pool - c#

I have 30+ tasks that can be executed in parallel.
I use ThreadPool for each task.
But parent-function should not return until all tasks has completed.
I need a thread sync handle that would release WaitOne when its count reaches 0.
Something like:
foo.StartWith(myTasks.Count);
foreach (var task in myTasks) {
ThreadPool.QueueUserWorkItem(state => { task(state); foo.Release(); });
}
foo.WaitOne();
Semaphore feels right, just can't figure out how to apply it here.

int running = myTasks.Count;
AutoResetEvent done = new AutoResetEvent(false);
foreach (var task in myTasks) {
ThreadPool.QueueUserWorkItem(state => {
task(state);
if (0 == Interlocked.Decrement(ref running))
done.Set ();
});
}
done.WaitOne();
With C# 4.0 you can use the new CountdownEvent primitive.

Joe Duffy wrote a great article on this sort of thing:
CLR Inside Out: 9 Reusable Parallel Data Structures and Algorithms
I'm eyballing the CountdownLatch as particularly appropriate for your requirements.

According to this article: Overview of Synchronization Primitives
C# has a built-in type for this scenario, CountDownEvent: CountdownEvent Class
Or a similiar one: Barrier (.NET Framework)
For newer versions, use the TPL (Task Parallel Library) , and for this scenerio this code is relevant:
// Create an ActionBlock<int> object that prints its input
// and throws ArgumentOutOfRangeException if the input
// is less than zero.
var throwIfNegative = new ActionBlock<int>(n =>
{
Console.WriteLine("n = {0}", n);
if (n < 0)
{
throw new ArgumentOutOfRangeException();
}
});
// Post values to the block.
throwIfNegative.Post(0);
throwIfNegative.Post(-1);
throwIfNegative.Post(1);
throwIfNegative.Post(-2);
throwIfNegative.Complete();
// Wait for completion in a try/catch block.
try
{
throwIfNegative.Completion.Wait();
}
catch (AggregateException ae)
{
// If an unhandled exception occurs during dataflow processing, all
// exceptions are propagated through an AggregateException object.
ae.Handle(e =>
{
Console.WriteLine("Encountered {0}: {1}",
e.GetType().Name, e.Message);
return true;
});
}
/* Output:
n = 0
n = -1
Encountered ArgumentOutOfRangeException: Specified argument was out of the range
of valid values.
*/
from https://msdn.microsoft.com/en-us/library/hh228603.aspx

Related

Interrupt Thread Child c#

How interrupt thread child in this case? I need interrupt foreach in thread child after throw exception in thread parent
Thread child;
Thread parent = new Thread(() =>
{
child = new Thread(() =>
{
for (int i = 0; i < int.MaxValue; i++)
{
Console.WriteLine("THREAD 1 = " + i);
}
});
child.Start();
try
{
child.Join();
}
catch (Exception ex)
{
System.Console.WriteLine(ex.ToString());
** // here????**
}
});
parent.Start();
parent.Interrupt();
Interrupt thread child
You cannot "interput" threads. Older .Net versions had Thread.Abort, but there are several good reasons why it should never be used.
What you should do is ask the thread to stop processing, and your thread should frequently check if needs to stop.
There are several ways to do this, one example would be the cancellation token. Let your thread take a cancellation token as input and in your loop add a call to
myCancellationToken.ThrowIfCancellationRequested();
Your caller would create a cancellation token source to generate the token and call .Cancel() to request the thread to abort.
However, using threads directly is very outdated, you should be using task based programming. There are common patterns to do most common tasks in a elegant and compact way, like DataFlow, Parallel.For, PLinq, concurrent collections etc.
Multi threaded programming is difficult. You need to have a fair bit of knowledge on the topic before you try to write anything intended for real world use. Thread safety bugs are quite easy to make, and are notoriously difficult to find and debug. So I would recommend studying thread safety, as well as modern threading patterns. This is not an area where trial and error is appropriate, since you are likely to create thing that work 99.9% of the time, or works 100% on your computer, but not in production.
JonasH's answer is a good one, but just for reference, here's how to use the CancellationToken with the original code:
CancellationTokenSource cts = new CancellationTokenSource();
CancellationToken ct = cts.Token;
Thread parent = new Thread(() =>
{
Thread child = new Thread(() =>
{
for (int i = 0; i < int.MaxValue; i++)
{
if (ct.IsCancellationRequested)
{
break;
}
Console.WriteLine("THREAD 1 = " + i);
}
});
child.Start();
try
{
child.Join();
}
catch (Exception ex)
{
System.Console.WriteLine(ex.ToString());
cts.Cancel();
}
});
parent.Start();
parent.Interrupt();

WaitAll vs WaitAny

I am little bit confuse for WaitAll and WaitAny. I am trying to get exception but when i do WaitAll it return exception but When use WaitAny returns nothing.And necessary is that if any of task complete work done.Is their any Replacement of WaitAny(). WaitAll and WhenAll are different becuase i dont want let all task done. Like
try
{
int i = 0;
Task t1 = Task.Factory.StartNew(() =>
{
i = 2 * 4;
});
Task<int> t2 = Task.Factory.StartNew(() =>
{
int a = 0;
int b = 100 / a;
return 0;
});
Task[] tasks = new Task[] { t1, t2 };
Task.WaitAny(tasks);
//Task.WaitAll(tasks);// Working
}
catch (AggregateException ae)
{
var message = ae.InnerException.Message;
Console.WriteLine(message);
}
Console.ReadLine();
This is called Unobserved Exception, which basically means that exception in newly created threads in Task Parallel Library (TPL) ThreadPool, is not caught by the other thread or as stated in aforementioned link:
While unobserved exceptions will still cause the
UnobservedTaskException event to be raised (not doing so would be a
breaking change), the process will not crash by default. Rather, the
exception will end up getting eaten after the event is raised,
regardless of whether an event handler observes the exception.
This means that when using WaitAny(), if one of the tasks completes without any exception, exceptions in other tasks will not be caught.
Maybe you want something like this. Note, this uses the Task-based Asynchronous Pattern.
public static Task<Task[]> WhenAllOrFirstException(params Task[] tasks)
{
var countdownEvent = new CountdownEvent(tasks.Length);
var completer = new TaskCompletionSource<Task[]>();
Action<Task> onCompletion = completed =>
{
if (completed.IsFaulted && completed.Exception != null)
{
completer.TrySetException(completed.Exception.InnerExceptions);
}
if(countdownEvent.Signal() && !completer.Task.IsCompleted)
{
completer.TrySetResult(tasks);
}
};
foreach(var task in tasks)
{
task.ContinueWith(onCompletion)
}
return completer.Task;
}
Your WaitAny implementation will throw error as well it depends what task will complete first. WaitAny does not wait till looser task (in most cases it would be t2) got completed - so it won't got exception from it.
Update
Indeed WaitAny will not return any error as descriobed here Task.WhenAny and Unobserved Exceptions

Cancel execution of command if it takes too long

I am currently working on a piece of software that uses an assembly from a different department.I call a Method from this assembly like this:
using (var connection = (IConnection) Factory.GetObject(typeof (IConnection)))
The code used to work perfectly fine. But for the last few minutes it seemed like my program was doing nothing when I tried to launch it. Pressing pause while debugging showed me it got "stuck" at the line above.My guess is they're just doing some maintenance or something but that's not the point here.
So I though it would be nice to tell the user what went wrong if the program doesn't start. Something simple like
MessageBox.Show("Could not connect", "Connection Error");
And then close the program. My question is:
How do I terminate the execution of a command after a set amount of time and jump somewhere else?My guess would be moving it to a separate thread and then putting the calling thread to sleep for a few seconds after which it disposes of the extra thread if it isn't completed yet. But that seems really dirty to me and there's got to be a better way.
Your question can be separated into two parts:
How to terminate the execution of a command?
The only way is to abort the thread. But don't do it. There is no guaranteed and safe way. There are such methods like Thread.Interrupt and Thread.Abort that can wake up the thread. But they will work only if the thread is in the WaitSleepJoin state and it hangs in managed code.
Seems like you already know it. But once again, if something inside the assembly hangs infinitely the execution of code then the thread is probably "gone". So you are right that the program should be closed.
... jump somewhere else?
Good approach is using of TPL and async model.
Here is an extension method to wrap up any Task and expires after timeout.
public static async Task TimeoutAfter(this Task task, int millisecondsTimeout)
{
if (task == await Task.WhenAny(task, Task.Delay(millisecondsTimeout)))
await task;
else
throw new TimeoutException();
}
Then use it
try
{
using (var result = await Task.Run(() => (IConnection)Factory.GetObject(typeof(IConnection))).TimeoutAfter(1000))
{
...
}
}
catch (TimeoutException ex)
{
//timeout
}
Here you can find more information
A simple way of doing it without extra libraries or extension methods:
using ( var task = new Task<IConnection>( () => Factory.GetObject( typeof( IConnection ) ) ) )
{
task.Start();
if( !task.Wait( timeoutMilliseconds ) )
{
throw new TimeoutException();
}
IConnection result = task.Result;
}
Task.Wait does what you want, because you can throw an exception if it returns false (task didn't complete in time.)
It's even simpler if you have an Action that doesn't return something:
if ( !Task.Run( action ).Wait( timeoutMilliseconds ) )
{
throw new TimeoutException();
}
Where action is some Action or lambda.
The easiest way to do this, if a native timeout is not implemented, as you mentioned, is a separate thread to load it on. Although this sounds like it'll be really dirty, it's as simple as (using Rx):
Task<IConnection> connectionTask = Observable.Start(() => Factory.GetObject(typeof (IConnection)), Scheduler.Default).Timeout(TimeSpan.FromSeconds(20)).ToTask());
using (var connection = connectionTask.Result)
{
...
}
You can tweak the Scheduler if you don't want it to run on the threadpool. It will throw a TimeoutException if the Factory.GetObject call takes longer than 20 seconds.
You could use CancellationTokenSource to set a timeout on the operation
var timeoutCts = new CancellationTokenSource();
try
{
timeoutCts.CancelAfter(300000); // Cancel after 5 minutes
// ... run your long term operation
}
catch (OperationCanceledException ex)
{
// Handle the timeout
}
Please see this documentation from Microsoft
using System;
using System.Threading.Tasks;
public class Example
{
public static void Main()
{
Task t = Task.Run(() =>
{
Random rnd = new Random();
long sum = 0;
int n = 5000000;
for (int ctr = 1; ctr <= n; ctr++)
{
int number = rnd.Next(0, 101);
sum += number;
}
Console.WriteLine("Total: {0:N0}", sum);
Console.WriteLine("Mean: {0:N2}", sum / n);
Console.WriteLine("N: {0:N0}", n);
});
TimeSpan ts = TimeSpan.FromMilliseconds(150);
if (!t.Wait(ts))
Console.WriteLine("The timeout interval elapsed.");
}
}

Is there a C# equivalent to Java CompletionService to react to completed threads?

There are lots of examples of how to wait for all 'worker' jobs to finish, but what about reacting as each worker completes in the same way as Java's CompletionService?
I've hacked together something that works, but this seems fragile:
int completedWorkers = 0;
while (completedWorkers < workerCount)
{
int eventId = WaitHandle.WaitAny(eventArray);
events[eventId].Reset();
completedWorkers++;
Console.WriteLine("Worker {0} has completed. {1} have now completed.", eventId, completedWorkers);
}
Console.WriteLine("All threads have finished");
This relies on an array of ManualResetEvent instances ('eventArray' in my example), similar to that shown in the Microsoft example, http://msdn.microsoft.com/en-us/library/3dasc8as.aspx (however their code uses WaitAll() to wait for all workers to finish, not reacting as each worker finishes).
Edit:
I've accepted Douglas's answer using the TPL, but for reference, Delegates give us this functionality out of the box:
IAsyncResult result = caller.BeginInvoke(
new AsyncCallback(CallbackMethod),
"The call executed on thread {0}, with return value \"{1}\".");
...
void CallbackMethod(IAsyncResult ar)
{
// 'react' to async delegates completing
}
If you use the Task Parallel Library, you could attach a continuation to each of your tasks.
int completedWorkers = 0;
var continuations = tasks.Select((task, index) => task.ContinueWith(antecedent =>
{
lock (tasks) // only required if you want to avoid races between incrementing and writing
{
int _index = index; // avoid closures issue
completedWorkers++; // if lock is removed, replace with: Interlocked.Increment(ref completedWorkers);
Console.WriteLine("Worker {0} has completed. {1} have now completed.", _index, completedWorkers);
}
}));
Task.WaitAll(continuations.ToArray());
Console.WriteLine("All threads have finished");
I have found an example that requires .NET 4.5 here: http://msdn.microsoft.com/en-us/library/jj155756.aspx
// ***Add a loop to process the tasks one at a time until none remain.
while (downloadTasks.Count > 0)
{
// Identify the first task that completes.
Task<int> firstFinishedTask = await Task.WhenAny(downloadTasks);
// ***Remove the selected task from the list so that you don't
// process it more than once.
downloadTasks.Remove(firstFinishedTask);
// Await the completed task.
int length = await firstFinishedTask;
resultsTextBox.Text += String.Format("\r\nLength of the download: {0}", length);
}

Logging exceptions for each item in Parallel.ForEach and Task.Factory.StartNew from it

I am trying to use Parallel.ForEach on a list and for each item in the list, trying to make a database call. I am trying to log each item with or without error. Just wanted to check with experts here If I am doing thinsg right way. For this example, I am simulating the I/O using the File access instead of database access.
static ConcurrentQueue<IdAndErrorMessage> queue = new ConcurrentQueue<IdAndErrorMessage>();
private static void RunParallelForEach()
{
List<int> list = Enumerable.Range(1, 5).ToList<int>();
Console.WriteLine("Start....");
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
Parallel.ForEach(list, (tempId) =>
{
string errorMessage = string.Empty;
try
{
ComputeBoundOperationTest(tempId);
try
{
Task[] task = new Task[1]
{
Task.Factory.StartNew(() => this.contentFactory.ContentFileUpdate(content, fileId))
};
}
catch (Exception ex)
{
this.tableContentFileConversionInfoQueue.Enqueue(new ContentFileConversionInfo(fileId, ex.ToString()));
}
}
catch (Exception ex)
{
errorMessage = ex.ToString();
}
if (queue.SingleOrDefault((IdAndErrorMessageObj) => IdAndErrorMessageObj.Id == tempId) == null)
{
queue.Enqueue(new IdAndErrorMessage(tempId, errorMessage));
}
}
);
Console.WriteLine("Stop....");
Console.WriteLine("Total milliseconds :- " + stopWatch.ElapsedMilliseconds.ToString());
}
Below are the helper methods :-
private static byte[] FileAccess(int id)
{
if (id == 5)
{
throw new ApplicationException("This is some file access exception");
}
return File.ReadAllBytes(Directory.GetFiles(Environment.SystemDirectory).First());
//return File.ReadAllBytes("Files/" + fileName + ".docx");
}
private static void ComputeBoundOperationTest(int tempId)
{
//Console.WriteLine("Compute-bound operation started for :- " + tempId.ToString());
if (tempId == 4)
{
throw new ApplicationException("Error thrown for id = 4 from compute-bound operation");
}
Thread.Sleep(20);
}
private static void EnumerateQueue(ConcurrentQueue<IdAndErrorMessage> queue)
{
Console.WriteLine("Enumerating the queue items :- ");
foreach (var item in queue)
{
Console.WriteLine(item.Id.ToString() + (!string.IsNullOrWhiteSpace(item.ErrorMessage) ? item.ErrorMessage : "No error"));
}
}
There is no reason to do this:
/*Below task is I/O bound - so do this Async.*/
Task[] task = new Task[1]
{
Task.Factory.StartNew(() => FileAccess(tempId))
};
Task.WaitAll(task);
By scheduling this in a separate task, and then immediately waiting on it, you're just tying up more threads. You're better off leaving this as:
/*Below task is I/O bound - but just call it.*/
FileAccess(tempId);
That being said, given that you're making a logged value (exception or success) for every item, you might want to consider writing this into a method and then just calling the entire thing as a PLINQ query.
For example, if you write this into a method that handles the try/catch (with no threading), and returns the "logged string", ie:
string ProcessItem(int id) { // ...
You could write the entire operation as:
var results = theIDs.AsParallel().Select(id => ProcessItem(id));
You might want to remove Console.WriteLine from thread code. Reason being there can be only one console per windows app. So if two or more threads going to write parallel to console, one has to wait.
In replacement to your custom error queue you might want to see .NET 4's Aggregate Exception and catch that and process exceptions accordingly. The InnerExceptions propery will give you the necessary list of exceptions. More here
And a general code review comment, don't use magic numbers like 4 in if (tempId == 4) Instead have some const defined which tells what 4 stands for. e.g. if (tempId == Error.FileMissing)
Parallel.ForEach runs an action/func concurrently up to a certain number of simultaneous instances. If what each of those iterations is doing is not inherently independent on one another, you're not getting any performance gains. And, likely are reducing performance by introducing expensive context switching and contention. You say that you want to do a "database call" and simulating it with a file operation. If each iteration uses the same resource (same row in a database table, for example; or try to write to the same file in the same location) then they're not really going to be run in parallel. only one will be running at a time, the others will simply be "waiting" to get a hold of the resource--needlessly making your code complex.
You haven't detailed what you want to do for each iteration; but when I've encountered situations like this with other programmers, they almost always aren't really doing things in parallel and they've simply gone through and replaced foreachs with Parallel.ForEach in the hopes of magically gaining performance or magically making use of multi-CPU/Core processors.

Categories

Resources