Develop Reference

Code Help for ThreadPool

Have created a class which implements ThreadPool. The code is as below:
public sealed class PyeThreadPool :
IDisposable
{
private readonly object _lock = new object();
private readonly int _minThreadCount;
private readonly int _maxThreadCount;
private readonly Queue<Action> _queue = new Queue<Action>();
private int _totalThreadCount;
private int _waitingThreadCount;
private bool _disposed;
public PyeThreadPool(int minThreadCount, int maxThreadCount)
{
if (minThreadCount < 0)
throw new ArgumentOutOfRangeException("minThreadCount");
if (maxThreadCount < 1 || maxThreadCount < minThreadCount)
throw new ArgumentOutOfRangeException("maxThreadCount");
_minThreadCount = minThreadCount;
_maxThreadCount = maxThreadCount;
}
public void Dispose()
{
lock (_lock)
{
_disposed = true;
// if there are thread waiting, they should stop waiting.
if (_waitingThreadCount > 0)
Monitor.PulseAll(_lock);
}
}
/// <summary>
/// Executes an action in a parallel thread.
/// </summary>
public void RunParallel(Action action)
{
if (action == null)
throw new ArgumentNullException("action");
lock (_lock)
{
if (_disposed)
throw new ObjectDisposedException(GetType().FullName);
bool queued = false;
if (_waitingThreadCount == 0)
{
if (_totalThreadCount < _maxThreadCount)
{
_totalThreadCount++;
var thread = new Thread(_ThreadRun);
thread.Name = "Worker Thread";
thread.Start(action);
queued = true;
}
}
if (!queued)
{
_queue.Enqueue(action);
Monitor.Pulse(_lock);
}
}
}
private void _ThreadRun(object firstAction)
{
Action action = (Action)firstAction;
firstAction = null;
// we always start a new thread with an action, so we get it immediately.
// but, as we don't know what that action really holds in memory, we set
// the initial action to null, so after it finishes and a new action is get,
// we will let the GC collect it.
while (true)
{
action();
lock (_lock)
{
if (_queue.Count == 0)
{
// we started waiting, so new threads don't need to be created.
_waitingThreadCount++;
while (_queue.Count == 0)
{
if (_disposed)
return;
if (_totalThreadCount > _minThreadCount)
{
_totalThreadCount--;
_waitingThreadCount--;
return;
}
action = null;
Monitor.Wait(_lock);
}
// we finished waiting.
_waitingThreadCount--;
}
action = _queue.Dequeue();
// we just get a new action, and we will release the lock and return
// to the while, where the action will be executed.
}
}
}
}
I have tried to use this and the test code is as:
PyeThreadPool MyPool;
int x = 1;
protected void Page_Load(object sender, EventArgs e)
{
MyPool = new PyeThreadPool(4, 6);
}
void showMessage(string message)
{
TxtMessage.Text = message;
}
protected void BtnStartThread_Click(object sender, EventArgs e)
{
x++;
int arg = x;
MyPool.RunParallel(() =>
{
showMessage(arg.ToString());
});
}
Problem is:
(1) When I execute this either in debug or release mode I do not see the result in textbox, on the other hand I see the result when I step through. What am I missing here, why I can not see the output.
(2) The RunParallel method shows only one thread even if I have set maxcount to more than 1. Is there any code logic missing or is it because the test application is simple?
Thanks !

You should have a look at SmartThreadPool library. It is one of the best alternative to ThreadPool.
Its features (copied from source link)
Smart Thread Pool is a thread pool written in C#. The implementation was first based on Stephan Toub's thread pool with some extra features, but now, it is far beyond the original. Here is a list of the thread pool features:
The number of threads dynamically changes according to the workload on the threads in the pool.
Work items can return a value.
A work item can be cancelled if it hasn't been executed yet.
The caller thread's context is used when the work item is executed (limited).
Usage of minimum number of Win32 event handles, so the handle count of the application won't explode.
The caller can wait for multiple or all the work items to complete.
A work item can have a PostExecute callback, which is called as soon the work item is completed.
The state object that accompanies the work item can be disposed automatically.
Work item exceptions are sent back to the caller.
Work items have priority.
Work items group.
The caller can suspend the start of a thread pool and work items group.
Threads have priority.
Threads have initialization and termination events.
WinCE platform is supported (limited).
Action and Func generic methods are supported.
Silverlight is supported.
Mono is supported.
Performance counters (Windows and internal).
Work item timeout (passive).
Threads ApartmentState
Threads IsBakcground
Threads name template
Windows Phone is supported (limited)
Threads MaxStackSize

The problem is you are attempting to update a UI control from a background thread. Not allowed.
You need to do a BeginInvoke or Invoke in your ShowMessage function.

Excessive CPU usage

I'm coping with a problem. In fact I need to use different timers in my programe and Timers from Framework .Net don't do what I expect. So I decided to create my own Timer, but my Timer uses too much CPU. This is my code :
using System;
using System.Threading;
namespace XXXXXXX.Common.Types
{
public delegate void TimerFinishedEventHandler(object sender, EventArgs e);
class Timer
{
#region Attributes
private long m_InitialTickCount;
private long m_Interval;
private Thread m_Thread;
private bool m_Enabled;
#endregion
#region Events
public event TimerFinishedEventHandler Finished;
#endregion
#region Constructors
public Timer(long interval, TimerFinishedEventHandler e)
{
Finished += e;
m_Interval = interval;
Start(m_Interval);
}
#endregion
#region Public methods
/// <summary>
/// Start the timer thread.
/// </summary>
public void Start(long interval)
{
m_Interval = interval;
m_Enabled = true;
m_InitialTickCount = Environment.TickCount;
if (m_Thread == null)
{
m_Thread = new Thread(Check);
m_Thread.Start();
}
}
/// <summary>
/// Stop the Timer.
/// </summary>
public void Stop()
{
m_Enabled = false;
}
/// <summary>
/// Restart the Timer.
/// </summary>
public void Restart()
{
m_InitialTickCount = Environment.TickCount;
}
#endregion
#region Private methods
/// <summary>
/// Check if the timer is finished or not.
/// </summary>
private void Check()
{
while (true)
{
if (!m_Enabled)
return;
if (Environment.TickCount > m_InitialTickCount + m_Interval)
{
OnFinished(EventArgs.Empty);
return;
}
}
}
/// <summary>
/// Called when the Timer is Finished.
/// </summary>
/// <param name="e">Event</param>
protected virtual void OnFinished(EventArgs e)
{
if (Finished != null)
Finished(this, e);
}
#endregion
}
}
Is there anybody who have a solution ? Because when I launch my program, 2 or 3 Timers are created, another thread run and I have my CPU 100% using.

There is absolutely no reason you can't have multiple timers. I have programs that have hundreds of timers, and at any time a handful of them can be actually doing work. The point of timers is that they allow you to schedule periodic actions and not consume any CPU resources except when the actions are actually being processed. That is, if you set a timer to tick once per minute, then that timer doesn't occupy a thread, doesn't consume any memory (beyond a token amount for the timer handle and the callback address), and doesn't consume any CPU resources. Only when the timer "ticks" once per minute is a thread allocated to execute code for it. And typically that is a pool thread that already exists, so thread startup time is negligible.
Using a timer is very easy: you create a method for the timer to execute, and you schedule the timer to run it. For example:
System.Threading.Timer myTimer =
new System.Threading.Timer(MyTimerProc, null, TimeSpan.FromMinutes(1), TimeSpan.FromMinutes(1));
void MyTimerProc(object state)
{
// do something here
}
You could have another timer that ticks every 30 seconds and executes a different timer proc:
System.Threading.Timer myOtherTimer =
new System.Threading.Timer(MyOtherTimerProc, null, TimeSpan.FromMinutes(1), TimeSpan.FromMinutes(1));
void MyOtherTimerProc(object state)
{
// do something else here
}
The timers won't interfere with each other. Of course, if code in the timer procs modifies shared data (for example, both procs try to update a list or a dictionary), then you'll have to synchronize access to that shared data or use a concurrent data structure.
You can get into trouble with reentrancy if the processing in your timer proc takes longer than the timer period. If the processing in MyTimerProc takes longer than 60 seconds, then another timer tick can come along and now you have two threads executing that timer proc. That can cause many different types of problems if your code is not set up to handle it. Typically you eliminate that problem by making the timer a one-shot and restarting it at the end of each processing cycle. There are examples of doing that here on Stack Overflow.
System.Timers.Timer is a component wrapper around System.Threading.Timer. The idea that it's "optimized for high performance threading" or some such is silly. System.Timers.Timer gives you a familiar event-oriented interface, and also provides a SynchronizingObject, which lets you have the event raised on a particular thread rather than having to explicitly Invoke as you would with System.Threading.Timer. Typically, that's only useful in UI applications.
System.Timers.Timer has one particularly ugly "feature" that I consider a bug: it squashes exceptions. As the documentation says:
In the .NET Framework version 2.0 and earlier, the Timer component catches and suppresses all exceptions thrown by event handlers for the Elapsed event.
That behavior still exists in .NET 4.5. The problem is that if you have your Elapsed event:
private static void OnTimedEvent(object source, ElapsedEventArgs e)
{
// do stuff here
}
If your event handler throws an exception, it will propagate back to the timer code, which squashes the exception and never tells you about it. In effect, the timer does this:
try
{
OnTimedEvent(source, args);
}
catch
{
// Squash those pesky exceptions. Who needs them?
}
It's a bug hider because you never know that the exception was thrown. So your program doesn't work and you can't figure out why. It's for this reason that I strongly recommend that you NOT use System.Timers.Timer. Use System.Threading.Timer instead; after all, it's the base that System.Timers.Timer is built on.

To answer your question directly, the reason your CPU usage is so high is that you are using a tight while loop to check your elapsed event. This is sometimes called a spin lock (called so because it is one, very inefficient, way to implement a semaphore where the thread checks a locking variable in a tight loop, thus it "spins").
Instead of a tight loop, you need to block and allow something else to run for a while:
private void Check()
{
while (true)
{
if (!m_Enabled)
return;
Thread.Sleep(10); //10 millisecond resolution for this timer
if (Environment.TickCount > m_InitialTickCount + m_Interval)
{
OnFinished(EventArgs.Empty);
return;
}
}
}
The resolution depends on how long you sleep for. That being said, the provided timers should always be sufficent, even using 2 System.Threading.Timers would work. I have personally used multiples of both System.Threading.Timer and System.Timers.Timer with no problems.
Of course, with all these timers, you need to be careful about accessing shared resources (perhaps what you mean by the existing timers blocking other threads?). Deadlock is a very real scenario in multi-threading, but doesn't have much to do with timers.

For #Jim Mischel.
In my class connecting to my website, checking datas :
#region Attributes
private static Timer m_TimerNextCheck;
#endregion
#region Méthodes publiques
public static void StartCheck()
{
Thread licenceThread = new Thread(Checking);
licenceThread.Start();
}
#endregion
#region Méthodes privées
private static void Checking()
{
//connect to the website
try
{
HttpWebResponse httpWebResponse = (HttpWebResponse) request.GetResponse();
StreamReader streamReader = new StreamReader(httpWebResponse.GetResponseStream(), Encoding.Default);
string response = streamReader.ReadToEnd();
httpWebResponse.Close();
if (//Some code)
{
//Some code
}
else
{
if (m_TimerNextCheck == null)
m_TimerNextCheck = new Timer(TimerNextCheckFinished, null, 300000, Timeout.Infinite);
else
m_TimerNextCheck.Change(300000, Timeout.Infinite);
}
}
catch (WebException exception)
{
//Some code
if (m_TimerNextCheck == null)
m_TimerNextCheck = new Timer(TimerNextCheckFinished, null, 60000, Timeout.Infinite);
else
m_TimerNextCheck.Change(60000, Timeout.Infinite);
}
}
private static void TimerNextCheckFinished(object statusInfos)
{
Checking();
}
#endregion
In another class :
#region Attributs
private Thread m_ConnectionThread;
private Timer m_TimerConnectionThread;
#endregion
#region Méthodes publiques
public void Launch()
{
m_ConnectionThread = new Thread(Connect);
m_ConnectionThread.Start();
}
public void GetNextMeal()
{
//Some code
if (//Some code)
{
//Some code
if (m_TimerConnectionThread == null)
m_TimerConnectionThread = new Timer(TimerConnectionThreadFinished, null,
(int)TimeSpan.FromHours(difference.Hour).TotalMilliseconds +
(int)TimeSpan.FromMinutes(difference.Minute).TotalMilliseconds, Timeout.Infinite);
else
m_TimerConnectionThread.Change((int)TimeSpan.FromHours(difference.Hour).TotalMilliseconds +
(int)TimeSpan.FromMinutes(difference.Minute).TotalMilliseconds, Timeout.Infinite);
}
else
{
//Some code
}
}
public void TryReconnect(int minute)
{
//Some code
if (m_TimerConnectionThread == null)
m_TimerConnectionThread = new Timer(TimerConnectionThreadFinished, null, (int)TimeSpan.FromMinutes(minute).TotalMilliseconds,
Timeout.Infinite);
else
m_TimerConnectionThread.Change((int)TimeSpan.FromMinutes(minute).TotalMilliseconds, Timeout.Infinite);
//Some code
}
//Some code
#endregion
#region Méthodes privées
private void Connect()
{
if (m_TimerConnectionThread != null)
m_TimerConnectionThread.Change(Timeout.Infinite, Timeout.Infinite);
//Some code
}
//Some code
private void TimerConnectionThreadFinished(object stateInfo)
{
Connect();
}
#endregion
And it works good !

Implementing a thread queue/wait, how?

I have a timer calling a function every 15 minutes, this function counts the amount of lines in my DGV and starts a thread for each lines (of yet another function), said thread parse a web page which can take anywhere from 1 second to 10 second to finish.
Whilst it does work fine as it is with 1-6 rows, anymore will cause the requests to time-out.
I want it to wait for the newly created thread to finish processing before getting back in the loop to create another thread without locking the main UI
for (int x = 0; x <= dataGridFollow.Rows.Count - 1; x++)
{
string getID = dataGridFollow.Rows[x].Cells["ID"].Value.ToString();
int ID = int.Parse(getID);
Thread t = new Thread(new ParameterizedThreadStart(UpdateLo));
t.Start(ID);
// <- Wait for thread to finish here before getting back in the for loop
}
I have googled a lot in the past 24 hours, read a lot about this specific issue and its implementations (Thread.Join, ThreadPools, Queuing, and even SmartThreadPool).
It's likely that I've read the correct answer somewhere but I'm not at ease enough with C# to decypher those Threading tools
Thanks for your time

to avoid the UI freeze the framework provide a class expressly for these purposes: have a look at the BackgroundWorker class (executes an operation on a separate thread), here's some infos : http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx
http://msdn.microsoft.com/en-us/magazine/cc300429.aspx
Btw looks if I understand correctly you don't want to parallelize any operation so just wait for the method parsing the page to be completed. Basically for each (foreach look) row of your grid you get the id and call the method. If you want to go parallel just reuse the same foreach loop and add make it Parallel
http://msdn.microsoft.com/en-us/library/dd460720.aspx

What you want is to set off a few workers that do some task.
When one finishes you can start a new one off.
I'm sure there is a better way using thread pools or whatever.. but I was bored so i came up with this.
using System;
using System.Collections.Generic;
using System.Linq;
using System.ComponentModel;
using System.Threading;
namespace WorkerTest
{
class Program
{
static void Main(string[] args)
{
WorkerGroup workerGroup = new WorkerGroup();
Console.WriteLine("Starting...");
for (int i = 0; i < 100; i++)
{
var work = new Action(() =>
{
Thread.Sleep(1000); //somework
});
workerGroup.AddWork(work);
}
while (workerGroup.WorkCount > 0)
{
Console.WriteLine(workerGroup.WorkCount);
Thread.Sleep(1000);
}
Console.WriteLine("Fin");
Console.ReadLine();
}
}
public class WorkerGroup
{
private List<Worker> workers;
private Queue<Action> workToDo;
private object Lock = new object();
public int WorkCount { get { return workToDo.Count; } }
public WorkerGroup()
{
workers = new List<Worker>();
workers.Add(new Worker());
workers.Add(new Worker());
foreach (var w in workers)
{
w.WorkCompleted += (OnWorkCompleted);
}
workToDo = new Queue<Action>();
}
private void OnWorkCompleted(object sender, EventArgs e)
{
FindWork();
}
public void AddWork(Action work)
{
workToDo.Enqueue(work);
FindWork();
}
private void FindWork()
{
lock (Lock)
{
if (workToDo.Count > 0)
{
var availableWorker = workers.FirstOrDefault(x => !x.IsBusy);
if (availableWorker != null)
{
var work = workToDo.Dequeue();
availableWorker.StartWork(work);
}
}
}
}
}
public class Worker
{
private BackgroundWorker worker;
private Action work;
public bool IsBusy { get { return worker.IsBusy; } }
public event EventHandler WorkCompleted;
public Worker()
{
worker = new BackgroundWorker();
worker.DoWork += new DoWorkEventHandler(OnWorkerDoWork);
worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(OnWorkerRunWorkerCompleted);
}
private void OnWorkerRunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
if (WorkCompleted != null)
{
WorkCompleted(this, EventArgs.Empty);
}
}
public void StartWork(Action work)
{
if (!IsBusy)
{
this.work = work;
worker.RunWorkerAsync();
}
else
{
throw new InvalidOperationException("Worker is busy");
}
}
private void OnWorkerDoWork(object sender, DoWorkEventArgs e)
{
work.Invoke();
work = null;
}
}
}
This would be just a starting point.
You could start it off with a list of Actions and then have a completed event for when that group of actions is finished.
then at least you can use a ManualResetEvent to wait for the completed event.. or whatever logic you want really.

Call a method directly or do a while loop (with sleep calls) to check the status of the thread.
There are also async events but the would call another method, and you want to continue from the same point.

I have no idea why the requests would timeout. That sounds like a different issue. However, I can make a few suggestions regarding your current approach.
Avoid creating threads in loops with nondeterministic bounds. There is a lot of overhead in creating threads. If the number of operations is not known before hand then use the ThreadPool or the Task Parallel Library instead.
You are not going to get the behavior you want by blocking the UI thread with Thread.Join. The cause the UI to become unresponsive and it will effectively serialize the operations and cancel out any advantage you were hoping to gain with threads.
If you really want to limit the number of concurrent operations then a better solution is to create a separate dedicated thread for kicking off the operations. This thread will spin around a loop indefinitely waiting for items to appear in a queue and when they do it will dequeue them and use that information to kick off an operation asynchronously (again using the ThreadPool or TPL). The dequeueing thread can contain the logic for limiting the number of concurrent operations. Search for information regarding the producer-consumer pattern to get a better understand of how you can implement this.
There is a bit of a learning curve, but who said threading was easy right?

If I understand correctly, what you're currently doing is looping through a list of IDs in the UI thread, starting a new thread to handle each one. The blocking issue you're seeing then could well be that it's taking too many resources to create unique threads. So, personally (without knowing more) would redesign the process like so:
//Somewhere in the UI Thread
Thread worker = new Thread(new ParameterizedThreadStart(UpdateLoWorker));
worker.Start(dataGridFollow.Rows);
//worker thread
private void UpdateLoWorker(DataRowCollection rows)
{
foreach(DataRow r in rows){
string getID = r.Cells["ID"].Value.ToString();
int ID = int.Parse(getID);
UpdateLo(ID);
}
}
Here you'd have a single non-blocking worker which sequentially handles each ID.

Consider using Asynchronous CTP. It's an asynch pattern Microsoft recently released for download. It should simplify asynch programming tremendouesly. The link is http://msdn.microsoft.com/en-us/vstudio/async.aspx. (Read the whitepaper first)
Your code would look something like the following. (I've not verified my syntax yet, sorry).
private async Task DoTheWork()
{
for(int x = 0; x <= dataGridFollow.Rows.Count - 1; x++)
{
string getID = dataGridFollow.Rows[x].Cells["ID"].Value.ToString();
int ID = int.Parse(getID);
task t = new Task(new Action<object>(UpdateLo), ID);
t.Start();
await t;
}
}
This method returns a Task that can be checked periodically for completion. This follows the pattern of "fire and forget" meaning you just call it and presumably, you don't care when it completes (as long as it does complete before 15 minutes).
EDIT
I corrected the syntax above, you would need to change UpdateLo to take an object instead of an Int.

For a simple background thread runner that will run one thread from a queue at a time you can do something like this:
private List<Thread> mThreads = new List<Thread>();
public static void Main()
{
Thread t = new Thread(ThreadMonitor);
t.IsBackground = true;
t.Start();
}
private static void ThreadMonitor()
{
while (true)
{
foreach (Thread t in mThreads.ToArray())
{
// Runs one thread in the queue and waits for it to finish
t.Start();
mThreads.Remove(t);
t.Join();
}
Thread.Sleep(2000); // Wait before checking for new threads
}
}
// Called from the UI or elsewhere to create any number of new threads to run
public static void DoStuff()
{
Thread t = new Thread(DoCorestuff);
t.IsBackground = true;
mActiveThreads.Add(t);
}
public static void DoStuffCore()
{
// Your code here
}

how to wait for all background threads to finish (in C#)?

how to make the foreground thread wait for all background (child) threads to finish in C#? I need to get list of pending jobs from the queue (database), start a new thread to execute each of them and finally wait for all the child threads to finish. how to do that in C#? Thanks in advance.

You could store each launched thread in an array. Then when you need to wait for them all, call Join method on each thread in an array in a loop.
Thread child = new Thread(...);
Threads.Add(child);
child.Start()
...
foreach(Thread t in Threads)
{
t.Join();
}
HTH

Consider using ThreadPool. Most of what you want is already done. There is an example from Microsoft which does pretty much your entire task. Replace "fibonacci" with "database task" and it sounds like your problem.

Using dynamic data you can pass your object and the WaitHandle (ActionResetEvent) that lets you wait for all the background threads to finish without declaring an extra class:
static void Main(string[] args)
{
List<AutoResetEvent> areList = new List<AutoResetEvent>();
foreach (MyObject o in ListOfMyObjects)
{
AutoResetEvent are = new AutoResetEvent(false);
areList.Add(are);
ThreadPool.QueueUserWorkItem(DoWork, new { o, are });
};
Console.WriteLine("Time: {0}", DateTime.Now);
WaitHandle.WaitAll(areList.ToArray());
Console.WriteLine("Time: {0}", DateTime.Now);
Console.ReadKey();
}
static void DoWork(object state)
{
dynamic o = state;
MyObject myObject = (MyObject)o.o;
AutoResetEvent are = (AutoResetEvent)o.are;
myObject.Execute();
are.Set();
}

This is incomplete code, but ManualResetEvent works for you
var waitEvents = new List<ManualResetEvent>();
foreach (var action in actions)
{
var evt = new ManualResetEvent(false);
waitEvents.Add(evt);
ThreadPool.RegisterWaitForSingleObject(asyncResult.AsyncWaitHandle, TimeoutCallback, state, 5000, true);
}
if (waitEvents.Count > 0)
WaitHandle.WaitAll(waitEvents.ToArray());

Create a structure to keep track of your worker threads
private struct WorkerThreadElement
{
public IAsyncResult WorkerThreadResult;
public AsyncActionExecution WorkerThread;
}
You also need to keep track the total number of threads expected to be created and the number of threads that have currently completed
private int _TotalThreads = 0;
private int _ThreadsHandled = 0;
private List<WorkerThreadElement> _WorkerThreadElements = new List<WorkerThreadElement>();
Then create an autoreset handle in order to wait for thread completion.
// The wait handle thread construct to signal the completion of this process
private EventWaitHandle _CompletedHandle = new AutoResetEvent(false);
You also need a delegate to create new threads - There are multiple ways of doing this but i have chosen a simple delegate for the sake of this example
// Delegate to asynchronously invoke an action
private delegate void AsyncActionExecution();
Lets asume that the Invoke method is the entrance point that will create all threads and wait for their execution. So we have:
public void Invoke()
{
_TotalThreads = N; /* Change with the total number of threads expected */
foreach (Object o in objects)
{
this.InvokeOneThread();
}
// Wait until execution has been completed
_CompletedHandle.WaitOne();
// Collect any exceptions thrown and bubble them up
foreach (WorkerThreadElement workerThreadElement in _WorkerThreadElements)
{
workerThreadElement.WorkerThread.EndInvoke(workerThreadElement.WorkerThreadResult);
}
}
InvokeOneThread is the method used to create a single thread for one operation. Here we need to create a worker thread element and invoke the actual thread.
private void InvokeOneThread()
{
WorkerThreadElement threadElement = new WorkerThreadElement();
threadElement.WorkerThread = new AsyncActionExecution();
threadElement.WorkerThreadResult = threadElement.WorkerThread.BeginInvoke(actionParameters, InvokationCompleted, null);
_WorkerThreadElements.Add(threadElement);
}
Callback from thread completion
private object _RowLocker = new object();
/// <summary>
/// Increment the number of rows that have been fully processed
/// </summary>
/// <param name="ar"></param>
private void InvokationCompleted(IAsyncResult ar)
{
lock (_RowLocker)
{
_RowsHandled++;
}
if (_TotalThreads == _ThreadsHandled)
_CompletedHandle.Set();
}
Done

Background worker synchronization

Lets say I have a class that is supposed to generate some ID (for example GUID) for me. Now unfortunately the ID generation is a somewhat long process and if I need a hundred of those I run into a problem of significant slowdowns. In order to avoid those, I keep a queue of pre-generated ID, and when this queue starts to run down on them I use the BackgroundWorker to generate new ones and place them in the queue. But there are some problems I've run into. The biggest one at the moment is how to make sure that in case the queue compleatelly runs out on IDs the main thread waits for the BackroundWorker to generate and place them in the queue. Heres the code that I have at the moment.
public class IdGenerator
{
private Queue<string> mIds = new Queue<string>();
private BackgroundWorker mWorker = new BackgroundWorker();
private static EventWaitHandle mWaitHandle = new AutoResetEvent(false);
public IdGenerator()
{
GenerateIds();
this.mWorker.DoWork += new DoWorkEventHandler(FillQueueWithIds);
}
private void GenerateIds()
{
List<string> ids = new List<string>();
for (int i = 0; i < 100; i++ )
{
ids.Add(Guid.NewGuid().ToString());
}
lock (this.mIds)
{
foreach (string id in ids)
{
this.mIds.Enqueue(id);
}
}
}
public string GetId()
{
string id = string.Empty;
lock (this.mIds)
{
if (this.mIds.Count > 0)
{
id = this.mIds.Dequeue();
}
if (this.mIds.Count < 100)
{
if (!this.mWorker.IsBusy)
{
this.mWorker.RunWorkerAsync();
}
}
}
if (this.mIds.Count < 1)
{
mWaitHandle.WaitOne();
}
return id;
}
void FillQueueWithIds(object sender, DoWorkEventArgs e)
{
GenerateIds();
mWaitHandle.Set();
}
}
Obviously it doesn't work correctly. It seems that I have a problem with proper timing for calling WaitOne and Set methods. And sometimes the IsBusy property returns true even though the worker has already completed his work.
EDIT:
Its a WinForm and I'm required to use .NET 2.0

The problem you have is the classic Producer-Consumer problem. Take a look at http://en.wikipedia.org/wiki/Producer-consumer_problem
A simple explanation is that you will have two threads. One will be the producer (the GUID generator) and the other will be the consumer.
You will keep these threads in synch through the use of semaphores. The semaphore will be the responsible to stop the producer when the queue is full and to stop the consumer when it is empty.
The process is all very well explained at the Wikipedia article and I bet you can find a basic implementation of Producer-Consumer in c# on the internet.

In .NET 4 you can use the BlockingCollection<T> and more generically IProducerConsumerCollection<T>
Here's an example of 2 tasks, one adding and the other taking, using it.
http://msdn.microsoft.com/en-us/library/dd997306.aspx

There are some bugs related to thread sync, see in changed code below.
When you apply lock sync to queue pay attention to put under lock all uses of queue.
I've changed GetId method to probe for new ids if there are none.
public class IdGenerator
{
private Queue<string> mIds = new Queue<string>();
private BackgroundWorker mWorker = new BackgroundWorker();
private static EventWaitHandle mWaitHandle = new AutoResetEvent(false);
public IdGenerator()
{
GenerateIds();
this.mWorker.DoWork += new DoWorkEventHandler(FillQueueWithIds);
}
private void GenerateIds()
{
List<string> ids = new List<string>();
for (int i = 0; i < 100; i++ )
{
ids.Add(Guid.NewGuid().ToString());
}
lock (this.mIds)
{
foreach (string id in ids)
{
this.mIds.Enqueue(id);
}
}
}
public string GetId()
{
string id = string.Empty;
//Indicates if we need to wait
bool needWait = false;
do
{
lock (this.mIds)
{
if (this.mIds.Count > 0)
{
id = this.mIds.Dequeue();
return id;
}
if (this.mIds.Count < 100 && this.mIds.Count > 0)
{
if (!this.mWorker.IsBusy)
{
this.mWorker.RunWorkerAsync();
}
}
else
{
needWait = true;
}
}
if (needWait)
{
mWaitHandle.WaitOne();
needWait = false;
}
} while(true);
return id;
}
void FillQueueWithIds(object sender, DoWorkEventArgs e)
{
GenerateIds();
mWaitHandle.Set();
}
}

Your main code (presumably WinForms) calls mWaitHandle.WaitOne() at a certain point. At that moment the Messagepump is blocked and the Bgw will be unable to call its Completed event. That means the IsBusy flag remain true: deadlock.
Similar issues can arise if code inside DoWork throws an exception.
Edit:
I would think that you could solve most problems by using a ThreadPool thread to replace the Bgw. And a simple volatile bool isbusy flag.

OK, heres the final solution I went with. This one doesn't use the BackgroundWorker, but it works. Thanks to Edu who pointed to the Producer-Consumer problem. I used the example provided by MSDN located here.