Develop Reference

"If two threads are using Pulse and Wait to interact, this could result in a deadlock."

Basically the load() is for the producer(there's one and only one dispatcher thread that loads the _tickQueue) and the Unload is for the consumer(there's one and only one dedicated thread executing the function). _tickQueue is a regular queue protected by a lock(I'm using itself as the argument to lock()). Surprisingly, it caused deadlock.
public void Load(Tick tick)
{
lock (_tickQueue)
{
while (_tickQueue.Count >= CapSize)
{
Monitor.Wait(_tickQueue);
}
_tickQueue.Enqueue(tick);
if (!_receivedTickCounts.ContainsKey(tick.Underlier))
{
_receivedTickCounts.Add(tick.Underlier, 0);
}
Console.WriteLine("Received {1} ticks for {0}", tick.Underlier, ++_receivedTickCounts[tick.Underlier]);
Monitor.Pulse(_tickQueue);
}
}
private void Unload()
{
while (true)
{
try
{
Tick tick;
lock (_tickQueue)
{
while (_tickQueue.Count == 0)
{
Monitor.Wait(_tickQueue);
}
tick = _tickQueue.Dequeue();
Monitor.Pulse(_tickQueue);
}
Persist(tick);
}
catch (Exception e)
{
Console.WriteLine(e);
}
}
}
The comment in the title was found here:
https://msdn.microsoft.com/en-us/library/system.threading.monitor.pulse%28v=vs.110%29.aspx
My understanding of the "Important" paragraph is: Monitor class not maintaining state (in the way ResetEvent does) implies deadlock. A specific example was given: when two threads interact using Pulse and Wait, if one thread pulses when the other thread is not on the wait queue, then deadlock happens.
Can someone SPECIFICALLY(e.g. give a scenario for deadlock to happen) point out where I did wrong in my program? I don't see any scenario that can possibly lead to deadlock.
Thanks.
===================EDIT====================
Specifically, I'm interested to know why the following coding pattern for monitor suddenly doesn't work - must be related to the monitor implementation in .net?
lock
while(wait condition is met)
{
wait()
}
// critical section: doing work
signal();// or broadcast()
unlock

I suspect you are imposing an un-ending wait upon both methods. You are surrounding your Monitor method calls with While loops continually checking a condition. For certain values of CapSize and _tickQueue.Count, both of your Load() and Unload() methods will be forever waiting. What isn't evident here is the value of CapSize, is it constant, or does it change? Is _tickQueue thread-safe?
What if we reach an error on tick = _tickQueue.Dequeue(); in Unload(), _tickQueue.Count reaches 0, and the Load() method was Waiting()'ing? Load() will be waiting forever.
I would avoid having your consumer method Pulse to notify that Producer method it's ready for more work. Your consumer should only be waiting when there is no more work for it to do (queue is empty). Your Producer would be better suited controlling it's own work schedule, and pulsing the consumer when new work has been queued. Why not put the Producer on a Timer?
In the end, I believe the supplied code simply provides too many points of failure. Could I suggest an alternate implementation? This uses the thread-safe ConcurrentQueue collection and eliminates the discussed issues.
public class StackOverflowMonitorExample
{
ConcurrentQueue<Tick> _tickQueue = new ConcurrentQueue<Tick>();
object locker = new object();
bool stopCondition = false;
public void Load(Tick tick)
{
_tickQueue.Enqueue(tick);
lock (locker)
{
Monitor.Pulse(locker);
}
}
private void Unload()
{
while (!stopCondition)
{
try
{
Tick nextWorkItem = null;
_tickQueue.TryDequeue(out nextWorkItem);
if (nextWorkItem != null)
{
Persist(nextWorkItem);
}
else
{
lock (locker)
{
Monitor.Wait(locker);
}
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
}
}
}
This eliminates the large locking sections, and removes most of the signals between the consumer and producer. The Producer will only ever add new items to the queue, and Pulse() to notify that new work is available. The Consumer will loop and continue to work as long as items remain in the queue, and stop condition has not been met. If queue count reaches 0, then the consumer will wait for new queue entries.

Code Help for ThreadPool

Have created a class which implements ThreadPool. The code is as below:
public sealed class PyeThreadPool :
IDisposable
{
private readonly object _lock = new object();
private readonly int _minThreadCount;
private readonly int _maxThreadCount;
private readonly Queue<Action> _queue = new Queue<Action>();
private int _totalThreadCount;
private int _waitingThreadCount;
private bool _disposed;
public PyeThreadPool(int minThreadCount, int maxThreadCount)
{
if (minThreadCount < 0)
throw new ArgumentOutOfRangeException("minThreadCount");
if (maxThreadCount < 1 || maxThreadCount < minThreadCount)
throw new ArgumentOutOfRangeException("maxThreadCount");
_minThreadCount = minThreadCount;
_maxThreadCount = maxThreadCount;
}
public void Dispose()
{
lock (_lock)
{
_disposed = true;
// if there are thread waiting, they should stop waiting.
if (_waitingThreadCount > 0)
Monitor.PulseAll(_lock);
}
}
/// <summary>
/// Executes an action in a parallel thread.
/// </summary>
public void RunParallel(Action action)
{
if (action == null)
throw new ArgumentNullException("action");
lock (_lock)
{
if (_disposed)
throw new ObjectDisposedException(GetType().FullName);
bool queued = false;
if (_waitingThreadCount == 0)
{
if (_totalThreadCount < _maxThreadCount)
{
_totalThreadCount++;
var thread = new Thread(_ThreadRun);
thread.Name = "Worker Thread";
thread.Start(action);
queued = true;
}
}
if (!queued)
{
_queue.Enqueue(action);
Monitor.Pulse(_lock);
}
}
}
private void _ThreadRun(object firstAction)
{
Action action = (Action)firstAction;
firstAction = null;
// we always start a new thread with an action, so we get it immediately.
// but, as we don't know what that action really holds in memory, we set
// the initial action to null, so after it finishes and a new action is get,
// we will let the GC collect it.
while (true)
{
action();
lock (_lock)
{
if (_queue.Count == 0)
{
// we started waiting, so new threads don't need to be created.
_waitingThreadCount++;
while (_queue.Count == 0)
{
if (_disposed)
return;
if (_totalThreadCount > _minThreadCount)
{
_totalThreadCount--;
_waitingThreadCount--;
return;
}
action = null;
Monitor.Wait(_lock);
}
// we finished waiting.
_waitingThreadCount--;
}
action = _queue.Dequeue();
// we just get a new action, and we will release the lock and return
// to the while, where the action will be executed.
}
}
}
}
I have tried to use this and the test code is as:
PyeThreadPool MyPool;
int x = 1;
protected void Page_Load(object sender, EventArgs e)
{
MyPool = new PyeThreadPool(4, 6);
}
void showMessage(string message)
{
TxtMessage.Text = message;
}
protected void BtnStartThread_Click(object sender, EventArgs e)
{
x++;
int arg = x;
MyPool.RunParallel(() =>
{
showMessage(arg.ToString());
});
}
Problem is:
(1) When I execute this either in debug or release mode I do not see the result in textbox, on the other hand I see the result when I step through. What am I missing here, why I can not see the output.
(2) The RunParallel method shows only one thread even if I have set maxcount to more than 1. Is there any code logic missing or is it because the test application is simple?
Thanks !

You should have a look at SmartThreadPool library. It is one of the best alternative to ThreadPool.
Its features (copied from source link)
Smart Thread Pool is a thread pool written in C#. The implementation was first based on Stephan Toub's thread pool with some extra features, but now, it is far beyond the original. Here is a list of the thread pool features:
The number of threads dynamically changes according to the workload on the threads in the pool.
Work items can return a value.
A work item can be cancelled if it hasn't been executed yet.
The caller thread's context is used when the work item is executed (limited).
Usage of minimum number of Win32 event handles, so the handle count of the application won't explode.
The caller can wait for multiple or all the work items to complete.
A work item can have a PostExecute callback, which is called as soon the work item is completed.
The state object that accompanies the work item can be disposed automatically.
Work item exceptions are sent back to the caller.
Work items have priority.
Work items group.
The caller can suspend the start of a thread pool and work items group.
Threads have priority.
Threads have initialization and termination events.
WinCE platform is supported (limited).
Action and Func generic methods are supported.
Silverlight is supported.
Mono is supported.
Performance counters (Windows and internal).
Work item timeout (passive).
Threads ApartmentState
Threads IsBakcground
Threads name template
Windows Phone is supported (limited)
Threads MaxStackSize

The problem is you are attempting to update a UI control from a background thread. Not allowed.
You need to do a BeginInvoke or Invoke in your ShowMessage function.

Waking a thread in C#

I am looking for a simple way to put a thread to sleep and to wake it. The thread runs in background in an infinite loop and sometimes does some work, sometimes just runs through. I have found out that there is no corresponding Wait() to the Sleep() and waking a thread with Interrupt() causes an exception. Apparently a sleeping thread is not meant to be disturbed.
Since I know when the work appears it seems a good idea to tell the thread, instead of having it check over and over again.
How can a thread be put to a 'lighter sleep' to be able to wake up alone each second or at a command from other thread?
//Thread to put to sleep and wake (thread1)
while (true)
{
if (thereIsWork)
{ DoWork(); }
//put thread to sleep in a way that other threads can wake it, and it wakes alone after some time (eg. 1000 ms)
// Thread.Sleep(1000); //nice, but not working as desired
}
-
//Other thread:
thereIsWork = true;
//thread1.Wake(); //Not existing

You can use an AutoResetEvent for this - just call Set() to signal work needs to be done and have your thread wait for it to be called using WaitOne().
This means the threads that are communicating this way share the same AutoResetEvent instance - you can pass it in as a dependency for the thread that does the actual work.

The thread shouldn't Sleep(), it should call WaitOne() on an AutoResetEvent or ManualResetEvent until some other thread calls Set() on that same resetevent object.

How about using a blocking queue, with Monitor Pulse and Wait:
class BlockingQueue<T>
{
private Queue<T> _queue = new Queue<T>();
public void Enqueue(T data)
{
if (data == null) throw new ArgumentNullException("data");
lock (_queue)
{
_queue.Enqueue(data);
Monitor.Pulse(_queue);
}
}
public T Dequeue()
{
lock (_queue)
{
while (_queue.Count == 0) Monitor.Wait(_queue);
return _queue.Dequeue();
}
}
}
Then thread 1 becomes
BlockingQueue<Action> _workQueue = new BlockingQueue<Action>();
while (true)
{
var workItem = _workQueue.Dequeue();
workItem();
}
And the other thread:
_workQueue.Enqueue(DoWork);
NB: you should probably use the built in type if you're using .Net 4 BlockingCollection using Add and Take instead of Enqueue and Dequeue.
Edit:
Ok. If you want it really simple...
//Thread to put to sleep and wake (thread1)
while (true)
{
lock(_lock)
{
while (!thereIsWork) Monitor.Wait(_lock);
DoWork();
}
//put thread to sleep in a way that other threads can wake it, and it wakes alone after some time (eg. 1000 ms)
// Thread.Sleep(1000); //nice, but not working as desired
}
and
//Other thread:
lock(_lock)
{
thereIsWork = true;
//thread1.Wake(); //Not existing
Monitor.Pulse(_lock);
}

I'n not an expert with threads, but maybe EventWaitHandle is what you're looking for. Check this link

Implementing a thread queue/wait, how?

I have a timer calling a function every 15 minutes, this function counts the amount of lines in my DGV and starts a thread for each lines (of yet another function), said thread parse a web page which can take anywhere from 1 second to 10 second to finish.
Whilst it does work fine as it is with 1-6 rows, anymore will cause the requests to time-out.
I want it to wait for the newly created thread to finish processing before getting back in the loop to create another thread without locking the main UI
for (int x = 0; x <= dataGridFollow.Rows.Count - 1; x++)
{
string getID = dataGridFollow.Rows[x].Cells["ID"].Value.ToString();
int ID = int.Parse(getID);
Thread t = new Thread(new ParameterizedThreadStart(UpdateLo));
t.Start(ID);
// <- Wait for thread to finish here before getting back in the for loop
}
I have googled a lot in the past 24 hours, read a lot about this specific issue and its implementations (Thread.Join, ThreadPools, Queuing, and even SmartThreadPool).
It's likely that I've read the correct answer somewhere but I'm not at ease enough with C# to decypher those Threading tools
Thanks for your time

to avoid the UI freeze the framework provide a class expressly for these purposes: have a look at the BackgroundWorker class (executes an operation on a separate thread), here's some infos : http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx
http://msdn.microsoft.com/en-us/magazine/cc300429.aspx
Btw looks if I understand correctly you don't want to parallelize any operation so just wait for the method parsing the page to be completed. Basically for each (foreach look) row of your grid you get the id and call the method. If you want to go parallel just reuse the same foreach loop and add make it Parallel
http://msdn.microsoft.com/en-us/library/dd460720.aspx

What you want is to set off a few workers that do some task.
When one finishes you can start a new one off.
I'm sure there is a better way using thread pools or whatever.. but I was bored so i came up with this.
using System;
using System.Collections.Generic;
using System.Linq;
using System.ComponentModel;
using System.Threading;
namespace WorkerTest
{
class Program
{
static void Main(string[] args)
{
WorkerGroup workerGroup = new WorkerGroup();
Console.WriteLine("Starting...");
for (int i = 0; i < 100; i++)
{
var work = new Action(() =>
{
Thread.Sleep(1000); //somework
});
workerGroup.AddWork(work);
}
while (workerGroup.WorkCount > 0)
{
Console.WriteLine(workerGroup.WorkCount);
Thread.Sleep(1000);
}
Console.WriteLine("Fin");
Console.ReadLine();
}
}
public class WorkerGroup
{
private List<Worker> workers;
private Queue<Action> workToDo;
private object Lock = new object();
public int WorkCount { get { return workToDo.Count; } }
public WorkerGroup()
{
workers = new List<Worker>();
workers.Add(new Worker());
workers.Add(new Worker());
foreach (var w in workers)
{
w.WorkCompleted += (OnWorkCompleted);
}
workToDo = new Queue<Action>();
}
private void OnWorkCompleted(object sender, EventArgs e)
{
FindWork();
}
public void AddWork(Action work)
{
workToDo.Enqueue(work);
FindWork();
}
private void FindWork()
{
lock (Lock)
{
if (workToDo.Count > 0)
{
var availableWorker = workers.FirstOrDefault(x => !x.IsBusy);
if (availableWorker != null)
{
var work = workToDo.Dequeue();
availableWorker.StartWork(work);
}
}
}
}
}
public class Worker
{
private BackgroundWorker worker;
private Action work;
public bool IsBusy { get { return worker.IsBusy; } }
public event EventHandler WorkCompleted;
public Worker()
{
worker = new BackgroundWorker();
worker.DoWork += new DoWorkEventHandler(OnWorkerDoWork);
worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(OnWorkerRunWorkerCompleted);
}
private void OnWorkerRunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
if (WorkCompleted != null)
{
WorkCompleted(this, EventArgs.Empty);
}
}
public void StartWork(Action work)
{
if (!IsBusy)
{
this.work = work;
worker.RunWorkerAsync();
}
else
{
throw new InvalidOperationException("Worker is busy");
}
}
private void OnWorkerDoWork(object sender, DoWorkEventArgs e)
{
work.Invoke();
work = null;
}
}
}
This would be just a starting point.
You could start it off with a list of Actions and then have a completed event for when that group of actions is finished.
then at least you can use a ManualResetEvent to wait for the completed event.. or whatever logic you want really.

Call a method directly or do a while loop (with sleep calls) to check the status of the thread.
There are also async events but the would call another method, and you want to continue from the same point.

I have no idea why the requests would timeout. That sounds like a different issue. However, I can make a few suggestions regarding your current approach.
Avoid creating threads in loops with nondeterministic bounds. There is a lot of overhead in creating threads. If the number of operations is not known before hand then use the ThreadPool or the Task Parallel Library instead.
You are not going to get the behavior you want by blocking the UI thread with Thread.Join. The cause the UI to become unresponsive and it will effectively serialize the operations and cancel out any advantage you were hoping to gain with threads.
If you really want to limit the number of concurrent operations then a better solution is to create a separate dedicated thread for kicking off the operations. This thread will spin around a loop indefinitely waiting for items to appear in a queue and when they do it will dequeue them and use that information to kick off an operation asynchronously (again using the ThreadPool or TPL). The dequeueing thread can contain the logic for limiting the number of concurrent operations. Search for information regarding the producer-consumer pattern to get a better understand of how you can implement this.
There is a bit of a learning curve, but who said threading was easy right?

If I understand correctly, what you're currently doing is looping through a list of IDs in the UI thread, starting a new thread to handle each one. The blocking issue you're seeing then could well be that it's taking too many resources to create unique threads. So, personally (without knowing more) would redesign the process like so:
//Somewhere in the UI Thread
Thread worker = new Thread(new ParameterizedThreadStart(UpdateLoWorker));
worker.Start(dataGridFollow.Rows);
//worker thread
private void UpdateLoWorker(DataRowCollection rows)
{
foreach(DataRow r in rows){
string getID = r.Cells["ID"].Value.ToString();
int ID = int.Parse(getID);
UpdateLo(ID);
}
}
Here you'd have a single non-blocking worker which sequentially handles each ID.

Consider using Asynchronous CTP. It's an asynch pattern Microsoft recently released for download. It should simplify asynch programming tremendouesly. The link is http://msdn.microsoft.com/en-us/vstudio/async.aspx. (Read the whitepaper first)
Your code would look something like the following. (I've not verified my syntax yet, sorry).
private async Task DoTheWork()
{
for(int x = 0; x <= dataGridFollow.Rows.Count - 1; x++)
{
string getID = dataGridFollow.Rows[x].Cells["ID"].Value.ToString();
int ID = int.Parse(getID);
task t = new Task(new Action<object>(UpdateLo), ID);
t.Start();
await t;
}
}
This method returns a Task that can be checked periodically for completion. This follows the pattern of "fire and forget" meaning you just call it and presumably, you don't care when it completes (as long as it does complete before 15 minutes).
EDIT
I corrected the syntax above, you would need to change UpdateLo to take an object instead of an Int.

For a simple background thread runner that will run one thread from a queue at a time you can do something like this:
private List<Thread> mThreads = new List<Thread>();
public static void Main()
{
Thread t = new Thread(ThreadMonitor);
t.IsBackground = true;
t.Start();
}
private static void ThreadMonitor()
{
while (true)
{
foreach (Thread t in mThreads.ToArray())
{
// Runs one thread in the queue and waits for it to finish
t.Start();
mThreads.Remove(t);
t.Join();
}
Thread.Sleep(2000); // Wait before checking for new threads
}
}
// Called from the UI or elsewhere to create any number of new threads to run
public static void DoStuff()
{
Thread t = new Thread(DoCorestuff);
t.IsBackground = true;
mActiveThreads.Add(t);
}
public static void DoStuffCore()
{
// Your code here
}

What is most CPU efficient method to make the worker threads wait for tasks?

In my current C#/NET 3.5 application, I have a task queue (thread safe) and I have 5 worker threads that has to constantly look for tasks in the queue. If a task is available, any one worker will dequeue the task and take required action.
My worker thread class is as follows:
public class WorkerThread
{
//ConcurrentQueue is my implementation of thread safe queue
//Essentially just a wrapper around Queue<T> with synchronization locks
readonly ConcurrentQueue<CheckPrimeTask> mQ;
readonly Thread mWorker;
bool mStop;
public WorkerThread (ConcurrentQueue<CheckPrimeTask> aQ) {
mQ = aQ;
mWorker = new Thread (Work) {IsBackground = true};
mStop = false;
}
private void Work () {
while (!mStop) {
if (mQ.Count == 0) {
Thread.Sleep (0);
continue;
}
var task = mQ.Dequeue ();
//Someone else might have been lucky in stealing
//the task by the time we dequeued it!!
if (task == null)
continue;
task.IsPrime = IsPrime (task.Number);
task.ExecutedBy = Thread.CurrentThread.ManagedThreadId;
//Ask the threadpool to execute the task callback to
//notify completion
ThreadPool.QueueUserWorkItem (task.CallBack, task);
}
}
private bool IsPrime (int number) {
int limit = Convert.ToInt32 (Math.Sqrt (number));
for (int i = 2; i <= limit; i++) {
if (number % i == 0)
return false;
}
return true;
}
public void Start () {
mStop = false;
mWorker.Start ();
}
public void Stop () {
mStop = true;
}
}
Problem is that when queue is empty, it consumes too much CPU (nearly 98%). I tried AutoResetEvent to notify the workers that queue has been changed. So they effectively wait for that signal to set. It has braught down the CPU to nearly 0% but I am not entirely sure whether this is the best method. Can you suggest a better method to keep the threads idle without hurting CPU usage?

Check out this implementation of a BlockingQueue. If the queue is empty, it uses Monitor.Wait() to put the thread to sleep. When an item is added, it uses Monitor.Pulse() to wake up a thread that is sleeping on the empty queue.
Another technique is to use a semaphore. Each time you add an item to a queue, call Release(). When you want an item from a queue, call WaitOne().

You currently have Thread.Sleep(0) in your Work method for where there are no queue items. Change it to anything greater than 0 and your CPU use will go down. Try 10 to start with...

You have a couple of options that I can think of.
One way is to place a small thread sleep during your loop. This will basically drop your CPU usage to 0 and is fairly standard way of doing this.
Another way is to use a reset (either auto or manual) as suggested by Mitch Wheat in the comments.
You could also devise some kind of IdleTask that has a thread sleep for a certain amount of time and if your queue is empty, just process the IdleTask (which will thread sleep).

If your Queue is thread safe then you would not need to do this...
//Someone else might have been lucky in stealing
//the task by the time we dequeued it!!
if (task == null)
continue;