I've reused the example producer consumer queue from the C# in a Nutshell book of Albahari (http://www.albahari.com/threading/part5.aspx#_BlockingCollectionT) and a colleague remarked:
"Why isn't the Dispose called on the BlockingCollection in the Dispose of the collection?"
I couldn't find an answer and the only reason I can come up with is that execution of the remaining workload of the queue wouldn't be processed. However, when I'm disposing the queue, why wouldn't it stop processing?
Besides the "Why you shouldn't Dispose the BlockingCollection?" I've got also a second question "Does it harm if you don't dispose a BlockingCollection?". I suppose when you are spawning/disposing a lot of producer consumer queues it gives problems (not that I want that but just for the cause of knowing).
According to What does BlockingCollection.Dispose actually do? BlockingCollection contains two wait handles (obviously) so not calling Dispose will give you some problems. Thanks ken2k for pointing this out.
The code I'm talking about:
public class PCQueue : IDisposable
{
BlockingCollection<Action> _taskQ = new BlockingCollection<Action>();
public PCQueue (int workerCount)
{
// Create and start a separate Task for each consumer:
for (int i = 0; i < workerCount; i++)
Task.Factory.StartNew (Consume);
}
public void Dispose() { _taskQ.CompleteAdding(); }
public void EnqueueTask (Action action) { _taskQ.Add (action); }
void Consume()
{
// This sequence that we’re enumerating will block when no elements
// are available and will end when CompleteAdding is called.
foreach (Action action in _taskQ.GetConsumingEnumerable())
action(); // Perform task.
}
}
Because that would be a bug. The collection cannot be disposed until all the consumer threads have exited. If that's not interlocked then those threads would bomb with an exception. The class does not in any way have awareness of what consumer threads might be pulling from the collection so it cannot reasonably know when it is safe to dispose. All it can do is prevent any more objects from being added by the producer, that's reasonable.
This is a common problem with threads, safely disposing requires knowing when the thread is complete. Which often defeats the point of using threads in the first place, you don't want to wait until a thread ends. This is most visible in the Thread class itself, it consumes five native operating system handles but doesn't have a Dispose() method. They need to be released by the finalizer. Same here.
Related
I have an infinite loop that is used to consume items from a BlockingCollection.
public class MessageFileLogger
{
private BlockingCollection<ILogItem> _messageQueue;
private Thread _worker;
private bool _enabled = false;
public MessageFileLogger()
{
_worker = new Thread(LogMessage);
_worker.IsBackground = true;
_worker.Start();
}
private void LogMessage()
{
while (_enabled)
{
if (_messageQueue.Count > 0)
{
itm = _messageQueue.Take();
processItem(itm);
}
else
{
Thread.Sleep(1000);
}
}
}
}
which is referenced by another object that gets instantiated every minute or couple of minutes (could be moved out to 1 hour increments or such).
public class Helper
{
MessageFileLogger _logger;
public Helper(string logFilePath, LogMode logMode)
{
_logger = new MessageFileLogger(logFilePath, logMode);
_logger.Enabled = true;
}
public void foo()
{
}
}
Question #1)
What can I do to ensure that the thread is exited when the object that references it is no longer needed?
Note: Helper only needs to call foo, so once it no longer needs to call foo, the object can be garbage collected. So, incorporating a using statement with Helper is certainly a possibility.
Question #2)
Does _messageQueue need to be disposed? If so, how do I dispose of it without it affecting the LogMessage thread? (I tried disposing of it while the thread was running and no surprise got an error).
I tried extending IDisposable (in MessageFileLogger):
public void Dispose()
{
_enabled = false;
_messageQueue.Dispose();
}
and I haven't had any issues with this but I'm not confident that I just haven't had an issue yet. Also, would this mean that Helper also needs to IDisposable and a using statement needs to be used with Helper?
Note: This question is based on the same code I had with another question of mine.
First off, your consumer shouldn't be calling Thread.Sleep. It also most certainly shouldn't be checking the count of the collection. The whole point of BlockingCollection is that when you call Take, it either gives you and item, or it waits until there is an item to give you, and then gives it to you. So you can just keep calling Take in a loop with nothing else. This prevents you from waiting some fraction of a second when there is already an item you could be processing.
Better still, you can simply use GetConsumingEnumerable to get a sequence of items.
Your consumer can now look like this:
foreach(var item in _messageQueue.GetConsumingEnumerable())
processItem(item);
Additionally, BlockingCollection has built in support for indicating that the queue is done. Simply have the producer call CompleteAdding to indicate that no more items will be added. After doing that, once the queue is empty, the Enumerable will end, and the foreach loop will finish. The consumer can do any clean up it needs to at that point in time.
In addition to the fact that using the BlockingCollection to determine when you're done is just generally more convenient, it's also correct, unlike your code. Since _enabled isn't volatile, even though you're reading and writing to it from different threads, you're not introducing the proper memory barriers, so the consumer is likely to be reading a stale value of that variable for some time. When you use mechanisms from the BCL specifically designed to handle these types of multithreaded situations you can be sure that they'll be handled properly on your behalf, without you needing to think about them.
(question revised): So far, the answers all include a single thread re-entering the lock region linearly, through things like recursion, where you can trace the steps of a single thread entering the lock twice. But is it possible somehow, for a single thread (perhaps from the ThreadPool, perhaps as a result of timer events or async events or a thread going to sleep and being awaken/reused in some other chunk of code separately) to somehow be spawned in two different places independently of each other, and hence, run into the lock re-entrance problem when the developer didn't expect it by simply reading their own code?
In the ThreadPool Class Remarks (click here) the Remarks seem to suggest that sleeping threads should be reused when they're not in use, or otherwise wasted by sleeping.
But on the Monitor.Enter reference page (click here) they say "It is legal for the same thread to invoke Enter more than once without it blocking." So I figure there must be something I'm supposed to be careful to avoid. What is it? How is it even possible for a single thread to enter the same lock region twice?
Suppose you have some lock region that takes an unfortunately long time. This might be realistic, for example, if you access some memory that has been paged out (or whatever.) The thread in the locked region might go to sleep or something. Does the same thread become eligible to run more code, which might accidentally step into the same lock region? The following does NOT, in my testing, get multiple instances of the same thread to run into the same lock region.
So how does one produce the problem? What exactly do you need to be careful to avoid?
class myClass
{
private object myLockObject;
public myClass()
{
this.myLockObject = new object();
int[] myIntArray = new int[100]; // Just create a bunch of things so I may easily launch a bunch of Parallel things
Array.Clear(myIntArray, 0, myIntArray.Length); // Just create a bunch of things so I may easily launch a bunch of Parallel things
Parallel.ForEach<int>(myIntArray, i => MyParallelMethod());
}
private void MyParallelMethod()
{
lock (this.myLockObject)
{
Console.Error.WriteLine("ThreadId " + Thread.CurrentThread.ManagedThreadId.ToString() + " starting...");
Thread.Sleep(100);
Console.Error.WriteLine("ThreadId " + Thread.CurrentThread.ManagedThreadId.ToString() + " finished.");
}
}
}
Suppose you have a queue that contains actions:
public static Queue<Action> q = whatever;
Suppose Queue<T> has a method Dequeue that returns a bool indicating whether the queue could be successfully dequeued.
And suppose you have a loop:
static void Main()
{
q.Add(M);
q.Add(M);
Action action;
while(q.Dequeue(out action))
action();
}
static object lockObject = new object();
static void M()
{
Action action;
lock(lockObject)
{
if (q.Dequeue(out action))
action();
}
}
Clearly the main thread enters the lock in M twice; this code is re-entrant. That is, it enters itself, through an indirect recursion.
Does this code look implausible to you? It should not. This is how Windows works. Every window has a message queue, and when a message queue is "pumped", methods are called corresponding to those messages. When you click a button, a message goes in the message queue; when the queue is pumped, the click handler corresponding to that message gets invoked.
It is therefore extremely common, and extremely dangerous, to write Windows programs where a lock contains a call to a method which pumps a message loop. If you got into that lock as a result of handling a message in the first place, and if the message is in the queue twice, then the code will enter itself indirectly, and that can cause all manner of craziness.
The way to eliminate this is (1) never do anything even slightly complicated inside a lock, and (2) when you are handling a message, disable the handler until the message is handled.
Re-Entrance is possible if you have a structure like so:
Object lockObject = new Object();
void Foo(bool recurse)
{
lock(lockObject)
{
Console.WriteLine("In Lock");
if (recurse) { foo(false); }
}
}
While this is a pretty simplistic example, it's possible in many scenarios where you have interdependent or recursive behaviour.
For example:
ComponentA.Add(): locks a common 'ComponentA' object, adds new item to ComponentB.
ComponentB.OnNewItem(): new item triggers data-validation on each item in list.
ComponentA.ValidateItem(): locks a common 'ComponentA' object to validate the item.
Same-thread re-entry on the same lock is needed to ensure you don't get deadlocks occurring with your own code.
One of the more subtle ways you can recurse into a lock block is in GUI frameworks. For example, you can asynchronously invoke code on a single UI thread (a Form class)
private object locker = new Object();
public void Method(int a)
{
lock (locker)
{
this.BeginInvoke((MethodInvoker) (() => Method(a)));
}
}
Of course, this also puts in an infinite loop; you'd likely have a condition by which you'd want to recurse at which point you wouldn't have an infinite loop.
Using lock is not a good way to sleep/awaken threads. I would simply use existing frameworks like Task Parallel Library (TPL) to simply create abstract tasks (see Task) to creates and the underlying framework handles creating new threads and sleeping them when needed.
IMHO, Re-entering a lock is not something you need to take care to avoid (given many people's mental model of locking this is, at best, dangerous, see Edit below). The point of the documentation is to explain that a thread cannot block itself using Monitor.Enter. This is not always the case with all synchronization mechanisms, frameworks, and languages. Some have non-reentrant synchronization in which case you have to be careful that a thread doesn't block itself. What you do need to be careful about is always calling Monitor.Exit for every Monitor.Enter call. The lock keyword does this for you automatically.
A trivial example with re-entrance:
private object locker = new object();
public void Method()
{
lock(locker)
{
lock(locker) { Console.WriteLine("Re-entered the lock."); }
}
}
The thread has entered the lock on the same object twice so it must be released twice. Usually it is not so obvious and there are various methods calling each other that synchronize on the same object. The point is that you don't have to worry about a thread blocking itself.
That said you should generally try to minimize the amount the time you need to hold a lock. Acquiring a lock is not computationally expensive, contrary to what you may hear (it is on the order of a few nanoseconds). Lock contention is what is expensive.
Edit
Please read Eric's comments below for additional details, but the summary is that when you see a lock your interpretation of it should be that "all activations of this code block are associated with a single thread", and not, as it is commonly interpreted, "all activations of this code block execute as a single atomic unit".
For example:
public static void Main()
{
Method();
}
private static int i = 0;
private static object locker = new object();
public static void Method()
{
lock(locker)
{
int j = ++i;
if (i < 2)
{
Method();
}
if (i != j)
{
throw new Exception("Boom!");
}
}
}
Obviously, this program blows up. Without the lock, it is the same result. The danger is that the lock leads you into a false sense of security that nothing could modify state on you between initializing j and evaluating the if. The problem is that you (perhaps unintentionally) have Method recursing into itself and the lock won't stop that. As Eric points out in his answer, you might not realize the problem until one day someone queues up too many actions simultaneously.
ThreadPool threads cannot be reused elsewhere just because they went to sleep; they need to finish before they're reused. A thread that is taking a long time in a lock region does not become eligible to run more code at some other independent point of control. The only way to experience lock re-entry is by recursion or executing methods or delegates inside a lock that re-enter the lock.
Let's think about something other than recursion.
In some of business logics, they would like to control the behaviors of synchronization.
One of these patterns, they invoke Monitor.Enter somewhere and would like to invoke Monitor.Exit elsewhere later. Here is the code to get the idea about that:
public partial class Infinity: IEnumerable<int> {
IEnumerator IEnumerable.GetEnumerator() {
return this.GetEnumerator();
}
public IEnumerator<int> GetEnumerator() {
for(; ; )
yield return ~0;
}
public static readonly Infinity Enumerable=new Infinity();
}
public partial class YourClass {
void ReleaseLock() {
for(; lockCount-->0; Monitor.Exit(yourLockObject))
;
}
void GetLocked() {
Monitor.Enter(yourLockObject);
++lockCount;
}
void YourParallelMethod(int x) {
GetLocked();
Debug.Print("lockCount={0}", lockCount);
}
public static void PeformTest() {
new Thread(
() => {
var threadCurrent=Thread.CurrentThread;
Debug.Print("ThreadId {0} starting...", threadCurrent.ManagedThreadId);
var intanceOfYourClass=new YourClass();
// Parallel.ForEach(Infinity.Enumerable, intanceOfYourClass.YourParallelMethod);
foreach(var i in Enumerable.Range(0, 123))
intanceOfYourClass.YourParallelMethod(i);
intanceOfYourClass.ReleaseLock();
Monitor.Exit(intanceOfYourClass.yourLockObject); // here SynchronizationLockException thrown
Debug.Print("ThreadId {0} finished. ", threadCurrent.ManagedThreadId);
}
).Start();
}
object yourLockObject=new object();
int lockCount;
}
If you invoke YourClass.PeformTest(), and get a lockCount greater than 1, you've reentered; not necessarily be concurrent.
If it was not safe for reentrancy, you will get stuck in the foreach loop.
In the code block where Monitor.Exit(intanceOfYourClass.yourLockObject) will throw you a SynchronizationLockException, it is because we are trying to invoke Exit more than the times it have entered. If you are about to use the lock keyword, you possibly would not encounter this situation except directly or indirectly of recursive calls. I guess that's why the lock keyword was provided: it prevents the Monitor.Exit to be omitted in a careless manner.
I remarked the calling of Parallel.ForEach, if you are interested then you can test it for fun.
To test the code, .Net Framework 4.0 is the least requirement, and following additional name spaces are required, too:
using System.Threading.Tasks;
using System.Diagnostics;
using System.Threading;
using System.Collections;
Have fun.
I have a producer-consumer scenario in ASP.NET. I designed a Producer class, a Consumer class and a class for holding the shared objects and responsible for communication between Producer and Consumer, lets call it Mediator. Because I fork the execution path at start-up (in parent object) and one thread would call Producer.Start() and another thread calls Consumer.Start(), I need to pass a reference of Mediator to both Producer and Consumer (via Constructor). Mediator is a smart class which will optimize many things like length of it's inner queue but for now consider it as a circular blocking queue. Producer would enqueues new objects to Mediator until the queue gets full and then Producer would block. Consumer dequeues objects from Mediator until there's nothing in the queue. For signaling between threads, I implemented two methods in Mediator class: Wait() and Pulse(). The code is something like this:
Class Mediator
{
private object _locker = new object();
public void Wait()
{
lock(_locker)
Monitor.Wait(_locker);
}
public void Pulse()
{
lock(_locker)
Monitor.Pulse(_locker);
}
}
// This way threads are signaling:
Class Consumer
{
object x;
if (Mediator.TryDequeue(out x))
// Do something
else
Mediator.Wait();
}
Inside Mediator I use this.Pulse() every time something is Enqueued or Dequeued so waiting threads would be signaled and continue their work.
But I encounter deadlocks and because I have never used this kind of design for signaling threads, I'm not sure if something is wrong with the design or I'm doing something wrong elsewhere ?
Thanks
There is not much code here to go on, but my best guess is that you have a live-lock problem. If Mediator.Pulse is called before Mediator.Wait then the signal gets lost even though there is something in the queue. Here is the standard pattern for implementing the blocking queue.
public class BlockingQueue<T>
{
private Queue<T> m_Queue = new Queue<T>();
public void Enqueue(T item)
{
lock (m_Queue)
{
m_Queue.Enqueue(item);
Monitor.Pulse(m_Queue);
}
}
public T Dequeue()
{
lock (m_Queue)
{
while (m_Queue.Count == 0)
{
Monitor.Wait(m_Queue);
}
return m_Queue.Dequeue();
}
}
}
Notice how Monitor.Wait is only called when the queue is empty. Also notice how it is being called in a while loop. This is because a Wait does not have priority over a Enter so a new thread coming into Dequeue could take the last item even though a call to Wait is ready to return. Without the loop a thread could attempt to remove an item from an empty queue.
If you can use .NET 4 your best bet would be to use BlockingCollection<T> (http://msdn.microsoft.com/en-us/library/dd267312.aspx) which handles queueing, dequeuing, and limits on queue length.
Nothing is wrong with design.
Problem raises when you use Monitor.Wait() and Monitor.Pulse() when you don't know which thread is going to do it's job first (producer or consumer). In that case using an AutoResetEvent resolves the problem. Think of consumer when it reaches the section where it should consume the data produced by producer. Maybe it reaches there before producer pulse it, then everything is OK but what if consumer reaches there after producer has signaled. Yes, then you encounter a deadlock because producer already called Monitor.Pulse() for that section and would not repeat it.
Using AutoResetEvent you sure consumer waits there for signal from producer and if producer already has signaled before consumer even reaches the section, the gate is open and consumer would continue.
It's OK to use Monitor.Wait() and Monitor.Pulse() inside Mediator for signaling waiting threads.
Is it possible that the deadlock is occurring because Pulse doesn't store any state? This means that if the Producer calls Pulse before/after Consumer calls Wait, then the Wait will block. This is the note in the documentation for Monitor.Pulse
Also, you should know that object x = new object(); is extraneous - an out call will initialize x, so the object created will fall out of scope with the TryDequeue call.
Difficult to tell with the code sample supplied.
Is the lock held elsewhere? Within Mediator?
Are the threads just parked on obtaining the lock and not on the actual Wait call?
Have you paused the threads in a debugger to see what the current state is?
Have you tried a simple test with just putting a simple single value on a queue and getting it to work? Or is Mediator pretty complex at this point?
Until a little more detail is available in the Mediator class and your producer class, it's some wild guessing. It seems like some thread may be holding the lock when you don't expect it to. Once you pulse, you do need to free the lock in whatever thread may have it by exiting the "lock" scope. So, if somewhere in Mediator you have the lock and then call Pulse, you need to exit the outer most scope where the lock is held and not just the one in Pulse.
Can you refactor to a normal consumer/ producer queue? That could then handle enqueing and dequing and thread-signalling in a single class, so no need to pass around public locks. Dequeing process could then be handled via a delegate. I can post an example if you wish.
I have developed a generic producer-consumer queue which pulses by Monitor in the following way:
the enqueue :
public void EnqueueTask(T task)
{
_workerQueue.Enqueue(task);
Monitor.Pulse(_locker);
}
the dequeue:
private T Dequeue()
{
T dequeueItem;
if (_workerQueue.Count > 0)
{
_workerQueue.TryDequeue(out dequeueItem);
if(dequeueItem!=null)
return dequeueItem;
}
while (_workerQueue.Count == 0)
{
Monitor.Wait(_locker);
}
_workerQueue.TryDequeue(out dequeueItem);
return dequeueItem;
}
the wait section produces the following SynchronizationLockException :
"object synchronization method was called from an unsynchronized block of code"
do i need to synch it? why ? Is it better to use ManualResetEvents or the Slim version of .NET 4.0?
Yes, the current thread needs to "own" the monitor in order to call either Wait or Pulse, as documented. (So you'll need to lock for Pulse as well.) I don't know the details for why it's required, but it's the same in Java. I've usually found I'd want to do that anyway though, to make the calling code clean.
Note that Wait releases the monitor itself, then waits for the Pulse, then reacquires the monitor before returning.
As for using ManualResetEvent or AutoResetEvent instead - you could, but personally I prefer using the Monitor methods unless I need some of the other features of wait handles (such as atomically waiting for any/all of multiple handles).
From the MSDN description of Monitor.Wait():
Releases the lock on an object and blocks the current thread until it reacquires the lock.
The 'releases the lock' part is the problem, the object isn't locked. You are treating the _locker object as though it is a WaitHandle. Doing your own locking design that's provably correct is a form of black magic that's best left to our medicine man, Jeffrey Richter and Joe Duffy. But I'll give this one a shot:
public class BlockingQueue<T> {
private Queue<T> queue = new Queue<T>();
public void Enqueue(T obj) {
lock (queue) {
queue.Enqueue(obj);
Monitor.Pulse(queue);
}
}
public T Dequeue() {
T obj;
lock (queue) {
while (queue.Count == 0) {
Monitor.Wait(queue);
}
obj = queue.Dequeue();
}
return obj;
}
}
In most any practical producer/consumer scenario you will want to throttle the producer so it cannot fill the queue unbounded. Check Duffy's BoundedBuffer design for an example. If you can afford to move to .NET 4.0 then you definitely want to take advantage of its ConcurrentQueue class, it has lots more black magic with low-overhead locking and spin-waiting.
The proper way to view Monitor.Wait and Monitor.Pulse/PulseAll is not as providing a means of waiting, but rather (for Wait) as a means of letting the system know that the code is in a waiting loop which can't exit until something of interest changes, and (for Pulse/PulseAll) as a means of letting the system know that code has just changed something that might cause satisfy the exit condition some other thread's waiting loop. One should be able to replace all occurrences of Wait with Sleep(0) and still have code work correctly (even if much less efficiently, as a result of spending CPU time repeatedly testing conditions that haven't changed).
For this mechanism to work, it is necessary to avoid the possibility of the following sequence:
The code in the wait loop tests the condition when it isn't satisfied.
The code in another thread changes the condition so that it is satisfied.
The code in that other thread pulses the lock (which nobody is yet waiting on).
The code in the wait loop performs a Wait since its condition wasn't satisfied.
The Wait method requires that the waiting thread have a lock, since that's the only way it can be sure that the condition it's waiting upon won't change between the time it's tested and the time the code performs the Wait. The Pulse method requires a lock because that's the only way it can be sure that if another thread has "committed" itself to performing a Wait, the Pulse won't occur until after the other thread actually does so. Note that using Wait within a lock doesn't guarantee that it's being used correctly, but there's no way that using Wait outside a lock could possibly be correct.
The Wait/Pulse design actually works reasonably well if both sides cooperate. The biggest weaknesses of the design, IMHO, are (1) there's no mechanism for a thread to wait until any of a number of objects is pulsed; (2) even if one is "shutting down" an object such that all future wait loops should exit immediately (probably by checking an exit flag), the only way to ensure that any Wait to which a thread has committed itself will get a Pulse is to acquire the lock, possibly waiting indefinitely for it to become available.
This isn't about the different methods I could or should be using to utilize the queues in the best manner, rather something I have seen happening that makes no sense to me.
void Runner() {
// member variable
queue = Queue.Synchronized(new Queue());
while (true) {
if (0 < queue.Count) {
queue.Dequeue();
}
}
}
This is run in a single thread:
var t = new Thread(Runner);
t.IsBackground = true;
t.Start();
Other events are "Enqueue"ing else where. What I've seen happen is over a period of time, the Dequeue will actually throw InvalidOperationException, queue empty. This should be impossible seeing as how the count guarantees there is something there, and I'm positive that nothing else is "Dequeue"ing.
The question(s):
Is it possible that the Enqueue actually increases the count before the item is fully on the queue (whatever that means...)?
Is it possible that the thread is somehow restarting (expiring, reseting...) at the Dequeue statement, but immediately after it already removed an item?
Edit (clarification):
These code pieces are part of a Wrapper class that implements the background helper thread. The Dequeue here is the only Dequeue, and all Enqueue/Dequeue are on the Synchronized member variable (queue).
Using Reflector, you can see that no, the count does not get increased until after the item is added.
As Ben points out, it does seem as you do have multiple people calling dequeue.
You say you are positive that nothing else is calling dequeue. Is that because you only have the one thread calling dequeue? Is dequeue called anywhere else at all?
EDIT:
I wrote a little sample code, but could not get the problem to reproduce. It just kept running and running without any exceptions.
How long was it running before you got errors? Maybe you can share a bit more of the code.
class Program
{
static Queue q = Queue.Synchronized(new Queue());
static bool running = true;
static void Main()
{
Thread producer1 = new Thread(() =>
{
while (running)
{
q.Enqueue(Guid.NewGuid());
Thread.Sleep(100);
}
});
Thread producer2 = new Thread(() =>
{
while (running)
{
q.Enqueue(Guid.NewGuid());
Thread.Sleep(25);
}
});
Thread consumer = new Thread(() =>
{
while (running)
{
if (q.Count > 0)
{
Guid g = (Guid)q.Dequeue();
Console.Write(g.ToString() + " ");
}
else
{
Console.Write(" . ");
}
Thread.Sleep(1);
}
});
consumer.IsBackground = true;
consumer.Start();
producer1.Start();
producer2.Start();
Console.ReadLine();
running = false;
}
}
Here is what I think the problematic sequence is:
(0 < queue.Count) evaluates to true, the queue is not empty.
This thread gets preempted and another thread runs.
The other thread removes an item from the queue, emptying it.
This thread resumes execution, but is now within the if block, and attempts to dequeue an empty list.
However, you say nothing else is dequeuing...
Try outputting the count inside the if block. If you see the count jump numbers downwards, someone else is dequeuing.
Here's a possible answer from the MSDN page on this topic:
Enumerating through a collection is
intrinsically not a thread-safe
procedure. Even when a collection is
synchronized, other threads can still
modify the collection, which causes
the enumerator to throw an exception.
To guarantee thread safety during
enumeration, you can either lock the
collection during the entire
enumeration or catch the exceptions
resulting from changes made by other
threads.
My guess is that you're correct - at some point, there's a race condition happening, and you end up dequeuing something that isn't there.
A Mutex or Monitor.Lock is probably appropriate here.
Good luck!
Are the other areas that are "Enqueuing" data also using the same synchronized queue object? In order for the Queue.Synchronized to be thread-safe, all Enqueue and Dequeue operations must use the same synchronized queue object.
From MSDN:
To guarantee the thread safety of the
Queue, all operations must be done
through this wrapper only.
Edited:
If you are looping over many items that involve heavy computation or if you are using a long-term thread loop (communications, etc.), you should consider having a wait function such as System.Threading.Thread.Sleep, System.Threading.WaitHandle.WaitOne, System.Threading.WaitHandle.WaitAll, or System.Threading.WaitHandle.WaitAny in the loop, otherwise it might kill system performance.
question 1: If you're using a synchronized queue, then: no, you're safe! But you'll need to use the synchronized instance on both sides, the supplier and the feeder.
question 2: Terminating your worker thread when there is no work to do, is a simple job. However, you either way need a monitoring thread or have the queue start a background worker thread whenever the queue has something to do. The last one sounds more like the ActiveObject Pattern, than a simple queue (which's Single-Responsibily-Pattern says that it should only do queueing).
In addition, I'd go for a blocking queue instead of your code above. The way your code works requires CPU processing power even if there is no work to do. A blocking queue lets your worker thread sleep whenever there is nothing to do. You can have multiple sleeping threads running without using CPU processing power.
C# doesn't come with a blocking queue implementation, but there a many out there. See this example and this one.
Another option for making thread-safe use of queues is the ConcurrentQueue<T> class that has been introduced since 2009 (the year of this question). This may help avoid having to write your own synchronization code or at least help making it much simpler.
From .NET Framework 4.6 onward, ConcurrentQueue<T> also implements the interface IReadOnlyCollection<T>.