I've created a custom thread pool utility, but there seems to be a problem that I cannot find.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Threading;
namespace iWallpaper.S3Uploader
{
public class QueueManager<T>
{
private readonly Queue queue = Queue.Synchronized(new Queue());
private readonly AutoResetEvent res = new AutoResetEvent(true);
private readonly AutoResetEvent res_thr = new AutoResetEvent(true);
private readonly Semaphore sem = new Semaphore(1, 4);
private readonly Thread thread;
private Action<T> DoWork;
private int Num_Of_Threads;
private QueueManager()
{
Num_Of_Threads = 0;
maxThread = 5;
thread = new Thread(Worker) {Name = "S3Uploader EventRegisterer"};
thread.Start();
// log.Info(String.Format("{0} [QUEUE] FileUploadQueueManager created", DateTime.Now.ToLongTimeString()));
}
public int maxThread { get; set; }
public static FileUploadQueueManager<T> Instance
{
get { return Nested.instance; }
}
/// <summary>
/// Executes multythreaded operation under items
/// </summary>
/// <param name="list">List of items to proceed</param>
/// <param name="action">Action under item</param>
/// <param name="MaxThreads">Maximum threads</param>
public void Execute(IEnumerable<T> list, Action<T> action, int MaxThreads)
{
maxThread = MaxThreads;
DoWork = action;
foreach (T item in list)
{
Add(item);
}
}
public void ExecuteNoThread(IEnumerable<T> list, Action<T> action)
{
ExecuteNoThread(list, action, 0);
}
public void ExecuteNoThread(IEnumerable<T> list, Action<T> action, int MaxThreads)
{
foreach (T wallpaper in list)
{
action(wallpaper);
}
}
/// <summary>
/// Default 10 threads
/// </summary>
/// <param name="list"></param>
/// <param name="action"></param>
public void Execute(IEnumerable<T> list, Action<T> action)
{
Execute(list, action, 10);
}
private void Add(T item)
{
lock (queue)
{
queue.Enqueue(item);
}
res.Set();
}
private void Worker()
{
while (true)
{
if (queue.Count == 0)
{
res.WaitOne();
}
if (Num_Of_Threads < maxThread)
{
var t = new Thread(Proceed);
t.Start();
}
else
{
res_thr.WaitOne();
}
}
}
private void Proceed()
{
Interlocked.Increment(ref Num_Of_Threads);
if (queue.Count > 0)
{
var item = (T) queue.Dequeue();
sem.WaitOne();
ProceedItem(item);
sem.Release();
}
res_thr.Set();
Interlocked.Decrement(ref Num_Of_Threads);
}
private void ProceedItem(T activity)
{
if (DoWork != null)
DoWork(activity);
lock (Instance)
{
Console.Title = string.Format("ThrId:{0}/{4}, {1}, Activity({2} left):{3}",
thread.ManagedThreadId, DateTime.Now, queue.Count, activity,
Num_Of_Threads);
}
}
#region Nested type: Nested
protected class Nested
{
// Explicit static constructor to tell C# compiler
// not to mark type as beforefieldinit
internal static readonly QueueManager<T> instance = new FileUploadQueueManager<T>();
}
#endregion
}
}
Problem is here:
Console.Title = string.Format("ThrId:{0}/{4}, {1}, Activity({2} left):{3}",
thread.ManagedThreadId, DateTime.Now, queue.Count, activity,
Num_Of_Threads);
There is always ONE thread id in title. And program seems to be working in one thread.
Sample usage:
var i_list = new int[] {1, 2, 4, 5, 6, 7, 8, 6};
QueueManager<int>.Instance.Execute(i_list,
i =>
{
Console.WriteLine("Some action under element number {0}", i);
}, 5);
P.S.: it's pretty messy, but I'm still working on it.
I looked through your code and here are a couple of issues I saw.
You lock the queue object even though it is synchronized queue. This is unnecessary
You inconsistently lock the queue object. It should either be locked for every access or not locked and depending on the Synchronized behavior.
The Proceed method is not thread safe. These two lines are the issue
if (queue.Count > 0) {
var item = (T)queue.Dequeue();
...
}
Using a synchronized queue only guarantees that individual accesses are safe. So both the .Count and the .Dequeue method won't mess with te internal structure of the queue. However imagine the scenario where two threads run these lines of code at the same time with a queue of count 1
Thread1: if (...) -> true
Thread2: if (...) -> true
Thread1: dequeue -> sucess
Thread2: dequeue -> fails because the queue is empty
There is a race condition between Worker and Proceed that can lead to deadlock. The following two lines of code should be switched.
Code:
res_thr.Set()
Interlocked.Decrement(ref Num_Of_Threads);
The first line will unblock the Worker method. If it runs quickly enough it will go back through the look, notice that Num_Of_Threads < maxThreads and go right back into res_thr.WaitOne(). If no other threads are currently running then this will lead to a deadlock in your code. This is very easy to hit with a low number of maximum threads (say 1). Inverting these two lines of code should fix the issue.
The maxThread count property does not seem to be useful beyond 4. The sem object is initialized to accept only 4 maximum concurrent entries. All code that actually executes an item must go through this semaphore. So you've effectively limited the maximum number of concurrent items to 4 regardless of how high maxThread is set.
Writing robust threaded code is not trivial. There are numerous thread-pools around that you might look at for reference, but also note that Parallel Extensions (available as CTP, or later in .NET 4.0) includes a lot of additional threading constructs out-of-the-box (in the TPL/CCR). For example, Parallel.For / Parallel.ForEach, which deal with work-stealing, and handling the available cores effectively.
For an example of a pre-rolled thread-pool, see Jon Skeet's CustomThreadPool here.
I think you can simply things considerably.
Here is a modified form (I didn't test the modifications) of the thread pool I use:
The only sync. primitive you need is a Monitor, locked on the thread pool. You don't need a semaphore, or the reset events.
internal class ThreadPool
{
private readonly Thread[] m_threads;
private readonly Queue<Action> m_queue;
private bool m_shutdown;
private object m_lockObj;
public ThreadPool(int numberOfThreads)
{
Util.Assume(numberOfThreads > 0, "Invalid thread count!");
m_queue = new Queue<Action>();
m_threads = new Thread[numberOfThreads];
m_lockObj = new object();
lock (m_lockObj)
{
for (int i = 0; i < numberOfWriteThreads; ++i)
{
m_threads[i] = new Thread(ThreadLoop);
m_threads[i].Start();
}
}
}
public void Shutdown()
{
lock (m_lockObj)
{
m_shutdown = true;
Monitor.PulseAll(m_lockObj);
if (OnShuttingDown != null)
{
OnShuttingDown();
}
}
foreach (var thread in m_threads)
{
thread.Join();
}
}
public void Enqueue(Action a)
{
lock (m_lockObj)
{
m_queue.Enqueue(a);
Monitor.Pulse(m_lockObj);
}
}
private void ThreadLoop()
{
Monitor.Enter(m_lockObj);
while (!m_shutdown)
{
if (m_queue.Count == 0)
{
Monitor.Wait(m_lockObj);
}
else
{
var a = m_queue.Dequeue();
Monitor.Pulse(m_lockObj);
Monitor.Exit(m_lockObj);
try
{
a();
}
catch (Exception ex)
{
Console.WriteLine("An unhandled exception occured!\n:{0}", ex.Message, null);
}
Monitor.Enter(m_lockObj);
}
}
Monitor.Exit(m_lockObj);
}
}
You should probally use the built in thread pool. When running your code I noticed that your spining up a bunch of threads but since the queue count is <1 you just exit, this continues until the queue is actually populated then your next thread processes everything. This is a very expensive process. You should only spin up threads if you have something to do.
Related
Background:
I have an application I am developing that deals with a large number of addons for another application. One if its primary uses is to safely modify file records in files with fewer records so that they may be treated as one file (almost as if it is combing the files together into one set of records. To do this safely it keeps track of vital information about those files and changes made to them so that those changes can be undone if they don't work as expected.
When my application starts, it analyzes those files and keeps essential properties in a cache (to reduce load times). If a file is missing from the cache, the most important stuff is retrieved and then a background worker must process the file for more information. If a file that was previously modified has been updated with a new version of the file, the UI must confirm this with the user and its modification data removed. All of this information, including information on its modification is stored in the cache.
My Problem:
My problem is that neither of these processes are guaranteed to run (the confirmation window or the background file processor). If either of them run, then the cache must be updated by the main thread. I don't know enough about worker threads, and which thread runs the BackgroundWorker.RunWorkerCompleted event handler in order to effectively decide how to approach guaranteeing that the cache updater is run after either (or both) processes are completed.
To sum up: if either process is run, they both must finish and (potentially) wait for the other to be completed before running the cache update code. How can I do this?
ADJUNCT INFO (My current intervention that doesn't seem to work very well):
I have a line in the RunWorkerCompleted handler that waits until the form reference is null before continuing and exiting but maybe this was a mistake as it sometimes locks my program up.
SpinWait.SpinUntil(() => overwriteForm == null);
I haven't included any more code because I anticipate that this is more of a conceptual question than a code one. However, if necessary, I can supply code if it helps.
I think CountDownTask is what you need
using System;
using System.Threading;
public class Program
{
public class AtomicInteger
{
protected int value = 0;
public AtomicInteger(int value)
{
this.value = value;
}
public int DecrementAndGet()
{
int answer = Interlocked.Decrement(ref value);
return answer;
}
}
public interface Runnable
{
void Run();
}
public class CountDownTask
{
private AtomicInteger count;
private Runnable task;
private Object lk = new Object();
private volatile bool runnable;
private bool cancelled;
public CountDownTask(Int32 count, Runnable task)
{
this.count = new AtomicInteger(count);
this.task = task;
this.runnable = false;
this.cancelled = false;
}
public void CountDown()
{
if (count.DecrementAndGet() == 0)
{
lock (lk)
{
runnable = true;
Monitor.Pulse(lk);
}
}
}
public void Await()
{
lock (lk)
{
while (!runnable)
{
Monitor.Wait(lk);
}
if (cancelled)
{
Console.WriteLine("Sorry! I was cancelled");
}
else {
task.Run();
}
}
}
public void Cancel()
{
lock (lk)
{
runnable = true;
cancelled = true;
Monitor.Pulse(lk);
}
}
}
public class HelloWorldTask : Runnable
{
public void Run()
{
Console.WriteLine("Hello World, I'm last one");
}
}
public static void Main()
{
Thread.CurrentThread.Name = "Main";
Console.WriteLine("Current Thread: " + Thread.CurrentThread.Name);
CountDownTask countDownTask = new CountDownTask(3, new HelloWorldTask());
Thread worker1 = new Thread(() => {
Console.WriteLine("Worker 1 run");
countDownTask.CountDown();
});
Thread worker2 = new Thread(() => {
Console.WriteLine("Worker 2 run");
countDownTask.CountDown();
});
Thread lastThread = new Thread(() => countDownTask.Await());
lastThread.Start();
worker1.Start();
worker2.Start();
//countDownTask.Cancel();
Console.WriteLine("Main Thread Run");
countDownTask.CountDown();
Thread.Sleep(1000);
}
}
let me explain (but you can refer Java CountDownLatch)
1. To ensure a task must run after another tasks, we need create a Wait function to wait for they done, so I used
while(!runnable) {
Monitor.Wait(lk);
}
2. When there is a task done, we need count down, and if count down to zero (it means all of the tasks was done) we will need notify to blocked thread to wake up and process task
if(count.decrementAndGet() == 0) {
lock(lk) {
runnable = true;
Monitor.Pulse(lk);
}
}
Let read more about volatile, thanks
While dung ta van's "CountDownTask" answer isn't quite what I needed, it heavily inspired the solution below (see it for more info). Basically all I did was add some extra functionality and most importantly: made it so that each task "vote" on the outcome (true or false). Thanks dung ta van!
To be fair, dung ta van's solution DOES work to guarantee execution which as it turns out isn't quite what I needed. My solution adds the ability to make that execution conditional.
This was my solution which worked:
public enum PendingBool
{
Unknown = -1,
False,
True
}
public interface IRunnableTask
{
void Run();
}
public class AtomicInteger
{
int integer;
public int Value { get { return integer; } }
public AtomicInteger(int value) { integer = value; }
public int Decrement() { return Interlocked.Decrement(ref integer); }
public static implicit operator int(AtomicInteger ai) { return ai.integer; }
}
public class TaskElectionEventArgs
{
public bool VoteResult { get; private set; }
public TaskElectionEventArgs(bool vote) { VoteResult = vote; }
}
public delegate void VoteEventHandler(object sender, TaskElectionEventArgs e);
public class SingleVoteTask
{
private AtomicInteger votesLeft;
private IRunnableTask task;
private volatile bool runTask = false;
private object _lock = new object();
public event VoteEventHandler VoteCast;
public event VoteEventHandler TaskCompleted;
public bool IsWaiting { get { return votesLeft.Value > 0; } }
public PendingBool Result
{
get
{
if (votesLeft > 0)
return PendingBool.Unknown;
else if (runTask)
return PendingBool.True;
else
return PendingBool.False;
}
}
public SingleVoteTask(int numberOfVotes, IRunnableTask taskToRun)
{
votesLeft = new AtomicInteger(numberOfVotes);
task = taskToRun;
}
public void CastVote(bool vote)
{
votesLeft.Decrement();
runTask |= vote;
VoteCast?.Invoke(this, new TaskElectionEventArgs(vote));
if (votesLeft == 0)
lock (_lock)
{
Monitor.Pulse(_lock);
}
}
public void Await()
{
lock(_lock)
{
while (votesLeft > 0)
Monitor.Wait(_lock);
if (runTask)
task.Run();
TaskCompleted?.Invoke(this, new TaskElectionEventArgs(runTask));
}
}
}
Implementing the above solution was as simple as creating the SingleVoteTask in the UI thread and then having each thread affecting the outcome cast a vote.
I am developing an application simulating a network comprising a number of nodes that exchange messages. I try to simulate the transmission channel with a Queue where every node can place a message. Then, another entity takes over the message and delivers it to the specified node. Then, i want to signal (with an event) the end of a transmission phase when the message queue is idle for a certain amount of time, say X, namely no new messages have been added to the queue for X mseconds.
I understand that my case follows the consumer/producer paradigm. So far, i have done the following:
public class Com<T>
{
private Thread dispatcher;
private Queue<T> queue;
private int waitTime;
private Object locker;
private Timer timer;
public event EventHandler EmptyQueueEvent;
public Com()
{
queue = new Queue<T>();
locker = new Object();
waitTime = X;
timer = new Timer(FireEmpty, null, Timeout.Infinite,Timeout.Infinite);
dispatcher = new Thread(Serve);
dispatcher.IsBackground = true;
dispatcher.Start();
}
private void Serve()
{
while (true)
{
try
{
if (queue.Count == 0)
{
timer.Change(waitTime, 0);
Thread.Sleep(Timeout.Infinite);
}
}
catch (ThreadInterruptedException)
{
}
while (queue.Count != 0)
{
lock (locker)
{
deliver(queue.Dequeue());
}
}
}
}
private void deliver(T item)
{
// Do stuff
}
public void Add(T item)
{
timer.Change(Timeout.Infinite, Timeout.Infinite);
lock (locker)
{
queue.Enqueue(item);
}
dispatcher.Interrupt();
}
private void FireEmpty(object o)
{
//Fire Event
}
}
However, running my simulations proves that my synchronization is not enough, since I am sometimes getting a "ThreadInterruptedException" while trying to dequeue my message (in method Serve()). Note that the exception does not occur each time i run the simulation, but rather rarely: approximately every 850-1000 executions (i am running the execution iteratively)..
Does anybody have an idea what it is wrong with my code? :)
Have you tried locking before you attempt to get the Queue count? Like:
private void Serve()
{
while (true)
{
try
{
int count = 0;
lock(locker)
count= queue.Count;
if (count == 0)
{
timer.Change(waitTime, 0);
Thread.Sleep(Timeout.Infinite);
}
}
catch (ThreadInterruptedException)
{
}
while (queue.Count != 0)
{
lock (locker)
{
deliver(queue.Dequeue());
}
}
}
}
It's possible that an add is getting called at the same time you're trying to count the number of items. Also, you might want to consider one of the collections from System.Collections.Concurrent if you're using .net 4.0.
** UPDATE **
I just took a closer look at your code and had an "Oh duh" moment. You should be getting a ThreadInterruptException because you're calling delegate.Interrupt(). Check the MSDN documentation on that. I think what you need to do is use something like a ManualResetEvent and instead of calling Interrupt() do a WaitOne() on that event.
** UPDATE2 **
Here's some sample code that includes my other locking suggestion as well (on Gist):
https://gist.github.com/1683547
I need to design perfect worker thread method. The method must do the following:
1) extract something from queue (let's say a queue of string) and do something
2) stop and return when class is disposed
3) wait for some event (that queue is not empty) and do not consume cpu
4) run in separate thread
Main thread will add string to queue and signal thread method to continue and do the job.
I would like you to provide me the the template with required syncronization objects.
class MyClass, IDisposable
{
// Thread safe queue from third party
private ThreadSafeQueue<string> _workerQueue;
private Thread _workerThread;
public bool Initialize()
{
_workerThread = new Thread(WorkerThread).Start();
}
public AddTask(string object)
{
_workerQueue.Enqueue(object);
// now we must signal worker thread
}
// this is worker thread
private void WorkerThread()
{
// This is what worker thread must do
List<string> objectList = _workerQueue.EnqueAll
// Do something
}
// Yeap, this is Dispose
public bool Dispose()
{
}
}
Try something like this. instantiate with type string and give it a delegate to process your string:
public class SuperQueue<T> : IDisposable where T : class
{
readonly object _locker = new object();
readonly List<Thread> _workers;
readonly Queue<T> _taskQueue = new Queue<T>();
readonly Action<T> _dequeueAction;
/// <summary>
/// Initializes a new instance of the <see cref="SuperQueue{T}"/> class.
/// </summary>
/// <param name="workerCount">The worker count.</param>
/// <param name="dequeueAction">The dequeue action.</param>
public SuperQueue(int workerCount, Action<T> dequeueAction)
{
_dequeueAction = dequeueAction;
_workers = new List<Thread>(workerCount);
// Create and start a separate thread for each worker
for (int i = 0; i < workerCount; i++)
{
Thread t = new Thread(Consume) { IsBackground = true, Name = string.Format("SuperQueue worker {0}",i )};
_workers.Add(t);
t.Start();
}
}
/// <summary>
/// Enqueues the task.
/// </summary>
/// <param name="task">The task.</param>
public void EnqueueTask(T task)
{
lock (_locker)
{
_taskQueue.Enqueue(task);
Monitor.PulseAll(_locker);
}
}
/// <summary>
/// Consumes this instance.
/// </summary>
void Consume()
{
while (true)
{
T item;
lock (_locker)
{
while (_taskQueue.Count == 0) Monitor.Wait(_locker);
item = _taskQueue.Dequeue();
}
if (item == null) return;
// run actual method
_dequeueAction(item);
}
}
/// <summary>
/// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
/// </summary>
public void Dispose()
{
// Enqueue one null task per worker to make each exit.
_workers.ForEach(thread => EnqueueTask(null));
_workers.ForEach(thread => thread.Join());
}
}
What you are describing is best accomplished with the producer-consumer pattern. This pattern is most easily implemented with a blocking queue. If you are using .NET 4.0 then you can take advantage of the BlockingCollection class. Here is how I am seeing your code working. In the following example I am using a null value as sentinel for gracefully ending the consumer, but you could also take advantage of the CancellationToken parameter on the Take method.
public class MyClass : IDisposable
{
private BlockingCollection<string> m_Queue = new BlockingCollection<string>();
public class MyClass()
{
var thread = new Thread(Process);
thread.IsBackground = true;
thread.Start();
}
public void Dispose()
{
m_Queue.Add(null);
}
public void AddTask(string item)
{
if (item == null)
{
throw new ArgumentNullException();
}
m_Queue.Add(item);
}
private void Process()
{
while (true)
{
string item = m_Queue.Take();
if (item == null)
{
break; // Gracefully end the consumer thread.
}
else
{
// Process the item here.
}
}
}
}
I think you should consider using BackgroundWorker class, which may fit well to your needs.
Sounds like BlockingQueue is what you need.
You should take a look at the new .Net 4 System.Collections.Concurrent Namespace. Also this little example should help you to get a better understanding on how to use it.
I have a class running the Producer-Consumer model like this:
public class SyncEvents
{
public bool waiting;
public SyncEvents()
{
waiting = true;
}
}
public class Producer
{
private readonly Queue<Delegate> _queue;
private SyncEvents _sync;
private Object _waitAck;
public Producer(Queue<Delegate> q, SyncEvents sync, Object obj)
{
_queue = q;
_sync = sync;
_waitAck = obj;
}
public void ThreadRun()
{
lock (_sync)
{
while (true)
{
Monitor.Wait(_sync, 0);
if (_queue.Count > 0)
{
_sync.waiting = false;
}
else
{
_sync.waiting = true;
lock (_waitAck)
{
Monitor.Pulse(_waitAck);
}
}
Monitor.Pulse(_sync);
}
}
}
}
public class Consumer
{
private readonly Queue<Delegate> _queue;
private SyncEvents _sync;
private int count = 0;
public Consumer(Queue<Delegate> q, SyncEvents sync)
{
_queue = q;
_sync = sync;
}
public void ThreadRun()
{
lock (_sync)
{
while (true)
{
while (_queue.Count == 0)
{
Monitor.Wait(_sync);
}
Delegate query = _queue.Dequeue();
query.DynamicInvoke(null);
count++;
Monitor.Pulse(_sync);
}
}
}
}
/// <summary>
/// Act as a consumer to the queries produced by the DataGridViewCustomCell
/// </summary>
public class QueryThread
{
private SyncEvents _syncEvents = new SyncEvents();
private Object waitAck = new Object();
private Queue<Delegate> _queryQueue = new Queue<Delegate>();
Producer queryProducer;
Consumer queryConsumer;
public QueryThread()
{
queryProducer = new Producer(_queryQueue, _syncEvents, waitAck);
queryConsumer = new Consumer(_queryQueue, _syncEvents);
Thread producerThread = new Thread(queryProducer.ThreadRun);
Thread consumerThread = new Thread(queryConsumer.ThreadRun);
producerThread.IsBackground = true;
consumerThread.IsBackground = true;
producerThread.Start();
consumerThread.Start();
}
public bool isQueueEmpty()
{
return _syncEvents.waiting;
}
public void wait()
{
lock (waitAck)
{
while (_queryQueue.Count > 0)
{
Monitor.Wait(waitAck);
}
}
}
public void Enqueue(Delegate item)
{
_queryQueue.Enqueue(item);
}
}
The code run smoothly but the wait() function.
In some case I want to wait until all the function in the queue were finished running so I made the wait() function.
The producer will fire the waitAck pulse at suitable time.
However, when the line "Monitor.Wait(waitAck);" is ran in the wait() function, all thread stop, includeing the producer and consumer thread.
Why would this happen and how can I solve it? thanks!
It seems very unlikely that all the threads will actually stop, although I should point out that to avoid false wake-ups you should probably have a while loop instead of an if statement:
lock (waitAck)
{
while(queryProducer.secondQueue.Count > 0)
{
Monitor.Wait(waitAck);
}
}
The fact that you're calling Monitor.Wait means that waitAck should be released so it shouldn't prevent the consumer threads from locking...
Could you give more information about the way in which the producer/consumer threads are "stopping"? Does it look like they've just deadlocked?
Is your producer using Notify or NotifyAll? You've got an extra waiting thread now, so if you only use Notify it's only going to release a single thread... it's hard to see whether or not that's a problem without the details of your Producer and Consumer classes.
If you could show a short but complete program to demonstrate the problem, that would help.
EDIT: Okay, now you've posted the code I can see a number of issues:
Having so many public variables is a recipe for disaster. Your classes should encapsulate their functionality so that other code doesn't have to go poking around for implementation bits and pieces. (For example, your calling code here really shouldn't have access to the queue.)
You're adding items directly to the second queue, which means you can't efficiently wake up the producer to add them to the first queue. Why do you even have multiple queues?
You're always waiting on _sync in the producer thread... why? What's going to notify it to start with? Generally speaking the producer thread shouldn't have to wait, unless you have a bounded buffer
You have a static variable (_waitAck) which is being overwritten every time you create a new instance. That's a bad idea.
You also haven't shown your SyncEvents class - is that meant to be doing anything interesting?
To be honest, it seems like you've got quite a strange design - you may well be best starting again from scratch. Try to encapsulate the whole producer/consumer queue in a single class, which has Produce and Consume methods, as well as WaitForEmpty (or something like that). I think you'll find the synchronization logic a lot easier that way.
Here is my take on your code:
public class ProducerConsumer
{
private ManualResetEvent _ready;
private Queue<Delegate> _queue;
private Thread _consumerService;
private static Object _sync = new Object();
public ProducerConsumer(Queue<Delegate> queue)
{
lock (_sync)
{
// Note: I would recommend that you don't even
// bother with taking in a queue. You should be able
// to just instantiate a new Queue<Delegate>()
// and use it when you Enqueue. There is nothing that
// you really need to pass into the constructor.
_queue = queue;
_ready = new ManualResetEvent(false);
_consumerService = new Thread(Run);
_consumerService.IsBackground = true;
_consumerService.Start();
}
}
public override void Enqueue(Delegate value)
{
lock (_sync)
{
_queue.Enqueue(value);
_ready.Set();
}
}
// The consumer blocks until the producer puts something in the queue.
private void Run()
{
Delegate query;
try
{
while (true)
{
_ready.WaitOne();
lock (_sync)
{
if (_queue.Count > 0)
{
query = _queue.Dequeue();
query.DynamicInvoke(null);
}
else
{
_ready.Reset();
continue;
}
}
}
}
catch (ThreadInterruptedException)
{
_queue.Clear();
return;
}
}
protected override void Dispose(bool disposing)
{
lock (_sync)
{
if (_consumerService != null)
{
_consumerService.Interrupt();
}
}
base.Dispose(disposing);
}
}
I'm not exactly sure what you're trying to achieve with the wait function... I'm assuming you're trying to put some type of a limit to the number of items that can be queued. In that case simply throw an exception or return a failure signal when you have too many items in the queue, the client that is calling Enqueue will keep retrying until the queue can take more items. Taking an optimistic approach will save you a LOT of headaches and it simply helps you get rid of a lot of complex logic.
If you REALLY want to have the wait in there, then I can probably help you figure out a better approach. Let me know what are you trying to achieve with the wait and I'll help you out.
Note: I took this code from one of my projects, modified it a little and posted it here... there might be some minor syntax errors, but the logic should be correct.
UPDATE: Based on your comments I made some modifications: I added another ManualResetEvent to the class, so when you call BlockQueue() it gives you an event which you can wait on and sets a flag to stop the Enqueue function from queuing more elements. Once all the queries in the queue are serviced, the flag is set to true and the _wait event is set so whoever is waiting on it gets the signal.
public class ProducerConsumer
{
private bool _canEnqueue;
private ManualResetEvent _ready;
private Queue<Delegate> _queue;
private Thread _consumerService;
private static Object _sync = new Object();
private static ManualResetEvent _wait = new ManualResetEvent(false);
public ProducerConsumer()
{
lock (_sync)
{
_queue = new Queue<Delegate> _queue;
_canEnqueue = true;
_ready = new ManualResetEvent(false);
_consumerService = new Thread(Run);
_consumerService.IsBackground = true;
_consumerService.Start();
}
}
public bool Enqueue(Delegate value)
{
lock (_sync)
{
// Don't allow anybody to enqueue
if( _canEnqueue )
{
_queue.Enqueue(value);
_ready.Set();
return true;
}
}
// Whoever is calling Enqueue should try again later.
return false;
}
// The consumer blocks until the producer puts something in the queue.
private void Run()
{
try
{
while (true)
{
// Wait for a query to be enqueued
_ready.WaitOne();
// Process the query
lock (_sync)
{
if (_queue.Count > 0)
{
Delegate query = _queue.Dequeue();
query.DynamicInvoke(null);
}
else
{
_canEnqueue = true;
_ready.Reset();
_wait.Set();
continue;
}
}
}
}
catch (ThreadInterruptedException)
{
_queue.Clear();
return;
}
}
// Block your queue from enqueuing, return null
// if the queue is already empty.
public ManualResetEvent BlockQueue()
{
lock(_sync)
{
if( _queue.Count > 0 )
{
_canEnqueue = false;
_wait.Reset();
}
else
{
// You need to tell the caller that they can't
// block your queue while it's empty. The caller
// should check if the result is null before calling
// WaitOne().
return null;
}
}
return _wait;
}
protected override void Dispose(bool disposing)
{
lock (_sync)
{
if (_consumerService != null)
{
_consumerService.Interrupt();
// Set wait when you're disposing the queue
// so that nobody is left with a lingering wait.
_wait.Set();
}
}
base.Dispose(disposing);
}
}
For a "log information for support" type of function I'd like to enumerate and dump active thread information.
I'm well aware of the fact that race conditions can make this information semi-inaccurate, but I'd like to try to get the best possible result, even if it isn't 100% accurate.
I looked at Process.Threads, but it returns ProcessThread objects, I'd like to have a collection of Thread objects, so that I can log their name, and whether they're background threads or not.
Is there such a collection available, even if it is just a snapshot of the active threads when I call it?
ie.
Thread[] activeThreads = ??
Note, to be clear, I am not asking about Process.Threads, this collection gives me a lot, but not all of what I want. I want to know how much time specific named threads in our application is currently using (which means I will have to look at connecting the two types of objects later, but the names is more important than the CPU time to begin with.)
If you're willing to replace your application's Thread creations with another wrapper class, said wrapper class can track the active and inactive Threads for you. Here's a minimal workable shell of such a wrapper:
namespace ThreadTracker
{
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Threading;
public class TrackedThread
{
private static readonly IList<Thread> threadList = new List<Thread>();
private readonly Thread thread;
private readonly ParameterizedThreadStart start1;
private readonly ThreadStart start2;
public TrackedThread(ParameterizedThreadStart start)
{
this.start1 = start;
this.thread = new Thread(this.StartThreadParameterized);
lock (threadList)
{
threadList.Add(this.thread);
}
}
public TrackedThread(ThreadStart start)
{
this.start2 = start;
this.thread = new Thread(this.StartThread);
lock (threadList)
{
threadList.Add(this.thread);
}
}
public TrackedThread(ParameterizedThreadStart start, int maxStackSize)
{
this.start1 = start;
this.thread = new Thread(this.StartThreadParameterized, maxStackSize);
lock (threadList)
{
threadList.Add(this.thread);
}
}
public TrackedThread(ThreadStart start, int maxStackSize)
{
this.start2 = start;
this.thread = new Thread(this.StartThread, maxStackSize);
lock (threadList)
{
threadList.Add(this.thread);
}
}
public static int Count
{
get
{
lock (threadList)
{
return threadList.Count;
}
}
}
public static IEnumerable<Thread> ThreadList
{
get
{
lock (threadList)
{
return new ReadOnlyCollection<Thread>(threadList);
}
}
}
// either: (a) expose the thread object itself via a property or,
// (b) expose the other Thread public methods you need to replicate.
// This example uses (a).
public Thread Thread
{
get
{
return this.thread;
}
}
private void StartThreadParameterized(object obj)
{
try
{
this.start1(obj);
}
finally
{
lock (threadList)
{
threadList.Remove(this.thread);
}
}
}
private void StartThread()
{
try
{
this.start2();
}
finally
{
lock (threadList)
{
threadList.Remove(this.thread);
}
}
}
}
}
and a quick test driver of it (note I do not iterate over the list of threads, merely get the count in the list):
namespace ThreadTracker
{
using System;
using System.Threading;
internal static class Program
{
private static void Main()
{
var thread1 = new TrackedThread(DoNothingForFiveSeconds);
var thread2 = new TrackedThread(DoNothingForTenSeconds);
var thread3 = new TrackedThread(DoNothingForSomeTime);
thread1.Thread.Start();
thread2.Thread.Start();
thread3.Thread.Start(15);
while (TrackedThread.Count > 0)
{
Console.WriteLine(TrackedThread.Count);
}
Console.ReadLine();
}
private static void DoNothingForFiveSeconds()
{
Thread.Sleep(5000);
}
private static void DoNothingForTenSeconds()
{
Thread.Sleep(10000);
}
private static void DoNothingForSomeTime(object seconds)
{
Thread.Sleep(1000 * (int)seconds);
}
}
}
Not sure if you can go such a route, but it will accomplish the goal if you're able to incorporate at an early stage of development.
Is it feasible for you to store thread information in a lookup as you create each thread in your application?
As each thread starts, you can get its ID using AppDomain.GetCurrentThreadId(). Later, you can use this to cross reference with the data returned from Process.Threads.