I have a class running the Producer-Consumer model like this:
public class SyncEvents
{
public bool waiting;
public SyncEvents()
{
waiting = true;
}
}
public class Producer
{
private readonly Queue<Delegate> _queue;
private SyncEvents _sync;
private Object _waitAck;
public Producer(Queue<Delegate> q, SyncEvents sync, Object obj)
{
_queue = q;
_sync = sync;
_waitAck = obj;
}
public void ThreadRun()
{
lock (_sync)
{
while (true)
{
Monitor.Wait(_sync, 0);
if (_queue.Count > 0)
{
_sync.waiting = false;
}
else
{
_sync.waiting = true;
lock (_waitAck)
{
Monitor.Pulse(_waitAck);
}
}
Monitor.Pulse(_sync);
}
}
}
}
public class Consumer
{
private readonly Queue<Delegate> _queue;
private SyncEvents _sync;
private int count = 0;
public Consumer(Queue<Delegate> q, SyncEvents sync)
{
_queue = q;
_sync = sync;
}
public void ThreadRun()
{
lock (_sync)
{
while (true)
{
while (_queue.Count == 0)
{
Monitor.Wait(_sync);
}
Delegate query = _queue.Dequeue();
query.DynamicInvoke(null);
count++;
Monitor.Pulse(_sync);
}
}
}
}
/// <summary>
/// Act as a consumer to the queries produced by the DataGridViewCustomCell
/// </summary>
public class QueryThread
{
private SyncEvents _syncEvents = new SyncEvents();
private Object waitAck = new Object();
private Queue<Delegate> _queryQueue = new Queue<Delegate>();
Producer queryProducer;
Consumer queryConsumer;
public QueryThread()
{
queryProducer = new Producer(_queryQueue, _syncEvents, waitAck);
queryConsumer = new Consumer(_queryQueue, _syncEvents);
Thread producerThread = new Thread(queryProducer.ThreadRun);
Thread consumerThread = new Thread(queryConsumer.ThreadRun);
producerThread.IsBackground = true;
consumerThread.IsBackground = true;
producerThread.Start();
consumerThread.Start();
}
public bool isQueueEmpty()
{
return _syncEvents.waiting;
}
public void wait()
{
lock (waitAck)
{
while (_queryQueue.Count > 0)
{
Monitor.Wait(waitAck);
}
}
}
public void Enqueue(Delegate item)
{
_queryQueue.Enqueue(item);
}
}
The code run smoothly but the wait() function.
In some case I want to wait until all the function in the queue were finished running so I made the wait() function.
The producer will fire the waitAck pulse at suitable time.
However, when the line "Monitor.Wait(waitAck);" is ran in the wait() function, all thread stop, includeing the producer and consumer thread.
Why would this happen and how can I solve it? thanks!
It seems very unlikely that all the threads will actually stop, although I should point out that to avoid false wake-ups you should probably have a while loop instead of an if statement:
lock (waitAck)
{
while(queryProducer.secondQueue.Count > 0)
{
Monitor.Wait(waitAck);
}
}
The fact that you're calling Monitor.Wait means that waitAck should be released so it shouldn't prevent the consumer threads from locking...
Could you give more information about the way in which the producer/consumer threads are "stopping"? Does it look like they've just deadlocked?
Is your producer using Notify or NotifyAll? You've got an extra waiting thread now, so if you only use Notify it's only going to release a single thread... it's hard to see whether or not that's a problem without the details of your Producer and Consumer classes.
If you could show a short but complete program to demonstrate the problem, that would help.
EDIT: Okay, now you've posted the code I can see a number of issues:
Having so many public variables is a recipe for disaster. Your classes should encapsulate their functionality so that other code doesn't have to go poking around for implementation bits and pieces. (For example, your calling code here really shouldn't have access to the queue.)
You're adding items directly to the second queue, which means you can't efficiently wake up the producer to add them to the first queue. Why do you even have multiple queues?
You're always waiting on _sync in the producer thread... why? What's going to notify it to start with? Generally speaking the producer thread shouldn't have to wait, unless you have a bounded buffer
You have a static variable (_waitAck) which is being overwritten every time you create a new instance. That's a bad idea.
You also haven't shown your SyncEvents class - is that meant to be doing anything interesting?
To be honest, it seems like you've got quite a strange design - you may well be best starting again from scratch. Try to encapsulate the whole producer/consumer queue in a single class, which has Produce and Consume methods, as well as WaitForEmpty (or something like that). I think you'll find the synchronization logic a lot easier that way.
Here is my take on your code:
public class ProducerConsumer
{
private ManualResetEvent _ready;
private Queue<Delegate> _queue;
private Thread _consumerService;
private static Object _sync = new Object();
public ProducerConsumer(Queue<Delegate> queue)
{
lock (_sync)
{
// Note: I would recommend that you don't even
// bother with taking in a queue. You should be able
// to just instantiate a new Queue<Delegate>()
// and use it when you Enqueue. There is nothing that
// you really need to pass into the constructor.
_queue = queue;
_ready = new ManualResetEvent(false);
_consumerService = new Thread(Run);
_consumerService.IsBackground = true;
_consumerService.Start();
}
}
public override void Enqueue(Delegate value)
{
lock (_sync)
{
_queue.Enqueue(value);
_ready.Set();
}
}
// The consumer blocks until the producer puts something in the queue.
private void Run()
{
Delegate query;
try
{
while (true)
{
_ready.WaitOne();
lock (_sync)
{
if (_queue.Count > 0)
{
query = _queue.Dequeue();
query.DynamicInvoke(null);
}
else
{
_ready.Reset();
continue;
}
}
}
}
catch (ThreadInterruptedException)
{
_queue.Clear();
return;
}
}
protected override void Dispose(bool disposing)
{
lock (_sync)
{
if (_consumerService != null)
{
_consumerService.Interrupt();
}
}
base.Dispose(disposing);
}
}
I'm not exactly sure what you're trying to achieve with the wait function... I'm assuming you're trying to put some type of a limit to the number of items that can be queued. In that case simply throw an exception or return a failure signal when you have too many items in the queue, the client that is calling Enqueue will keep retrying until the queue can take more items. Taking an optimistic approach will save you a LOT of headaches and it simply helps you get rid of a lot of complex logic.
If you REALLY want to have the wait in there, then I can probably help you figure out a better approach. Let me know what are you trying to achieve with the wait and I'll help you out.
Note: I took this code from one of my projects, modified it a little and posted it here... there might be some minor syntax errors, but the logic should be correct.
UPDATE: Based on your comments I made some modifications: I added another ManualResetEvent to the class, so when you call BlockQueue() it gives you an event which you can wait on and sets a flag to stop the Enqueue function from queuing more elements. Once all the queries in the queue are serviced, the flag is set to true and the _wait event is set so whoever is waiting on it gets the signal.
public class ProducerConsumer
{
private bool _canEnqueue;
private ManualResetEvent _ready;
private Queue<Delegate> _queue;
private Thread _consumerService;
private static Object _sync = new Object();
private static ManualResetEvent _wait = new ManualResetEvent(false);
public ProducerConsumer()
{
lock (_sync)
{
_queue = new Queue<Delegate> _queue;
_canEnqueue = true;
_ready = new ManualResetEvent(false);
_consumerService = new Thread(Run);
_consumerService.IsBackground = true;
_consumerService.Start();
}
}
public bool Enqueue(Delegate value)
{
lock (_sync)
{
// Don't allow anybody to enqueue
if( _canEnqueue )
{
_queue.Enqueue(value);
_ready.Set();
return true;
}
}
// Whoever is calling Enqueue should try again later.
return false;
}
// The consumer blocks until the producer puts something in the queue.
private void Run()
{
try
{
while (true)
{
// Wait for a query to be enqueued
_ready.WaitOne();
// Process the query
lock (_sync)
{
if (_queue.Count > 0)
{
Delegate query = _queue.Dequeue();
query.DynamicInvoke(null);
}
else
{
_canEnqueue = true;
_ready.Reset();
_wait.Set();
continue;
}
}
}
}
catch (ThreadInterruptedException)
{
_queue.Clear();
return;
}
}
// Block your queue from enqueuing, return null
// if the queue is already empty.
public ManualResetEvent BlockQueue()
{
lock(_sync)
{
if( _queue.Count > 0 )
{
_canEnqueue = false;
_wait.Reset();
}
else
{
// You need to tell the caller that they can't
// block your queue while it's empty. The caller
// should check if the result is null before calling
// WaitOne().
return null;
}
}
return _wait;
}
protected override void Dispose(bool disposing)
{
lock (_sync)
{
if (_consumerService != null)
{
_consumerService.Interrupt();
// Set wait when you're disposing the queue
// so that nobody is left with a lingering wait.
_wait.Set();
}
}
base.Dispose(disposing);
}
}
Related
I need to code my own FIFO/strong semaphore in C#, using a semaphore class of my own as a base. I found this example, but it's not quite right since I'm not supposed to be using Monitor.Enter/Exit yet.
These are the methods for my regular semaphore, and I was wondering if there was a simple way to adapt it to be FIFO.
public virtual void Acquire()
{
lock (this)
{
while (uintTokens == 0)
{
Monitor.Wait(this);
}
uintTokens--;
}
}
public virtual void Release(uint tokens = 1)
{
lock (this)
{
uintTokens += tokens;
Monitor.PulseAll(this);
}
}
So SemaphoreSlim gives us a good starting place, so we'll begin by wrapping one of those in a new class, and directing everything but the wait method to that semaphore.
To get a queue like behavior we'll want a queue object, and to make sure it's safe in the face of multithreaded access, we'll use a ConcurrentQueue.
In this queue we'll put TaskCompletionSource objects. When we want to have something start waiting it can create a TCS, add it to the queue, and then inform the semaphore to asynchronously pop the next item off of the queue and mark it as "completed" when the wait finishes. We'll know that there will always be an equal or lesser number of continuations as there are items in the queue.
Then we just wait on the Task from the TCS.
We can also trivially create a WaitAsync method that returns a task, by just returning it instead of waiting on it.
public class SemaphoreQueue
{
private SemaphoreSlim semaphore;
private ConcurrentQueue<TaskCompletionSource<bool>> queue =
new ConcurrentQueue<TaskCompletionSource<bool>>();
public SemaphoreQueue(int initialCount)
{
semaphore = new SemaphoreSlim(initialCount);
}
public SemaphoreQueue(int initialCount, int maxCount)
{
semaphore = new SemaphoreSlim(initialCount, maxCount);
}
public void Wait()
{
WaitAsync().Wait();
}
public Task WaitAsync()
{
var tcs = new TaskCompletionSource<bool>();
queue.Enqueue(tcs);
semaphore.WaitAsync().ContinueWith(t =>
{
TaskCompletionSource<bool> popped;
if (queue.TryDequeue(out popped))
popped.SetResult(true);
});
return tcs.Task;
}
public void Release()
{
semaphore.Release();
}
}
I have created a FifoSemaphore class and I am successfully using it in my solutions. Current limitation is that it behaves like a Semaphore(1, 1).
public class FifoSemaphore
{
private readonly object lockObj = new object();
private List<Semaphore> WaitingQueue = new List<Semaphore>();
private Semaphore RequestNewSemaphore()
{
lock (lockObj)
{
Semaphore newSemaphore = new Semaphore(1, 1);
newSemaphore.WaitOne();
return newSemaphore;
}
}
#region Public Functions
public void Release()
{
lock (lockObj)
{
WaitingQueue.RemoveAt(0);
if (WaitingQueue.Count > 0)
{
WaitingQueue[0].Release();
}
}
}
public void WaitOne()
{
Semaphore semaphore = RequestNewSemaphore();
lock (lockObj)
{
WaitingQueue.Add(semaphore);
semaphore.Release();
if(WaitingQueue.Count > 1)
{
semaphore.WaitOne();
}
}
semaphore.WaitOne();
}
#endregion
}
Usage is just like with a regular semaphore:
FifoSemaphore fifoSemaphore = new FifoSemaphore();
On each thread:
fifoSemaphore.WaitOne();
//do work
fifoSemaphore.Release();
I wonder how to abort my Thread after my function ends Thread.Abort();
My application running files and each file is opened is different thread
int _counter;
int _parallelThreads
_queue = new Queue();
public void transmit()
{
while (_counter < _parallelThreads)
{
lock (_queue)
{
string file = (string)_queue.Dequeue();
ThreadStart ts = delegate { processFile(file); };
Thread thread = new Thread(ts);
thread.IsBackground = true;
thread.Start();
_counter++;
}
}
}
private void processFile(string file)
{
WiresharkFile wf = new WiresharkFile(file, _selectedOutputDevice, 1);
wf.OnFinishPlayEvent += wf_OnFinishPlayEvent;
wf.sendBuffer();
}
and this is the event that my file finished
private void wf_OnFinishPlayEvent(MyClass class)
{
// here i want to abort my thread
}
The reason i want to abort my thread when it finished is because i think this is my memory lack reason in case i open a lot of parallels thread and run it over ond over (my application memory usage read more than 1 giga)
when you abort a thread, a lot of unexpected things can go wrong. particularly when you work with files. when i had to do that (for example, a "cancel" button) i used a litlle trick.
i had a flag IsCanceled on a scope both threads can see be set to true, and on the worker thread, every few statement, will check that flag and close all open files and end itself.
this might not work well for your situation, depending on wf.sendBuffer(); logic. let me know
Example:
private void processFile(string file)
{
WiresharkFile wf = new WiresharkFile(file, _selectedOutputDevice, 1);
wf.OnFinishPlayEvent += wf_OnFinishPlayEvent;
if(IsCanceled == false)
{
wf.sendBuffer();
}
}
and if the sendBuffer() method logic is too long, then
public void sendBuffer()
{
// some logic
if(IsCanceled)
{
// close open streams
return;
}
// some logic
}
as for the flag itself, a singleton class could do just fine for that, or a class all the other classes know
public class Singleton
{
private static Singleton instance;
private bool isCanceled;
private Singleton()
{
isCanceled = false;
}
public static Singleton Instance
{
get
{
if (instance == null)
{
instance = new Singleton();
}
return instance;
}
}
public bool IsCanceled
{
get
{
return isCanceled;
}
set
{
isCanceled = value;
}
}
}
notice that the singleton class is open to everyone, and you might want to use a class only known by the threads that needs to check it. that is something that depend on your security needs.
You should not abort the threads, threads will quit automatically when the code in it finishes. Maybe you just want to wait the thread to finish, after that do something else.
You can use an array to store the thread, and use Thread.Join() to wait all the threads end.
List<Thread> threadList = new List<Thread>();
public void transmit()
{
while (_counter < _parallelThreads)
{
lock (_queue)
{
string file = (string)_queue.Dequeue();
ThreadStart ts = delegate { processFile(file); };
Thread thread = new Thread(ts);
thread.IsBackground = true;
threadList.Add(thread); //add thread to list
thread.Start();
_counter++;
}
}
//wait threads to end
foreach(Thread t in threadList)
t.Join();
}
private void processFile(string file)
{
WiresharkFile wf = new WiresharkFile(file, _selectedOutputDevice, 1);
wf.OnFinishPlayEvent += wf_OnFinishPlayEvent;
wf.sendBuffer();
}
I am developing an application simulating a network comprising a number of nodes that exchange messages. I try to simulate the transmission channel with a Queue where every node can place a message. Then, another entity takes over the message and delivers it to the specified node. Then, i want to signal (with an event) the end of a transmission phase when the message queue is idle for a certain amount of time, say X, namely no new messages have been added to the queue for X mseconds.
I understand that my case follows the consumer/producer paradigm. So far, i have done the following:
public class Com<T>
{
private Thread dispatcher;
private Queue<T> queue;
private int waitTime;
private Object locker;
private Timer timer;
public event EventHandler EmptyQueueEvent;
public Com()
{
queue = new Queue<T>();
locker = new Object();
waitTime = X;
timer = new Timer(FireEmpty, null, Timeout.Infinite,Timeout.Infinite);
dispatcher = new Thread(Serve);
dispatcher.IsBackground = true;
dispatcher.Start();
}
private void Serve()
{
while (true)
{
try
{
if (queue.Count == 0)
{
timer.Change(waitTime, 0);
Thread.Sleep(Timeout.Infinite);
}
}
catch (ThreadInterruptedException)
{
}
while (queue.Count != 0)
{
lock (locker)
{
deliver(queue.Dequeue());
}
}
}
}
private void deliver(T item)
{
// Do stuff
}
public void Add(T item)
{
timer.Change(Timeout.Infinite, Timeout.Infinite);
lock (locker)
{
queue.Enqueue(item);
}
dispatcher.Interrupt();
}
private void FireEmpty(object o)
{
//Fire Event
}
}
However, running my simulations proves that my synchronization is not enough, since I am sometimes getting a "ThreadInterruptedException" while trying to dequeue my message (in method Serve()). Note that the exception does not occur each time i run the simulation, but rather rarely: approximately every 850-1000 executions (i am running the execution iteratively)..
Does anybody have an idea what it is wrong with my code? :)
Have you tried locking before you attempt to get the Queue count? Like:
private void Serve()
{
while (true)
{
try
{
int count = 0;
lock(locker)
count= queue.Count;
if (count == 0)
{
timer.Change(waitTime, 0);
Thread.Sleep(Timeout.Infinite);
}
}
catch (ThreadInterruptedException)
{
}
while (queue.Count != 0)
{
lock (locker)
{
deliver(queue.Dequeue());
}
}
}
}
It's possible that an add is getting called at the same time you're trying to count the number of items. Also, you might want to consider one of the collections from System.Collections.Concurrent if you're using .net 4.0.
** UPDATE **
I just took a closer look at your code and had an "Oh duh" moment. You should be getting a ThreadInterruptException because you're calling delegate.Interrupt(). Check the MSDN documentation on that. I think what you need to do is use something like a ManualResetEvent and instead of calling Interrupt() do a WaitOne() on that event.
** UPDATE2 **
Here's some sample code that includes my other locking suggestion as well (on Gist):
https://gist.github.com/1683547
I need to create an method invoker that any thread (Thread B for example sake) can call, which will execute on the main executing thread (Thead A) at a specific given point in its execution.
Example usage would be as follows:
static Invoker Invoker = new Invoker();
static void ThreadA()
{
new Thread(ThreadB).Start();
Thread.Sleep(...); // Hypothetic Alpha
Invoker.Invoke(delegate { Console.WriteLine("Action"); }, true);
Console.WriteLine("Done");
Console.ReadLine();
}
static void ThreadB()
{
Thread.Sleep(...); // Hypothetic Beta
Invoker.Execute();
}
The Invoker class looks like this:
public class Invoker
{
private Queue<Action> Actions { get; set; }
public Invoker()
{
this.Actions = new Queue<Action>();
}
public void Execute()
{
while (this.Actions.Count > 0)
{
this.Actions.Dequeue()();
}
}
public void Invoke(Action action, bool block = true)
{
ManualResetEvent done = new ManualResetEvent(!block);
this.Actions.Enqueue(delegate
{
action();
if (block) done.Set();
});
if (block)
{
done.WaitOne();
}
}
}
This works fine in most cases, although it won't if, for any reason, the execution (and therefore the Set) is done before the WaitOne, in which case it will just freeze (it allows for the thread to proceed, then blocks). That could be reproduced if Alpha >> Beta.
I can use booleans and whatnot, but I'm never getting a real atomic safety here. I tried some fixes, but they wouldn't work in the case where Beta >> Alpha.
I also thought of locking around both the Invoker.Execute and Invoker.Invoke methods so that we are guaranteed that the execution does not occur between enqueing and waiting. However, the problem is that the lock also englobes the WaitOne, and therefore never finishes (deadlock).
How should I go about getting absolute atomic safety in this paradigm?
Note: It really is a requirement that I work with this design, from external dependencies. So changing design is not a real option.
EDIT: I did forget to mention that I want a blocking behaviour (based on bool block) until the delegate is executed on the Invoke call.
Use a Semaphore(Slim) instead of the ManualResetEvent.
Create a semaphore with an maximum count of 1, call WaitOne() in the calling thread, and call Release() in the delegate.
If you've already called Release(), WaitOne() should return immediately.
Make sure to Dispose() it when you're done, preferably in a using block.
If block is false, you shouldn't create it in the first place (although for SemaphoreSlim, that's not so bad).
You can use my technique:
public void BlockingInvoke(Action action)
{
volatile bool isCompleted = false;
volatile bool isWaiting = false;
ManualResetEventSlim waiter = new ManualResetEventSlim();
this.Actions.Enqueue(delegate
{
action();
isCompleted = true;
Thread.MemoryBarrier();
if (!isWaiting)
waiter.Dispose();
else
waiter.Set();
});
isWaiting = true;
Thread.MemoryBarrier();
if (!isCompleted)
waiter.Wait();
waiter.Dispose();
}
Untested
I'm answering only to show the implementation SLaks described and my solution to ensure proper and unique disposal with locks. It's open to improvement and criticism, but it actually works.
public class Invoker
{
private Queue<Action> Actions { get; set; }
public Invoker()
{
this.Actions = new Queue<Action>();
}
public void Execute()
{
while (this.Actions.Count > 0)
{
this.Actions.Dequeue()();
}
}
public void Invoke(Action action, bool block = true)
{
if (block)
{
SemaphoreSlim semaphore = new SemaphoreSlim(1);
bool disposed = false;
this.Actions.Enqueue(delegate
{
action();
semaphore.Release();
lock (semaphore)
{
semaphore.Dispose();
disposed = true;
}
});
lock (semaphore)
{
if (!disposed)
{
semaphore.Wait();
semaphore.Dispose();
}
}
}
else
{
this.Actions.Enqueue(action);
}
}
}
Greetings, I am trying to play some audio files without holding up the GUI. Below is a sample of the code:
if (audio)
{
if (ThreadPool.QueueUserWorkItem(new WaitCallback(CoordinateProc), fireResult))
{
}
else
{
MessageBox.Show("false");
}
}
if (audio)
{
if (ThreadPool.QueueUserWorkItem(new WaitCallback(FireProc), fireResult))
{
}
else
{
MessageBox.Show("false");
}
}
if (audio)
{
if (ThreadPool.QueueUserWorkItem(new WaitCallback(HitProc), fireResult))
{
}
else
{
MessageBox.Show("false");
}
}
The situation is the samples are not being played in order. some play before the other and I need to fix this so the samples are played one after another in order.
How do I implement this please?
Thank you.
EDIT: ThreadPool.QueueUserWorkItem(new WaitCallback(FireAttackProc), fireResult);
I have placed all my sound clips in FireAttackProc. What this does not do and I want is: wait until the thread stops running before starting a new thread so the samples dont overlap.
Why not just create one "WorkItem" and do everything there?
You can't guarrantee the order of execution of thread pool threads. Rather than that, as suggested by others, use a single thread to run the procs in order. Add the audio procs to a queue, run a single thread that pulls each proc off the queue in order and calls them. Use an event wait handle to signal the thread each time a proc is added to the queue.
An example (this doesn't completely implement the Dispose pattern... but you get the idea):
public class ConcurrentAudio : IDisposable
{
public ConcurrentAudio()
{
_queue = new ConcurrentQueue<WaitCallback>();
_waitHandle = new AutoResetEvent(false);
_disposed = false;
_thread = new Thread(RunAudioProcs);
_thread.IsBackground = true;
_thread.Name = "run-audio";
_thread.Start(null); // pass whatever "state" you need
}
public void AddAudio(WaitCallback proc)
{
_queue.Enqueue(proc);
_waitHandle.Set();
}
public void Dispose()
{
_disposed = true;
_thread.Join(1000); // don't feel like waiting forever
GC.SuppressFinalize(this);
}
private void RunAudioProcs(object state)
{
while (!_disposed)
{
try
{
WaitCallback proc = null;
if (_queue.TryDequeue(out proc))
proc(state);
else
_waitHandle.WaitOne();
}
catch (Exception x)
{
// Do something about the error...
Trace.WriteLine(string.Format("Error: {0}", x.Message), "error");
}
}
}
private ConcurrentQueue<WaitCallback> _queue;
private EventWaitHandle _waitHandle;
private bool _disposed;
private Thread _thread;
}
You should have a look at the BackgroundWorker option !