I'm trying to use the producer consumer pattern to process and save some data. I'm using AutoResetEvent for signalling between the two therads here is the code I have
Here is the producer function
public Results[] Evaluate()
{
processingComplete = false;
resultQueue.Clear();
for (int i = 0; i < data.Length; ++i)
{
if (saveThread.ThreadState == ThreadState.Unstarted)
saveThread.Start();
//-....
//Process data
//
lock (lockobject)
{
resultQueue.Enqueue(result);
}
signal.Set();
}
processingComplete = true;
}
And here is the consumer function
private void SaveResults()
{
Model dataAccess = new Model();
while (!processingComplete || resultQueue.Count > 0)
{
if (resultQueue.Count == 0)
signal.WaitOne();
ModelResults result;
lock (lockobject)
{
result = resultQueue.Dequeue();
}
dataAccess.Save(result);
}
SaveCompleteSignal.Set();
}
So my issue is sometimes resultQueue.Dequeue() throws InvalidOperation exception because the Queue is empty. I'm not sure what I'm doing wrong shouldn't the signal.WaitOne() above that block the the queue is empty?
You have synchronization issues due to a lack of proper locking.
You should lock all of the queue access, including the count check.
In addition, using Thread.ThreadState in this manner is a "bad idea". From the MSDN docs for ThreadState:
"Thread state is only of interest in debugging scenarios. Your code should never use thread state to synchronize the activities of threads."
You can't rely on this as a means of handling synchronization. You should redesign to make sure the thread will be started before it's used. If it's not started, just don't initialize it. (You can always use a null check - if the thread's null, create it and start it).
You check the Queue's Count outside of a synchronized context. Since the Queue is not threadsafe, this could be a problem (possibly while Enqueue is in process Count return 1 but no item can be dequeued), and it would go seriously wrong if you were to use more than one consumer anyways.
You may want to read the threading articles written by Joseph Albahari, he has also a good sample for your problem as well as a "better" solution without OS synchronization objects.
You have to put lock() around all references to the queue. You also have some issues around identifying processing complete (at the end of the queue you'll get a signal but the queue will be empty).
public Results[] Evaluate()
{
processingComplete = false;
lock(lockobject)
{
resultQueue.Clear();
}
for (int i = 0; i < data.Length; ++i)
{
if (saveThread.ThreadState == ThreadState.Unstarted)
saveThread.Start();
//-....
//Process data
//
lock (lockobject)
{
resultQueue.Enqueue(result);
}
signal.Set();
}
processingComplete = true;
}
private void SaveResults()
{
Model dataAccess = new Model();
while (true)
{
int count;
lock(lockobject)
{
count = resultQueue.Count;
}
if (count == 0)
signal.WaitOne();
lock(lockobject)
{
count = resultQueue.Count;
}
// we got a signal, but queue is empty, processing is complete
if (count == 0)
break;
ModelResults result;
lock (lockobject)
{
result = resultQueue.Dequeue();
}
dataAccess.Save(result);
}
SaveCompleteSignal.Set();
}
Related
Suppose we have many randomly incoming threads accessing same resource in parallel. To access the resource thread needs to acquire a lock. If we could pack N incoming threads into one request resource usage would be N times more efficient. Also we need to answer individual request as fast as possible. What is the best way/pattern to do that in C#?
Currently I have something like that:
//batches lock
var ilock = ModifyBatch.GetTableDeleteBatchLock(table_info.Name);
lock (ilock)
{
// put the request into requests batch
if (!ModifyBatch._delete_batch.ContainsKey(table_info.Name))
{
ModifyBatch._delete_batch[table_info.Name] = new DeleteData() { Callbacks = new List<Action<string>>(), ids = ids };
}
else
{
ModifyBatch._delete_batch[table_info.Name].ids.UnionWith(ids);
}
//this callback will get called once the job is done by a thread that will acquire resource lock
ModifyBatch._delete_batch[table_info.Name].Callbacks.Add(f =>
{
done = true;
error = f;
});
}
bool lockAcquired = false;
int maxWaitMs = 60000;
DeleteData _delete_data = null;
//resource lock
var _write_lock = GetTableWriteLock(typeof(T).Name);
try
{
DateTime start = DateTime.Now;
while (!done)
{
lockAcquired = Monitor.TryEnter(_write_lock, 100);
if (lockAcquired)
{
if (done) //some other thread did our job
{
Monitor.Exit(_write_lock);
lockAcquired = false;
break;
}
else
{
break;
}
}
Thread.Sleep(100);
if ((DateTime.Now - start).TotalMilliseconds > maxWaitMs)
{
throw new Exception("Waited too long to acquire write lock?");
}
}
if (done) //some other thread did our job
{
if (!string.IsNullOrEmpty(error))
{
throw new Exception(error);
}
else
{
return;
}
}
//not done, but have write lock for the table
lock (ilock)
{
_delete_data = ModifyBatch._delete_batch[table_info.Name];
var oval = new DeleteData();
ModifyBatch._delete_batch.TryRemove(table_info.Name, out oval);
}
if (_delete_data.ids.Any())
{
//doing the work with resource
}
foreach (var cb in _delete_data.Callbacks)
{
cb(null);
}
}
catch (Exception ex)
{
if (_delete_data != null)
{
foreach (var cb in _delete_data.Callbacks)
{
cb(ex.Message);
}
}
throw;
}
finally
{
if (lockAcquired)
{
Monitor.Exit(_write_lock);
}
}
If it is OK to process the task outside the scope of the current request, i.e. to queue it for later, then you can think of a sequence like this1:
Implement a resource lock (monitor) and a List of tasks.
For each request:
Lock the List, Add current task to the List, remember nr. of tasks in the List, unlock the List.
Try to acquire the lock.
If unsuccessful:
If the nr. of tasks in the list < threshold X, then Return.
Else Acquire the Lock (will block)
Lock the List, move it's contents to a temp list, unlock the List.
If temp list is not empty
Execute the tasks in the temp list.
Repeat from step 5.
Release the lock.
The first request will go through the whole sequence. Subsequent requests, if the first is still executing, will short-circuit at step 4.
Tune for the optimal threshold X (or change it to a time-based threshold).
1 If you need to wait for the task in the scope of the request, then you need to extend the process slightly:
Add two fields to the Task class: completion flag and exception.
At step 4, before Returning, wait for the task to complete (Monitor.Wait) until its completion flag becomes true. If exception is not null, throw it.
At step 6, for each task, set the completion flag and optionally the exception and then notify the waiters (Monitor.PulseAll).
I have been trying to use a Semaphore to control number of requests that my service can handle. I.E.
class Service : IDisposable {
SemaphoreSlim s = new SemaphoreSlim(InitialCapacity);
....
async void ProcessRequest() {
await s.WaitAsync();
try {
......
} finally {
s.Release();
}
}
}
There are 2 problems I am encountering that I am not sure how to solve. I have been using similar hacks to resolve these problems, but I wonder if there is any better way
I want to be able to dynamically change the capacity of my service class, so I have something like this.
void ChangeCapacity(int newCapacity) {
int extraRequestsCount = newCapacity - oldCapacity;
if (extraRequestsCount > 0) {
s.Release(extraRequestsCount);
}
else if (extraRequestsCount < 0) {
for (int i = 0; i < -extraRequestsCount; i++) {
s.WaitAsync(); // try to steal some resources, over time...
}
}
}
At the dispose method, I want to make sure all the request processing completes before I dispose the semaphore, otherwise the s.Release() call in my ProcessRequest() would throw ObjectDisposedException so I did the following
public void Dispose() {
if (s!= null) {
for (int i = 0; i < oldCapacity; i++) {
s.Wait();
}
s.Dispose();
}
}
Note that I have been using loop to manually wait many times. This is really slow if the capacity is large. Is there a better way to do this? There's a Release(int count) for semaphore why isn't there a Wait(int count)?
What I'd probably do is when the adjustment is made, replace your semaphore instance with a new semaphore with the desired capacity, and assign all future work to that semaphore. The existing and now dereferenced semaphore will not be garbage collected until all threads are done referencing it, so it should be safe to do this as long as you assign the semaphore variable locally to each thread.
I have a question, and I could do with some code examples to help me, and I feel it may help to give some background.
I have the need to create an engine of 3 Queues (in C#, winforms). The 3 Queues merely contain an "action" object. Actions get thrown into the engine, and stick themselves to the "most available" Queue (basically, the Queue with the lowest count). Almost all of the time the Queues can run discretely and asynchronously with no harm. However there is one "Action" situation which may happen, and when that type of "Action" occurs and does bubble to the front of a Queue, it must :
wait for the other queues to stop their current actions
lock/pause them when they are finished on their current Action
run the Action alone until it finishes
release the lock on the other 2 queues.
With the added issue that any of the 3 queues can lock the other 2.
Does anyone have any experience of this?
I hope so, it seems a bit painful :-) Thanks in advance
This is a combination of the single queue approach recommended by Servy and the ReaderWriterLock suggestion by Casperah.
ReaderWriterLockSlim throttler = new ReaderWriterLockSlim();
for (int i = 0; i < numWorkers; i++)
{
Task.Factory.StartNew(() =>
{
foreach (Action nextAction in queue.GetConsumingEnumerable())
{
if (mustBeExectutedSerially(nextAction))
{
try
{
throttler.EnterWriteLock();
nextAction();
}
finally
{
throttler.ExitWriteLock();
}
}
else
{
try
{
throttler.EnterReadLock();
nextAction();
}
finally
{
throttler.ExitReadLock();
}
}
}
});
}
First off, I wouldn't suggest using three queues. I'd suggest using one queue and just have 3 different tasks reading from it. I'd also suggest using BlockingCollection<T> (which is just a wrapper for a ConcurrentQueue as it's easier to work with.
As for the rest, a ReaderWriterLockSlim (Thanks Casperah) should handle it easy enough. A Writer requires an exclusive lock, and a reader only locks out other writers, which is exactly your use case.
var queue = new BlockingCollection<Action>();
int numWorkers = 3;
ReaderWriterLockSlim throttler = new ReaderWriterLockSlim();
for (int i = 0; i < numWorkers; i++)
{
Task.Factory.StartNew(() =>
{
foreach (Action nextAction in queue.GetConsumingEnumerable())
{
if (mustBeExectutedSerially(nextAction))
{
try
{
throttler.EnterWriteLock();
nextAction();
}
finally
{
throttler.ExitWriteLock();
}
}
else
{
try
{
throttler.EnterReadLock();
nextAction();
}
finally
{
throttler.ExitReadLock();
}
}
}
});
}
It seems that a System.Threading.ReaderWriterLock will do the job for you.
A normal task should do this:
readerWriterLock.AcquireReaderLock(timeout);
try
{
RunNormalAction();
}
finally
{
readerWriterLock.ReleaseReaderLock();
}
And the advanced task should do this:
readerWriterLock.AcquireWriterLock(timeout);
try
{
RunSpecialAction();
}
finally
{
readerWriterLock.ReleaseWriterLock();
}
You can start as many ReaderLocks as you want, and they will keep running as expected.
When a WriterLock is Acquired all the ReaderLocks has been released and only one WriterLock will run at a time.
My humble sugestion:
Create three objects
object threadlock1 = new object();
object threadlock2 = new object();
object threadlock3 = new object();
Each thread acquires lock over one object before running any action.
lock (threadlock1) // On thread 1, for example
{ //Run Action }
When THE action comes, the thread with THE action must acquire lock over the three objects, thus waiting for the other threads to finish their work, and preventing them from doing any more.
lock (threadlock1) // On thread 1, for example
{
lock (threadlock2)
{
lock (threadlock3)
{
//Run THE Action
}
}
}
When THE action is finished, you release all three locks, and all is back to normal, with each thread holding it's own lock, and resuming actions.
If there are multiple threads all waiting on the same lock is it possible to have the Main thread have higher priority in acquiring the lock. Meaning that if worker threads go to the lock statement before the main thread, the main thread would acquire the lock before the other threads that were already waiting on it.
No, the lock statement maps to System.Threading.Monitor.Enter() (MSDN) and there is no overload that accepts a priority parameter.
The closest thing I can think of is a ReaderWriterLock(Slim) but I would seriously reconsider the design that leads to this request. There probably are better ways to achieve what you need.
Through a native lock statement, no. Through your own custom locking mechanism, sure, if you're willing to spend the time and effort to develop it.
Here's my draft a a solution. It may or may not work, and may not be super efficient, but it's at least a starting place:
public class Lock
{
bool locked = false;
private object key = new object();
SortedDictionary<int, Queue<ManualResetEvent>> notifiers =
new SortedDictionary<int, Queue<ManualResetEvent>>();
ManualResetEvent specialNotifier = null;
public void Lock()
{
lock (key)
{
if (locked)
{
ManualResetEvent notifier = new ManualResetEvent(false);
int priority = getPriorityForThread();
Queue<ManualResetEvent> queue = notifiers[priority];
if (queue == null)
{
queue = new Queue<ManualResetEvent>();
notifiers[priority] = queue;
}
queue.Enqueue(notifier);
notifier.WaitOne();
}
else
{
locked = true;
}
}
}
private static int getPriorityForThread()
{
return 0;
}
public void Release()
{
lock (key)
{
foreach (var queue in notifiers.Values)
{
if (queue.Any())
{
var notifier = queue.Dequeue();
notifier.Set();
return;
}
}
locked = false;
}
}
}
Here is another solution. I has a lot of lines, but it is pretty simple. The function DoSomethingSingle will be called only one thread at a time, and those with the highPriority flag will get preference.
static int numWaiting = 0;
static object single = new object();
ResultType DoSomething(string[] argList, bool highPriority = false)
{
try
{
if (highPriority)
{
Interlocked.Increment(ref numWaiting);
}
for (;;)
{
lock (single)
{
if (highPriority || numWaiting == 0)
{
return DoSomethingSingle(argList);
}
}
// Sleep gives other threads a chance to enter the lock
Thread.Sleep(0);
}
}
finally
{
if (highPriority)
{
Interlocked.Decrement(ref numWaiting);
}
}
}
This allows two priority levels. Guaranteed that a low priority thread will gain access to the resource only if there are no high priority threads waiting for it.
edit: change to interlock incr/dec
I'm trying to implement a concurrent producer-consumer collection (multiple producers and consumers) that supports timeouts for consumers.
Now the actual collection is pretty complicated (nothing in System.Collections.Concurrent that does the job unfortunately), but I have a minimal sample here that demonstrates my problem (looks a bit like BlockingCollection<T>).
public sealed class ProducerConsumerQueueDraft<T>
{
private readonly Queue<T> queue = new Queue<T>();
private readonly object locker = new object();
public void Enqueue(T item)
{
lock (locker)
{
queue.Enqueue(item);
/* This "optimization" is broken, as Nicholas Butler points out.
if(queue.Count == 1) // Optimization
*/
Monitor.Pulse(locker); // Notify any waiting consumer threads.
}
}
public T Dequeue(T item)
{
lock (locker)
{
// Surprisingly, this needs to be a *while* and not an *if*
// which is the core of my problem.
while (queue.Count == 0)
Monitor.Wait(locker);
return queue.Dequeue();
}
}
// This isn't thread-safe, but is how I want TryDequeue to look.
public bool TryDequeueDesired(out T item, TimeSpan timeout)
{
lock (locker)
{
if (queue.Count == 0 && !Monitor.Wait(locker, timeout))
{
item = default(T);
return false;
}
// This is wrong! The queue may be empty even though we were pulsed!
item = queue.Dequeue();
return true;
}
}
// Has nasty timing-gymnastics I want to avoid.
public bool TryDequeueThatWorks(out T item, TimeSpan timeout)
{
lock (locker)
{
var watch = Stopwatch.StartNew();
while (queue.Count == 0)
{
var remaining = timeout - watch.Elapsed;
if (!Monitor.Wait(locker, remaining < TimeSpan.Zero ? TimeSpan.Zero : remaining))
{
item = default(T);
return false;
}
}
item = queue.Dequeue();
return true;
}
}
}
The idea is straightforward: consumers who find an empty queue wait to be signaled, and producers Pulse (note: not PulseAll, which would be inefficient) them to notify them of a waiting item.
My problem is this property of Monitor.Pulse:
When the thread that invoked Pulse releases the lock, the next
thread in the ready queue (which is not necessarily the thread that
was pulsed) acquires the lock.
What this means is that consumer-thread C1 could be woken up by a producer-thread to consume an item, but another consumer-thread C2 could acquire the lock before C1 has a chance to reacquire it, and consume the item, leaving C1 with an empty queue when it is given control.
This means I have to defensively check in the consumer code on every pulse if the queue is indeed non-empty, and go back and wait empty-handed if this not the case.
My primary issue with this is that it inefficient - threads may be woken up to do work and then promptly sent back to wait again. A related consequence of this is that implementing a TryDequeue with a timeout is unnecessarily difficult and inefficient (see TryDequeueThatWorks) when it should be elegant (see TryDequeueDesired).
How can I twist Monitor.Pulse to do what I want? Alternatively, is there another synchronization primitive that does? Is there a more efficient and/or elegant way to implement a TryDequeue timeout than what I have done?
FYI, here's a test that demonstrates the issues with my desired solution:
var queue = new ProducerConsumerQueueDraft<int>();
for (int consumer = 0; consumer < 3; consumer++)
new Thread(() =>
{
while (true)
{
int item;
// This call should occasionally throw an exception.
// Switching to queue.TryDequeueThatWorks should make
// the problem go away.
if (queue.TryDequeueDesired(out item, TimeSpan.FromSeconds(1)))
{
// Do nothing.
}
}
}).Start();
Thread.Sleep(1000); // Let consumers get up and running
for (int itemIndex = 0; itemIndex < 50000000; itemIndex++)
{
queue.Enqueue(0);
}
My primary issue with this is that it inefficient
It is not. You assume that this is a common occurrence but this kind of race happens very rarely. Once in a Blue Moon, at best. The while loop is necessary to ensure nothing goes wrong when it does occur. And it will. Don't mess with it.
It is in fact the opposite, the lock design is efficient because it does allow a race to occur. And deals with it. Tinkering with locking designs is so very dangerous because the races don't happen frequently enough. They are horribly random which prevents sufficient testing to prove that the alterations don't cause failure. Adding any instrumenting code doesn't work either, it alters the timing.
I wrote an article about this that may help:
Thread synchronization: Wait and Pulse demystified
In particular, it explains why a while loop is necessary.
Here's a simple key-based conflating producer-consumer queue:
public class ConflatingConcurrentQueue<TKey, TValue>
{
private readonly ConcurrentDictionary<TKey, Entry> entries;
private readonly BlockingCollection<Entry> queue;
public ConflatingConcurrentQueue()
{
this.entries = new ConcurrentDictionary<TKey, Entry>();
this.queue = new BlockingCollection<Entry>();
}
public void Enqueue(TValue value, Func<TValue, TKey> keySelector)
{
// Get the entry for the key. Create a new one if necessary.
Entry entry = entries.GetOrAdd(keySelector(value), k => new Entry());
// Get exclusive access to the entry.
lock (entry)
{
// Replace any old value with the new one.
entry.Value = value;
// Add the entry to the queue if it's not enqueued yet.
if (!entry.Enqueued)
{
entry.Enqueued = true;
queue.Add(entry);
}
}
}
public bool TryDequeue(out TValue value, TimeSpan timeout)
{
Entry entry;
// Try to dequeue an entry (with timeout).
if (!queue.TryTake(out entry, timeout))
{
value = default(TValue);
return false;
}
// Get exclusive access to the entry.
lock (entry)
{
// Return the value.
value = entry.Value;
// Mark the entry as dequeued.
entry.Enqueued = false;
entry.Value = default(TValue);
}
return true;
}
private class Entry
{
public TValue Value { get; set; }
public bool Enqueued { get; set; }
}
}
(This may need a code review or two, but I think in general it's sane.)