Develop Reference

.Net Mutex, lock, not working on worker threads

We have a function being called within a ASP.Net (Blazor) application simply within the processing of an HTTP request. It is on a Scoped, injected object though this is irrelevant to the problem at hand. Also the function is synchronous (does not have async) though it may be called from async functions.
A part of this functions code needs to be run in mutual exclusion. I thought this should be the simplest thing in the world and wrote the following code
using (Mutex mutex = new(true, "SyncObject")) { ... }
Basically I created a named Mutex, which should globally prevent more than 1 thread entering the block. To my surprise this did not work and with breakpoints I could see that multiple WorkerThreads entered the block of code.
After a lot of research I found that .Net has two namespaces 'Local' and 'Global' for synchronization object, since I was only interested in the 'Local' I did not need to make any change but out of frustration, I added it to the Global namespace and tried it but no luck.
using (Mutex mutex = new(true, "Global\\SyncObject")) { ... }
The above code did not work either and multiple threads entered the code.
I considered the possibility that the Worker threads may not be System threads and therefore the ownership if the mutex is always granted, then how to synchronize across two async methods becomes a question. Also since the function is synchronous a single thread would not be able to re-enter it until it completes.
using (Mutex mutex = new(false, "Global\\SyncObject")) {
mutex.WaitOne()
...
mutex.ReleaseMutex()
}
No luck.
Since the named mutex refused to work I tried creating a static mutex object as
private static Object mutex = new();
I tried using the above in a lock statement as
lock(mutex) {...}
This did not work either.
I found this to be amazingly strange. The behaviour of the sync objects indicates a single system thread, but then what can be done to sync whatever artificial threads .Net is creating and how can an artificial thread re-enter the function, this is not logical.
After digging in a bit I was able to see that these are indeed 2 threads, which makes sense and is as per expectation, but why won't the mutex / lock work?

The documentation suggests that you have a static readonly mutex object (static so that it is shared between instances)
class Test13
{
// Create a new Mutex. The creating thread does not own the
// Mutex.
private static Mutex mut = new Mutex();
private const int numIterations = 1;
private const int numThreads = 3;
static void Main()
{
// Create the threads that will use the protected resource.
for(int i = 0; i < numThreads; i++)
{
Thread myThread = new Thread(new ThreadStart(MyThreadProc));
myThread.Name = String.Format("Thread{0}", i + 1);
myThread.Start();
}
// The main thread exits, but the application continues to
// run until all foreground threads have exited.
}
private static void MyThreadProc()
{
for(int i = 0; i < numIterations; i++)
{
UseResource();
}
}
// This method represents a resource that must be synchronized
// so that only one thread at a time can enter.
private static void UseResource()
{
// Wait until it is safe to enter.
mut.WaitOne();
Console.WriteLine("{0} has entered the protected area",
Thread.CurrentThread.Name);
// Place code to access non-reentrant resources here.
// Simulate some work.
Thread.Sleep(500);
Console.WriteLine("{0} is leaving the protected area\r\n",
Thread.CurrentThread.Name);
// Release the Mutex.
mut.ReleaseMutex();
}
}

Understanding the C# lock [duplicate]

I am having hard time in understanding Wait(), Pulse(), PulseAll(). Will all of them avoid deadlock? I would appreciate if you explain how to use them?

Short version:
lock(obj) {...}
is short-hand for Monitor.Enter / Monitor.Exit (with exception handling etc). If nobody else has the lock, you can get it (and run your code) - otherwise your thread is blocked until the lock is aquired (by another thread releasing it).
Deadlock typically happens when either A: two threads lock things in different orders:
thread 1: lock(objA) { lock (objB) { ... } }
thread 2: lock(objB) { lock (objA) { ... } }
(here, if they each acquire the first lock, neither can ever get the second, since neither thread can exit to release their lock)
This scenario can be minimised by always locking in the same order; and you can recover (to a degree) by using Monitor.TryEnter (instead of Monitor.Enter/lock) and specifying a timeout.
or B: you can block yourself with things like winforms when thread-switching while holding a lock:
lock(obj) { // on worker
this.Invoke((MethodInvoker) delegate { // switch to UI
lock(obj) { // oopsiee!
...
}
});
}
The deadlock appears obvious above, but it isn't so obvious when you have spaghetti code; possible answers: don't thread-switch while holding locks, or use BeginInvoke so that you can at least exit the lock (letting the UI play).
Wait/Pulse/PulseAll are different; they are for signalling. I use this in this answer to signal so that:
Dequeue: if you try to dequeue data when the queue is empty, it waits for another thread to add data, which wakes up the blocked thread
Enqueue: if you try and enqueue data when the queue is full, it waits for another thread to remove data, which wakes up the blocked thread
Pulse only wakes up one thread - but I'm not brainy enough to prove that the next thread is always the one I want, so I tend to use PulseAll, and simply re-verify the conditions before continuing; as an example:
while (queue.Count >= maxSize)
{
Monitor.Wait(queue);
}
With this approach, I can safely add other meanings of Pulse, without my existing code assuming that "I woke up, therefore there is data" - which is handy when (in the same example) I later needed to add a Close() method.

Simple recipe for use of Monitor.Wait and Monitor.Pulse. It consists of a worker, a boss, and a phone they use to communicate:
object phone = new object();
A "Worker" thread:
lock(phone) // Sort of "Turn the phone on while at work"
{
while(true)
{
Monitor.Wait(phone); // Wait for a signal from the boss
DoWork();
Monitor.PulseAll(phone); // Signal boss we are done
}
}
A "Boss" thread:
PrepareWork();
lock(phone) // Grab the phone when I have something ready for the worker
{
Monitor.PulseAll(phone); // Signal worker there is work to do
Monitor.Wait(phone); // Wait for the work to be done
}
More complex examples follow...
A "Worker with something else to do":
lock(phone)
{
while(true)
{
if(Monitor.Wait(phone,1000)) // Wait for one second at most
{
DoWork();
Monitor.PulseAll(phone); // Signal boss we are done
}
else
DoSomethingElse();
}
}
An "Impatient Boss":
PrepareWork();
lock(phone)
{
Monitor.PulseAll(phone); // Signal worker there is work to do
if(Monitor.Wait(phone,1000)) // Wait for one second at most
Console.Writeline("Good work!");
}

No, they don't protect you from deadlocks. They are just more flexible tools for thread synchronization. Here is a very good explanation how to use them and very important pattern of usage - without this pattern you will break all the things:
http://www.albahari.com/threading/part4.aspx

Something that total threw me here is that Pulse just gives a "heads up" to a thread in a Wait. The Waiting thread will not continue until the thread that did the Pulse gives up the lock and the waiting thread successfully wins it.
lock(phone) // Grab the phone
{
Monitor.PulseAll(phone); // Signal worker
Monitor.Wait(phone); // ****** The lock on phone has been given up! ******
}
or
lock(phone) // Grab the phone when I have something ready for the worker
{
Monitor.PulseAll(phone); // Signal worker there is work to do
DoMoreWork();
} // ****** The lock on phone has been given up! ******
In both cases it's not until "the lock on phone has been given up" that another thread can get it.
There might be other threads waiting for that lock from Monitor.Wait(phone) or lock(phone). Only the one that wins the lock will get to continue.

They are tools for synchronizing and signaling between threads. As such they do nothing to prevent deadlocks, but if used correctly they can be used to synchronize and communicate between threads.
Unfortunately most of the work needed to write correct multithreaded code is currently the developers' responsibility in C# (and many other languages). Take a look at how F#, Haskell and Clojure handles this for an entirely different approach.

Unfortunately, none of Wait(), Pulse() or PulseAll() have the magical property which you are wishing for - which is that by using this API you will automatically avoid deadlock.
Consider the following code
object incomingMessages = new object(); //signal object
LoopOnMessages()
{
lock(incomingMessages)
{
Monitor.Wait(incomingMessages);
}
if (canGrabMessage()) handleMessage();
// loop
}
ReceiveMessagesAndSignalWaiters()
{
awaitMessages();
copyMessagesToReadyArea();
lock(incomingMessages) {
Monitor.PulseAll(incomingMessages); //or Monitor.Pulse
}
awaitReadyAreaHasFreeSpace();
}
This code will deadlock! Maybe not today, maybe not tomorrow. Most likely when your code is placed under stress because suddenly it has become popular or important, and you are being called to fix an urgent issue.
Why?
Eventually the following will happen:
All consumer threads are doing some work
Messages arrive, the ready area can't hold any more messages, and PulseAll() is called.
No consumer gets woken up, because none are waiting
All consumer threads call Wait() [DEADLOCK]
This particular example assumes that producer thread is never going to call PulseAll() again because it has no more space to put messages in. But there are many, many broken variations on this code possible. People will try to make it more robust by changing a line such as making Monitor.Wait(); into
if (!canGrabMessage()) Monitor.Wait(incomingMessages);
Unfortunately, that still isn't enough to fix it. To fix it you also need to change the locking scope where Monitor.PulseAll() is called:
LoopOnMessages()
{
lock(incomingMessages)
{
if (!canGrabMessage()) Monitor.Wait(incomingMessages);
}
if (canGrabMessage()) handleMessage();
// loop
}
ReceiveMessagesAndSignalWaiters()
{
awaitMessagesArrive();
lock(incomingMessages)
{
copyMessagesToReadyArea();
Monitor.PulseAll(incomingMessages); //or Monitor.Pulse
}
awaitReadyAreaHasFreeSpace();
}
The key point is that in the fixed code, the locks restrict the possible sequences of events:
A consumer threads does its work and loops
That thread acquires the lock
And thanks to locking it is now true that either:
a. Messages haven't yet arrived in the ready area, and it releases the lock by calling Wait() BEFORE the message receiver thread can acquire the lock and copy more messages into the ready area, or
b. Messages have already arrived in the ready area and it receives the messages INSTEAD OF calling Wait(). (And while it is making this decision it is impossible for the message receiver thread to e.g. acquire the lock and copy more messages into the ready area.)
As a result the problem of the original code now never occurs:
3. When PulseEvent() is called No consumer gets woken up, because none are waiting
Now observe that in this code you have to get the locking scope exactly right. (If, indeed I got it right!)
And also, since you must use the lock (or Monitor.Enter() etc.) in order to use Monitor.PulseAll() or Monitor.Wait() in a deadlock-free fashion, you still have to worry about possibility of other deadlocks which happen because of that locking.
Bottom line: these APIs are also easy to screw up and deadlock with, i.e. quite dangerous

This is a simple example of monitor use :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp4
{
class Program
{
public static int[] X = new int[30];
static readonly object _object = new object();
public static int count=0;
public static void PutNumbers(int numbersS, int numbersE)
{
for (int i = numbersS; i < numbersE; i++)
{
Monitor.Enter(_object);
try
{
if(count<30)
{
X[count] = i;
count++;
Console.WriteLine("Punt in " + count + "nd: "+i);
Monitor.Pulse(_object);
}
else
{
Monitor.Wait(_object);
}
}
finally
{
Monitor.Exit(_object);
}
}
}
public static void RemoveNumbers(int numbersS)
{
for (int i = 0; i < numbersS; i++)
{
Monitor.Enter(_object);
try
{
if (count > 0)
{
X[count] = 0;
int x = count;
count--;
Console.WriteLine("Removed " + x + " element");
Monitor.Pulse(_object);
}
else
{
Monitor.Wait(_object);
}
}
finally
{
Monitor.Exit(_object);
}
}
}
static void Main(string[] args)
{
Thread W1 = new Thread(() => PutNumbers(10,50));
Thread W2 = new Thread(() => PutNumbers(1, 10));
Thread R1 = new Thread(() => RemoveNumbers(30));
Thread R2 = new Thread(() => RemoveNumbers(20));
W1.Start();
R1.Start();
W2.Start();
R2.Start();
W1.Join();
R1.Join();
W2.Join();
R2.Join();
}
}
}

Can a second thread enter the same critical section just because the first thread called Monitor.Wait using the same sync lock?

Please tell me if I am thinking it alright.
A different thread cannot enter the same critical section using
the same lock just because the first thread called Monitor.Wait, right? The Wait method only allows a different thread to acquire
the same monitor, i.e. the same synchronization lock but only for a different critical section and never for the same critical
section.
Is my understanding correct?
Because if the Wait method meant that anyone can now enter this
same critical section using this same lock, then that would defeat
the whole purpose of synchronization, right?
So, in the code below (written in notepad, so please forgive any
typos), ThreadProc2 can only use syncLock to enter the code in
ThreadProc2 and not in ThreadProc1 while the a previous thread
that held and subsequently relinquished the lock was executing
ThreadProc1, right?
Two or more threads can use the same synchronization lock to run
different pieces of code at the same time, right? Same question as
above, basically, but just confirming for the sake of symmetry with
point 3 below.
Two or more threads can use a different synchronization lock to
run the same piece of code, i.e. to enter the same critical section.
Boilerplate text to correct the formatting.
class Foo
{
private static object syncLock = new object();
public void ThreadProc1()
{
try
{
Monitor.Enter(syncLock);
Monitor.Wait(syncLock);
Thread.Sleep(1000);
}
finally
{
if (Monitor.IsLocked(syncLock))
{
Monitor.Exit(syncLock);
}
}
}
public void ThreadProc2()
{
bool acquired = false;
try
{
// Calling TryEnter instead of
// Enter just for the sake of variety
Monitor.TryEnter(syncLock, ref acquired);
if (acquired)
{
Thread.Sleep(200);
Monitor.Pulse(syncLock);
}
}
finally
{
if (acquired)
{
Monitor.Exit(syncLock);
}
}
}
}
Update
The following illustration confirms that #3 is correct although I don't think it will be a nice thing to do.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace DifferentSyncLockSameCriticalSection
{
class Program
{
static void Main(string[] args)
{
var sathyaish = new Person { Name = "Sathyaish Chakravarthy" };
var superman = new Person { Name = "Superman" };
var tasks = new List<Task>();
// Must not lock on string so I am using
// an object of the Person class as a lock
tasks.Add(Task.Run( () => { Proc1(sathyaish); } ));
tasks.Add(Task.Run(() => { Proc1(superman); }));
Task.WhenAll(tasks);
Console.WriteLine("Press any key to exit.");
Console.ReadKey();
}
static void Proc1(object state)
{
// Although this would be a very bad practice
lock(state)
{
try
{
Console.WriteLine((state.ToString()).Length);
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
}
class Person
{
public string Name { get; set; }
public override string ToString()
{
return Name;
}
}
}

When a thread calls Monitor.Wait it is suspended and the lock released. This will allow another thread to acquire the lock, update some state, and then call Monitor.Pulse in order to communicate to other threads that something has happened. You must have acquired the lock in order to call Pulse. Before Monitor.Wait returns the framework will reacquire the lock for the thread that called Wait.
In order for two threads to communicate with each other they need to use the same synchronization primitive. In your example you've used a monitor, but you usually need to combine this with some kind of test that the Wait returned in response to a Pulse. This is because it is technically possible to Wait to return even if Pulse wasn't called (although this doesn't happen in practice).
It's also worth remembering that a call to Pulse isn't "sticky", so if nobody is waiting on the monitor then Pulse does nothing and a subsequent call to Wait will miss the fact that Pulse was called. This is another reason why you tend to record the fact that something has been done before calling Pulse (see the example below).
It's perfectly valid for two different threads to use the same lock to run different bits of code - in fact this is the typical use-case. For example, one thread acquires the lock to write some data and another thread acquires the lock to read the data. However, it's important to realize that they don't run at the same time. The act of acquiring the lock prevents another thread from acquiring the same lock, so any thread attempting to acquire the lock when it is already locked will block until the other thread releases the lock.
In point 3 you ask:
Two or more threads can use a different synchronization lock to run
the same piece of code, i.e. to enter the same critical section.
However, if two threads are using different locks then they are not entering the same critical section. The critical section is denoted by the lock that protects it - if they're different locks then they are different sections that just happen to access some common data within the section. You should avoid doing this as it can lead to some difficult to debug data race conditions.
Your code is a bit over-complicated for what you're trying to accomplish. For example, let's say we've got 2 threads, and one will signal when there is data available for another to process:
class Foo
{
private readonly object syncLock = new object();
private bool dataAvailable = false;
public void ThreadProc1()
{
lock(syncLock)
{
while(!dataAvailable)
{
// Release the lock and suspend
Monitor.Wait(syncLock);
}
// Now process the data
}
}
public void ThreadProc2()
{
LoadData();
lock(syncLock)
{
dataAvailable = true;
Monitor.Pulse(syncLock);
}
}
private void LoadData()
{
// Gets some data
}
}
}

Stopping a multi-threaded windows service

I have a multi-thread windows service in .Net 3.5, and I am having some trouble to stop the service properly when more than one thread is created.
This service used to create only one thread to do all the work, and I just changed it to be multi-threaded. It works perfectly, but when the service is stopped, if more than one thread is being executed, it will hang the service until all the threads are completed.
When the service is started, I create a background thread to handle the main process:
protected override void OnStart(string[] args)
{
try
{
//Global variable that is checked by threads to learn if service was stopped
DeliveryConstant.StopService = false;
bool SetMaxThreadsResult = ThreadPool.SetMaxThreads(10, 10);
ThreadStart st = new ThreadStart(StartThreadPool);
workerThread = new Thread(st);
workerThread.IsBackground = true;
serviceStarted = true;
workerThread.Start();
}
catch (Exception ex)
{
//Log something;
}
Here is the StartThreadPool method:
//Tried with and without this attribute with no success...
[System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.Synchronized)]
public void StartThreadPool()
{
while (serviceStarted)
{
ProcessInfo input = new ProcessInfo();
try
{
int? NumPendingRequests = GetItems(50, (Guid?)input.ProcessID);
if (NumPendingRequests > 0)
{
input.ProcessType = 1;
input.ProcessID = Guid.NewGuid();
ThreadPool.QueueUserWorkItem(new WaitCallback(new DispatchManager().ProcessRequestList), input);
}
}
catch (Exception ex)
{
//Some Logging here
}
}
DeliveryConstant.StopService = true;
}
I created a static variable in a separated class to notify the threads that the service was stopped. When the value for this variable is true, all threads should stop the main loop (a for each loop):
public static bool StopService;
Finally, the OnStop method:
protected override void OnStop()
{
DeliveryConstant.StopService = true;
//flag to tell the worker process to stop
serviceStarted = false;
workerThread.Join(TimeSpan.FromSeconds(30));
}
In the ProcessRequestList method, at the end of every foreach, I check for the value of the StopService variable. If true, I break the loop.
Here is the problem:
The threads are created in chunks of 50 items. When I have 50 items or less in the database, only one thread is created, and everything works beautifully.
When I have more than 50 items, multiple threads will be created, and when I try to stop the service, it doesn't stop until all the background threads are completed.
From the logs, I can see that the method OnStop is only executed AFTER all threads are completed.
Any clue what could be changed to fix that?

This blog answer states that OnStop isn't called until all ThreadPool tasks complete, which is news to me but would explain your issue.
I've fielded many multi-threaded Windows Services but I prefer to create my own background threads rather than use the ThreadPool since these are long-running threads. I instantiate worker classes and launch their DoWork() method on the thread. I also prefer to use callbacks to the launching class to check for a stop signal and pass status rather than just test against a global variable.

You are missing memory barriers around accesses to StopService, which may be a problem if you have multiple CPUs. Better lock any reference object for ALL accesses to the shared variable. For example:
object #lock;
...
lock (#lock)
{
StopService = true;
}
Edit: As another answer has revealed, this issue was not a locking problem, but I am leaving this answer here as a thing to check with multithread synchronization schemes.
Making the shared variable volatile would work in many cases as well, but it is more complex to prove correct because it does not emit full fences.

Shared Object Pool without Thread.Sleep?

I have developed an "object pool" and cannot seem to do it without using Thread.Sleep() which is "bad practice" I believe.
This relates to my other question "Is there a standard way of implementing a proprietary connection pool in .net?". The idea behind the object pool is similar to the one behind the connection pool used for database connections. However, in my case I am using it to share a limited resource in a standard ASP.NET Web Service (running in IIS6). This means that many threads will be requesting access to this limited resource. The pool would dish out these objects (a "Get) and once all the available pool objects have been used, the next thread requesting one would simply waits a set amount of time for one of these object to become available again (a thread would do a "Put" once done with the object). If an object does not become available in this set time, a timeout error will occur.
Here is the code:
public class SimpleObjectPool
{
private const int cMaxGetTimeToWaitInMs = 60000;
private const int cMaxGetSleepWaitInMs = 10;
private object fSyncRoot = new object();
private Queue<object> fQueue = new Queue<object>();
private SimpleObjectPool()
{
}
private static readonly SimpleObjectPool instance = new SimpleObjectPool();
public static SimpleObjectPool Instance
{
get
{
return instance;
}
}
public object Get()
{
object aObject = null;
for (int i = 0; i < (cMaxGetTimeToWaitInMs / cMaxGetSleepWaitInMs); i++)
{
lock (fSyncRoot)
{
if (fQueue.Count > 0)
{
aObject = fQueue.Dequeue();
break;
}
}
System.Threading.Thread.Sleep(cMaxGetSleepWaitInMs);
}
if (aObject == null)
throw new Exception("Timout on waiting for object from pool");
return aObject;
}
public void Put(object aObject)
{
lock (fSyncRoot)
{
fQueue.Enqueue(aObject);
}
}
}
To use use it, one would do the following:
public void ExampleUse()
{
PoolObject lTestObject = (PoolObject)SimpleObjectPool.Instance.Get();
try
{
// Do something...
}
finally
{
SimpleObjectPool.Instance.Put(lTestObject);
}
}
Now the question I have is: How do I write this so I get rid of the Thread.Sleep()?
(Why I want to do this is because I suspect that it is responsible for the "false" timeout I am getting in my testing. My test application has a object pool with 3 objects in it. It spins up 12 threads and each thread gets an object from the pool 100 times. If the thread gets an object from the pool, it holds on to if for 2,000 ms, if it does not, it goes to the next iteration. Now logic dictates that 9 threads will be waiting for an object at any point in time. 9 x 2,000 ms is 18,000 ms which is the maximum time any thread should have to wait for an object. My get timeout is set to 60,000 ms so no thread should ever timeout. However some do so something is wrong and I suspect its the Thread.Sleep)

Since you are already using lock, consider using Monitor.Wait and Monitor.Pulse
In Get():
lock (fSyncRoot)
{
while (fQueue.Count < 1)
Monitor.Wait(fSyncRoot);
aObject = fQueue.Dequeue();
}
And in Put():
lock (fSyncRoot)
{
fQueue.Enqueue(aObject);
if (fQueue.Count == 1)
Monitor.Pulse(fSyncRoot);
}

you should be using a semaphore.
http://msdn.microsoft.com/en-us/library/system.threading.semaphore.aspx
UPDATE:
Semaphores are one of the basic constructs of multi-threaded programming.
A semaphore can be used different ways, but the basic idea is when you have a limited resource and many clients who want to use that resource, you can limit the number of clients that can access the resource at any given time.
below is a very crude example. I didn't add any error checking or try/finally blocks but you should.
You can also check:
http://en.wikipedia.org/wiki/Semaphore_(programming)
Say you have 10 buckets and 100 people who want to use those buckets.
We can represent the buckets in a queue.
At the start, add all of your buckets to the queue
for(int i=0;i<10;i++)
{
B.Push(new Bucket());
}
Now create a semaphore to guard your bucket queue. This semaphore is created with no items triggered and a capacity of 10.
Semaphore s = new Semaphore(0, 10);
All clients should check the semaphore before accessing the queue. You might have 100 threads running the thread method below. The first 10 will pass the semaphore. All others will wait.
void MyThread()
{
while(true)
{
// thread will wait until the semaphore is triggered once
// there are other ways to call this which allow you to pass a timeout
s.WaitOne();
// after being triggered once, thread is clear to get an item from the queue
Bucket b = null;
// you still need to lock because more than one thread can pass the semaphore at the sam time.
lock(B_Lock)
{
b = B.Pop();
}
b.UseBucket();
// after you finish using the item, add it back to the queue
// DO NOT keep the queue locked while you are using the item or no other thread will be able to get anything out of it
lock(B_Lock)
{
B.Push(b);
}
// after adding the item back to the queue, trigger the semaphore and allow
// another thread to enter
s.Release();
}
}