I'm working on some big multi threaded project, now yesterday I had a deadlock (my first one ever), and I traced it by adding a Console.WriteLine("FunctionName: Lock on VariableName") and Console.WriteLine("FunctionName: Unlocking VariableName"). Adding all those was quite some work.
First of all, the program has a main loop that runs 2 times per second, that loop pulses some other threads to complete their work after the main loop has processed. Now what happened was that I had one thread in wait state to be pulsed, when it was pulsed it called another method that'd also wait to get pulsed, but the pulse already happened, and the thread won't pulse again until the action is actually completed.
Now what I want to do is override the Monitor.Enter and Monitor.Exit functions, without wrapping them in a class.
I've heard a lot about Reflection, but I have no idea how to apply it for this purpose, I know the easiest way to achieve it all is by just using a wrapper class, but then the lock keyword won't work anymore, and I'd have to convert all locks into Monitor.Enter try { } finally { Monitor.Exit }, that's huge amount of work.
So my question: How to override the Monitor.Enter and Monitor.Exit functions, while keeping access to the base function to do the actual lock?
And if that's impossible: How to override the lock statement to call my wrapper class instead of the Monitor.Enter and Monitor.Exit functions?
EDIT FOR CLARITY:
I request this just for allowing me to log when the locks happen, to make the debugging process easier, that also means I don't want to create my own locking mechanism, I just want to log when a lock is established and when it's released.
The close will also not be executed most of the time, only when I come across a threading problem.
It sounds like you're looking for lock helpers. Jon Skeet's MiscUtil has some:
http://www.yoda.arachsys.com/csharp/miscutil/usage/locking.html
The idea is that you replace your lock statements with using statements and thus preserve the try-finally structure:
class Example
{
SyncLock padlock = new SyncLock();
void Method1
{
using (padlock.Lock())
{
// Now own the padlock
}
}
void Method2
{
using (padlock.Lock())
{
// Now own the padlock
}
}
}
With regards to deadlock prevention, the library offers a specialized ordered lock:
class Example
{
OrderedLock inner = new OrderedLock("Inner");
OrderedLock outer = new OrderedLock("Outer");
Example()
{
outer.InnerLock = inner;
}
}
Of course, you could extend Jon's helpers, or simply create your own (for logging purposes, etc). Check out the link above for more information.
Don't do it! That sounds bonkers ;-)
A deadlock occurs when 2 (or more) threads are all waiting to simultaneously hold 2 (or more) locks. And each thread gets a lock and waits for the other one.
You can often redesign your code so each thread only requires a single lock - which makes deadlock impossible.
Failing that, you can make a thread give up the first lock if it can't acquire the second lock.
That's a very bad idea. I never had to override Monitor.Enter / Exit or lock to overcome a deadlock. Please consider redesigning your code!
For example, use ManualResetEvent for the pulsing.
Related
I'm trying to understand how the AsyncLock works.
First of all, here's a snippet to prove that it actually works:
var l = new AsyncLock();
var tasks = new List<Task>();
while (true)
{
Console.ReadLine();
var i = tasks.Count + 1;
tasks.Add(Task.Run(async () =>
{
Console.WriteLine($"[{i}] Acquiring lock ...");
using (await l.LockAsync())
{
Console.WriteLine($"[{i}] Lock acquired");
await Task.Delay(-1);
}
}));
}
By "works" I mean that you can run as many tasks as you want (by hitting Enter) and the number of threads doesn't grow. If you replace it with traditional lock, you'll see that the new threads are started, which is what we try to avoid.
But the first thing you see in the source code is... the lock
Can somebody please explain me how this works, why it doesn't block, and what am I missing here?
Can somebody please explain me how this works, why it doesn't block, and what am I missing here?
The short answer is that lock is just an internal mechanism used to guarantee thread safety. The lock is never exposed in any way, and there's no way for any thread to hold that lock for any real amount of time. In this way, it's similar to the locks used internally by various concurrent collections.
There is an alternate approach that uses lock-free programming, but I have found lock-free programming to be extremely difficult to write, read, and maintain. A great example of this (which is sadly not online) was a bunch of Dr. Dobb's articles in the late '90s, each one trying to out-do the last with a better lock-free queue implementation. It turns out they were all faulty - in some cases, the bugs took more than a decade to find.
For my own code, I do not use lock-free programming, except where the correctness of the code is trivially obvious.
As far as the async lock vs lock concepts, I'm going to take a stab at explaining this. There's a feeling I get that I have only felt when working with asynchronous coordination primitives. It's something I've thought a lot about writing a blog post on, but I don't have the right words to make it understandable. That said, here goes...
Asynchronous coordination primitives exist on a completely different plane than normal coordination primitives. Synchronous primitives block threads and signal threads. Asynchronous primitives just work on plain objects; the blocking or signaling is just "by convention".
So, with a normal lock, the calling code must take the lock immediately. But with an asynchronous "lock", the attempted lock is just a request, just an object. The calling code doesn't even need to await it. It's possible to request several locks and await them all together with Task.WhenAll. Or even combine them with other things; code can do crazy things like (a)wait for two locks to both be free or for a signal (like AsyncManualResetEvent) to be sent, and then cancel the lock requests if the signal comes in first.
From a thread perspective, it's kinda-sorta like user-mode thread scheduling. There's also some similarities to cooperative multitasking (as opposed to preemptive). But overall, the asynchronous primitives are "lifted" to a different plane, where one works only with objects and blocks of code, not threads.
The lock inside AsyncLock is beeing released very quickly. Each task which tries to acquire AsyncLock, successfully acquires it's internal lock and the actual locking logic is done with a queue.
By wrapping LockAsync() within using block, the lock is being released when the block ends since LockAsync returns a disposable object Key which will be disposed at the end of the using block, and upon disposing the lock will be released. see https://github.com/StephenCleary/AsyncEx/blob/master/src/Nito.AsyncEx.Coordination/AsyncLock.cs#L182-L185
Currently, I'm learning for a multithreading exam. I read the good threading article of albahari. I've got a question at the monitor usage - why is here used a loop in place of an if?
lock (_locker)
{
while (!_go) //why while and not if?
Monitor.Wait (_locker); // _lock is released
// lock is regained
...
}
I think, that an if would be sufficient.
I'm afraid, that I don't understand the article completely.
//Edit
Example-Code:
class SimpleWaitPulse
{
static readonly object _locker = new object();
static bool _go;
static void Main()
{ // The new thread will block
new Thread (Work).Start(); // because _go==false.
Console.ReadLine(); // Wait for user to hit Enter
lock (_locker) // Let's now wake up the thread by
{ // setting _go=true and pulsing.
_go = true;
Monitor.Pulse (_locker);
}
}
static void Work()
{
lock (_locker)
while (!_go)
Monitor.Wait (_locker); // Lock is released while we’re waiting
Console.WriteLine ("Woken!!!");
}
}
It just depends on the situation. In this case the code is just waiting for _go to be true.
Every time _locker is pulsed it will check to see if _go has been set to true. If _go is still false, it will wait for the next pulse.
If an if was used instead of a while, it would only wait once (or not at all if _go was already true), and would then continue on after a pulse, regardless of the new state of _go.
So how you use Monitor.Wait() depends entirely on your specific needs.
It really just depends on the situation. But first, we need to clarify how Monitors work. When a thread proceeds to signal a thread through Monitor.Pulse(), there is usually no guarantee that the signaled thread will actually run next. This means that it is possible for other threads to run before the signaled thread and change the condition under which it was okay for the signaled thread to proceed. This means that the signaled thread still needs to check if is safe for it to proceed after being woken up (ie the while loop). However, certain rare synchronization problems allow you to make the assumption that once a thread has been signaled to wake up (ie Monitor.Pulse()), no other thread has the ability to change the condition under which it is safe to proceed (ie. the if condition).
I wrote an article that might help here: Wait and Pulse demystified
There's more going on than is immediately obvious.
I've got a question at the monitor usage - why is here used a loop in
place of an if?
There is a well known rule when working with Pulse and Wait that states that when in doubt prefer while over an if. Clearly, either one will work in this case, but in almost every other situation while is required. In fact, there are very few (if any) scenarios where using a while loop would produce an incorrect result. That is the basis for this general rule. The author used a while loop because he was trying to stick with the tried-and-true pattern. He even provides the template in the same article. Here is it is:
lock (_locker)
while ( <blocking-condition> )
Monitor.Wait (_locker);
The simplest way to write correct code with Monitor.Wait is to assume the system will regard it as "advisory", and assume that the system may arbitrarily wake any waiting thread any time it can acquire the lock, without regard for whether Pulse has been called. The system usually won't do so, of course, but if a program is using Wait and Pulse properly, its correctness should not be affected by having Wait calls arbitrarily exit early for no reason. Essentially, one should regard Wait as a means of telling the system "Continuing execution past here will be a waste of time unless or until someone else calls Pulse".
I need a synchronization primitive which is similar to Monitor but doesn't require to Exit it as many times as I have enter it. If I enter Monitor by one thread and then reenter it by the same thread I will need to call Monitor.Exit twice. But I need to exit it by one call.
Now I use a some kind of Monitor wrapper which doesn't enter Monitor if it was already entered by current thread (and that's why then I can exit it by one call). But may be .NET Framework contains one?
I'm curious to know why you would ever call Monitor.Enter multiple times without an equal number of calls to Monitor.Exit. Typically any such synchronization code would look like this:
try
{
Monitor.Enter(lockObject);
// some code that needs to be synchronized
}
finally
{
Monitor.Exit(lockObject);
}
Assuming you are using try/finally wherever you acquire a lock using Monitor.Enter (which you should be), I'm having trouble seeing why you would need this "lock-once" class you're asking about.
In fact, you should basically never have to do this yourself anyway, as a much simpler approach that does essentially the same thing is to use a lock statement:
lock (lockObject)
{
// some code that needs to be synchronized
}
That said, I could certainly just be missing something.
how do you know it's the same thread and how to you ensure that when this threads leaves it is going to call exit before it leaves?
From the looks of it, you just need something else (in an outer level) that has the lock. Maybe an "entry point" method that locks and calls another method that has the meat of the work then you can call this other method many times without going past the lock.
public static void MethodOne()
{
lock (lockObj)
{
MethodTwo();
}
}
private static void MethodTwo()
{
//This method can be called multiple times
//without going past MethodOne and so you only
//lock once
}
private static void MethodThree()
{
}
I have developed a generic producer-consumer queue which pulses by Monitor in the following way:
the enqueue :
public void EnqueueTask(T task)
{
_workerQueue.Enqueue(task);
Monitor.Pulse(_locker);
}
the dequeue:
private T Dequeue()
{
T dequeueItem;
if (_workerQueue.Count > 0)
{
_workerQueue.TryDequeue(out dequeueItem);
if(dequeueItem!=null)
return dequeueItem;
}
while (_workerQueue.Count == 0)
{
Monitor.Wait(_locker);
}
_workerQueue.TryDequeue(out dequeueItem);
return dequeueItem;
}
the wait section produces the following SynchronizationLockException :
"object synchronization method was called from an unsynchronized block of code"
do i need to synch it? why ? Is it better to use ManualResetEvents or the Slim version of .NET 4.0?
Yes, the current thread needs to "own" the monitor in order to call either Wait or Pulse, as documented. (So you'll need to lock for Pulse as well.) I don't know the details for why it's required, but it's the same in Java. I've usually found I'd want to do that anyway though, to make the calling code clean.
Note that Wait releases the monitor itself, then waits for the Pulse, then reacquires the monitor before returning.
As for using ManualResetEvent or AutoResetEvent instead - you could, but personally I prefer using the Monitor methods unless I need some of the other features of wait handles (such as atomically waiting for any/all of multiple handles).
From the MSDN description of Monitor.Wait():
Releases the lock on an object and blocks the current thread until it reacquires the lock.
The 'releases the lock' part is the problem, the object isn't locked. You are treating the _locker object as though it is a WaitHandle. Doing your own locking design that's provably correct is a form of black magic that's best left to our medicine man, Jeffrey Richter and Joe Duffy. But I'll give this one a shot:
public class BlockingQueue<T> {
private Queue<T> queue = new Queue<T>();
public void Enqueue(T obj) {
lock (queue) {
queue.Enqueue(obj);
Monitor.Pulse(queue);
}
}
public T Dequeue() {
T obj;
lock (queue) {
while (queue.Count == 0) {
Monitor.Wait(queue);
}
obj = queue.Dequeue();
}
return obj;
}
}
In most any practical producer/consumer scenario you will want to throttle the producer so it cannot fill the queue unbounded. Check Duffy's BoundedBuffer design for an example. If you can afford to move to .NET 4.0 then you definitely want to take advantage of its ConcurrentQueue class, it has lots more black magic with low-overhead locking and spin-waiting.
The proper way to view Monitor.Wait and Monitor.Pulse/PulseAll is not as providing a means of waiting, but rather (for Wait) as a means of letting the system know that the code is in a waiting loop which can't exit until something of interest changes, and (for Pulse/PulseAll) as a means of letting the system know that code has just changed something that might cause satisfy the exit condition some other thread's waiting loop. One should be able to replace all occurrences of Wait with Sleep(0) and still have code work correctly (even if much less efficiently, as a result of spending CPU time repeatedly testing conditions that haven't changed).
For this mechanism to work, it is necessary to avoid the possibility of the following sequence:
The code in the wait loop tests the condition when it isn't satisfied.
The code in another thread changes the condition so that it is satisfied.
The code in that other thread pulses the lock (which nobody is yet waiting on).
The code in the wait loop performs a Wait since its condition wasn't satisfied.
The Wait method requires that the waiting thread have a lock, since that's the only way it can be sure that the condition it's waiting upon won't change between the time it's tested and the time the code performs the Wait. The Pulse method requires a lock because that's the only way it can be sure that if another thread has "committed" itself to performing a Wait, the Pulse won't occur until after the other thread actually does so. Note that using Wait within a lock doesn't guarantee that it's being used correctly, but there's no way that using Wait outside a lock could possibly be correct.
The Wait/Pulse design actually works reasonably well if both sides cooperate. The biggest weaknesses of the design, IMHO, are (1) there's no mechanism for a thread to wait until any of a number of objects is pulsed; (2) even if one is "shutting down" an object such that all future wait loops should exit immediately (probably by checking an exit flag), the only way to ensure that any Wait to which a thread has committed itself will get a Pulse is to acquire the lock, possibly waiting indefinitely for it to become available.
Will the following code result in a deadlock using C# on .NET?
class MyClass
{
private object lockObj = new object();
public void Foo()
{
lock(lockObj)
{
Bar();
}
}
public void Bar()
{
lock(lockObj)
{
// Do something
}
}
}
No, not as long as you are locking on the same object. The recursive code effectively already has the lock and so can continue unhindered.
lock(object) {...} is shorthand for using the Monitor class. As Marc points out, Monitor allows re-entrancy, so repeated attempts to lock on an object on which the current thread already has a lock will work just fine.
If you start locking on different objects, that's when you have to be careful. Pay particular attention to:
Always acquire locks on a given number of objects in the same sequence.
Always release locks in the reverse sequence to how you acquire them.
If you break either of these rules you're pretty much guaranteed to get deadlock issues at some point.
Here is one good webpage describing thread synchronisation in .NET: http://dotnetdebug.net/2005/07/20/monitor-class-avoiding-deadlocks/
Also, lock on as few objects at a time as possible. Consider applying coarse-grained locks where possible. The idea being that if you can write your code such that there is an object graph and you can acquire locks on the root of that object graph, then do so. This means you have one lock on that root object and therefore don't have to worry so much about the sequence in which you acquire/release locks.
(One further note, your example isn't technically recursive. For it to be recursive, Bar() would have to call itself, typically as part of an iteration.)
Well, Monitor allows re-entrancy, so you can't deadlock yourself... so no: it shouldn't do
If a thread is already holding a lock, then it will not block itself. The .Net framework ensures this. You only have to make sure that two threads do not attempt to aquire the same two locks out of sequence by whatever code paths.
The same thread can aquire the same lock multiple times, but you have to make sure you release the lock the same number of times that you aquire it. Of course, as long as you are using the "lock" keyword to accomplish this, it happens automatically.
No, this code will not have dead locks.
If you really want to create deadlock simplest one requires at-least 2 resources.
Consider dog and the bone scenario.
1. A dog has full control over 1 bone so any other dog has to wait.
2. 2 dog with 2 bones are minimum required to create a deadlock when they lock their bones respectively and seek others bone too.
.. so on and so forth n dogs and m bones and cause more sophisticated deadlocks.