Develop Reference

c# lock function during async file write [duplicate]

I want to start some new threads each for one repeating operation. But when such an operation is already in progress, I want to discard the current task. In my scenario I need very current data only - dropped data is not an issue.
In the MSDN I found the Mutex class but as I understand it, it waits for its turn, blocking the current thread. Also I want to ask you: Does something exist in the .NET framework already, that does the following:
Is some method M already being executed?
If so, return (and let me increase some counter for statistics)
If not, start method M in a new thread

The lock(someObject) statement, which you may have come across, is syntactic sugar around Monitor.Enter and Monitor.Exit.
However, if you use the monitor in this more verbose way, you can also use Monitor.TryEnter which allows you to check if you'll be able to get the lock - hence checking if someone else already has it and is executing code.
So instead of this:
var lockObject = new object();
lock(lockObject)
{
// do some stuff
}
try this (option 1):
int _alreadyBeingExecutedCounter;
var lockObject = new object();
if (Monitor.TryEnter(lockObject))
{
// you'll only end up here if you got the lock when you tried to get it - otherwise you'll never execute this code.
// do some stuff
//call exit to release the lock
Monitor.Exit(lockObject);
}
else
{
// didn't get the lock - someone else was executing the code above - so I don't need to do any work!
Interlocked.Increment(ref _alreadyBeingExecutedCounter);
}
(you'll probably want to put a try..finally in there to ensure the lock is released)
or dispense with the explicit lock althogether and do this
(option 2)
private int _inUseCount;
public void MyMethod()
{
if (Interlocked.Increment(ref _inUseCount) == 1)
{
// do dome stuff
}
Interlocked.Decrement(ref _inUseCount);
}
[Edit: in response to your question about this]
No - don't use this to lock on. Create a privately scoped object to act as your lock.
Otherwise you have this potential problem:
public class MyClassWithLockInside
{
public void MethodThatTakesLock()
{
lock(this)
{
// do some work
}
}
}
public class Consumer
{
private static MyClassWithLockInside _instance = new MyClassWithLockInside();
public void ThreadACallsThis()
{
lock(_instance)
{
// Having taken a lock on our instance of MyClassWithLockInside,
// do something long running
Thread.Sleep(6000);
}
}
public void ThreadBCallsThis()
{
// If thread B calls this while thread A is still inside the lock above,
// this method will block as it tries to get a lock on the same object
// ["this" inside the class = _instance outside]
_instance.MethodThatTakesLock();
}
}
In the above example, some external code has managed to disrupt the internal locking of our class just by taking out a lock on something that was externally accessible.
Much better to create a private object that you control, and that no-one outside your class has access to, to avoid these sort of problems; this includes not using this or the type itself typeof(MyClassWithLockInside) for locking.

One option would be to work with a reentrancy sentinel:
You could define an int field (initialize with 0) and update it via Interlocked.Increment on entering the method and only proceed if it is 1. At the end just do a Interlocked.Decrement.
Another option:
From your description it seems that you have a Producer-Consumer-Scenario...
For this case it might be helpful to use something like BlockingCollection as it is thread-safe and mostly lock-free...
Another option would be to use ConcurrentQueue or ConcurrentStack...

You might find some useful information on the following site (the PDf is also downlaodable - recently downloaded it myself). The Adavnced threading Suspend and Resume or Aborting chapters maybe what you are inetrested in.

You should use Interlocked class atomic operations - for best performance - since you won't actually use system-level sychronizations(any "standard" primitive needs it, and involve system call overhead).
//simple non-reentrant mutex without ownership, easy to remake to support //these features(just set owner after acquiring lock(compare Thread reference with Thread.CurrentThread for example), and check for matching identity, add counter for reentrancy)
//can't use bool because it's not supported by CompareExchange
private int lock;
public bool TryLock()
{
//if (Interlocked.Increment(ref _inUseCount) == 1)
//that kind of code is buggy - since counter can change between increment return and
//condition check - increment is atomic, this if - isn't.
//Use CompareExchange instead
//checks if 0 then changes to 1 atomically, returns original value
//return true if thread succesfully occupied lock
return CompareExchange(ref lock, 1, 0)==0;
return false;
}
public bool Release()
{
//returns true if lock was occupied; false if it was free already
return CompareExchange(ref lock, 0, 1)==1;
}

implementing a c# read write lock where some reads produce writes

I have to implement some .Net code involving a shared resource accessed by different threads. In principle, this should be solved with a simple read write lock. However, my solution requires that some of the read accessions do end up producing a write operation. I first checked the ReaderWriterLockSlim, but by itself it does not solve the problem, because it requires that I know in advance if a read operation can turn into a write operation, and this is not my case. I finally opted by simply using a ReaderWriterLockSlim, and when the read operation "detects" that needs to do a write operation, release the read lock and acquire a write lock. I am not sure if there is a better solution, or event if this solution could lead to some synchronization issue (I have experience with Java, but I am fairly new to .Net).
Below some sample code illustrating my solution:
public class MyClass
{
private int[] data;
private readonly ReaderWriterLockSlim syncLock = new ReaderWriterLockSlim();
public void modifyData()
{
try
{
syncLock.EnterWriteLock();
// clear my array and read from database...
}
finally
{
syncLock.ExitWriteLock();
}
}
public int readData(int index)
{
try
{
syncLock.EnterReadLock();
// some initial preprocessing of the arguments
try
{
_syncLock.ExitReadLock();
_syncLock.EnterWriteLock();
// check if a write is needed <--- this operation is fast, and, in most cases, the result will be false
// if true, perform the write operation
}
finally
{
_syncLock.ExitWriteLock();
_syncLock.EnterReadLock();
}
return data[index];
}
finally
{
syncLock.ExitReadLock();
}
}
}

When should & shouldn't I use this C# utility class to control threads via Interlocked

I'm trying to understand the logic behind how this class was written, and when I should and shouldn't use it. Any insight would be appreciated
internal struct SpinLock
{
private volatile int lockHeld;
private readonly static int processorCount;
public bool IsHeld
{
get
{
return this.lockHeld != 0;
}
}
static SpinLock()
{
SpinLock.processorCount = Environment.ProcessorCount;
}
public void Enter()
{
if (Interlocked.CompareExchange(ref this.lockHeld, 1, 0) != 0)
{
this.EnterSpin();
}
}
private void EnterSpin()
{
int num = 0;
while (this.lockHeld != null || Interlocked.CompareExchange(ref this.lockHeld, 1, 0) != 0)
{
if (num >= 20 || SpinLock.processorCount <= 1)
{
if (num >= 25)
{
Thread.Sleep(1);
}
else
{
Thread.Sleep(0);
}
}
else
{
Thread.SpinWait(100);
}
num++;
}
}
public void Exit()
{
this.lockHeld = 0;
}
}
Update: I found a sample usage in my source code... This indicates how to use the above object, though I don't understand "why"
internal class FastReaderWriterLock
{
private SpinLock myLock;
private uint numReadWaiters;
private uint numWriteWaiters;
private int owners;
private EventWaitHandle readEvent;
private EventWaitHandle writeEvent;
public FastReaderWriterLock()
{
}
public void AcquireReaderLock(int millisecondsTimeout)
{
this.myLock.Enter();
while (this.owners < 0 || this.numWriteWaiters != 0)
{
if (this.readEvent != null)
{
this.WaitOnEvent(this.readEvent, ref this.numReadWaiters, millisecondsTimeout);
}
else
{
this.LazyCreateEvent(ref this.readEvent, false);
}
}
FastReaderWriterLock fastReaderWriterLock = this;
fastReaderWriterLock.owners = fastReaderWriterLock.owners + 1;
this.myLock.Exit();
}
private void WaitOnEvent(EventWaitHandle waitEvent, ref uint numWaiters, int millisecondsTimeout)
{
waitEvent.Reset();
uint& numPointer = numWaiters;
bool flag = false;
this.myLock.Exit();
try
{
if (waitEvent.WaitOne(millisecondsTimeout, false))
{
flag = true;
}
else
{
throw new TimeoutException("ReaderWriterLock timeout expired");
}
}
finally
{
this.myLock.Enter();
uint& numPointer1 = numWaiters;
if (!flag)
{
this.myLock.Exit();
}
}
}
}

SpinLocks in general are a form of lock that keeps waiting threads awake (cycling over and over on a check condition in a tight loop- think "Mommy are we there yet?") rather than relying on heavier, slower, kernel mode signals. They are usually intended for situations where the expected wait time is very short, where they outperform the overhead of creating and waiting on an OS handle for a traditional lock. They incur more CPU cost than a traditional lock though, so for more than a very short wait time a traditional lock (like the Monitor class or the various WaitHandle implementations) is preferred.
This short wait time concept is demonstrated in your code above:
waitEvent.Reset();
// All that we are doing here is setting some variables.
// It has to be atomic, but it's going to be *really* fast
uint& numPointer = numWaiters;
bool flag = false;
// And we are done. No need for an OS wait handle for 2 lines of code.
this.myLock.Exit();
There is a perfectly good SpinLock built into the BCL, however it is only in v4.0+, so if you are working in an older version of the .NET framework or on code that was migrated from an older version, someone may have written their own implementation.
To answer your question:
You should use the built-in SpinLock if you are writing new code on .NET 4.0 or higher. For code on 3.5 or older, especially if you are extending Nesper, I'd argue that this implementation is time-tested and appropriate. Only use a SpinLock where you know that the time a thread may wait on it is very small, as in the example above.
EDIT: Looks like your implementation came from Nesper- the .NET port of the Esper CEP library:
https://svn.codehaus.org/esper/esper/tagsnet/release_1.12.0_beta_1/trunk/NEsper/compat/SpinLock.cs
and
https://svn.codehaus.org/esper/esper/tagsnet/release_1.12.0_beta_1/trunk/NEsper/compat/FastReaderWriterLock.cs
I can confirm that Nesper existed long before the .NET framework 4, so that explains the need for a home-spun SpinLock.

It appears that the original author wanted a faster version of ReaderWriterLock. This old class was painfully slow. My own tests (which I did a long time ago) indicates that RWL had ~8x the overhead of a plain old lock. ReaderWriterLockSlim improved things quite a bit (though it still has ~2x the overhead as compared to lock). At this point I would say ditch the custom code and just use the newer ReaderWriterLockSlim class.
But, for what it is worth let me explain some of that custom SpinLock code.
Interlocked.CompareExchange is .NET's version of a CAS operation. It is the most fundamental synchronization primitive. You can literally build everything else from this single operation including your own custom Monitor-like class, reader writer locks, etc. Obviously it was used here to create a spin lock.
Thread.Sleep(0) yields to any thread of with the same or higher priority on any processor.
Thread.Sleep(1) yields to any thread on any processor.
Thread.SpinWait puts the thread into a tight loop for the specified number of iterations.
Although it was not used in the code you posted there is another useful mechanism for creating spin locks (or other low lock strategies).
Thread.Yield yields to any thread on the same processor.
Microsoft uses all of these calls in their highly concurrent synchronization mechanisms and collections. If you decompile SpinLock, SpinWait, ManualResetEventSlim, etc. you will see a fairly complex song-and-dance going on with these calls...much more complex than the code you posted.
Again, ditch the custom code and just use ReaderWriterLockSlim instead of that custom FastReaderWriterLock class.
By the way, this.lockHeld != null should produce a compiler warning since lockHeld is a value type.

How to avoid double check locking when adding items to a Dictionary<> object in .NET?

I have a question about improving the efficiency of my program. I have a Dictionary<string, Thingey> defined to hold named Thingeys. This is a web application that will create multiple named Thingey’s over time. Thingey’s are somewhat expensive to create (not prohibitively so) but I’d like to avoid it whenever possible. My logic for getting the right Thingey for the request looks a lot like this:
private Dictionary<string, Thingey> Thingeys;
public Thingey GetThingey(Request request)
{
string thingeyName = request.ThingeyName;
if (!this.Thingeys.ContainsKey(thingeyName))
{
// create a new thingey on 1st reference
Thingey newThingey = new Thingey(request);
lock (this.Thingeys)
{
if (!this.Thingeys.ContainsKey(thingeyName))
{
this.Thingeys.Add(thingeyName, newThingey);
}
// else - oops someone else beat us to it
// newThingey will eventually get GCed
}
}
return this. Thingeys[thingeyName];
}
In this application, Thingeys live forever once created. We don’t know how to create them or which ones will be needed until the app starts and requests begin coming in. The question I have is in the above code is there are occasional instances where newThingey is created because we get multiple simultaneous requests for it before it’s been created. We end up creating 2 of them but only adding one to our collection.
Is there a better way to get Thingeys created and added that doesn’t involve check/create/lock/check/add with the rare extraneous thingey that we created but end up never using? (And this code works and has been running for some time. This is just the nagging bit that has always bothered me.)
I'm trying to avoid locking the dictionary for the duration of creating a Thingey.

This is the standard double check locking problem. The way it is implemented here is unsafe and can cause various problems - potentially up to the point of a crash in the first check if the internal state of the dictionary is screwed up bad enough.
It is unsafe because you are checking it without synchronization and if your luck is bad enough you can hit it while some other thread is in the middle of updating internal state of the dictionary
A simple solution is to place the first check under a lock as well. A problem with this is that this becomes a global lock and in web environment under heavy load it can become a serious bottleneck.
If we are talking about .NET environment, there are ways to work around this issue by piggybacking on the ASP.NET synchronization mechanism.
Here is how I did it in NDjango rendering engine: I keep one global dictionary and one dictionary per rendering thread. When a request comes I check the local dictionary first - this check does not have to be synchronized and if the thingy is there I just take it
If it is not I synchronize on the global dictionary check if it is there and if it is add it to my thread dictionary and release the lock. If it is not in the global dictionary I add it there first while still under lock.

Well, from my point of view simpler code is better, so I'd only use one lock:
private readonly object thingeysLock = new object();
private readonly Dictionary<string, Thingey> thingeys;
public Thingey GetThingey(Request request)
{
string key = request.ThingeyName;
lock (thingeysLock)
{
Thingey ret;
if (!thingeys.TryGetValue(key, out ret))
{
ret = new Thingey(request);
thingeys[key] = ret;
}
return ret;
}
}
Locks are really cheap when they're not contended. The downside is that this means that occasionally you will block everyone for the whole duration of the time you're creating a new Thingey. Clearly to avoid creating redundant thingeys you'd have to at least block while multiple threads create the Thingey for the same key. Reducing it so that they only block in that situation is somewhat harder.
I would suggest you use the above code but profile it to see whether it's fast enough. If you really need "only block when another thread is already creating the same thingey" then let us know and we'll see what we can do...
EDIT: You've commented on Adam's answer that you "don't want to lock while a new Thingey is being created" - you do realise that there's no getting away from that if there's contention for the same key, right? If thread 1 starts creating a Thingey, then thread 2 asks for the same key, your alternatives for thread 2 are either waiting or creating another instance.
EDIT: Okay, this is generally interesting, so here's a first pass at the "only block other threads asking for the same item".
private readonly object dictionaryLock = new object();
private readonly object creationLocksLock = new object();
private readonly Dictionary<string, Thingey> thingeys;
private readonly Dictionary<string, object> creationLocks;
public Thingey GetThingey(Request request)
{
string key = request.ThingeyName;
Thingey ret;
bool entryExists;
lock (dictionaryLock)
{
entryExists = thingeys.TryGetValue(key, out ret);
// Atomically mark the dictionary to say we're creating this item,
// and also set an entry for others to lock on
if (!entryExists)
{
thingeys[key] = null;
lock (creationLocksLock)
{
creationLocks[key] = new object();
}
}
}
// If we found something, great!
if (ret != null)
{
return ret;
}
// Otherwise, see if we're going to create it or whether we need to wait.
if (entryExists)
{
object creationLock;
lock (creationLocksLock)
{
creationLocks.TryGetValue(key, out creationLock);
}
// If creationLock is null, it means the creating thread has finished
// creating it and removed the creation lock, so we don't need to wait.
if (creationLock != null)
{
lock (creationLock)
{
Monitor.Wait(creationLock);
}
}
// We *know* it's in the dictionary now - so just return it.
lock (dictionaryLock)
{
return thingeys[key];
}
}
else // We said we'd create it
{
Thingey thingey = new Thingey(request);
// Put it in the dictionary
lock (dictionaryLock)
{
thingeys[key] = thingey;
}
// Tell anyone waiting that they can look now
lock (creationLocksLock)
{
Monitor.PulseAll(creationLocks[key]);
creationLocks.Remove(key);
}
return thingey;
}
}
Phew!
That's completely untested, and in particular it isn't in any way, shape or form robust in the face of exceptions in the creating thread... but I think it's the generally right idea :)

If you're looking to avoid blocking unrelated threads, then additional work is needed (and should only be necessary if you've profiled and found that performance is unacceptable with the simpler code). I would recommend using a lightweight wrapper class that asynchronously creates a Thingey and using that in your dictionary.
Dictionary<string, ThingeyWrapper> thingeys = new Dictionary<string, ThingeyWrapper>();
private class ThingeyWrapper
{
public Thingey Thing { get; private set; }
private object creationLock;
private Request request;
public ThingeyWrapper(Request request)
{
creationFlag = new object();
this.request = request;
}
public void WaitForCreation()
{
object flag = creationFlag;
if(flag != null)
{
lock(flag)
{
if(request != null) Thing = new Thingey(request);
creationFlag = null;
request = null;
}
}
}
}
public Thingey GetThingey(Request request)
{
string thingeyName = request.ThingeyName;
ThingeyWrapper output;
lock (this.Thingeys)
{
if(!this.Thingeys.TryGetValue(thingeyName, out output))
{
output = new ThingeyWrapper(request);
this.Thingeys.Add(thingeyName, output);
}
}
output.WaitForCreation();
return output.Thing;
}
While you are still locking on all calls, the creation process is much more lightweight.
Edit
This issue has stuck with me more than I expected it to, so I whipped together a somewhat more robust solution that follows this general pattern. You can find it here.

IMHO, if this piece of code is called from many thread simultaneous, it is recommended to check it twice.
(But: I'm not sure that you can safely call ContainsKey while some other thread is call Add. So it might not be possible to avoid the lock at all.)
If you just want to avoid the Thingy is created but not used, just create it within the locking block:
private Dictionary<string, Thingey> Thingeys;
public Thingey GetThingey(Request request)
{
string thingeyName = request.ThingeyName;
if (!this.Thingeys.ContainsKey(thingeyName))
{
lock (this.Thingeys)
{
// only one can create the same Thingy
Thingey newThingey = new Thingey(request);
if (!this.Thingeys.ContainsKey(thingeyName))
{
this.Thingeys.Add(thingeyName, newThingey);
}
}
}
return this. Thingeys[thingeyName];
}

You have to ask yourself the question whether the specific ContainsKey operation and the getter are themselfes threadsafe (and will stay that way in newer versions), because those may and willbe invokes while another thread has the dictionary locked and is performing the Add.
Typically, .NET locks are fairly efficient if used correctly, and I believe that in this situation you're better of doing this:
bool exists;
lock (thingeys) {
exists = thingeys.TryGetValue(thingeyName, out thingey);
}
if (!exists) {
thingey = new Thingey();
}
lock (thingeys) {
if (!thingeys.ContainsKey(thingeyName)) {
thingeys.Add(thingeyName, thingey);
}
}
return thingey;

Well I hope not being to naive at giving this answer. but what I would do, as Thingyes are expensive to create, would be to add the key with a null value. That is something like this
private Dictionary<string, Thingey> Thingeys;
public Thingey GetThingey(Request request)
{
string thingeyName = request.ThingeyName;
if (!this.Thingeys.ContainsKey(thingeyName))
{
lock (this.Thingeys)
{
this.Thingeys.Add(thingeyName, null);
if (!this.Thingeys.ContainsKey(thingeyName))
{
// create a new thingey on 1st reference
Thingey newThingey = new Thingey(request);
Thingeys[thingeyName] = newThingey;
}
// else - oops someone else beat us to it
// but it doesn't mather anymore since we only created one Thingey
}
}
return this.Thingeys[thingeyName];
}
I modified your code in a rush so no testing was done.
Anyway, I hope my idea is not so naive. :D

You might be able to buy a little bit of speed efficiency at the expense of memory. If you create an immutable array that lists all of the created Thingys and reference the array with a static variable, then you could check the existance of a Thingy outside of any lock, since immutable arrays are always thread safe. Then when adding a new Thingy, you can create a new array with the additional Thingy and replace it (in the static variable) in one (atomic) set operation. Some new Thingys may be missed, because of race conditions, but the program shouldn't fail. It just means that on rare occasions extra duplicate Thingys will be made.
This will not replace the need for duplicate checking when creating a new Thingy, and it will use a lot of memory resources, but it will not require that the lock be taken or held while creating a Thingy.
I'm thinking of something along these lines, sorta:
private Dictionary<string, Thingey> Thingeys;
// An immutable list of (most of) the thingeys that have been created.
private string[] existingThingeys;
public Thingey GetThingey(Request request)
{
string thingeyName = request.ThingeyName;
// Reference the same list throughout the method, just in case another
// thread replaces the global reference between operations.
string[] localThingyList = existingThingeys;
// Check to see if we already made this Thingey. (This might miss some,
// but it doesn't matter.
// This operation on an immutable array is thread-safe.
if (localThingyList.Contains(thingeyName))
{
// But referencing the dictionary is not thread-safe.
lock (this.Thingeys)
{
if (this.Thingeys.ContainsKey(thingeyName))
return this.Thingeys[thingeyName];
}
}
Thingey newThingey = new Thingey(request);
Thiney ret;
// We haven't locked anything at this point, but we have created a new
// Thingey that we probably needed.
lock (this.Thingeys)
{
// If it turns out that the Thingey was already there, then
// return the old one.
if (!Thingeys.TryGetValue(thingeyName, out ret))
{
// Otherwise, add the new one.
Thingeys.Add(thingeyName, newThingey);
ret = newThingey;
}
}
// Update our existingThingeys array atomically.
string[] newThingyList = new string[localThingyList.Length + 1];
Array.Copy(localThingyList, newThingey, localThingyList.Length);
newThingey[localThingyList.Length] = thingeyName;
existingThingeys = newThingyList; // Voila!
return ret;
}

Mutual Exclusion Required

Code that not affecting the collection needs Mutual Exclusion
List<string> _itemColection = new List<string>();
object _criticalSection = new object();
private void Add(string item)
{
lock (_criticalSection)
{
_itemColection.Add(item);
}
}
private void Remove(string item)
{
lock (_criticalSection)
{
_itemColection.Remove(item);
}
}
private void GetCount()
{
///Is it lock is reuired here?
return _itemColection.Count;
}
//Thread method
private void Run()
{
lock (_criticalSection)
{
foreach (string item in _itemColection)
{
///Some operation
}
}
}
Is it Mutex is required in GetCount() method. The collection value's are not changed

Yes, you should lock there. You're requesting access to shared data, and if you don't have some sort of memory barrier there'll be no guarantee that it'll be "fresh" information. The memory model can be a real mind-bender sometimes :)
In addition, while I'd expect List<T>.Count to be a pretty simple operation, it could theoretically be complicated - and if another thread is mutating stuff (e.g. adding an item, which then requires a buffer expansion) while it's working out the count, you could theoretically run into trouble.
Basically, unless a type claims to be thread-safe for your particular scenario, I'd always make sure you don't perform two operations on it at the same time.

In practise its probably not needed, but I would include it myself to be on the safe side. _itemCollection.Count is a property, so effectively you are calling a function on the collection, and you have no guarantee what this function is doing.

Simple answer is yes because the values might get changed and you won't have up-to-date data.

That's an excelent question about concurency..
In my opinion it's always necessary to use lock when you are in presence of some concurrency.
EDIT
We just use lock in operations that need to change some information, and never need to use that in read-only objects.
Best regards