I have List<string> collection called List<string> list.
I have two threads.
One thread is enumerating through all list elements and adding to collection.
Second thread is enumerating through all list elements and removing from it.
How can make it thread safe?
I tried creating global Object "MyLock" and using lock(MyLock) block in each thread function but it didn't work.
Can you help me?
If you have access to .NET 4.0 you can use the class ConcurrentQueue or a BlockingCollection with a ConcurrentQueue backing it. It does exactly what you are trying to do and does not require any locking. The BlockingCollection will make your thread wait if there is no items available in the list.
A example of removing from the ConcurrentQueue you do something like
ConcurrentQueue<MyClass> cq = new ConcurrentQueue<MyClass>();
void GetStuff()
{
MyClass item;
if(cq.TryDeqeue(out item))
{
//Work with item
}
}
This will try to remove a item, but if there are none available it does nothing.
BlockingCollection<MyClass> bc = BlockingCollection<MyClass>(new ConcurrentQueue<MyClass>());
void GetStuff()
{
if(!bc.IsCompleated) //check to see if CompleatedAdding() was called and the list is empty.
{
try
{
MyClass item = bc.Take();
//Work with item
}
catch (InvalidOpperationExecption)
{
//Take is marked as completed and is empty so there will be nothing to take
}
}
}
This will block and wait on the Take till there is something available to take from the list. Once you are done you can call CompleteAdding() and Take will throw a execption when the list becomes empty instead of blocking.
Without knowing more about your program and requirements, I'm going say that this is a "Bad Idea". Altering a List<> while iterating through it's contents will most likely throw an exception.
You're better off using a Queue<> instead of a List<>, as a Queue<> was designed with synchronization in mind.
You should be able to lock directly on your list:
lock(list) {
//work with list here
}
However adding/removing from the list while enumerating it will likely cause an exception...
Lock on the SyncRoot of your List<T>:
lock(list.SyncRoot)
{
}
More information on how to use it properly can be found here
You could implement your own version of IList<T> that wraps the underlying List<T> to provide locking on every method call.
public class LockingList<T> : IList<T>
{
public LockingList(IList<T> inner)
{
this.Inner = inner;
}
private readonly object gate = new object();
public IList<T> Inner { get; private set; }
public int IndexOf(T item)
{
lock (gate)
{
return this.Inner.IndexOf(item);
}
}
public void Insert(int index, T item)
{
lock (gate)
{
this.Inner.Insert(index, item);
}
}
public void RemoveAt(int index)
{
lock (gate)
{
this.Inner.RemoveAt(index);
}
}
public T this[int index]
{
get
{
lock (gate)
{
return this.Inner[index];
}
}
set
{
lock (gate)
{
this.Inner[index] = value;
}
}
}
public void Add(T item)
{
lock (gate)
{
this.Inner.Add(item);
}
}
public void Clear()
{
lock (gate)
{
this.Inner.Clear();
}
}
public bool Contains(T item)
{
lock (gate)
{
return this.Inner.Contains(item);
}
}
public void CopyTo(T[] array, int arrayIndex)
{
lock (gate)
{
this.Inner.CopyTo(array, arrayIndex);
}
}
public int Count
{
get
{
lock (gate)
{
return this.Inner.Count;
}
}
}
public bool IsReadOnly
{
get
{
lock (gate)
{
return this.Inner.IsReadOnly;
}
}
}
public bool Remove(T item)
{
lock (gate)
{
return this.Inner.Remove(item);
}
}
public IEnumerator<T> GetEnumerator()
{
lock (gate)
{
return this.Inner.ToArray().AsEnumerable().GetEnumerator();
}
}
IEnumerator IEnumerable.GetEnumerator()
{
lock (gate)
{
return this.Inner.ToArray().GetEnumerator();
}
}
}
You would use this code like this:
var list = new LockingList<int>(new List<int>());
If you're using large lists and/or performance is an issue then this kind of locking may not be terribly performant, but in most cases it should be fine.
It is very important to notice that the two GetEnumerator methods call .ToArray(). This forces the evaluation of the enumerator before the lock is released thus ensuring that any modifications to the list don't affect the actual enumeration.
Using code like lock (list) { ... } or lock (list.SyncRoot) { ... } do not cover you against list changes occurring during enumerations. These solutions only cover against concurrent modifications to the list - and that's only if all callers do so within a lock. Also these solutions can cause your code to die if some nasty bit of code takes a lock and doesn't release it.
In my solution you'll notice I have a object gate that is a private variable internal to the class that I lock on. Nothing outside the class can lock on this so it is safe.
I hope this helps.
As others already said, you can use concurrent collections from the System.Collections.Concurrent namespace. If you can use one of those, this is preferred.
But if you really want a list which is just synchronized, you could look at the SynchronizedCollection<T>-Class in System.Collections.Generic.
Note that you had to include the System.ServiceModel assembly, which is also the reason why I don't like it so much. But sometimes I use it.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I'm looking for best practices / standard on how to deal with this situation.
We have our code (MyClass) that consumes another class (ItemGenerator).
ItemGenerator is a blackbox for us so we don't know the implementation (we do but we don't want to rely on that because it could change from underneath).
ItemGenerator has a method, GetItems(), that returns an IEnumerable of Item.
Item class implements IDisposable so we should dispose of the object when we are done.
When we (MyClass) iterates through the list of items, if an exception (any exception) occurs, we want to stop processing and release control (bubble up the exception).
My question is this:
Should we keep iterating through the items in order to dispose of all of them? It might seem silly but what happens with the rest of the items if they are not disposed of?
At the same time, based on the code below, we should definitely not iterate through the rest of the items because they are yield return. So why generate them just so we can dispose of them (it could significantly affect the performance).
The problem is that we do not know if GetItems() returns the items on demand (yield) or not. And I don't think we should care, right?
So how should we handle the situation when an exception occurs in the middle of the list (for example)?
Below is an example of the code that illustrates the gist of it.
This is our code:
public class MyClass
{
public void VerifyAllItems()
{
ItemGenerator generator = new ItemGenerator();
foreach (Item item in generator.GetItems())
{
try
{
// Do some work with "item" here. Though an exception could occur.
// If an exception occurs, we don't care about processing the rest of the items and just want to bubble up the exception
}
finally
{
// Always dispose of the
item?.Dispose();
}
}
}
}
And this is the blackbox code
public class ItemGenerator
{
private long _itemsToGenerate = 0;
public ItemGenerator()
{
_itemsToGenerate = new Random().Next(10, 100);
}
public IEnumerable<Item> GetItems()
{
while (_itemsToGenerate > 0)
{
yield return HeavyWork();
_itemsToGenerate--;
}
}
private Item HeavyWork()
{
// Doing a lot of work here
return new Item();
}
}
public class Item : IDisposable
{
private bool _isDisposed = false;
public virtual void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
private void Dispose(bool isDisposing)
{
if (!_isDisposed)
{
if (isDisposing)
{
// Dispose of any resources
}
_isDisposed = true;
}
}
}
The problem is worse than you state. Not only can you not be sure whether to enumerate the collection, you can't be sure whether to dispose of any of the items at all. Just because something implements IDisposable doesn't mean you should be disposing it, e.g. if you have a factory that always returns the same instance of something.
This kind of a problem is exactly why the code that allocates something is generally held responsible for deallocating it. In this case, the ItemGenerator creates the items, so it should dispose them.
class ItemGenerator : IDiposable
{
protected readonly List<Item> _instances = new List<Item>();
IEnumerable<Item> GetItems()
{
for ( some; condition; here; )
{
var item = new Item();
_instances.Add(item);
yield return item;
}
}
public void Dispose()
{
foreach (var item in _instances) item.Dispose();
}
}
Now all you have to do is put your ItemGenerator in a using block and you're good to go.
public void VerifyAllItems()
{
using (ItemGenerator generator = new ItemGenerator())
{
foreach (Item item in generator.GetItems())
{
try
{
// Do some work with "item" here. Though an exception could occur.
// If an exception occurs, we don't care about processing the rest of the items and just want to bubble up the exception
}
finally
{
//Don't need to dispose anything here
}
}
} //Disposal happens here because of the using statement
}
With this patterns, any items that were allocated by the ItemGenerator will get disposed when you exit the using block.
Now the caller doesn't have to care about the implementation of the item generator at all, or worry about disposing anything other than the generator itself. And of course you should dispose the generator if you're the one who allocated it.
Slightly bizarre, but...
internal class DisposingEnumerator : IEnumerator<Item>
{
private readonly List<IDisposable> deallocationQueue = new List<IDisposable>();
private readonly IEnumerable<Item> source;
private IEnumerator<Item> sourceEnumerator;
public DisposingEnumerator(IEnumerable<Item> source)
{
this.source = source;
}
public bool MoveNext()
{
if (sourceEnumerator == null)
{
sourceEnumerator = source.GetEnumerator();
}
bool hasNext = sourceEnumerator.MoveNext();
if (hasNext)
{
deallocationQueue.Add(Current);
}
return hasNext;
}
public Item Current => sourceEnumerator.Current;
object IEnumerator.Current => Current;
public void Reset()
{
throw new NotSupportedException();
}
// Will be called within "foreach" statement
// You can implement IDisposable in ItemCollection as well
public void Dispose()
{
foreach (var item in deallocationQueue)
{
item.Dispose();
}
}
}
public class ItemCollection : IEnumerable<Item>
{
private IEnumerator<Item> enumerator;
public ItemCollection(IEnumerable<Item> source)
{
this.enumerator = new DisposingEnumerator(source);
}
public IEnumerator<Item> GetEnumerator()
{
return enumerator;
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
Usage:
var items = generator.GetItems();
// Equivalent of using statement
foreach (var item in new ItemCollection(items))
{
}
Proof
I need to make a class that stores a List of int and read/write from it asynchronously.
here's my class:
public class ConcurrentIntegerList
{
private readonly object theLock = new object();
List<int> theNumbers = new List<int>();
public void AddNumber(int num)
{
lock (theLock)
{
theNumbers.Add(num);
}
}
public List<int> GetNumbers()
{
lock (theLock)
{
return theNumbers;
}
}
}
But it is not thread-safe until here. when I do multiple operations from different threads I get this error:
Collection was modified; enumeration operation may not execute.
What I missed here?
public List<int> GetNumbers()
{
lock (theLock)
{
// return theNumbers;
return theNumbers.ToList();
}
}
But the performance won't be very good this way, and GetNumbers() now returns a snapshot copy.
I've got a public static List<MyDoggie> DoggieList;
DoggieList is appended to and written to by multiple processes throughout my application.
We run into this exception pretty frequently:
Collection was modified; enumeration operation may not execute
Assuming there are multiple classes writing to DoggieList how do we get around this exception?
Please note that this design is not great, but at this point we need to quickly fix it in production.
How can we perform mutations to this list safely from multiple threads?
I understand we can do something like:
lock(lockObject)
{
DoggieList.AddRange(...)
}
But can we do this from multiple classes against the same DoggieList?
you can also create you own class and encapsulate locking thing in that only, you can try like as below ,
you can add method you want like addRange, Remove etc.
class MyList {
private object objLock = new object();
private List<int> list = new List<int>();
public void Add(int value) {
lock (objLock) {
list.Add(value);
}
}
public int Get(int index) {
int val = -1;
lock (objLock) {
val = list[0];
}
return val;
}
public void GetAll() {
List<int> retList = new List<int>();
lock (objLock) {
retList = new List<T>(list);
}
return retList;
}
}
Good stuff : Concurrent Collections very much in detail :http://www.albahari.com/threading/part5.aspx#_Concurrent_Collections
making use of concurrent collection ConcurrentBag Class can also resolve issue related to multiple thread update
Example
using System.Collections.Concurrent;
using System.Threading.Tasks;
public static class Program
{
public static void Main()
{
var items = new[] { "item1", "item2", "item3" };
var bag = new ConcurrentBag<string>();
Parallel.ForEach(items, bag.Add);
}
}
Using lock a the disadvantage of preventing concurrent readings.
An efficient solution which does not require changing the collection type is to use a ReaderWriterLockSlim
private static readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
With the following extension methods:
public static class ReaderWriterLockSlimExtensions
{
public static void ExecuteWrite(this ReaderWriterLockSlim aLock, Action action)
{
aLock.EnterWriteLock();
try
{
action();
}
finally
{
aLock.ExitWriteLock();
}
}
public static void ExecuteRead(this ReaderWriterLockSlim aLock, Action action)
{
aLock.EnterReadLock();
try
{
action();
}
finally
{
aLock.ExitReadLock();
}
}
}
which can be used the following way:
_lock.ExecuteWrite(() => DoggieList.Add(new Doggie()));
_lock.ExecuteRead(() =>
{
// safe iteration
foreach (MyDoggie item in DoggieList)
{
....
}
})
And finally if you want to build your own collection based on this:
public class SafeList<T>
{
private readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
private readonly List<T> _list = new List<T>();
public T this[int index]
{
get
{
T result = default(T);
_lock.ExecuteRead(() => result = _list[index]);
return result;
}
}
public List<T> GetAll()
{
List<T> result = null;
_lock.ExecuteRead(() => result = _list.ToList());
return result;
}
public void ForEach(Action<T> action) =>
_lock.ExecuteRead(() => _list.ForEach(action));
public void Add(T item) => _lock.ExecuteWrite(() => _list.Add(item));
public void AddRange(IEnumerable<T> items) =>
_lock.ExecuteWrite(() => _list.AddRange(items));
}
This list is totally safe, multiple threads can add or get items in parallel without any concurrency issue. Additionally, multiple threads can get items in parallel without locking each other, it's only when writing than 1 single thread can work on the collection.
Note that this collection does not implement IEnumerable<T> because you could get an enumerator and forget to dispose it which would leave the list locked in read mode.
make DoggieList of type ConcurrentStack and then use pushRange method. It is thread safe.
using System.Collections.Concurrent;
var doggieList = new ConcurrentStack<MyDoggie>();
doggieList.PushRange(YourCode)
I came across this puzzle in a programming contest where I am not supposed to use any inbuilt concurrent .NET 4.0 data structures.
I have to override the ToString() method and this method should support concurrent readers.
This is the solution which I came up with but I strongly believe it is does not support concurrent readers. How can I support concurrent readers without locking the list?
class Puzzle
{
private List<string> listOfStrings = new List<string>();
public void Add(string item)
{
lock (listOfStrings)
{
if (item != null)
{
listOfStrings.Add(item);
}
}
}
public override string ToString()
{
lock (listOfStrings)
{
return string.Join(",", listOfStrings);
}
}
}
Because List<T> is just a T[] under the hood, there's nothing unsafe about reading the list from multiple threads. Your issue is that reading while writing and writing concurrently are unsafe. Because of this, you should use ReaderWriterLockSlim.
class Puzzle
{
private List<string> listOfStrings = new List<string>();
private ReaderWriterLockSlim listLock = new ReaderWriterLockSlim();
public void Add(string item)
{
listLock.EnterWriteLock();
try
{
listOfStrings.Add(item);
}
finally
{
listlock.ExitWriteLock();
}
}
public override string ToString()
{
listLock.EnterReadLock();
try
{
return string.Join(",", listOfStrings);
}
finally
{
listLock.ExitReadLock();
}
}
}
ReaderWriterLockSlim will allow multiple threads to enter the lock in read mode, but they will all block if/while something is in the lock in write mode. Likewise, entering the lock in write mode will block until all threads have exited the lock in read mode. The practical outworking is that multiple reads can happen at the same time as long as nothing is writing, and one write operation can happen at a time as long as nothing is reading.
Since it doesn't look like Remove isn't a requirement, why can't you just return a string?
class Puzzle
{
private string Value { get; set; }
public Puzzle()
{
Value = String.Empty;
}
public void Add(String item)
{
Value += "," + item;
}
public override string ToString()
{
return Value;
}
}
I recently had a interview question in a test that was similar to the below, I do not have very much experience of development using threads can someone please help advise me how to approach this question?:
public class StringQueue
{
private object _lockObject = new object();
private List<string> _items = new List<string>();
public bool IsEmpty()
{
lock (_lockObject)
return _items.Count == 0;
}
public void Enqueue(string item)
{
lock (_lockObject)
_items.Add(item);
}
public string Dequeue()
{
lock (_lockObject)
{
string result = _items[0];
_items.RemoveAt(0);
return result;
}
}
}
Is the following method thread safe with the above implementation and why?
public string DequeueOrNull()
{
if (IsEmpty())
return null;
return Dequeue();
}
It seems to me the answer is no.
While isEmpty() procedure locks the object, it is released as soon as the call is returned -
a different thread could potentially call DequeueOrNull() between the call to IsEmpty() and Dequeue() (at which time the object is unlocked), thus removing the only item that existed, making Dequeue() invalid at that time.
A plausible fix would be to put the lock over both statements in DequeueOrNull(), so no other thread could call DeQueue() after the check but before the DeQueue().
It is not threadsafe. At the marked line it is possible that the Dequeue method is called from another thread and thus, the consequent Dequeue return a wrong value:
public string DequeueOrNull()
{
if (IsEmpty())
return null;
/// << it is possible that the Dequeue is called from another thread here.
return Dequeue();
}
The thread safe code would be:
public string DequeueOrNull()
{
lock(_lockObject) {
if (IsEmpty())
return null;
return Dequeue();
}
}
No, because the state of _items could potentially change between the thread-safe IsEmpty() and the thread-safe Dequeue() calls.
Fix it with something like the following, which ensures that _items is locked during the whole operation:
public string DequeueOrNull()
{
lock (_lockObject)
{
if (IsEmpty())
return null;
return Dequeue();
}
}
Note: depending in the implementation of _lock, you may wish to avoid double-locking the resource by moving the guts of IsEmpty() and Dequeue into separate helper functions.