I have a Generic List as below
public static readonly List<Customer> Customers = new List<Customer>();
I'm using the below methods for it:
.Add
.Find
.FirstOrDefault
The last 2 are LINQ extensions.
I'd need to make this thread-safe to be able to run multiple instances of the container class.
How to achieve that?
If those are the only functions you are using on List<T> then the easiest way is to write a quick wrapper that synchronizes access with a lock
class MyList<T> {
private List<T> _list = new List<T>();
private object _sync = new object();
public void Add(T value) {
lock (_sync) {
_list.Add(value);
}
}
public bool Find(Predicate<T> predicate) {
lock (_sync) {
return _list.Find(predicate);
}
}
public T FirstOrDefault() {
lock (_sync) {
return _list.FirstOrDefault();
}
}
}
I highly recommend the approach of a new type + private lock object. It makes it much more obvious to the next guy who inherits your code what the actual intent was.
Also note that .Net 4.0 introduced a new set of collections specifically aimed at being used from multiple threads. If one of these meets your needs I'd highly recommend using it over rolling your own.
ConcurrentStack<T>
ConcurrentQueue<T>
To expand on #JaradPar's answer, here is a full implementation with a few extra features, as described in the summary
/// <summary>
/// a thread-safe list with support for:
/// 1) negative indexes (read from end). "myList[-1]" gets the last value
/// 2) modification while enumerating: enumerates a copy of the collection.
/// </summary>
/// <typeparam name="TValue"></typeparam>
public class ConcurrentList<TValue> : IList<TValue>
{
private object _lock = new object();
private List<TValue> _storage = new List<TValue>();
/// <summary>
/// support for negative indexes (read from end). "myList[-1]" gets the last value
/// </summary>
/// <param name="index"></param>
/// <returns></returns>
public TValue this[int index]
{
get
{
lock (_lock)
{
if (index < 0)
{
index = this.Count - index;
}
return _storage[index];
}
}
set
{
lock (_lock)
{
if (index < 0)
{
index = this.Count - index;
}
_storage[index] = value;
}
}
}
public void Sort()
{
lock (_lock)
{
_storage.Sort();
}
}
public int Count
{
get
{
lock (_lock) return _storage.Count;
}
}
bool ICollection<TValue>.IsReadOnly
{
get
{
return ((IList<TValue>)_storage).IsReadOnly;
}
}
public void Add(TValue item)
{
lock (_lock)
{
_storage.Add(item);
}
}
public void Clear()
{
lock (_lock)
{
_storage.Clear();
}
}
public bool Contains(TValue item)
{
lock (_lock)
{
return _storage.Contains(item);
}
}
public void CopyTo(TValue[] array, int arrayIndex)
{
lock (_lock)
{
_storage.CopyTo(array, arrayIndex);
}
}
public int IndexOf(TValue item)
{
lock (_lock)
{
return _storage.IndexOf(item);
}
}
public void Insert(int index, TValue item)
{
lock (_lock)
{
_storage.Insert(index, item);
}
}
public bool Remove(TValue item)
{
lock (_lock)
{
return _storage.Remove(item);
}
}
public void RemoveAt(int index)
{
lock (_lock)
{
_storage.RemoveAt(index);
}
}
public IEnumerator<TValue> GetEnumerator()
{
lock (_lock)
{
return _storage.ToArray().AsEnumerable().GetEnumerator();
}
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
If you're using version 4 or greater of the .NET framework you can use the thread-safe collections.
You can replace List<T> with ConcurrentBag<T>:
namespace Playground.Sandbox
{
using System.Collections.Concurrent;
using System.Threading.Tasks;
public static class Program
{
public static void Main()
{
var items = new[] { "Foo", "Bar", "Baz" };
var bag = new ConcurrentBag<string>();
Parallel.ForEach(items, bag.Add);
}
}
}
You will need to use locks in every place where the collection gets modified or iterated over.
Either that or use one of the new thread-safe data structures, like ConcurrentBag.
Use the lock keyword when you manipulate the collection, ie: your Add/Find:
lock(Customers) {
Customers.Add(new Customer());
}
Never use ConcurrangBag for ordered data. Use Array instead
Make your Action as accessible by one only by using lock on any private object
Refer to : Thread Safe Generic Queue Class
http://www.codeproject.com/Articles/38908/Thread-Safe-Generic-Queue-Class
Ok, so I had to completely rewrite my answer. After 2 days of testing I have to say that the JasonS's code has some defects, I guess because of Enumerators. While one thread uses foreach, and the other other changes the list, it throws exceptions.
So I found this answer, and it works for me fine the last 48 hours non-stop, I guess more than 100k threads were created in my application, and used that lists.
The only thing I changed - I've moved entering the locks outside the try-finally section. Read here about the possible exceptions. Also, if you will read MSDN, they have the same approach.
But, as were mentioned in link below, List can not be 100% thread safe, probably that is why there is no default ConcurentList implementation in c#.
Related
I'm looking for a C# collection type that allow me to define a maximum capacity. I would like to add objects to this collection and when my capacity is reached the oldest object should be replaced by the new one.
Specifically, I would like to create a collection that memorizes my last 10 or 20 actions.
I already read articles on Google but I'm looking for answers from this community.
The only fixed collection type in .NET is an array, so that is the only one meeting your requirement.
You can keep an index to do the rotating. You just have to remember what your next location to write to is.
Something like this:
int index = 0;
string[] collection = new string[10];
public void Write(string text)
{
index %= collection.Length; // prevent overflowing
collection[index++] = text;
}
If your application is not performance-sensitive, then you can use a generic collection Queue.
Here is the example wrapper which can solve your problem:
public class LimitedQueue<T>
{
private readonly Queue<T> _queue;
private readonly int _limit;
public LimitedQueue(int limit)
{
_queue = new Queue<T>();
_limit = limit;
}
public void Enqueue(T item)
{
if (_queue.Count == _limit) _queue.Dequeue();
_queue.Enqueue(item);
}
public T Dequeue()
{
return _queue.Dequeue();
}
public T Peek()
{
return _queue.Peek();
}
public T[] GetAll()
{
return _queue.ToArray();
}
}
It is less performant than an array but it lets you do something useful things like getting all items from the Queue.
The type you are describing I've always referred to as a FixedQueue or a fixed size FIFO. The idea is the first in first out but you discard the first out if the size is exceeded:
public class FixedQueue<T>
{
private readonly ConcurrentQueue<T> _innerQueue;
private int _length;
public FixedQueue(int length)
{
_length = length;
_innerQueue = new ConcurrentQueue<T>(length);
}
public void Enqueue(T obj)
{
lock (_innerQueue)
{
if (_innerQueue.Length == _length)
_innerQueue.Dequeue();
_innerQueue.Enqueue(obj);
}
}
public T Dequeue()
{
lock (_innerQueue)
{
return _innerQueue.Dequeue();
}
}
// etc...
}
If you are looking for a more oriented object solution with inheritance and you don't care about maintenance you can write this:
public class FixedQueue<T> : Queue<T>
{
//private readonly ConcurrentQueue<T> _innerQueue;
private int _capacity;
public FixedQueue(int capacity) : base()
{
_capacity = capacity;
}
public void Enqueue(T obj)
{
lock(this)
{
if (this.Count == _capacity)
base.Dequeue();
base.Enqueue(obj);
}
}
public T Dequeue()
{
lock (this)
{
return this.Dequeue();
}
}
}
this solution doesn't respect the new trend that says you should prefer composition over inheritance.
I have a singleton IEnumerable that generates a sequence of numbers. The sequence is interable (basically indefinitely) and I generate the next number in the sequence only when needed.
public class Generator:IEnumerable<long> {
private Generator() { }
private static volatile Generator instance=new Generator();
private static readonly object syncRoot=new object();
public static Generator Instance { get { return instance; } }
private static List<long> numsList=new List<long>();
private void GenerateNextNumber() {
long number;
//Code to generate next number
numsList.Add(number);
}
private long GenerateToNthNumber(int n) {
lock(syncRoot) {
while(numsList.Count<n)
GenerateNextNumber();
}
return numsList[n-1];
}
public static long GetNthNumber(int n) {
return Instance.GenerateToNthNumber(n);
}
private class GeneratorEnumerator:IEnumerator<long> {
private int index=0;
public long Current { get { return GetNthNumber(index); } }
public void Dispose() { }
object System.Collections.IEnumerator.Current { get { return GetNthNumber(index); } }
public bool MoveNext() {
index++;
return true;
}
public void Reset() {
index=0;
}
}
public IEnumerator<long> GetEnumerator() {
return new GeneratorEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() {
return GetEnumerator();
}
}
This code works enumerating through and summing the numbers in concurrent threads. Is there a way to prevent having to lock every time GenerateToNthNumber is called? I tried this code:
private long GenerateToNthNumber(int n) {
if(numsList.Count<n) {
lock(syncRoot) {
while(numsList.Count<n)
GenerateNextNumber();
}
}
return numsList[n-1];
}
But when testing enumerating through and summing the numbers in multiple concurrent threads, not all the results end up with the same sum. My objective is to have non-blocking reads if the number being asked for is already generated, if that is even possible. Is there a better way to do this?
The way List is implemented, it cannot be safely read in one thread while it is being written in another. I would suggest that instead you use nested known-size arrays which, once allocated, are never abandoned (e.g. once an array is allocated that will hold theList[15691], the item will never be held by any other array). Such things may easily be used to implement an add-only list which requires locking when adding items, but is inherently thread-safe for reading without locking.
Have you thought about using a thread safe collection?
http://msdn.microsoft.com/en-us/library/dd997305.aspx
I have a list where it is entries can be updated, new data inserted or removed from 2 different threads.
Is it ok to use a public readonly object to lock when it is being used to interact to the other thread as to when it is locked or not or what would be the correct way to use this list across the 2 threads ?
You should always use a lock when accessing the list on different threads.
public class Sample
{
object synch = new object();
List<Something> list = new List<Something>();
void Add(Something something)
{
lock (synch) { list.Add(something); }
}
// Add the methods for update and delete.
}
You should wrap this in a class that handles the locking for you, or use a thread-safe collection, such as ConcurrentQueue<T> or one of the other collections in System.Collections.Concurrent.
Exposing the synchronization object to a public API is dangerous, and not a good practice.
First, read this article to understand why it's bad: http://blogs.msdn.com/b/jaredpar/archive/2009/02/11/why-are-thread-safe-collections-so-hard.aspx
Then, do it anyway like I did:
public abstract class ConcurrentCollection<T> : ICollection<T>
{
private List<T> List { get; set; }
public ConcurrentCollection()
{
this.List = new List<T>();
}
public T this[int index]
{
get
{
return this.List[index];
}
}
protected virtual void AddUnsafe(T item)
{
this.List.Add(item);
}
protected virtual void RemoveUnsafe(T item)
{
this.List.Remove(item);
}
protected virtual void ClearUnsafe()
{
this.List.Clear();
}
public void Add(T item)
{
lock (this.List)
{
this.AddUnsafe(item);
}
}
public bool Remove(T item)
{
lock (this.List)
{
this.RemoveUnsafe(item);
return true;
}
}
public void Clear()
{
lock (this.List)
{
this.ClearUnsafe();
}
}
public int Count
{
get
{
lock (this.List)
{
return this.List.Count;
}
}
}
public bool IsReadOnly
{
get
{
return false;
}
}
public bool Contains(T item)
{
lock (this.List)
{
return this.List.Contains(item);
}
}
public void CopyTo(T[] array, int arrayIndex)
{
lock (this.List)
{
this.List.CopyTo(array, arrayIndex);
}
}
public IEnumerator<T> GetEnumerator()
{
return new ConcurrentEnumerator<T>(this.List, this.List);
}
IEnumerator IEnumerable.GetEnumerator()
{
throw new NotImplementedException("Abstract concurrent enumerators not implemented.");
}
}
public class ConcurrentEnumerator<T> : IEnumerator<T>
{
private int Position = -1;
private List<T> Duplicate;
private object Mutex;
private ICollection<T> NonConcurrentCollection;
internal ConcurrentEnumerator(ICollection<T> nonConcurrentCollection, object mutex)
{
this.NonConcurrentCollection = nonConcurrentCollection;
this.Mutex = mutex;
lock (this.Mutex)
{
this.Duplicate = new List<T>(this.NonConcurrentCollection);
}
}
public T Current
{
get
{
return this.Duplicate[this.Position];
}
}
object IEnumerator.Current
{
get
{
return this.Current;
}
}
public bool MoveNext()
{
this.Position++;
lock (this.Mutex)
{
while (this.Position < this.Duplicate.Count && !this.NonConcurrentCollection.Contains(this.Current))
{
this.Position++;
}
}
return this.Position < this.Duplicate.Count;
}
public void Reset()
{
this.Position = -1;
}
public void Dispose() { }
}
// Standards have List as derived Collection...
public class ConcurrentList<T> : ConcurrentCollection<T> { }
This code is still not fully safe, for instance the Count example may still crash, but it allows for iteration, adding and removing across threads. If you want to expose the mutex, do so, then lock around it for your other code constructs like count and contains.
But it's still a bad idea.
Edit: Example usage.
ConcurrentList<string> list = new ConcurrentList<string>();
list.Add("hello");
list.Add("world");
list.Add("foo");
list.Add("bar");
foreach (string word in list)
{
if (word == "world")
{
list.Remove("bar"); // Will not crash the foreach!
}
Console.WriteLine(word);
}
Output:
hello
world
foo
I have an enumerable that takes a long time to get the next value.
I'm trying to wrap that enumerable so that I get an enumerable that caches the results.
I'd also like it to do additional loading on another thread (reporting it reached the end of the collection). i.e. if it has 10 cached values, and i enumerate it reports 10, then starts a thread to get the next one so when enumerated again there's 11 cached values.
What i have so far is below (with code that tests it at the bottom):
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace ConsoleApplication1
{
public class CachedEnumerable<T> : IEnumerable<T>
{
public class CachedEnumerator : IEnumerator<T>
{
private IEnumerator<T> _UnderlyingEnumerator;
public event EventHandler Disposed;
public CachedEnumerator(IEnumerator<T> UnderlyingEnumerator)
{
_UnderlyingEnumerator = UnderlyingEnumerator;
}
public T Current
{
get { return _UnderlyingEnumerator.Current; }
}
public void Dispose()
{
_UnderlyingEnumerator.Dispose();
if (Disposed != null)
Disposed(this, new EventArgs());
}
object System.Collections.IEnumerator.Current
{
get { return _UnderlyingEnumerator.Current; }
}
public bool MoveNext()
{
return _UnderlyingEnumerator.MoveNext();
}
public void Reset()
{
_UnderlyingEnumerator.Reset();
}
}
// The slow enumerator.
private IEnumerator<T> _SourceEnumerator;
// Whether we're currently already getting the next item.
private bool _GettingNextItem = false;
// Whether we've got to the end of the source enumerator.
private bool _EndOfSourceEnumerator = false;
// The list of values we've got so far.
private List<T> _CachedValues = new List<T>();
// An object to lock against, to protect the cached value list.
private object _CachedValuesLock = new object();
// A reset event to indicate whether the cached list is safe, or whether we're currently enumerating over it.
private ManualResetEvent _CachedValuesSafe = new ManualResetEvent(true);
public CachedEnumerable(IEnumerable<T> Source)
{
_SourceEnumerator = Source.GetEnumerator();
}
private void Enum_Disposed(object sender, EventArgs e)
{
// The cached list is now safe (because we've finished enumerating).
_CachedValuesSafe.Set();
if (!_EndOfSourceEnumerator && !_GettingNextItem)
{
_GettingNextItem = true;
ThreadPool.QueueUserWorkItem((SourceEnumeratorArg) =>
{
var SourceEnumerator = SourceEnumeratorArg as IEnumerator<T>;
if (SourceEnumerator.MoveNext())
{
_CachedValuesSafe.WaitOne(); // Wait for any enumerator to finish
lock (_CachedValuesLock)
{
_CachedValues.Add(SourceEnumerator.Current);
}
}
else
{
_EndOfSourceEnumerator = true;
}
_GettingNextItem = false;
}, _SourceEnumerator);
}
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
public IEnumerator<T> GetEnumerator()
{
lock (_CachedValuesLock)
{
var Enum = new CachedEnumerator(_CachedValues.GetEnumerator());
Enum.Disposed += new EventHandler(Enum_Disposed);
_CachedValuesSafe.Reset();
return Enum;
}
}
}
class Program
{
public static IEnumerable<int> SlowNumbers()
{
int i = 0;
while (true)
{
Thread.Sleep(1000);
yield return i++;
}
}
static void Main(string[] args)
{
var SlowNumbersEnumerator = new CachedEnumerable<int>(SlowNumbers());
while (true)
{
foreach (var i in SlowNumbersEnumerator)
{
Console.WriteLine(i);
Thread.Sleep(100);
}
}
}
}
}
My problem is that i get the The collection has been modified error, because the worker thread is adding to the list while it's being enumerated. However, through my use of a ManualResetEvent i thought i was guarding against this.
What am i doing wrong?
(making my comment an answer, no way to format comments correctly)
Thread A is enumerating | Thread B is on _CachedValuesSafe.WaitOne();
Thread A finish enumerating (releasing the WaitHandle)
Thread A started a new enumeration getting the lock | Thread B want to aquire lock
Thread A initialize the enumeration| Thread B wait for the lock
Thread A return from the GetEnumerator() call and release the lock | Thread B acquire the lock
Thread A is enumerating the collection | Thread B execute Add, modifying the collection
Thread A enumerate the next value and throw an exception
I have requirement in which I have to back and fort with record. So I am using IEnumerator to that. But I can move forward by movenext but there no way to move back
Here's one way you could wrap an IEnumerator<T>, by capturing its contents in a List<T> as it moves along:
public interface ITwoWayEnumerator<T> : IEnumerator<T>
{
bool MovePrevious();
}
public class TwoWayEnumerator<T> : ITwoWayEnumerator<T>
{
private IEnumerator<T> _enumerator;
private List<T> _buffer;
private int _index;
public TwoWayEnumerator(IEnumerator<T> enumerator)
{
if (enumerator == null)
throw new ArgumentNullException("enumerator");
_enumerator = enumerator;
_buffer = new List<T>();
_index = -1;
}
public bool MovePrevious()
{
if (_index <= 0)
{
return false;
}
--_index;
return true;
}
public bool MoveNext()
{
if (_index < _buffer.Count - 1)
{
++_index;
return true;
}
if (_enumerator.MoveNext())
{
_buffer.Add(_enumerator.Current);
++_index;
return true;
}
return false;
}
public T Current
{
get
{
if (_index < 0 || _index >= _buffer.Count)
throw new InvalidOperationException();
return _buffer[_index];
}
}
public void Reset()
{
_enumerator.Reset();
_buffer.Clear();
_index = -1;
}
public void Dispose()
{
_enumerator.Dispose();
}
object System.Collections.IEnumerator.Current
{
get { return Current; }
}
}
Then I would expose this kind of enumerator using an extension method:
public static class TwoWayEnumeratorHelper
{
public static ITwoWayEnumerator<T> GetTwoWayEnumerator<T>(this IEnumerable<T> source)
{
if (source == null)
throw new ArgumentNullExceptions("source");
return new TwoWayEnumerator<T>(source.GetEnumerator());
}
}
Note that this is definitely overkill if the collection you're dealing with is already an indexed collection such as a T[] or a List<T>. It makes more sense for scenarios such as when you're enumerating over a sequence that isn't already in a conveniently indexed form and you want to be able to go backwards as well as forwards.
The IEnumerator (and IEnumerator<T>) interfaces only implement a forward only enumerator. You'll need to make your own class or interface if you want to allow bi-directional iteration through your collection.
You can't go backwards with IEnumerator. Either suck the entire set into a List or cache the current element on each pass through the loop, so it's available to the next pass.