I have multithreads application and i get this error
************** Exception Text **************
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Collections.Generic.List`1.Enumerator.MoveNext()
...
I probably have problem with my collection, because on one thread i read my collection and on another thread i modify collection.
public readonly ObservableCollectionThreadSafe<GMapMarker> Markers = new ObservableCollectionThreadSafe<GMapMarker>();
public void problem()
{
foreach (GMapMarker m in Markers)
{
...
}
}
I am trying to lock collection with this code, but doesn't work.
public void problem()
{
lock(Markers)
{
foreach (GMapMarker m in Markers)
{
...
}
}
}
Any ideas to fix that problem?
This is a pretty common mistake - modifying a collection whilst iterating it using foreach, keep in mind that foreach uses readonly IEnumerator instance.
Try to loop through the collection using for() with an extra index check so if the index is out of bounds you would be able to apply additional logic to handle it. You can also use LINQ's Count() as another loop exit condition by evaluating the Count value each time if the underlying enumeration does not implement ICollection:
If Markers implements IColletion - lock on SyncRoot:
lock (Markers.SyncRoot)
Use for():
for (int index = 0; index < Markers.Count(); index++)
{
if (Markers>= Markers.Count())
{
// TODO: handle this case to avoid run time exception
}
}
You might find this post useful: How do foreach loops work in C#?
You need to lock both on the reading and the writing side. Otherwise one of the threads will not know about the lock and will try to read/modify the collection, while the other is modifying/reading (respectively) with the lock held
Try to read a clone of your collection
foreach (GMapMarker m in Markers.Copy())
{
...
}
this will create a new copy of your collection that will not be affected by another thread but may cause a performance issue in case of huge collection.
So I think it will be better if you locked the collection while reading and writing processes.
This worked for me. Perform a ToList() operation on Markers:
foreach (GMapMarker m in Markers.ToList())
You can use a foreach but you have to cast the collection to a list and use the dot operator to access the behavior methods.
Example: Markers.Tolist().ForEach(i => i.DeleteObject())
Not totally sure what you're doing with your collection. My example is assuming you just wanted to delete all items from the collection, but it can be applied to any behavior you're trying to do with your collection.
Related
static List<int> numbers = new List<int>();
static void Main(string[] args)
{
foreach(var number in GetNumbers())
{
if (number == 1)
{
Thread t = new Thread(() =>
{
numbers.Add(234567);
});
t.Start();
}
Console.WriteLine(number);
}
}
public static IEnumerable<int> GetNumbers()
{
for(int i =0; i <=10;i++)
{
numbers.Add(i);
}
foreach (var number in numbers)
{
yield return number;
}
}
In the above example, I was iterating over collection using yield and added an item into the collection to get the updated number.
I understand modifying a collection which we are iterating throws a collection modified exception but with IEnumerable, I get deferred execution and I should be able to add to the main collection as yield would return data one by one.
I understand, removing an item could be problematic but adding an item to iterating collection should not be problem. However,if it is not allowed as what I have seen in above example(even it is deferred execution), what if I have situation like this:
"There is a large collection and so many consumers are iterating over it using IEnumerable and yield. They take each item, do some processing with this item.
and If there is any new item added into main list or collection, then client should get the latest item too.
Even though you are using yield, you are using it inside a foreach.
The docs state:
It is safe to perform multiple read operations on a List, but
issues can occur if the collection is modified while it’s being read.
This is the fundamental issue here - you are reading the collection (as evidenced by the foreach) while writing to it. That just isn't allowed.
This may be worth reading - https://social.msdn.microsoft.com/Forums/vstudio/en-US/a90c87be-9553-4d48-9892-d482ee325f02/why-cant-change-value-in-foreach?forum=csharpgeneral
You likely want to consider using ConcurrentBag, ConcurrentQueue or ConcurrentStack as alternatives.
I am using a thread party data model which uses it's custom data model. Hierarchy of the data model is as below:
Model
---Tables(type of Table)
-----Rows(type of Row)
-------Cells( type of Cell)
Table has property Rows as like DataTable and I have to access this property in more than tasks. Now I need a row from the table which has a column value to the specified value.
To do this, I have created a method which has lock statement to make it accessible from only one thread once.
public static Row GetRowWithColumnValue(Model model, string tableKey, string indexColumnKey, string indexColumnValue)
{
Row simObj = null;
lock (syncRoot)
{
SimWrapperFromValueFactory wrapperSimSystem = new SimWrapperFromValueFactory(model, tableKey, indexColumnKey);
simObj = wrapperSimSystem.GetWrapper(indexColumnValue);
}
return simObj;
}
To create the lookup for one of the column in Table, I have create a method which always try to create a copy of the rows to avoid collection modified exception:
Private Function GetTableRows(table As Table) As List(Of Row)
Dim rowsList As New List(Of Row)(table.Rows) 'Case 1
'rowsList.AddRange(table.Rows) 'Case 2
' Case 3
'For i As Integer = 0 To table.Rows.Count - 1
'rowsList.Add(table.Rows.ElementAt(i))
'Next
Return rowsList
End Function
but other threads can modify the table(e.g. add, remove rows or update column value in any rows). I am getting below "Collection modified exception":
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
I cannot modify this third party library to concurrent collections and this same Data Model shared between multiple project.
Question: I hunting for the solution that let me allow multiple readers on this collection either it modified in another threads.. Is it possible to Get a copy of the collection without getting exception??
Referenced below SO threads but did not find exact solution:
Lock vs. ToArray for thread safe foreach access of List collection
Can ToArray() throw an exception?
Is returning an IEnumerable<> thread-safe?
The simplest solution is to retry on exception, like this:
private List<Row> CopyVolatileList(IEnumerable<Row> original)
{
while (true)
{
try
{
List<Row> copy = new List<Row>();
foreach (Row row in original) {
copy.Add(row);
}
// Validate.
if (copy.Count != 0 && copy[copy.Count - 1] == null) // Assuming Row is a reference type.
{
// At least one element was removed from the list while were copying.
continue;
}
return copy;
}
catch (InvalidOperationException)
{
// Check ex.Message?
}
// Keep trying.
}
}
Eventually you'll get a run where the exception isn't thrown and the data integrity validation passes.
Alternatively, you can dive deep (and I mean very, very deep).
DISCLAIMER: Never ever use this in production. Unless you're desperate and really have no other option.
So we've established that you're working with a custom collection (TableRowCollection) which ultimately uses List<Row>.Enumerator to iterate through the rows. This strongly suggests that your collection is backed by a List<Row>.
First things first, you need to get a reference to that list. Your collection will not expose it publicly, so you'll need to fiddle a bit. You will need to use Reflection to find and get the value of the backing list. I recommend looking at your TableRowCollection in the debugger. It will show you non-public members and you will know what to reflect.
If you can't find your List<Row>, then take a closer look at TableRowCollection.GetEnumerator() - specifically GetEnumerator().GetType(). If that returns List<Row>.Enumerator, then bingo: we can get the backing list out of it, like so:
List<Row> list;
using (IEnumerator<Row> enumerator = table.GetEnumerator())
{
list = (List<Row>)typeof(List<Row>.Enumerator)
.GetField("list", BindingFlags.Instance | BindingFlags.NonPublic)
.GetValue(enumerator);
}
If the above methods of getting your List<Row> have failed, there is no need to read further. You might as well give up.
In case you've succeeded, now that you have the backing List<Row>, we'll have to look at Reference Source for List<T>.
What we see is 3 fields being used:
private T[] _items;
private int _size; // Accessible via "Count".
private int _version;
Our goal is to copy the items whose indexes are between zero and _size - 1 from the _items array into a new array, and to do so in between _version changes.
Observations re thread safety: List<T> does not use locks, none of the fields are marked as volatile and _version is incremented via ++, not Interlocked.Increment. Long story short this means that it is impossible to read all 3 field values and confidently say that we're looking at stable data. We'll have to read the field values repeatedly in order to be somewhat confident that we're looking at a reasonable snapshot (we will never be 100% confident, but you might choose to settle for "good enough").
using System;
using System.Collections.Generic;
using System.Linq.Expressions;
using System.Reflection;
using System.Threading;
private Row[] CopyVolatileList(List<Row> original)
{
while (true)
{
// Get _items and _size values which are safe to use in tandem.
int version = GetVersion(original); // _version.
Row[] items = GetItems(original); // _items.
int count = original.Count; // _size.
if (items.Length < count)
{
// Definitely a torn read. Copy will fail.
continue;
}
// Copy.
Row[] copy = new Row[count];
Array.Copy(items, 0, copy, 0, count);
// Stabilization window.
Thread.Sleep(1);
// Validate.
if (version == GetVersion(original)) {
return copy;
}
// Keep trying.
}
}
static Func<List<Row>, int> GetVersion = CompilePrivateFieldAccessor<List<Row>, int>("_version");
static Func<List<Row>, Row[]> GetItems = CompilePrivateFieldAccessor<List<Row>, Row[]>("_items");
static Func<TObject, TField> CompilePrivateFieldAccessor<TObject, TField>(string fieldName)
{
ParameterExpression param = Expression.Parameter(typeof(TObject), "o");
MemberExpression fieldAccess = Expression.PropertyOrField(param, fieldName);
return Expression
.Lambda<Func<TObject, TField>>(fieldAccess, param)
.Compile();
}
Note re stabilization window: the bigger it is, the more confidence you have that you're not dealing with a torn read (because the list is in process of modifying all 3 fields). I've settled on the smallest value I couldn't fail in my tests where I called CopyVolatileList in a tight loop on one thread, and used another thread to add items to the list, remove them or clear the list at random intervals between 0 and 20ms.
If you remove the stabilization window, you will occasionally get a copy with uninitialized elements at the end of the array because the other thread has removed a row while you were copying - that's why it's needed.
You should obviously validate the copy once it's built, to the best of your ability (at least check for uninitialized elements at the end of the array in case the stabilization window fails).
Good luck.
Recently I was running into the following exception when using a generic dictionary
An InvalidOperationException has occurred. A collection was modified
I realized that this error was primarily because of thread safety issues on the static dictionary I was using.
A little background: I currently have an application which has 3 different methods that are related to this issue.
Method A iterates through the dictionary using foreach and returns a value.
Method B adds data to the dictionary.
Method C changes the value of the key in the dictionary.
Sometimes while iterating through the dictionary, data is also being added, which is the cause of this issue. I keep getting this exception in the foreach part of my code where I iterate over the contents of the dictionary. In order to resolve this issue, I replaced the generic dictionary with the ConcurrentDictionary and here are the details of what I did.
Aim : My main objective is to completely remove the exception
For method B (which adds a new key to the dictionary) I replaced .Add with TryAdd
For method C (which updates the value of the dictionary) I did not make any changes. A rough sketch of the code is as follows :
static public int ChangeContent(int para)
{
foreach (KeyValuePair<string, CustObject> pair in static_container)
{
if (pair.Value.propA != para ) //Pending cancel
{
pair.Value.data_id = prim_id; //I am updating the content
return 0;
}
}
return -2;
}
For method A - I am simply iterating over the dictionary and this is where the running code stops (in debug mode) and Visual Studio informs me that this is where the error occured.The code I am using is similar to the following
static public CustObject RetrieveOrderDetails(int para)
{
foreach (KeyValuePair<string, CustObject> pair in static_container)
{
if (pair.Value.cust_id.Equals(symbol))
{
if (pair.Value.OrderStatus != para)
{
return pair.Value; //Found
}
}
}
return null; //Not found
}
Are these changes going to resolve the exception that I am getting.
Edit:
It states on this page that the method GetEnumerator allows you to traverse through the elements in parallel with writes (although it may be outdated). Isnt that the same as using foreach ?
For modification of elements, one option is to manually iterate the dictionary using a for loop, e.g.:
Dictionary<string, string> test = new Dictionary<string, string>();
int dictionaryLength = test.Count();
for (int i = 0; i < dictionaryLength; i++)
{
test[test.ElementAt(i).Key] = "Some new content";
}
Be weary though, that if you're also adding to the Dictionary, you must increment dictionaryLength (or decrement it if you move elements) appropriately.
Depending on what exactly you're doing, and if order matters, you may wish to use a SortedDictionary instead.
You could extend this by updating dictionaryLength explicitly by recalling test.Count() at each iteration, and also use an additional list containing a list of keys you've already modified and so on and so forth if there's a danger of missing any, it really depends what you're doing as much as anything and what your needs are.
You can further get a list of keys using test.Keys.ToList(), that option would work as follows:
Dictionary<string, string> test = new Dictionary<string, string>();
List<string> keys = test.Keys.ToList();
foreach (string key in keys)
{
test[key] = "Some new content";
}
IEnumerable<string> newKeys = test.Keys.ToList().Except(keys);
if(newKeys.Count() > 0)
// Do it again or whatever.
Note that I've also shown an example of how to find out whether any new keys were added between you getting the initial list of keys, and completing iteration such that you could then loop round and handle the new keys.
Hopefully one of these options will suit (or you may even want to mix and match- for loop on the keys for example updating that as you go instead of the length) - as I say, it's as much about what precisely you're trying to do as much as anything.
Before doing foreach() try out copying container to a new instance
var unboundContainer = static_container.ToList();
foreach (KeyValuePair<string, CustObject> pair in unboundContainer)
Also I think updating Value property is not right from thread safety perspectives, refactor your code to use TryUpdate() instead.
This seems like it should be answered but potential dupes I found were asking different things...
I noticed that this seems to work fine (sourceDirInclusion is a simple Dictionary<X,Y>)
foreach (string dir in sourceDirInclusion.Keys)
{
if (sourceDirInclusion[dir] == null)
sourceDirInclusion.Remove(dir);
}
Does that mean removing items from a collection in foreach is safe, or that I got lucky?
What about if I was adding more elements to the dictionary rather than removing?
The problem I'm trying to solve is that sourceDirInclusion is initially populated, but then each value can contribute new items to the dictionary in a second pass. e.g what I want to do is like:
foreach (string dir in sourceDirInclusion.Keys)
{
X x = sourceDirInclusion[dir];
sourceDirInclusion.Add(X.dir,X.val);
}
Short answer: This is not safe.
Long answer: From the IEnumerator<T> documentation:
An enumerator remains valid as long as the collection remains unchanged. If changes are made to the collection, such as adding, modifying, or deleting elements, the enumerator is irrecoverably invalidated and its behavior is undefined.
Note that the docs say the behavior is undefined, which means that it might work and it might not. One should never rely on undefined behavior.
In this case, it depends on the behavior of the Keys enumerable, regarding whether or not it creates a copy of the list of keys when you begin enumerating. In this specific case, we know from the docs that the return value from Dictionary<,>.Keys is a collection that refers back to the dictionary:
The returned Dictionary<TKey, TValue>.KeyCollection is not a static copy; instead, the Dictionary<TKey, TValue>.KeyCollection refers back to the keys in the original Dictionary<TKey, TValue>. Therefore, changes to the Dictionary<TKey, TValue> continue to be reflected in the Dictionary<TKey, TValue>.KeyCollection.
So it should be considered unsafe to modify the dictionary while enumerating the dictionary's keys.
You can correct this with one change. Alter this line:
foreach (string dir in sourceDirInclusion.Keys)
To this:
foreach (string dir in sourceDirInclusion.Keys.ToList())
The ToList() extension method will create an explicit copy of the list of keys, making it safe to modify the dictionary; the "underlying collection" will be the copy and not the original.
If will throw
InvalidOperationException: Message="Collection was modified; enumeration operation may not execute
To avoid that add candidates for removal to an external list. Then loop over it and remove from target container (dictionary).
List<string> list = new List<string>(sourceDirInclusion.Keys.Count);
foreach (string dir in sourceDirInclusion.Keys)
{
if (sourceDirInclusion[dir] == null)
list.Add(dir);
}
foreach (string dir in list)
{
sourceDirInclusion.Remove(dir);
}
check this out: What is the best way to modify a list in a 'foreach' loop?
In short:
The collection used in foreach is immutable. This is very much by design.
As it says on MSDN:
The foreach statement is used to iterate through the collection to get the information that you want, but can not be used to add or remove items from the source collection to avoid unpredictable side effects. If you need to add or remove items from the source collection, use a for loop.
UPDATE:
You can use a for loop instead:
for (int index = 0; index < dictionary.Count; index++) {
var item = dictionary.ElementAt(index);
var itemKey = item.Key;
var itemValue = item.Value;
}
This works because you are traversing sourceDirInclusion.Keys.
However, just to be sure with future versions of the FrameWork I recommend that you use sourceDirInclusion.Keys.ToArray() in the foreach statement this way you will create a copy of the keys that you loop through.
This will however not work:
foreach(KeyValuePair<string, object> item in sourceDirInclusion)
{
if (item.Value == null)
sourceDirInclusion.Remove(item.Key);
}
As a rule, you cannot modify a collection while it is traversed, but often you can make a new collection by using .ToArray() or .ToList() and traverse that while modifying the original collection.
Good luck with your quest.
I have System.Collections.Generic.SynchronizedCollection shared collection. Our code uses .Net 4.0 Task library to span threads and pass the synchronized collection to the thread. So far threads has not been adding or removing items into the collection. But the new requirement which requires one of the thread has to remove items from the collection while the other thread just read the collection. Do I need to add lock before removing the items from the Collection? If so, would reader thread be thread safe? Or Suggest best way to get the thread safety?
No it is not fully thread-safe. Try the following in a simple Console-Application and see how it crashes with an exception:
var collection = new SynchronizedCollection<int>();
var n = 0;
Task.Run(
() =>
{
while (true)
{
collection.Add(n++);
Thread.Sleep(5);
}
});
Task.Run(
() =>
{
while (true)
{
Console.WriteLine("Elements in collection: " + collection.Count);
var x = 0;
if (collection.Count % 100 == 0)
{
foreach (var i in collection)
{
Console.WriteLine("They are: " + i);
x++;
if (x == 100)
{
break;
}
}
}
}
});
Console.ReadKey();
Note, that if you replace the SynchronizedCollection with a ConcurrentBag, you will get thread-safety:
var collection = new ConcurrentBag<int>();
SynchronizedCollection is simply not thread-safe in this application. Use Concurrent Collections instead.
As Alexander already pointed out the SynchronizedCollection is not thread safe for this scenario.
The SynchronizedCollection actually wraps a normal generic list and just delegates every call to the underlying list with a lock surrounding the call. This is also done in GetEnumerator. So the getting of the enumerator is synchronized but NOT the actual enumeration.
var collection = new SynchronizedCollection<string>();
collection.Add("Test1");
collection.Add("Test2");
collection.Add("Test3");
collection.Add("Test4");
var enumerator = collection.GetEnumerator();
enumerator.MoveNext();
collection.Add("Test5");
//The next call will throw a InvalidOperationException ("Collection was modified")
enumerator.MoveNext();
When using a foreach an enumerator will be called in this way. So adding a ToArray() before enumerating through this array will not work either as this will first enumerate into this array.
This enumeration could be faster when what you are doing inside of your foreach so it could reduce the probability of getting a concurrency issue.
As Richard pointed out: for true thread safety go for the System.Collections.Concurrent classes.
Yes, SynchronizedCollection will do the locking for you.
If you have multiple readers and just one writer, you may want to look at using a ReaderWriterLock, instead of SynchronizedCollection.
Also, if you are .Net 4+ then take a look at System.Collections.Concurrent. These classes have much better performance than SynchronizedCollection.