Hi guys I have found an issue that I'm unable to explain logically. In the following snippet flpRecordIndexes is a FlowLayoutPabel that contains lots of RecordIndexControl (a user control that I created). I want to delete everything except the first control. The same idea with flpRecordContainer.
If I execute this (without the ToList call), it only removes half of the controls, if it was a sequence for example it will remove (2,4,6,8) etc.
foreach (var recordIndexControl in flpRecordIndexes.Controls.Cast<RecordIndexControl>().Skip(1))
{
flpRecordIndexes.Controls.Remove(recordIndexControl);
}
foreach (var recordControl in flpRecordContainer.Controls.Cast<RecordControl>().Skip(1))
{
flpRecordContainer.Controls.Remove(recordControl);
}
If I execute this (with the ToList), it removes everything except the first control, what I wanted.
foreach (var recordIndexControl in flpRecordIndexes.Controls.Cast<RecordIndexControl>().ToList().Skip(1))
{
flpRecordIndexes.Controls.Remove(recordIndexControl);
}
foreach (var recordControl in flpRecordContainer.Controls.Cast<RecordControl>().ToList().Skip(1))
{
flpRecordContainer.Controls.Remove(recordControl);
}
Why calling Cast without ToList produce this behavior?
This is entirely normal, you are modifying the collection you are iterating with the Controls.Remove() call. The Controls collection behaves different from other framework collections, it doesn't throw an exception when you do this. So in effect you remove every other control, depending on the mix.
The ToList() call creates a copy of the Controls collection, it is no longer affected by the Remove() calls. It is the correct workaround.
Do keep in mind that you most likely have a nasty leak. The controls you remove must be disposed. You can no longer rely on Winforms doing this for you, it can't since they are no longer in the Controls collection. Failure to dispose them is a permanent leak, the garbage collector cannot help.
Why calling Cast without ToList produce this behavior?
Invoking ToList() materializes the collection, whereas Cast<T> does not. Once ToList() is invoked the list is solidified so to speak and you have a finite number in the list.
I would suggest iterating the Control.Controls via a for loop instead of a foreach. This will avoid the issue you're seeing entirely and is actually more performant. The ControlCollection class inherits IList so you should be good with that.
for (var index = Controls.Count - 1; index >= 1; -- index)
{
flpRecordContainer.Controls.RemoveAt(index);
}
Note the index >= 1 to ensure we leave the first control in the list.
Related
I have a ConcurrentBag of objects, and I want to do following over it:
enumerate all items with a where filtering.
for each item, check some properties, and based on the values, make some method call. After the method call, it's better to remove the item form the bag.
modify some properties' value and save it to the bag.
So basically I need something like following:
foreach (var item in myBag.Where(it => it.Property1 = true))
{
if (item.Property2 = true)
{
SomeMethodToReadTheItem(item);
//it's better to remove this item from the bag here, but
//there is a permeance hit, then just leave it.
}
else
{
item.Property3= "new value";
//now how do I save the item back to the bag?
}
}
Of cause it should be done in a thread-safe way. I know that the enumeration over a ConcurrentBag is actually over a "snapshot" of the real bag, but how about with a where clause filter? Should I do a ToList to prevent it form making a new "snapshot"?
Also if you want to modify one specific item, you just bag.TryTake(out item). But since I've already get the item in the enumeration, should I "take" it again?
Any explanation/comment/sample would be very much apricated.
Thank you.
I'll try to answer specific parts of your question without addressing the performance.
First off, the Where method takes an IEnumerable<T> as its first parameter and will itself iterate over the enumerable which will call GetEnumerator() once so you will only take one snapshot of the underlying ConcurrentBag.
Secondly the thread-safety of your code is not very clear, there may be some implicit guarantees in the rest of your code which are not specified. For example you have a ConcurrentBag so your collection is thread-safe however you modify the items contained within that collection without any thread synchronisation. If there is other code that runs the same method or in another method that reads/modifies the items in the ConcurrentBag concurrently then you may see data races.
Note that it is not necessary to call TryTake if you already have a reference to the item as it will only return the same reference.
I recommend you just create a new list and, if the WHERE filter, add it to this new list.
It would look something like this:
List<T> myNewList = new List<T>();
foreach (var item in myBag.Where(it => it.Property1))
{
if (!item.Property2)
{
myNewList.Add(item);
}
}
attention to " ! "
In C#, the IEnumerator interface defines a way to traverse a collection and look at the elements. I think this is tremendously useful because if you pass IEnumerable<T> to a method, it's not going to modify the original source.
However, in Java, Iterator defines the remove operation to (optionally!) allow deleting elements. There's no advantage in passing Iterable<T> to a method because that method can still modify the original collection.
remove's optionalness is an example of the refused bequest smell, but ignoring that (already discussed here) I'd be interested in the design decisions that prompted a remove event to be implemented on the interface.
What are the design decisions that led to remove being added to Iterator?
To put another way, what is the C# design decision that explicitly doesn't have remove defined on IEnumerator?
Iterator is able to remove elements during iteration. You cannot iterate collection using iterator and remove elements from target collection using remove() method of that collection. You will get ConcurrentModificationException on next call of Iterator.next() because iterator cannot know how exactly the collection was changed and cannot know how to continue to iterate.
When you are using remove() of iterator it knows how the collection was changed. Moreover actually you cannot remove any element of collection but only the current one. This simplifies continuation of iterating.
Concerning to advantages of passing iterator or Iterable: you can always use Collection.unmodifireableSet() or Collection.unmodifireableList() to prevent modification of your collection.
It is probably due to the fact that removing items from a collection while iterating over it has always been a cause for bugs and strange behaviour. From reading the documentation it would suggest that Java enforces at runtime remove() is only called once per call to next() which makes me think it has just been added to prevent people messing up removing data from a list when iterating over it.
There are situations where you want to be able to remove elements using the iterator because it is the most efficient way to do it. For example, when traversing a linked data structure (e.g. a linked list), removing using the iterator is an O(1) operation ... compared to O(N) via the List.remove() operations.
And of course, many collections are designed so that modifying the collection during a collection by any other means than Iterator.remove() will result in a ConcurrentModificationException.
If you have a situation where you don't want to allow modification via a collection iterator, wrapping it using Collection.unmodifiableXxxx and using it's iterator will have the desired effect. Alternatively, I think that Apache Commons provides a simple unmodifiable iterator wrapper.
By the way IEnumerable suffers from the same "smell" as Iterator. Take a look at the reset() method. I was also curious as to how the C# LinkedList class deals with the O(N) remove problem. It appears that it does this by exposing the internals of the list ... in the form of the First and Last properties whose values are LinkedListNode references. That violates another design principle ... and is (IMO) far more dangerous than Iterator.remove().
This is actually an awesome feature of Java. As you may well know, when iterating through a list in .NET to remove elements (of which there are a number of use cases for) you only have two options.
var listToRemove = new List<T>(originalList);
foreach (var item in originalList)
{
...
if (...)
{
listToRemove.Add(item)
}
...
}
foreach (var item in listToRemove)
{
originalList.Remove(item);
}
or
var iterationList = new List<T>(originalList);
for (int i = 0; i < iterationList.Count; i++)
{
...
if (...)
{
originalList.RemoveAt(i);
}
...
}
Now, I prefer the second, but with Java I don't need all of that because while I'm on an item I can remove it and yet the iteration will continue! Honestly, though it may seem out of place, it's really an optimization in a lot of ways.
Sometimes it is useful to enumerate a list while it is changing.
e.g.
foreach (var item in listOfEntities)
item.Update();
// somewhere else (with someEntity contained in listOfEntities)
// an add or remove is made:
someEntity.OnUpdate += (s,e) => listOfEntities.Remove(someEntity);
This will fail if listOfEntities is a List<T>.
There are workarounds like making a copy or a simple for-loop, each with different drawbacks, but I would like to know if there is a list type in the framework (or open source) that supports this.
Look at the collections in System.Collections.Concurrent. There's no list there, but the collections' enumerators do "represents a moment-in-time snapshot of the contents of the [collection]".
These collections are designed for access from multiple threads, so they will be better suited to applications like the code sample you posted.
This has nothing to do with List<T>; it is a limitation of the enumerator. If you change the state of the collection underneath the enumerator it will throw, period.
You could use a for loop, but you will then run into logical errors as you index into a collection after the number of items have changed.
It's probably a bad idea to swap items in and out of a collection while you are enumerating it in another thread. I would stick with the tried and true method of recording the items to be removed in another collection or locking the collection while it is being enumerated.
I'm not claiming this is an impossible problem to solve, I just don't know of an easy way to do it.
I came across a method to change a list in a foreach loop by converting to a list in itself like this:
foreach (var item in myList.ToList())
{
//add or remove items from myList
}
(If you attempt to modify myList directly an error is thrown since the enumerator basically locks it)
This works because it's not the original myList that's being modified. My question is, does this method create garbage when the loop is over (namely from the List that's returned from the ToList method? For small loops, would it be preferable to using a for loop to avoid the creation of garbage?
The second list is going to be garbage, there will be garbage for an enumerator that is used in building the second list, and add in the enumerator that the foreach would spawn, which you would have had with or without the second list.
Should you switch to a for? Maybe, if you can point to this region of code being a true performance bottleneck. Otherwise, code for simplicity and maintainability.
Yes. ToList() would create another list that would need to be garbage collected.
That's an interesting technique which I will keep in mind for the future! (I can't believe I've never thought of that!)
Anyway, yes, the list that you are building doesn't magically unallocate itself. The possible performance problems with this technique are:
Increased memory usage (building a List, separate from the IEnumerable). Probably not that big of a deal, unless you do this very frequently, or the IEnumerable is very large.
Decreased speed, since it has to go through the IEnumerable at once to build the List.
Also, if enumerating the IEnumerable has side effects, they will all be triggered by this process.
Unless this is actually inside an inner loop, or you're working with very large data sets, you can probably do this without any problems.
Yes, the ToList() method creates "garbage". I would just indexing.
for (int i = MyList.Count - 1; 0 <= i; --i)
{
var item = MyList[i];
//add or remove items from myList
}
It's non-deterministic. But the reference created from the call ToList() will be GCd eventually.
I wouldn't worry about it too much, since all it would be holding at most would be references or small value types.
Does
foreach(T value in new List<T>(oldList) )
is dangerous (costly) when oldList contains 1 millions of object T ?
More generaly what is the best way to enumerate over oldList given that elements can be added/removed during the enumeration...
The general rule is, you should not modify the same collection in which you are enumerating. If you want to do something like that, keep another collection which will keep track of which elements to add/remove from the original collection and then after exiting from the loop, perform the add/remove operation on the original collection.
I usually just create a list for all the objects to be removed or added.
Within the foreach I just add the items to the appropriate collections and modify the original collection after the foreach have completed (loop through the removeItems and addItems collection)
just like this
var itemsToBeRemoved = new List<T>();
foreach (T item in myHugeList)
{
if (/*<condition>*/)
itemsToBeRemoved.Add(item);
}
myHugeList.RemoveRange(itemsToBeRemoved);
You could iterate through the list without using an enumerator, so do something like...
for(int i = 0;i<oldList.Count;i++) {
var value = oldList[i];
...
if(itemRemoveCondition) {
oldList.RemoveAt(i--);
}
}
If you mean you can add/remove objects from another thread, I would:
1-synchronize the threads
2- in the add/remove threads, create a list of items to be added or deleted
3- and then delete these items in a critical section (so it is small - you don't have to synch while adding the items to the delete list)
If you dont want to do that, you can use for instead of foreach, that would avoid the exception, but you would have to take extra care so you do not get other kinds of exceptions
foreach(T value in new List(oldList).ToList() ) - give a try
For me, first thing is you should consider using some kind of data paging, because having such 1-milion-items-large list could be dangerous itself.
Have you heard about Unit of Work pattern?
You can implement it so you mark objects for create, update or delete, and later, you call "SaveChanges", "Commit" or any other doing the job of "apply changes", and you'll get done.
For example, you iterate over the enumerable (oldList) and you mark them as "delete". Later, you call "SaveChanges" and the more abstract, generic unit of work will iterate over the small, filtered list of objects to work with.
http://martinfowler.com/eaaCatalog/unitOfWork.html
Anyway, avoid lists of a milion items. You should work with paged lists of objects.
It will be 'slow' but there is not much more you can do about it, except running it on a background thread. E.g. using a BackgroundWorker.
If your operations on the list only occur on one thread, the correct approach is to add the items to add/remove to seperate lists, and perform those operations after your iterations has finished.
If you use multiple threads you will have to look into multithreaded programming, and e.g. use locks or probably better a ReaderWriterLock.
UPDATE:
As mentioned in another Stack Overflow question, this is now possible without any effort in .NET 4.0 when using concurrent collections.
If you are using Foreach loop for modifying collection then you will get this error as below.
List<string> li = new List<string>();
li.Add("bhanu");
li.Add("test");
foreach (string s in li)
{
li.Remove(s);
}
Solution - use For Loop as below.
for (int i = 0; i < li.Count; i++)
{
li.RemoveAt(i);
i--;
}
you can use a flag to switch the modification to a temporary list while the original is being enumerated.
/// where you are enumerating
isBeingEnumerated = true
foreach(T value in new List<T>(oldList) )
isBeingEnumerated = false
SyncList(oldList with temporaryList)
/// where you are modifying while enumerating
if isBeingEnumerated then
use a temporaryList to make the changes.