I using a ConcurrentBag as Collection that is thread safe, to avoid conflicts when i am updating my collection in differents threads.
But i notice that sometimes the itens get inverse, my firts item goes to the last position of my collection.
I just wanna know if this may be happening due to change the collection in concurrency. If it's not possible what could may be messing up my collection?
Edit: I'm adding some sample code.
When i need to add a item i make this:
var positionToInsert = (int)incremental.MDEntryPositionNo - 1;
concurrentList.ToList().Insert(positionToInsert, myListInfoToInsert);
In some cases i need to update a position so i do like this
var foundPosition = concurrentList.ToList()
.FirstOrDefault(orderBook => orderBook.BookPosition == incremental.MDEntryPositionNo);
var index = concurrentList.ToList().IndexOf(foundPosition);
if (index != -1)
{
concurrentList.ToList()[index] = infoToUpdate;
}
Thaks!
Edited: Just use sorting, don't use insertion it's a slow operation.
var orderedBooks = concurrentList.OrderBy(x=>x.BookPosition).ToList();
ConcurrentBag is implemented as a linked list, and the ToList code is shown below.
For each input thread, created own ThreadLocalList or reused free.
The head of a linked list is always the same, and I don't understand how the order can change in this situation. However you can't guarantee that the last item added won't be in the first bucket.
Please add your sample code.
private List<T> ToList()
{
List<T> objList = new List<T>();
for (ConcurrentBag<T>.ThreadLocalList threadLocalList = this.m_headList; threadLocalList != null; threadLocalList = threadLocalList.m_nextList)
{
for (ConcurrentBag<T>.Node node = threadLocalList.m_head; node != null; node = node.m_next)
objList.Add(node.m_value);
}
return objList;
}
Related
I would like to learn if it is possible to modify the list being iterated over using forEach so that there is no need to maintain an index.
var scanResults = await someFunction();
for (int i = 0; i < scanResults.Data.Count(); i++)
{
if ((scanResults.Data.ToList()[i].Filters.Count() == 0) != (scanResults.Data.ToList()[i].SubscribedFilters.Count() == 0))
{
scanResults.Data.ToList()[i] = await AddFilters(scanResults.Data.ToList()[i]);
}
}
return scanResults;
Note as mentioned by John and myself in the comment that in your code you are using ToList() (presumably System.Linq) in the nested check and statements (which is likely a logical mistake); which means you are creating a new list each time. Assuming that you reference one list throughout your nested statement, you will run into InvalidOperationException with a message of Collection Was Modified.
Scenario 1: Stick with for(;;) loop
The benefit of this is you don't need to use additional memory to create a temporary list or an alternative list.
Scenario 2: foreach with a temporary duplicate list to modify
If you really insist on using foreach loop then one simple option is to create an identical list with the same data and iterate through that. Depending on what you are doing within the list this might not work. The downside with this approach is you are using additional memory to store the duplicate list.
Lots of the code is not given in the problem so we can't guarantee which would make more sense in your situation. However in most cases you would try to stick with the for(;;) loop.
I can't see the rest of your code so I can only guess at the data types you are using but it'd be something like this.
foreach (var scanResult in ScanResults)
{
if ((scanResult.Data.ToList().Filters.Count() == 0) != (scanResult.Data.ToList().SubscribedFilters.Count() == 0))
{
scanResult.Data.ToList() = await AddFilters(scanResult.Data.ToList());
}
}
I have a list and want to iterate smoothly through it while removing one element after another. I thought I could do it like this:
List<Point> open = new List<Point>();
...
while (!(open == null))
{
Point p = open.RemoveAt(0);
...
However, it is not quite working how I would like it to, starting with "Cannot implicitly convert type 'void' to 'Point'". But shouldn't the call of RemoveAt give the point to P before removing it/making it void?
List.RemoveAt does not return item which you are removing. Also list will not become null when you'll remove all items. It will become empty, i.e. with Count equal to 0. I would suggest you to use Queue<T> instead of List<T>. Thus you will be able to remove fist added item and get it at same time:
Queue<Point> open = new Queue<Point>();
while(open.Count > 0)
{
var point = open.Dequeue();
// ...
}
If you want to use list, and remove first items, then you should retrieve item by index, and only then remove it from list:
List<Point> open = new List<Point>();
while (open.Count > 0) // or open.Any()
{
Point p = open[0];
open.RemoveAt(0);
// ...
}
No, it does not. It does not return anything, as per the specification. Try using a Queue<Point> instead. Also, removing the first item in a List<T> does force a copy of the array-contents as far as I know (If somebody knows, please add relevant reference), so always avoid removing the first element in list and try to always find the best data structure to solve your particular issue!
Example:
var open = new Queue<Point>();
// ... Fill it
// Any() is in general faster than Count() for checking that collection has data
// It is a good practice to use it in general, although Count (the property) is as fast
// but not all enumerables has that one
while (open.Any()) {
Point p = open.Dequeue();
// ... Do stuff
}
For now, the best I could think of is:
bool oneMoreTime = true;
while (oneMoreTime)
{
ItemType toDelete=null;
oneMoreTime=false;
foreach (ItemType item in collection)
{
if (ShouldBeDeleted(item))
{
toDelete=item;
break;
}
}
if (toDelete!=null)
{
collection.Remove(toDelete);
oneMoreTime=true;
}
}
I know that I have at least one extra variable here, but I included it to improve the readability of the algorithm.
The "RemoveAll" method is best.
Another common technique is:
var itemsToBeDeleted = collection.Where(i=>ShouldBeDeleted(i)).ToList();
foreach(var itemToBeDeleted in itemsToBeDeleted)
collection.Remove(itemToBeDeleted);
Another common technique is to use a "for" loop, but make sure you go backwards:
for (int i = collection.Count - 1; i >= 0; --i)
if (ShouldBeDeleted(collection[i]))
collection.RemoveAt(i);
Another common technique is to add the items that are not being removed to a new collection:
var newCollection = new List<whatever>();
foreach(var item in collection.Where(i=>!ShouldBeDeleted(i))
newCollection.Add(item);
And now you have two collections. A technique I particularly like if you want to end up with two collections is to use immutable data structures. With an immutable data structure, "removing" an item does not change the data structure; it gives you back a new data structure (that re-uses bits from the old one, if possible) that does not have the item you removed. With immutable data structures you are not modifying the thing you're iterating over, so there's no problem:
var newCollection = oldCollection;
foreach(var item in oldCollection.Where(i=>ShouldBeDeleted(i))
newCollection = newCollection.Remove(item);
or
var newCollection = ImmutableCollection<whatever>.Empty;
foreach(var item in oldCollection.Where(i=>!ShouldBeDeleted(i))
newCollection = newCollection.Add(item);
And when you're done, you have two collections. The new one has the items removed, the old one is the same as it ever was.
Just as I finished typing I remembered that there is lambda-way to do it.
collection.RemoveAll(i=>ShouldBeDeleted(i));
Better way?
A forward variation on the backward for loop:
for (int i = 0; i < collection.Count; )
if (ShouldBeDeleted(collection[i]))
collection.RemoveAt(i)
else
i++;
You cannot delete from a collection inside a foreach loop (unless it is a very special collection having a special enumerator). The BCL collections will throw exceptions if the collection is modified while it is being enumerated.
You could use a for loop to delete individual elements and adjust the index accordingly. However, doing that can be error prone. Depending on the implementation of the underlying collection it may also be expensive to delete individual elements. For instance deleting the first element of a List<T> will copy all the remaning elements in the list.
The best solution is often to create a new collection based on the old:
var newCollection = collection.Where(item => !ShouldBeDeleted(item)).ToList();
Use ToList() or ToArray() to create the new collection or initialize your specific collection type from the IEnumerable returned by the Where() clause.
The lambda way is good. You could also use a regular for loop, you can iterate lists that a for loop uses within the loop itself, unlike a foreach loop.
for (int i = collection.Count-1; i >= 0; i--)
{
if(ShouldBeDeleted(collection[i])
collection.RemoveAt(i);
}
I am assuming that collection is an arraylist here, the code might be a bit different if you are using a different data structure.
I have the classic case of trying to remove an item from a collection while enumerating it in a loop:
List<int> myIntCollection = new List<int>();
myIntCollection.Add(42);
myIntCollection.Add(12);
myIntCollection.Add(96);
myIntCollection.Add(25);
foreach (int i in myIntCollection)
{
if (i == 42)
myIntCollection.Remove(96); // The error is here.
if (i == 25)
myIntCollection.Remove(42); // The error is here.
}
At the beginning of the iteration after a change takes place, an InvalidOperationException is thrown, because enumerators don’t like when the underlying collection changes.
I need to make changes to the collection while iterating. There are many patterns that can be used to avoid this, but none of them seems to have a good solution:
Do not delete inside this loop, instead keep a separate “Delete List”, that you process after the main loop.
This is normally a good solution, but in my case, I need the item to be gone instantly as “waiting” till after
the main loop to really delete the item changes the logic flow of my code.
Instead of deleting the item, simply set a flag on the item and mark it as inactive. Then add the functionality of pattern 1 to clean up the list.
This would work for all of my needs, but it means that a lot of code will have to change in order to check the inactive flag every time an item is accessed. This is far too much administration for my liking.
Somehow incorporate the ideas of pattern 2 in a class that derives from List<T>. This Superlist will handle the inactive flag, the deletion of objects after the fact and also will not expose items marked as inactive to enumeration consumers. Basically, it just encapsulates all the ideas of pattern 2 (and subsequently pattern 1).
Does a class like this exist? Does anyone have code for this? Or is there a better way?
I’ve been told that accessing myIntCollection.ToArray() instead of myIntCollection will solve the problem and allow me to delete inside the loop.
This seems like a bad design pattern to me, or maybe it’s fine?
Details:
The list will contain many items and I will be removing only some of them.
Inside the loop, I will be doing all sorts of processes, adding, removing etc., so the solution needs to be fairly generic.
The item that I need to delete may not be the current item in the loop. For example, I may be on item 10 of a 30 item loop and need to remove item 6 or item 26. Walking backwards through the array will no longer work because of this. ;o(
The best solution is usually to use the RemoveAll() method:
myList.RemoveAll(x => x.SomeProp == "SomeValue");
Or, if you need certain elements removed:
MyListType[] elems = new[] { elem1, elem2 };
myList.RemoveAll(x => elems.Contains(x));
This assume that your loop is solely intended for removal purposes, of course. If you do need to additional processing, then the best method is usually to use a for or while loop, since then you're not using an enumerator:
for (int i = myList.Count - 1; i >= 0; i--)
{
// Do processing here, then...
if (shouldRemoveCondition)
{
myList.RemoveAt(i);
}
}
Going backwards ensures that you don't skip any elements.
Response to Edit:
If you're going to have seemingly arbitrary elements removed, the easiest method might be to just keep track of the elements you want to remove, and then remove them all at once after. Something like this:
List<int> toRemove = new List<int>();
foreach (var elem in myList)
{
// Do some stuff
// Check for removal
if (needToRemoveAnElement)
{
toRemove.Add(elem);
}
}
// Remove everything here
myList.RemoveAll(x => toRemove.Contains(x));
If you must both enumerate a List<T> and remove from it then I suggest simply using a while loop instead of a foreach
var index = 0;
while (index < myList.Count) {
if (someCondition(myList[index])) {
myList.RemoveAt(index);
} else {
index++;
}
}
I know this post is old, but I thought I'd share what worked for me.
Create a copy of the list for enumerating, and then in the for each loop, you can process on the copied values, and remove/add/whatever with the source list.
private void ProcessAndRemove(IList<Item> list)
{
foreach (var item in list.ToList())
{
if (item.DeterminingFactor > 10)
{
list.Remove(item);
}
}
}
When you need to iterate through a list and might modify it during the loop then you are better off using a for loop:
for (int i = 0; i < myIntCollection.Count; i++)
{
if (myIntCollection[i] == 42)
{
myIntCollection.Remove(i);
i--;
}
}
Of course you must be careful, for example I decrement i whenever an item is removed as otherwise we will skip entries (an alternative is to go backwards though the list).
If you have Linq then you should just use RemoveAll as dlev has suggested.
As you enumerate the list, add the one you want to KEEP to a new list. Afterward, assign the new list to the myIntCollection
List<int> myIntCollection=new List<int>();
myIntCollection.Add(42);
List<int> newCollection=new List<int>(myIntCollection.Count);
foreach(int i in myIntCollection)
{
if (i want to delete this)
///
else
newCollection.Add(i);
}
myIntCollection = newCollection;
Let's add you code:
List<int> myIntCollection=new List<int>();
myIntCollection.Add(42);
myIntCollection.Add(12);
myIntCollection.Add(96);
myIntCollection.Add(25);
If you want to change the list while you're in a foreach, you must type .ToList()
foreach(int i in myIntCollection.ToList())
{
if (i == 42)
myIntCollection.Remove(96);
if (i == 25)
myIntCollection.Remove(42);
}
For those it may help, I wrote this Extension method to remove items matching the predicate and return the list of removed items.
public static IList<T> RemoveAllKeepRemoved<T>(this IList<T> source, Predicate<T> predicate)
{
IList<T> removed = new List<T>();
for (int i = source.Count - 1; i >= 0; i--)
{
T item = source[i];
if (predicate(item))
{
removed.Add(item);
source.RemoveAt(i);
}
}
return removed;
}
How about
int[] tmp = new int[myIntCollection.Count ()];
myIntCollection.CopyTo(tmp);
foreach(int i in tmp)
{
myIntCollection.Remove(42); //The error is no longer here.
}
If you're interested in high performance, you can use two lists. The following minimises garbage collection, maximises memory locality and never actually removes an item from a list, which is very inefficient if it's not the last item.
private void RemoveItems()
{
_newList.Clear();
foreach (var item in _list)
{
item.Process();
if (!item.NeedsRemoving())
_newList.Add(item);
}
var swap = _list;
_list = _newList;
_newList = swap;
}
Just figured I'll share my solution to a similar problem where i needed to remove items from a list while processing them.
So basically "foreach" that will remove the item from the list after it has been iterated.
My test:
var list = new List<TempLoopDto>();
list.Add(new TempLoopDto("Test1"));
list.Add(new TempLoopDto("Test2"));
list.Add(new TempLoopDto("Test3"));
list.Add(new TempLoopDto("Test4"));
list.PopForEach((item) =>
{
Console.WriteLine($"Process {item.Name}");
});
Assert.That(list.Count, Is.EqualTo(0));
I solved this with a extension method "PopForEach" that will perform a action and then remove the item from the list.
public static class ListExtensions
{
public static void PopForEach<T>(this List<T> list, Action<T> action)
{
var index = 0;
while (index < list.Count) {
action(list[index]);
list.RemoveAt(index);
}
}
}
Hope this can be helpful to any one.
Currently you are using a list. If you could use a dictionary instead, it would be much easier. I'm making some assumptions that you are really using a class instead of just a list of ints. This would work if you had some form of unique key. In the dictionary, object can be any class you have and int would be any unique key.
Dictionary<int, object> myIntCollection = new Dictionary<int, object>();
myIntCollection.Add(42, "");
myIntCollection.Add(12, "");
myIntCollection.Add(96, "");
myIntCollection.Add(25, "");
foreach (int i in myIntCollection.Keys)
{
//Check to make sure the key wasn't already removed
if (myIntCollection.ContainsKey(i))
{
if (i == 42) //You can test against the key
myIntCollection.Remove(96);
if (myIntCollection[i] == 25) //or you can test against the value
myIntCollection.Remove(42);
}
}
Or you could use
Dictionary<myUniqueClass, bool> myCollection; //Bool is just an empty place holder
The nice thing is you can do anything you want to the underlying dictionary and the key enumerator doesn't care, but it also doesn't update with added or removed entries.
I have a String Collection that is populated with ID's like so -->
12345
23456
34567
and so on. What I need to do is at the user's request, and when certain parameters are met, go through that list, starting at the top, and perform a method() using that ID. If successful I would remove it from the list and move on.
I, embarrassingly, have never worked with a collection before in this manner. Can someone point me in the right direction. Examples all seem to be of the Console.Writeline(""); variety.
My base, ignorant, attempt looks like this -->
var driUps = Settings.Default.DRIUpdates.GetEnumerator();
while (driUps.MoveNext())
{
var wasSuccessfull = PerformDRIUpdate(driUps.Current);
if (wasSuccessfull)
{
driUps.Current.Remove(driUps.Current.IndexOf(driUps.Current));
}
}
The part I am most concerned with is the Remove(); Isn't there a better way to get the Current Index? Any and all Tips, Hints, Criticism, Pointers, etc....welcome. Thanks!
You are quite right to be concerned about the 'remove' during enumeration. How about somethign like this:
int idx = 0;
while (idx < strCol.Count)
{
var wasSuccessful = PerformDRIUpdate(strCol[idx]);
if (wasSuccessful)
strCol.RemoveAt(idx);
else
++idx;
}
As suggested by n8wrl, using RemoveAt solves the issue of trying to remove an item whilst enumerating the collection, but for large collections removing items from the front can cause performance issues as the underlying collection is re-built. Work your way from the end of the collection and remove items from that end:
//Loop backwards, as removing from the beginning
//causes underlying collection to be re-built
int index = (strCol.Count - 1);
while (index >= 0)
{
if (PerformDRIUpdate(strCol[index]))
{
strCol.RemoveAt(index);
}
--index;
}
Iterating an enumerator is best done with the foreach(), it does a GetEnumerator() and creates a similar block under the covers to what you're getting at, the syntax is:
foreach(ObjectType objectInstance in objectInstanceCollection)
{
do something to object instance;
}
for you,
List<DRIUpdate> updatesToRemove = new List<DRIUpdate>();
foreach(DRIUpdate driUpdate in Settings.Default.DRIUpdates)
{
if (PerformDRIUpdate(driUpdate))
{
updatesToRemove.Add(driUpdate);
}
}
foreach(DRIUpdate driUpdate in updatesToRemove)
{
Settings.Default.DRIUpdates.Remove(driUpdate);
}
If driUps is an IEnumerable<T>, try this:
driUps = driUps.Where(elem => !PerformDRIUpdate(elem));
Update:
From the example, it seems this is more appropriate:
Settings.Default.DRIUpdates =
Settings.Default.DRIUpdates.Where(elem => !PerformDRIUpdate(elem));
For a List<T>, it's simpler:
list.RemoveAll(PerformDRIUpdate);