I have a list of Item entities that are being processed in a batch, something like this:
foreach (var itemId in ItemIdList)
{
var item = getById(itemId); //load line
if(item != null)
{
//...do some processing
delete(item)
}
}
The problem is, the same itemId could be listed in ItemIdList multiple times, so after it is deleted, and I try to load the item a second time, the load line fails with the error
Unexpected row count: 0; expected: 1 (stale state exception)
I understand that the entity is not there any more, but I would have expected my get function to just return null. Here is my getById function:
var item = (from i in UnitOfWork.CurrentSession.QueryOver<T>()
where i.Id == id
select i
).SingleOrDefault();
Why isn't SingleOrDefault just returning null?
My Item entity only has one autogenerated key and the hash function looks like this:
public override int GetHashCode()
{
int hashCode = 0;
hashCode = hashCode ^ Id.GetHashCode();
return hashCode;
}
Edit:
Here is my delete method
public void Delete(T t) //T would be Item in this case
{
UnitOfWork.CurrentSession.Delete(t);
}
I haven't seen your delete method but since you state these items are being done in a 'batch' I'll assume you wait to flush until everything has been deleted.
Since the flush doesn't occur until you've attempted to delete all the items, it still exists in the database. Therefore, when you retrieve the item a second time it thinks that it should be 'deleted' but, behold, it still exists. This is why NHibernate thinks that the state is 'stale.'
One easy way to fix this would be to have the ItemIdList be a set (like a HashSet). This will prevent duplicate IDs from being present and should fix the problem.
On another note, if you are attempting to delete all entities within a list of ids, there are a lot more efficient ways than reading each one, one at a time, from the database and then deleting them.
Related
I have a method that takes an IEnumerable, filters it further and loops through the filtered collection
to modify one property.
I am observing a very weird behaviour.
While the method loops through the filtered IEnumerable<Entity>, after a few iterations (I've not exactly counted how many),
one of the items in it gets deleted.
private async Task<bool> UpdateSomeValue(IEnumerable<BusinessEntity> entities, BusinessEntity entityToDelete)
{
//FIlter the IENumerable
var entitiesToUpdateSequence = entities
.Where(f => f.Sequence > entityToDelete.Sequence);
if (entitiesToUpdateSequence.Any())
{
var testList = new List<FormBE>(entitiesToUpdateSequence);
Debug.WriteLine(entitiesToUpdateSequence.Count()); // 5
//DUring this loop, after a few iterations, one item gets deleted
foreach (var entity in testList)
{
entity.Sequence -= 1;
}
Debug.WriteLine(entitiesToUpdateSequence.Count()); // 4
return await _someRepo.UpdateEntitySequence(entityToDelete.Id1, entityToDelete.ID2, testList);
}
return await Task.FromResult(true);
}
THis method is called like this:
var entities = await entitiesTask.ConfigureAwait(false);
var entityToDelete = entities.Single(f => f.Key.Equals("someValue"));
var updated = await UpdateSomeValue(entities, entityToDelete);
and that's it, there's no other reference to the entities collection. Therefore, it cannot be modified from any other thread.
I've temprorarily found a word around by copy the filtered IEnumerable in a List and then using the List for further operation
(List content remains the same after the loop).
What may be causing this issue?
Check out the documentation on Enumerable.Where. Specifically, the Remarks.
This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action. The query represented by this method is not executed until the object is enumerated either by calling its GetEnumerator method directly or by using foreach in Visual C# or For Each in Visual Basic.
Which means that when you call Where you're not necessarily getting back an object such as a List or Array that just has X number of items in it. You're getting back an object that knows how to filter the IEnumerable<T> you called Where on, based on the predicate you provided. When you iterate that object, such as with a foreach loop or a call to Enumerable.Count() each item in the source IEnumerable<T> is evaluated against the predicate you provided and only the items that satisfy that predicate are returned.
Since the predicate you're providing checks the Sequence property, and you're modifying that property inside the first foreach loop, the second time you iterate entitiesToUpdateSequence fewer items match the predicate you provided and so you get a lower count. If you were to increment Sequence instead of decrement it, you might end up with a higher count the second time you iterate entitiesToUpdateSequence.
I am using a thread party data model which uses it's custom data model. Hierarchy of the data model is as below:
Model
---Tables(type of Table)
-----Rows(type of Row)
-------Cells( type of Cell)
Table has property Rows as like DataTable and I have to access this property in more than tasks. Now I need a row from the table which has a column value to the specified value.
To do this, I have created a method which has lock statement to make it accessible from only one thread once.
public static Row GetRowWithColumnValue(Model model, string tableKey, string indexColumnKey, string indexColumnValue)
{
Row simObj = null;
lock (syncRoot)
{
SimWrapperFromValueFactory wrapperSimSystem = new SimWrapperFromValueFactory(model, tableKey, indexColumnKey);
simObj = wrapperSimSystem.GetWrapper(indexColumnValue);
}
return simObj;
}
To create the lookup for one of the column in Table, I have create a method which always try to create a copy of the rows to avoid collection modified exception:
Private Function GetTableRows(table As Table) As List(Of Row)
Dim rowsList As New List(Of Row)(table.Rows) 'Case 1
'rowsList.AddRange(table.Rows) 'Case 2
' Case 3
'For i As Integer = 0 To table.Rows.Count - 1
'rowsList.Add(table.Rows.ElementAt(i))
'Next
Return rowsList
End Function
but other threads can modify the table(e.g. add, remove rows or update column value in any rows). I am getting below "Collection modified exception":
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
I cannot modify this third party library to concurrent collections and this same Data Model shared between multiple project.
Question: I hunting for the solution that let me allow multiple readers on this collection either it modified in another threads.. Is it possible to Get a copy of the collection without getting exception??
Referenced below SO threads but did not find exact solution:
Lock vs. ToArray for thread safe foreach access of List collection
Can ToArray() throw an exception?
Is returning an IEnumerable<> thread-safe?
The simplest solution is to retry on exception, like this:
private List<Row> CopyVolatileList(IEnumerable<Row> original)
{
while (true)
{
try
{
List<Row> copy = new List<Row>();
foreach (Row row in original) {
copy.Add(row);
}
// Validate.
if (copy.Count != 0 && copy[copy.Count - 1] == null) // Assuming Row is a reference type.
{
// At least one element was removed from the list while were copying.
continue;
}
return copy;
}
catch (InvalidOperationException)
{
// Check ex.Message?
}
// Keep trying.
}
}
Eventually you'll get a run where the exception isn't thrown and the data integrity validation passes.
Alternatively, you can dive deep (and I mean very, very deep).
DISCLAIMER: Never ever use this in production. Unless you're desperate and really have no other option.
So we've established that you're working with a custom collection (TableRowCollection) which ultimately uses List<Row>.Enumerator to iterate through the rows. This strongly suggests that your collection is backed by a List<Row>.
First things first, you need to get a reference to that list. Your collection will not expose it publicly, so you'll need to fiddle a bit. You will need to use Reflection to find and get the value of the backing list. I recommend looking at your TableRowCollection in the debugger. It will show you non-public members and you will know what to reflect.
If you can't find your List<Row>, then take a closer look at TableRowCollection.GetEnumerator() - specifically GetEnumerator().GetType(). If that returns List<Row>.Enumerator, then bingo: we can get the backing list out of it, like so:
List<Row> list;
using (IEnumerator<Row> enumerator = table.GetEnumerator())
{
list = (List<Row>)typeof(List<Row>.Enumerator)
.GetField("list", BindingFlags.Instance | BindingFlags.NonPublic)
.GetValue(enumerator);
}
If the above methods of getting your List<Row> have failed, there is no need to read further. You might as well give up.
In case you've succeeded, now that you have the backing List<Row>, we'll have to look at Reference Source for List<T>.
What we see is 3 fields being used:
private T[] _items;
private int _size; // Accessible via "Count".
private int _version;
Our goal is to copy the items whose indexes are between zero and _size - 1 from the _items array into a new array, and to do so in between _version changes.
Observations re thread safety: List<T> does not use locks, none of the fields are marked as volatile and _version is incremented via ++, not Interlocked.Increment. Long story short this means that it is impossible to read all 3 field values and confidently say that we're looking at stable data. We'll have to read the field values repeatedly in order to be somewhat confident that we're looking at a reasonable snapshot (we will never be 100% confident, but you might choose to settle for "good enough").
using System;
using System.Collections.Generic;
using System.Linq.Expressions;
using System.Reflection;
using System.Threading;
private Row[] CopyVolatileList(List<Row> original)
{
while (true)
{
// Get _items and _size values which are safe to use in tandem.
int version = GetVersion(original); // _version.
Row[] items = GetItems(original); // _items.
int count = original.Count; // _size.
if (items.Length < count)
{
// Definitely a torn read. Copy will fail.
continue;
}
// Copy.
Row[] copy = new Row[count];
Array.Copy(items, 0, copy, 0, count);
// Stabilization window.
Thread.Sleep(1);
// Validate.
if (version == GetVersion(original)) {
return copy;
}
// Keep trying.
}
}
static Func<List<Row>, int> GetVersion = CompilePrivateFieldAccessor<List<Row>, int>("_version");
static Func<List<Row>, Row[]> GetItems = CompilePrivateFieldAccessor<List<Row>, Row[]>("_items");
static Func<TObject, TField> CompilePrivateFieldAccessor<TObject, TField>(string fieldName)
{
ParameterExpression param = Expression.Parameter(typeof(TObject), "o");
MemberExpression fieldAccess = Expression.PropertyOrField(param, fieldName);
return Expression
.Lambda<Func<TObject, TField>>(fieldAccess, param)
.Compile();
}
Note re stabilization window: the bigger it is, the more confidence you have that you're not dealing with a torn read (because the list is in process of modifying all 3 fields). I've settled on the smallest value I couldn't fail in my tests where I called CopyVolatileList in a tight loop on one thread, and used another thread to add items to the list, remove them or clear the list at random intervals between 0 and 20ms.
If you remove the stabilization window, you will occasionally get a copy with uninitialized elements at the end of the array because the other thread has removed a row while you were copying - that's why it's needed.
You should obviously validate the copy once it's built, to the best of your ability (at least check for uninitialized elements at the end of the array in case the stabilization window fails).
Good luck.
I have two tables: Transactions and TransactionAgents. TransactionAgents has a foreign key to Transactions called TransactionID. Pretty standard.
I also have this code:
BrokerManagerDataContext db = new BrokerManagerDataContext();
var transactions = from t in db.Transactions
where t.SellingPrice != 0
select t;
var taAgents = from ta in db.TransactionAgents
select ta;
foreach (var transaction in transactions)
{
foreach(var agent in taAgents)
{
agent.AgentCommission = ((transaction.CommissionPercent / 100) * (agent.CommissionPercent / 100) * transaction.SellingPrice) - agent.BrokerageSplit;
}
}
dataGridView1.DataSource = taAgents;
Basically, a TransactionAgent has a property/column named AgentCommission, which is null for all TransactionAgents in my database.
My goal is to perform the math you see in the foreach(var agent in taAgents) to patch up the value for each agent so that it isn't null.
Oddly, when I run this code and break-point on agent.AgentCommission = (formula) it shows the value is being calculated for AgentCommissision and the object is being updated but after it displays in my datagrid (used only for testing), it does not show the value it calculated.
So, to me, it seems that the Property isn't being permanently set on the object. What's more, If I persist this newly updated object back to the database with an update, I doubt the calculated AgentCommission will be set there.
Without having my table set up the same way, is there anyone that can look at the code and see why I am not retaining the property's value?
IEnumerable<T>s do not guarantee that updated values will persist across enumerations. For instance, a List will return the same set of objects on every iteration, so if you update a property, it will be saved across iterations. However, many other implementations of IEnumerables return a new set of objects each time, so any changes made will not persist.
If you need to store and update the results, pull the IEnumerable<T> down to a List<T> using .ToList() or project it into a new IEnumerable<T> using .Select() with the changes applied.
To specifically apply that to your code, it would look like this:
var transactions = (from t in db.Transactions
where t.SellingPrice != 0
select t).ToList();
var taAgents = (from ta in db.TransactionAgents
select ta).ToList();
foreach (var transaction in transactions)
{
foreach(var agent in taAgents)
{
agent.AgentCommission = ((transaction.CommissionPercent / 100) * (agent.CommissionPercent / 100) * transaction.SellingPrice) - agent.BrokerageSplit;
}
}
dataGridView1.DataSource = taAgents;
Specifically, the problem is that each time you access the IEnumerable, it enumerates over the collection. In this case, the collection is a call to the database. In the first part, you're getting the values from the database and updating them. In the second part, you're getting the values from the database again and setting that as the datasource (or, pedantically, you're setting the enumerator as the datasource, and then that is getting the values from the database).
Use .ToList() or similar to keep the results in memory, and access the same collection every time.
Assuming you are using LINQ to SQL, if EnableObjectTracking is false, then the objects will be constructed new every time the query is run. Otherwise, you would be getting the same object instances each time and your changes would survive. However, like others have shown, instead of having the query execute multiple times, cache the results in a list. Not only will you get what you want working, you'll have fewer database round trips.
I found that I had to locate the item in the list that I wanted to modify, extract the copy, modify the copy (by incrementing its count property), remove the original from the list and add the modified copy.
var x = stats.Where(d => d.word == s).FirstOrDefault();
var statCount = stats.IndexOf(x);
x.count++;
stats.RemoveAt(statCount);
stats.Add(x);
It is helpful to rewrite your LINQ expression using lambdas so that we can consider the code in more explicit terms.
//Original code from question
var taAgents = from ta in db.TransactionAgents
select ta;
//Rewritten to explicitly call attention to what Select() is actually doing
var taAgents = db.TransactionAgents.Select(ta => new TransactionAgents(/*database row's data*/)});
In the rewritten code, we can clearly see that Select() is constructing a new object based on each row returned from the database. What's more, this object construction occurs every time the IEnumerable taAgents is iterated through.
So, explained more concretely, if there are 5 TransactionAgents rows in the database, in the following example, the TransactionAgents() constructor is called a total of 10 times.
// Assume there are 5 rows in the TransactionAgents table
var taAgents = from ta in db.TransactionAgents
select ta;
//foreach will iterate through the IEnumerable, thus calling the TransactionAgents() constructor 5 times
foreach(var ta in taAgents)
{
Console.WriteLine($"first iteration through taAgents - element {ta}");
}
// these first 5 TransactionAgents objects are now out of scope and are destroyed by the GC
//foreach will iterate through the IEnumerable, thus calling the TransactionAgents() constructor 5 MORE times
foreach(var ta in taAgents)
{
Console.WriteLine($"second iteration through taAgents - element {ta}");
}
// these second 5 TransactionAgents objects are now out of scope and are destroyed by the GC
As we can see, all 10 of our TransactionAgents objects were created by the lambda in our Select() method, and do not exist outside of the scope of the foreach statement.
I'm building a repository method (Entity Framework) to take in a collection of ids supplied by checkboxes in a form as part of a CMS, and updating a lookup table (entity set) that relates topics to publications.
I have this method in a respository:
public void AttachToTopics(int pubId, IQueryable<int> topicsForAssociation, IQueryable<int> topicsSubset, int primaryTopicId)
{
// EVERYTHING IS FINE IF I INSERT A MANUAL COLLECTION OF int LIKE THIS:
// var priorAssociatedTopics = new[] { 2 }.AsQueryable(); //
// BUT WHAT I REALLY NEED TO WORK IS THIS:
IQueryable<int> priorAssociatedTopics = ListTopicIdsForPublication(pubId);
var priorAssociatedTopicsToExamine = priorAssociatedTopics.Intersect(topicsSubset);
var topicsToAdd =
associatedTopics.Intersect(topicsSubset).Except(priorAssociatedTopicsToExamine);
foreach (var topicToAdd in topicsToAdd)
AttachToTopic(pubId, topicToAdd);
foreach (var topicToRemove in priorAssociatedTopicsToExamine.Except(associatedTopics))
DetachFromTopic(pubId, topicToRemove);
}
AttachToTopics chokes on the first foreach loop, yielding this error message:
This method supports the LINQ to Entities infrastructure and is not intended to be used directly from your code.
But the problem is really with the first line: the repository method called on that line provides the appropriately typed collection into priorAssociatedTopics, and Intellisense has no problem with explicitly typing it as IQueryable (normally, I'd use var), and the debugger shows that this variable holds a collection of integers.
public IQueryable<int> ListTopicIdsForPublication(int pubId)
{
var topics = from x in DataContext.TopicPublications where x.PublicationId == pubId select x;
return topics.Select(t => t.Id);
}
However, back in attachToTopics, my topicsToAdd collection doesn't get populated, and its Results View in debug holds the aforementioned error message.
Curiously, if I switch in a manually generated IQueryable collection of ints for priorAssociatedTopics (see comment in code, above) the foreach loop works fine. So I beleive I need to find some other way to get priorAssociatedTopics populated with ints from a method call in my repository.
Any clue out there?
Is there any reason that ListTopicIdsForPublication can't return an IEnumerable<int> or an IList<int> in this case? If so, adding a .ToList() at the end of topics.Select(t => t.ID) will ensure that the query gets run at that point.
It's not so much that it's IQueryable<int> that's causing the problem, but rather IQueryable<int> from DataContext.TopicPublications. It looks like it's losing it's contextual data information and that's why you're getting the exception.
I wanted to generate a unique identifier for the results of a Linq query i did on some date.
Initially i thought of using Guid for that but stumbling upon this problem i had to improvise.
However I'd like to see if anyone could have a solution using Guid so here we go.
Imagine we have:
class Query
{
public class Entry
{
public string Id { get; set; }
public int Value { get; set; }
}
public static IEnumerable<Entry> GetEntries( IEnumerable<int> list)
{
var result =
from i in list
select new Entry
{
Id = System.Guid.NewGuid().ToString("N"),
Value = i
};
return result;
}
}
Now we want Id to be unique for each entry, but we need this value to be the same for each traversal of the IEnumerable we get from GetEntries. This means that we want calling the following code:
List<int> list = new List<int> { 1, 2, 3, 4, 5 };
IEnumerable<Query.Entry> entries = Query.GetEntries(list);
Console.WriteLine("first pass");
foreach (var e in entries) { Console.WriteLine("{0} {1}", e.Value, e.Id); }
Console.WriteLine("second pass");
foreach (var e in entries) { Console.WriteLine("{0} {1}", e.Value, e.Id); }
to give us something like:
first pass
1 47f4a21a037c4ac98a336903ca9df15b
2 f339409bde22487e921e9063e016b717
3 8f41e0da06d84a58a61226a05e12e519
4 013cddf287da46cc919bab224eae9ee0
5 6df157da4e404b3a8309a55de8a95740
second pass
1 47f4a21a037c4ac98a336903ca9df15b
2 f339409bde22487e921e9063e016b717
3 8f41e0da06d84a58a61226a05e12e519
4 013cddf287da46cc919bab224eae9ee0
5 6df157da4e404b3a8309a55de8a95740
However we get:
first pass
1 47f4a21a037c4ac98a336903ca9df15b
2 f339409bde22487e921e9063e016b717
3 8f41e0da06d84a58a61226a05e12e519
4 013cddf287da46cc919bab224eae9ee0
5 6df157da4e404b3a8309a55de8a95740
second pass
1 a9433568e75f4f209c688962ee4da577
2 2d643f4b58b946ba9d02b7ba81064274
3 2ffbcca569fb450b9a8a38872a9fce5f
4 04000e5dfad340c1887ede0119faa16b
5 73a11e06e087408fbe1909f509f08d03
Now taking a second look at my code above I realized where my error was:
The assignment of Id to Guid.NewGuid().ToString("N") gets called every time we traverse the collection and thus is different everytime.
So what should i do then?
Is there a way i can reassure that i will get with only one copy of the collection everytime?
Is there a way that i'm sure that i won't be getting the new instances of the result of the query?
Thank you for your time in advance :)
This is a inherent to all LINQ queries. Being repeatable is coincidental, not guaranteed.
You can solve it with a .ToList() , like:
IEnumerable<Query.Entry> entries = Query.GetEntries(list).ToList();
Or better, move the .ToList() inside GetEntries()
Perhaps you need to produce the list of entries once, and return the same list each time in GetEntries.
Edit:
Ah no, you get each time the different list! Well, then it depends on what you want to get. If you want to get the same Id for each specific Value, maybe in different lists, you need to cache Ids: you should have a Dictionary<int, Guid> where you'll store the already allocated GUIDs. If you want your GUIDs be unique for each source list, you would perhaps need to cache the input the return IEnumerables, and always check if this input list was already returned or not.
Edit:
If you don't want to share the same GUIDs for different runs of GetEntries, you should just "materialize" the query (replacing return result; with return result.ToList();, for example), as it was suggested in the comment to your question.
Otherwise the query will run each time you traverse your list. This is what is called lazy evaluation. The lazy evaluation is usually not a problem, but in your case it leads to recalculating the GUID each query run (i.e., each loop over the result sequence).
Any reason you have to use LINQ? The following seems to work for me:
public static IEnumerable<Entry> GetEntries(IEnumerable<int> list)
{
List<Entry> results = new List<Entry>();
foreach (int i in list)
{
results.Add(new Entry() { Id = Guid.NewGuid().ToString("N"), Value = i });
}
return results;
}
That's because of the way linq works. When you return just the linq query, it is executed every time you enumerate over it. Therefore, for each list item Guid.NewGuid will be executed as many times as you enumerate over the query.
Try adding an item to the list after you iterated once over the query and you will see, that when iterating a second time, the just added list item will be also in the result set. That's because the linq query holds an instance of your list and not an independent copy.
To get always the same result, return an array or list instead of the linq query, so change the return line of the GetEntries method to something like that:
return result.ToArray();
This forces immediate execution, which also happens only once.
Best Regards,
Oliver Hanappi
You might think not using Guid, at least not with "new".
Using GetHashCode() returns unique values that don't change when you traverse the list multiple times.
The problem is that your list is IEnumerable<int>, so the hash code of each item coincides with its value.
You should re-evaluate your approach and use a different strategy. One thing that comes into my mind is to use a pseudo-random number generator initialized with the hash code of the collection. It will return you always the same numbers as soon as it's initialized with the same value. But, again, forget Guid
One suggestion: (Don't know if that's your case or not though)
If you want to save the entries in database, Try to assign your entry's primary key a Guid at the database level. This way, each entry will have a unique and persisted Guid as its primary key. Checkout this link for more info.