I am trying to do some checks on my database with an automated process. On a schedule the process goes out to a service and checks all the entries in the database against a list.
I want to re-insert the records that may have been deleted and update ones that are out of date
foreach (Category x in CustomeClass)
{
Category exists = Context.SSActivewear_Category
.Where(b => b.CategoryID == x.CategoryID)
.FirstOrDefault();
if (exists == null)
Context.Add(x);
else
Context.Update(x);
}
Not sure but I keep getting messages about tracking an instance with the same key etc. Can someone point me to a best practice on something like this
Danka!
This type of error is common when re-using the same instance of EF dbContext, especially when trying to load the same entity more then once from database using the same context.
If this is the case, then simply recreate the context (either new it up or use context factory) and then try update or modify your database data.
Creating new context is cheap, so no worries there.
After updating, save changes and dispose of the context (or use it in using statement to begin with).
If you are modifying the same entity multiple times using the same context, then do not load it from database multiple times.
In Your particular code example I would check if there is no duplication of Category objects in CustomeClass collection.
Is there duplication of CategoryID?
Is CategoryID for sure auto-generated when saving data (entity configuration)?
Is code not trying to update multiple entities with same id?
etc.
Entity framework works with references. Two instances of a class with the same data amount to two different references and only one reference can be associated with a DbContext, otherwise you get errors like that.
As per your example below:
foreach (Category x in CustomeClass)
{
Category exists = Context.SSActivewear_Category
.Where(b => b.CategoryID == x.CategoryID)
.FirstOrDefault();
if (exists == null)
Context.Add(x);
else
Context.Update(x);
}
Category is assumed to be your Entity, so "CustomeClass" would be a collection of instances that are not associated with your Context. These are Detached instances.
When your "exists" comes back as #null, this will appear to work as the Category "x" gets added and tracked by the Context. However, when "exists" comes back as not #null, you now have two instances for the same entity. "exists" is tracked by the DbContext, while "x" is not. You cannot use Update() with "x", you must copy the values across.
The simplest way to do this would be Automapper where you can create a map from Category to Category, then use Map to copy all values from "x" over into "exists":
var config = new MapperConfiguration(cfg => cfg.CreateMap<Category, Category>());
var mapper = config.CreateMapper();
mapper.Map(x, exists);
This is purely an example, you'll probably want to configure and inject a mapper that handles your entity copying. You can configure the CreateMap to exclude columns that shouldn't ever change. (Using ForMember, etc.)
Alternatively you can copy the values across manually:
// ...
else
{
exists.Name = x.Name,
exists.SomeValue = x.SomeValue,
// ...
}
In general you should avoid using the Update method in EF as this will result in a statement that overwrites all columns in a table, rather than updating just the column(s) that changes. (If no columns changed then no UPDATE SQL will actually get run)
On another side note, when getting "exists", you should use SingleOrDefault() not FirstOrDefault() as you expect 0 or 1 row back. First* methods should be used in cases where you expect there can be multiple matches but only want the first match, and should always be used with an OrderBy*() method to ensure the results are predictable.
You can use Update by performing an Exists check query that doesn't load a tracked entity. Examples would be:
Category exists = Context.SSActivewear_Category
.AsNoTracking()
.Where(b => b.CategoryID == x.CategoryID)
.SingleOrDefault();
or better:
bool exists = Context.SSActivewear_Category
.Where(b => b.CategoryID == x.CategoryID)
.Any();
Then you could use Context.Update(x). AsNoTracking() tells EF to load an instance but not track it. This would really be a waste in this case as it's a round trip to the DB to return everything in the Category only to check if something was returned or not. The Any() call would be a round trip but just does an EXISTS db query to return true or false.
However, these are not fool-proof as there is no guarantee that the Context instance isn't already tracking an instance for that Category from some other operation. Something as trivial as having a Category appear in CustomeClass twice for any reason would be enough to trip the above examples up as once you call Context.Update(x) that instance is now tracked, so the loop iteration for the second instance would fail.
Related
I guess I just don't understanding EF tracking. I have the context added via dependency injection via:
builder.Services.AddDbContext<OracleContext>(options => options.UseOracle(OracleConnectionString, b => b.UseOracleSQLCompatibility("11"))
.LogTo(s => System.Diagnostics.Debug.WriteLine(s))
.EnableDetailedErrors(Settings.Dev_System)
.EnableSensitiveDataLogging(Settings.Dev_System)
.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking));
I set the tracking behavior to NoTracking here (at least so I thought).
I have a .NET Controller that has the context in its constructor. It passes this context to my class constructor containing my methods. Pretty much everything works fine... except for one:
I have a method that does a context.RawSqlQuery to get a list of objects. I iterate over these objects calling two separate methods from a different class that was generated the same way (using the injected context). This method first does a EF query to verify the object does not already exist, if it does it returns it and we move on - no issues. On the query to check if it exists I also added .AsNoTracking() for SnGs. However, if the object does not exist, and I try to make a new one... every time I do an context.add I get
"The instance of entity type 'Whatever' cannot be tracked because another instance with the key value '{MfrId: 90}' is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached."
I have tried adding
db.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking - no change
I have tried adding context.Entry(NewItem).State = EntityState.Detached; before and after the call - no change.
I tried a loop in the context that gets all tracked objects and sets them to detached - no change.
What am I missing here? First - why is it tracking at all? Second - any suggestions on how to get passed this? Should I just give up using dependency injection for the context (suck... lots of rework for this)?
As requested - here is the class & method that is failing (non related stuff removed):
public class AssetMethods : IAssetMethods
{
public OracleContext db;
public AssetMethods(OracleContext context)
{
db = context;
}
public CcpManufacturer? CreateNewManufacturer(CcpManufacturer NewMan, string ActorID)
{
...blah blah non DB validation stuff removed...
//Check if it exists already
CcpManufacturer? OldMan = db.CcpManufacturers.Where(m=>m.MfrName == NewMan.MfrName).AsNoTracking().FirstOrDefault();
if (OldMan != null) {
return OldMan;
}
//Who done did it
NewMan.CreatedBy = ActorID;
NewMan.CreationDate = DateTime.Now;
NewMan.Isenabled = true;
//save
db.CcpManufacturers.Add(NewMan);
db.SaveChanges(); //fails here
//Prevent XSS Reflection
return db.CcpManufacturers.Find(NewMan.MfrId);
}
}
this method is called from this code. The OM is also using the injected context
List<MasterItem> Items = OM.RawSqlQuery(Query, x => new MasterItem { MFR_NAME = (string)x[0], MODEL_NUMBER = (string)x[1], LEGAL_NAME= (string)x[2]});
foreach (MasterItem item in Items)
{
CcpManufacturer? Man = new() {
MfrName = item.MFR_NAME,
Displayname = item.MFR_NAME
};
Man = AM.CreateNewManufacturer(Man,System.Id); //if its new.. it never get passed here because of the discussed error...
if (Man == null || Man.MfrId == 0)
{
continue;
}
.... other stuff
}
So the mfr id is added to a new object that's passed to a pretty much identical methods to create a item (where the mfr id is attached). Now - if I detach THAT item - I am ok. But why is it tracking when I have it turned off pretty much everywhere?
Yes, you found your problem.
Turning off Tracking affects what EF does when querying for entities. This means when I tell EF to read data from the DB and give me entities, it will not hang onto references of those entities.
However, entities you tell a DBContext to ADD to a DbSet and related entities will be tracked, regardless of your tracking setting.
So if I do something like:
var entity1 = context.Entities.Single(x => x.Id == entityId).AsNoTracking();
var entity2 = context.Entities.Single(x => x.Id == entityId).AsNoTracking();
The references to entity1 and entity2 will be 2 distinct references to the same record. Both are detached, so the DbContext isn't tracking either of them. I can use and attach either of them to perform an update, but that entity would be from that point considered Attached until I explicitly detach it again. Attempting to use the other reference for an update would result in that error. If I query specifying NoTracking after I have attached and updated that first entity reference, I will get back a new untracked entity reference. The DbContext doesn't return it's tracked reference, but it doesn't discard it either.
The exact same thing happens if I add a new entity then query for it specifying NoTracking. The query returns an untracked reference. So if you try and attach it to update a row, EF will complain about the reference it is already tracking.
I don't recommend diving down the rabbit hole of passing around detached entities unless you're keen to spend the time to really understand what is going on behind the scenes and prepared for the pretty deliberate and careful handling of references. The implications aren't just things not working as expected, it's having things work or not work on a completely situational basis which can be a nasty thing to debug and fix, even when you know what to look for.
I am deleting hundreds of rows from my table. Using ADO.Net code i.e. Delete from table where somecolumn = somestring takes less than a second while using Entity Framwork i.e.
MyDbContext context = new MyDbContext()
context.SomeEntity.RemoveRange(context.SomeEntity.Where(i => i.somecolumn == somestring));
context.SaveChanges();
is taking 8-10 seconds.
can anybody explain it or I am doing something wrong.
In these situations which you want to delete among of records (read thousand or more) , the most efficient way would be a so called "bulk delete". EFCore.BulkExtensions allows that. Some thing like below code:
var recordsToRemove = context.SomeEcontity.Where(i => i.somecolumn == somestring));
context.BulkDelete(recordsToRemove);
EF is designed around providing relational data mapped to an object model. It isn't ideally suited to large bulk operations. Still, you can facilitate one-off requirements like this a number of ways.
The first method would be to use stubs for the delete. For this you would want to ensure that the DbContext instance is "clean" of any tracked instances that might be deleted, so ideally a DbContext that is scoped to that method.
using (var context = new SomeDbContext())
{
var stubs = context.SomeEntities
.Where(x => x.SomeColumn == someString)
.Select(x => x.Id)
.ToList()
.Select(x => new SomeEntity { Id == x })
.ToList();
}
Now you might be able to simplify that down to:
var stubs = context.SomeEntities
.Where(x => x.SomeColumn == someString)
.Select(x => new SomeEntity { Id == x.Id })
.ToList();
However, you would probably want to test that to ensure that the resulting SQL is just selecting the ID (not the entire entity) and that context.SomeEntities.Local.Any() is still false... The first example will just ensure that the query load the IDs, then proceeds to build stub Entities using that ID. This makes the "selection" for our data as efficient as possible.
From here you should be able to use RemoveRange on the untracked stubs.
context.SomEntities.RemoveRamge(stubs);
context.SaveChanges();
The important detail is that the DbContext cannot be tracking any of these entities as this will temporarily attach these stubs to the DbContext. If the context was already tracking an instance with one of those ids then you would receive an error that one or more entities with the same ID was already being tracked. (Hence the locally scoped DbContext to avoid that)
The other way to perform this deletion would be to issue a direct SQL operation. If you have a DbContext that is scoped to a request or longer than this single operation then this should be done after dealing with any currently tracked instances.
Step 1. Deal with any tracked instances if you have an injected DbContext:
var trackedInstances = context.SomeEntities.Local
.Where(x => x.SomeColumn == someString);
.ToList();
if (trackedInstances.Any())
context.SomeInstances.RemoveRange(trackedInstances);
This will check the DbContext for any tracked instances without hitting the DB. We will want to remove these instances to avoid possibly having any of these marked as Modified and triggering an exception later during a SaveChanges call.
Step 2. Build and run a parameterized raw SQL statement to clean off all remaining rows in the DB.
context.Database.ExecuteSqlCommand(#"DELETE FROM dbo.SomeEntities
WHERE SomeColumn = #someString", new SqlParameter("someString", someString));
context.SaveChanges();
The important detail here is to use a parameterized query. Do not execute raw SQL with the parameters embedded in the string as this leaves the door open to SQL injection attacks.
I.e. do not use anything like:
context.Database.ExecuteSqlCommand($"DELETE FROM dbo.SomeEntities
WHERE SomeColumn = '{someString}'");
// or
context.Database.ExecuteSqlCommand("DELETE FROM dbo.SomeEntities
WHERE SomeColumn = '" + someString + "'");
I am updating some existing code of a former colleague and have a strange issue where an unused line is causing an error with Entity Framework. If I comment out the code tagged with //This Line!, everything works.
foreach (Place item in ListOfPlaces)
{
//This line!
List<Place> PlacesList = context.Places.Where(x => x.PlaceNumberID == item.PlaceNumberID).ToList();
long PlaceId = context.Places
.Where(x => x.PlaceNumberID == item.PlaceNumberID)
.Select(x => x.PlaceId)
.FirstOrDefault();
if (PlaceId != 0)
{
item.ID = PlaceId;
context.Places.Attach(item);
context.Entry(item).State = System.Data.Entity.EntityState.Modified;
}
}
If I include that line, I get the error shown here on the Attach(item) line:
Attaching an entity of type 'Place' failed because another entity of the same type already has the same primary key value. This can happen when using the 'Attach' method or setting the state of an entity to 'Unchanged' or 'Modified' if any entities in the graph have conflicting key values. This may be because some entities are new and have not yet received database-generated key values. In this case use the 'Add' method or the 'Added' entity state to track the graph and then set the state of non-new entities to 'Unchanged' or 'Modified' as appropriate.
I know how to fix this from a code point of view (remove the line!), but I can't work out why its breaking the application if somebody could kindly explain please.
I can't work out why its breaking the application
Looks to me like the line causes the download of some Place with ID N - see the ToList on the end? It will trigger the query to run and download data. EF creates objects from every row it receives because that's the default behavior(it can be disabled with eg AsNoTracking)
Later you try to create another object with the same primary key value and attach it to the context, but the context already knows about some object with ID 123 (for example) because the first line caused it to have been downloaded/tracked so you get an error when you try and associate another - if EF allowed both into its tracking memory it wouldn't know which one was the true authority of record that should be saved back to the db
Your interim query doesn't cause the problem, I believe, because it doesn't trigger the download of an entire entity, seeing as it just retrieves an ID
If you're trying to implement insert-if-not-exists style behavior, you should attempt to download an entity with ID x using some XOrDefault or Find, and if it results in null/default then create and add a new entity (you don't need to attach). In essence, ditch the first line, just do the ID check and if the returned ID is default, do a context.Places.Add(new Place{...}).
If you're looking for upsert, it's probably easiest to download the whole entity and then inspect if it was default or not; if it is, then make a new one otherwise edit the downloaded one.
If you're trying for "update without download" then skip the querying part entirely and attach an entity you declare as modified.
If you're after some hybrid upsets without download, I think you'll struggle, because you have to at least quiz the db as to whether it knows of an entity before you decide what to do.. or you run a raw MERGE
That entire loop makes no sense. You repeat the same twice. And as soon as you select one of the items, EF marks it as a tracked. And you can't update using another item, before the first one will be untracked or you can use the tracked item.
Try this code
foreach (Place item in ListOfPlaces)
{
var placesList = context.Places.Where(x => x.PlaceNumberID == item.PlaceNumberID).ToList();
if(placesList!=null && placesList.Count ==1)
{
var existedPlace = placesList.FirstOrDefault();
context.Entry(existedPlace).CurrentValues.SetValues(item);
}
// and maybe this
else context.Places.Add(item)
}
context.SaveChanges();
UPDATE
Thanks to #CaiusJard for a hint, in this case it is more efficient to use SingleOrDefault instead of ToList
.....
var existedPlace = context.Places.Where(x => x.PlaceNumberID == item.PlaceNumberID).SingleOrDefault();
if(existedPlace!=null)
{
context.Entry(existedPlace).CurrentValues.SetValues(item);
}
.....
I´m trying to update a list of objects, but when I run context.SaveChanges(), I get an error
Cannot insert explicit value for identity column in table 'TABLE' when IDENTITY_INSERT is set to OFF
I know when the Identity key is generated you cannot register a new entry with this KEY but I'm trying to update my registry not insert a new one and this exception appears. I don't know why this happens now.
My code:
foreach (int i in deletes)
{
var p = context.Item.FirstOrDefault(g => g.participant_id == i);
if (p != null)
{
p.group_id = 0;
context.Entry(p).State = Microsoft.EntityFrameworkCore.EntityState.Modified;
context.SaveChanges();
}
}
group_id is not my identity column, it is just another column I added:
ALTER TABLE Items
ADD group_id INTEGER DEFAULT(0);
The error specifically is the result of a configuration mismatch somewhere in your entity definitions where a table is set to use an Identity PK but EF is not configured for an identity column and the DbContext is attempting to insert a row into that table. This is not necessarily the table you think the DbContext is attempting to update. Chances are somewhere prior to SaveChanges being called there has been an entity incorrectly created or associated and treated as a new entity.
Try this, provided your code is roughly identical to your example:
using (var tempContext = new YourAppDbContext())
{
foreach (int i in deletes)
{
var p = tempContext.Item.FirstOrDefault(g => g.participant_id == i);
if (p != null)
p.group_id = 0;
}
tempContext.SaveChanges();
}
Here we isolate this operation to a clean, temporary DbContext instance to ensure there are no polluting tracked changes. This is not a "fix" for the issue, just to identify whether the context was polluted. You can inspect context pending changes to see what it might be tracking. (ChangeTracker) Your issue may also stem from setting the entire entity to Modified state and a misconfigured related entity definition. (unlikely if the entity and associated were read from this DbContext instance) When updating an entity, don't set the EntityState to Modified. This sets EF up to effectively update all columns on that table. In the above example if we just update 1 column, EF will generate an Update statement for just that single column, and will only generate an update statement if that value actually changes.
If the above code works without an issue, then you have rogue tracked changes polluting your DbContext. This either means you have something else going on before this code is called that you need to fix, or your DbContext lifetime scope is far longer than it should be. (or both)
The problem with your example is that we cannot see the scope that your context is covering. Whenever you make any kind of change to a DbContext, those changes are tracked by default by that DbContext. If you have code that is working with detached entities anywhere that suddenly get re-associated with a DbContext via association with an Added entity and treated as new entities themselves, then calling SaveChanges() when you expect to update a single row all the sudden results in strange unrelated exceptions because the DbContext wants to persist queued up changes.
A typical culprit to look out for with issues like this would be first to check over all of your entity definitions for tables using Identity PKs and ensure those entities are configured with DatabaseGeneratedOption.Identity. This alone would prevent the exception, but you'd probably find that EF would be inserting new, duplicate rows somewhere with new PKs. The culprit behind that is usually detached entities being re-associated to an entity.
Given a Order entity associated to a Product. We want to create an order so since we already loaded the Products collection to select and associated to the new Order, we set that reference client side and send the Order to the server. We assume that since we loaded the Products and it's children from a DbContext on a GET, that when we call POST and send that parent back to the server, all should be good.
public void CreateOrder(Order order)
{
context.Orders.Add(order);
context.SaveChanges();
}
order had a Product reference with ProductId = 22 since we selected product #22 for the order. However, the DbContext instance when we call CreateOrder doesn't know Product ID 22 represents a known entity, it isn't tracking Product ID 22, so it gets treated as a new entity. If the Product entity ID isn't configured as an Identity column but is in the DB, you get the error you saw. If it is configured as an Identity column then you'd find the Order saves successfully, but is pointing at a new, duplicate Product with an ID of 56 or whatever the next available ProductId is.
Passing detached entities is a big cause of headaches like this. The safe way to do something like the CreateOrder would be:
public void CreateOrder(Order order)
{
// TODO: Obviously add null checks, validations, etc.
var product = context.Products.Single(x => x.ProductId == order.Product.ProductId);
order.Product = product; // replace with the product tracked by this DbContext.
context.Orders.Add(order);
context.SaveChanges();
}
All associations on a detached entity need to be checked.
Devs don't like the hit to the database so they can be tempted to merely attach the Product:
public void CreateOrder(Order order)
{
// TODO: Obviously add null checks, validations, etc.
context.Products.Attach(order.Product);
context.Orders.Add(order);
context.SaveChanges();
}
.. and in most cases that will appear to work. Until it doesn't. Attach will throw an exception the moment you attempt to attach an entity with an ID that the DbContext is already tracking. If any code prior had loaded or attached an instance of that Product, then the call will fail. This can result in random-like exceptions appearing at runtime. The safe way to attach the entity would be:
public void CreateOrder(Order order)
{
// TODO: Obviously add null checks, validations, etc.
var existingProduct = context.Product.Local.SingleOrDefault(x => x.ProductId == order.Product.ProductId);
if (existingProduct == null)
context.Products.Attach(order.Product);
else
order.Product = existingProduct;
context.Orders.Add(order);
context.SaveChanges();
}
This checks the local tracking cache to see if the DbContext is tracking that product. (Does not hit DB) If it isn't then we can safely attach it. If it is, we replace the product reference on the order with the tracked one.
This would have to be done for every reference. Any that are missed would result in potential errors or duplicate rows.
Try
var p = context.DbSet<Participant>().Where(g => g.participant_id == i).FirstOrDefault();
And did you check your Participant class in EF? It still can be a foreign key there.
Lets say, I have such structure:
Branch: Id
Group: Id
BranchGroup: BranchId, GroupId, Since, Till
Relations are Branch 1..N BranchGroup 1..1 Group, so that generated Branch class has an ICollection<BranchGroup> BranchGroupsnavigation property.
And there is a property inside Branch POCO, returning current groups (simplified a bit):
public List<Group> Groups => BranchGroups.Where(bg => bg.Since <= DateHelper.Current &&
bg.Till >= DateHelper.Current)
.Select(bg => bg.Group).ToList();
I call it and db is exploded with thousands of queries, each for one group. Is it possible to somehow optimize it without having access to dbContext (or at least reach current context from inside entity)?
UPD: simplest real query looks like this:
public List<Group> Groups => BranchGroups.Where(bg => bg.Since <= DateHelper.Current &&
bg.Till >= DateHelper.Current)
.Select(bg => bg.Group)
.Where(g => !g.Deleted).ToList();
That seems to be not possible - whenever you access navigation property (BranchGroups in this case), whole related set is pulled from database, so the rest of your query (Where, Select, Where) is performed over data set in memory, so it not translated to database query. Then, bg.Group is also navigation property, so to get one, separate database query is performed, just as you observe.
Real POCO objects should not have any relation to DbContext, they are just plain objects. So you cannot (and should not be able to) reach to the context from inside POCO (or that is not POCO any more).
So your options are:
Fill Groups property when constructing Branch object (not good if you don't always need to load those Groups).
Fetch Groups with the help of external context reference (not from inside POCO).
Maybe, when constructing Branch object, set Groups property like this:
branch.Groups = context.BranchGroups.Where(c => c.BranchID = branch.ID && ...); // the rest of your query
Then it will still be POCO (Groups are just IEnumerable) but contains query you need which is lazy loaded on access (and no explicit references to context).