I have an issue using EF7 in a web application with which I could use some help. I'm currently using EF7 RC1.
Here are some models that illustrate my problem.
Contact
public class Contact
{
public Guid Id { get; set; }
public string Desc { get; set; }
public ContactType ContactType { get; set; }
}
ContactType
public class ContactType
{
public Guid Id { get; set; }
public string Desc { get; set; }
public ICollection<Contact> Contacts { get; set; }
}
These models are related via Fluent API like this:
modelBuilder.Entity<Contact>(entity => {
// abridged for clarity
entity
.HasOne(c => c.ContactType)
.WithMany(ct => ct.Contacts)
.IsRequired();
});
My needs are to be able to retrieve a collection of Contact entities from the database with their ContactType property loaded. EF makes this quite easy:
using(var context = new MyDbContext()) {
var contacts = await context
.Contacts
.Include(c => c.ContactTypes)
.Where(/* some search criteria */)
.ToListAsync();
}
The issue is that in loading the ContactType properties of the Contact entities (which happens due to the call to .Include() in the query), EF also helpfully loads the Contacts property of each ContactType entity, resulting in an infinite chain of Contacts pointing at ContactTypes and ContactTypes pointing at Contacts. I understand why this is the default behavior and that it's helpful in many cases, but my needs are to serialize these entities to JSON and send them down to the client - it's a read-only situation.
My desired behavior is for EF to return a collection of Contacts with loaded (non-null) ContactType properties that have their Contacts property set to null. Is this something EF can do? Is there any way to end up with the object graph I want short of manually nulling out properties I don't want populated?
Things I've tried:
Appending .AsNoTracking() to the EF query (which doesn't seem to stop
the Contacts property of the ContactType entity from being loaded)
Telling Json.NET not to serialize infinite reference loops (which is
required to avoid infinite recursion during serialization, but still
results in a lot of extra data being serialized)
You can't avoid EF to load ContactType.Contacts collection, as it's not actually loading it but filling the collection with the loaded Contact instances.
This is why using AsNoTracking has no efect, because is not a problem of lazy loading nor ChangeTracker.
You have three possible solutions:
Use Json.NET ReferenceLoopHandling = ReferenceLoopHandling.Ignore, but as you stated it will generate lot of unnecesary data, as you will get the collection of Contacts for every ContactType
Use [JsonIgnore] attribute on ContactType.Contacts so it will be ignored by the serializer. But it will ignore it always, I don't know if you need it in other situations
Define a DTO, use something like Automapper to map your data in it (without Contacts collection) and serialize it
I would prefer the 3rd option as I don't like sending domain model objects to the client, and it avoid adding attributes to domain model not related with domain.
I have same question Entity Framework 7 Core disable auto loading
I add AsNoTracking()
IQueryable<ScheduleModel> q = _db.Schedules;
q = q.AsNoTracking();
q = q.Include(x => x.ElementItem);
q = q.Include(x => x.ScheduleHours);
Properties not populate automatic now.
Related
If I simply do this:
var medical = ctx.Medicals.FirstOrDefault(p => p.ID == medicalViewModel.ID);
var sizeClinics = medical.Clinics.Count;
The amount is (for example) 10 (i.e. I have 10 clinics for that medical).
Now, if I do this:
var medical = mapper.Map<MedicalViewModel, Medicals>(medicalViewModel);
ctx.Entry(medical).State = medical.ID == 0 ? EntityState.Added : EntityState.Modified;
ctx.SaveChanges();
medical = ctx.Medicals.FirstOrDefault(p => p.ID == medicalViewModel.ID);
var sizeClinics = medical.Clinics.Count;
The size is 0. Why? It seems it remove relationship after SaveChanges?
Here's the Medicals object:
public partial class Medicals
{
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
public Medicals()
{
this.Activities = new HashSet<Activities>();
this.MedicalsRefunds = new HashSet<MedicalsRefunds>();
this.Clinics = new HashSet<Clinics>();
}
public int ID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string Email { get; set; }
public string Phone { get; set; }
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<Activities> Activities { get; set; }
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<MedicalsRefunds> MedicalsRefunds { get; set; }
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<Clinics> Clinics { get; set; }
}
I thing I've noticed: if I analyze medical object with QuickWatch the first time (without SaveChanges part) its as {System.Data.Entity.DynamicProxies.Medicals_650D310387E78A83885649345ED0FB2870EC304BF647B59321DFA0E4FBC78047}.
Instead, if I do SaveChanges and then I retrieve that medical, it is as {MyNamespace.Models.Medicals}.
What can it be?
This question is answered by understanding how Entity Framework works internally. I'll try to highlight the key features here.
Change tracking
Entity Framework has a sort of cache of entities in-memory, called the change tracker.
In your first example, when you fetch an entity from the database:
var medical = ctx.Medicals.FirstOrDefault(p => p.ID == medicalViewModel.ID);
Entity Framework creates the Medicals instance that you receive. When it does so, it also uses that opportunity to store a reference to that object, for its own reasons. It will keep an eye on those objects and track any changes made to them.
For example, if you now call ctx.SaveChanges(); at any point, it's going to look at everything in its change tracker, see which things have been changed, and update those in the database.
There are several benefits attached to this: you don't have to explicitly tell EF that you made changes to some of the entities it was already tracking in its cache, and EF can also spot which specific fields have changed, so it only has to update those specific fields and it can ignore the unchanged fields.
Update from comments: EF only allows the tracking of one instance of a given entity, based on the PK value. So if you've already tracked the Medical with ID 123, you can't track another instance of the same Medical entity with ID 123.
Lazy loading
The code you use suggests that you are lazy loading. I'm going to gloss over the intricate details here, to keep it simple. If you don't know what lazy/eager loading is, I suggest you look this up, as the explanation is too long to write down here. Lazy/eager loading is a key concept in Entity Framework for dealing with entity relations and how to fetch related entities.
When dealing with lazy loading, EF slightly tinkers with your entity when it fetches it for you. It puts a special lazy collection in all the entity's navigational properties (such as medical.Clinics), so that it will fetch the related data only when you actually try to access it, i.e. by enumerating the collection in any way.
Comparatively, if you were using eager loading, EF wouldn't do this for you and the nav prop simply wouldn't be filled in with anything unless you explicitly called Include on it.
Updating untracked entities
In your second example, you are working with an entity object which was not created by Entity Framework. You made it yourself:
var medical = mapper.Map<MedicalViewModel, Medicals>(medicalViewModel);
And now you manually add it to the change tracker:
ctx.Entry(medical).State = medical.ID == 0 ? EntityState.Added : EntityState.Modified;
There's nothing wrong with this, but you have to realize that the entity in the change tracker was not generated by EF, and therefore it doesn't contain these special "lazy navigational properties". And because it doesn't contain these lazy navigational properties...
var sizeClinics = medical.Clinics.Count;
... the above code doesn't actually try to fetch the data from the database. It simply works with the entity object you generated and what it already contains in-memory.
And since you didn't add anything to medical.Clinics yourself, the collection is therefore empty.
The answer
Lazy loading only works on entity objects generated by EF, not on entity objects generated by you, regardless of whether you manually added it to EF's change tracker afterwards or not.
So to get the count, you can specifically query the clinics from the database:
var medical = mapper.Map<MedicalViewModel, Medicals>(medicalViewModel);
var clinicCount = ctx.Clinics.Count(p => p.MedicalId == medical.ID);
Or you could detach the entity and fetch it from the db, though I'm not a fan of this:
var medical = mapper.Map<MedicalViewModel, Medicals>(medicalViewModel);
ctx.Entry(medical).State = medical.ID == 0 ? EntityState.Added : EntityState.Modified;
ctx.SaveChanges();
// Detach
ctx.Entry(medical).State = EntityState.Detached;
// Now fetch from db
var medical2 = ctx.Medicals.FirstOrDefault(p => p.ID == medical.ID);
var sizeClinics = medical2.Clinics.Count;
Why detach? Remember how I mentioned that EF only allows tracking of one entity of a given type and PK. Since the object referred to by medical is already being tracked, you can't fetch and track another new instance of Medicals with the same PK.
By detaching the first, medical2 can be fetched and tracked since the change tracker "forgot" the other instance.
But to be honest, it would be easier to just open a new context instead of trying to manually detach and re-query.
var medical = mapper.Map<MedicalViewModel, Medicals>(medicalViewModel);
ctx.Entry(medical).State = medical.ID == 0 ? EntityState.Added : EntityState.Modified;
ctx.SaveChanges();
using(var ctx2 = new MyContext())
{
var medical2 = ctx2.Medicals.FirstOrDefault(p => p.ID == medical.ID);
var sizeClinics = medical2.Clinics.Count;
}
More info if you're interested
If you're using code first, lazy loading is why EF requires you to make these properties virtual. EF needs to be able to inherit from your entity class and make a special derived class which overrides the navigational property behavior.
You already stumbled on this, when you said:
I thing I've noticed: if I analyze medical object with QuickWatch the first time (without SaveChanges part) its as {System.Data.Entity.DynamicProxies.Medicals_650D310387E78A83885649345ED0FB2870EC304BF647B59321DFA0E4FBC78047}.
Instead, if I do SaveChanges and then I retrieve that medical, it is as {MyNamespace.Models.Medicals}.
That System.Data.Entity.DynamicProxies.Medicals_65 (and so on) class was dynamically generated by Entity Framework, inherits the Medicals class, and overrides the virtual navigational properties so that it lazily loads this information when the collection is enumerated.
This is the hidden magic of how EF achieves lazy loading.
I wrote a query which is pretty simple:
var locations = await _context.Locations
.Include(x => x.LocationsOfTheUsers)
.Include(x => x.Address)
.ThenInclude(x => x.County)
.Where(CalculateFilters(searchObj))
.ToListAsync(cancellationToken);
And everytime LocationsOfTheUsers were null so I decided to .Include(x => x.LocationsOfTheUsers) and I received results as expected but I'm not sure why do I have to include this collections since it's defined like this:
public class Location
{
public string Title { get; set; }
public long? RegionId { get; set; }
public Region Region { get; set; }
public long? AddressId { get; set; }
public Address Address { get; set; }
public long? CountyId { get; set; }
public County County { get; set; }
public ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
}
I thought this will be automatically included since it exist as ICollection in Location class.
So why is .Include() on LocationsOfTheUsers needed here?
Thanks guys
Cheers
In entity framework the non-virtual properties represent the columns of the tables, the virtual properties represent the relations between the tables (one-to-many, many-to-many, ...)
So your property should have been defined as:
public virtual ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
One of the slower parts of a database query is the transfer of the selected data from the database management system to your local process. Hence it is wise to limit the selected data to the values you actually plan to use.
If you have a one-to-many relation between Schools and Students, and you ask for School [10] you don't want automatically to fetch its 2000 Students.
Even if you would like to have "School [10] with all its Students" it would not be efficient to use Include to also fetch the Students. Every Student will have a foreign key SchoolId with a Value of [10]. If you would use Include you would transfer this foreign key 2000 times. What a waste!
When using entity framework always use Select to fetch data and select only the properties that you actually plan to use. Only use Include if you plan to change the included items.
This way you can separate your database table structure from the actual query. If your database structure changes, only the query changes, users of your query don't notice the internal changes.
Apart from better performance and more robustness against changes, readers of your code can more easily see what values are in their query.
Certainly don't use Include to save you some typing. Having to debug one error after future changes will take way more time than you will ever save by typeing include instead of Select
Finally: limit your data early in your process, so put the Where in front.
So your query should be:
var predicate = CalculateFilters(searchObj)
var queryLocations = dbContext.Locations
.Where(predicate)
.Select(location => new
{
// Select only the location properties that you plan to use
Id = location.Id,
Name = location.Name,
// Locations Of the users:
UserLocations = location.LocationsOfTheUsers
.Select(userLocation => new
{
// again: only the properties that you plan to use
Id = userLocation.Id,
...
// Not needed, you already know the value
// LocationId = userLocation.LocationId
})
.ToList(),
Address = new
{
Street = location.Address.Street,
PostCode = location.Addrress.PostCode,
...
County = location.Address.County.Name // if you only want one property
// or if you want more properties:
County = new
{
Name = location.Address.County.Name,
Abbr = location.Address.Count.Abbr,
...
}),
},
});
I thought this will be automatically included since it exist as ICollection in Location class.
Well, it's not automatically included, probably for performance reasons as the graph of related entities and their recursive child entities may be rather deep.
That's why you use eager loading to explicitly include the related entities that you want using the Include method.
The other option is to use lazy loading which means that the related entities are loaded as soon as you access the navigation property in your code, assuming some prerequisites are fulfilled and that the context is still around when this happens.
Please refer to the docs for more information.
I believe you are using EntityFrameworkCore. In EntityFramework (EF6), lazy loading is enabled by default, However, in EntityFrameworkCore, lazy loading related entities is handled by a separate package Microsoft.EntityFrameworkCore.Proxies.
To enable the behaviour you are seeking, install the above package and add the following code
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseLazyLoadingProxies();
}
After this, the related entities will be loaded without the Include call.
I am using Entity Framework Core 2.2.6. I'm going to try and make this question concise and apologies in advance if it ends up being a wall of text.
The error I am seeing is an ambiguous column name in the SQL Entity Framework Core generates.
So my situation is this: I have two entities with a many-to-one relationship. The "parent" entity implements
an interface that has a property that is of type IChildEntity. Here are the interfaces:
public interface IParentEntity
{
IChildEntity Child { get; set; }
string Prop1 { get; set; }
string Prop2 { get; set; }
}
public interface IChildEntity
{
string ChildProp1 { get; set; }
string ChildProp2 { get; set; }
}
I am using ef core's fluent api and in order to set up the relationship between parent and child
I am using a concrete type of ChildEntity and defining a IChildEntity property to conform to the
interface and just passing things through to the concrete type:
public class ChildEntity : IChildEntity
{
public long ID {get; set;}
public string ChildProp1 { get; set; }
public string ChildProp2 { get; set; }
}
public class ParentEntity : IParentEntity
{
public long ID { get; set; }
public string Prop1 { get; set; }
public string Prop2 { get; set; }
public long ChildID { get; set; }
// Navigation property so EF Core can create the relationship
public ChildEntity MappedChild { get; private set; }
// this is to adhere to the interface
// just pass this through to the backing concrete instance
[NotMapped]
public IChildEntity Child
{
get => MappedChild;
set => MappedChild = (ChildEntity)value;
}
}
Then in OnModelCreating I set up the relationship like so:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<ParentEntity>()
.HasOne(e => e.MappedChild)
.WithMany()
.HasForeignKey(e => e.ChildID);
}
This works and the relationship gets set up as expected, however I am finding when I do a query it can generate
some SQL that can result in an ambigous column error in some database engines. Here is the example query:
MyContext.ParentEntity
.Include(p => p.MappedChild)
.Where(p => p.Prop1.Equals("somestring")
.FirstOrDefault()
The SQL that gets generated is similar to:
SELECT p."ID", p."ChildID", p."Prop1", p."Prop1", "p.MappedChild"."ID", "p.MappedChild"."ChildProp1", "p.MappedChild"."ChildProp2"
FROM "ParentEntity" AS p
INNER JOIN "ChildEntity" AS "p.MappedChild" ON p."ChildID" = "p.MappedChild"."ID"
WHERE p."Prop1" = 'somestring'
ORDER BY "p.MappedChild"."ID"
LIMIT 1
The problem here is we are selecting two columns with the name ID and not aliasing. Some databases will be ok with this
but some will not. A work around I can do for this is to do two separate queries to get the entity and the child entity:
var parent = MyContext.ParentEntity
.Where(p => p.Prop1.Equals("somestring")
.FirstOrDefault()
MyContext.Entry(parent).Reference(p => s.MappedChild).Load();
But this is less than ideal since it does multiple queries and is a bit less elegant than just using Include()
Because this seems like such a common use case and I couldn't find any bug reports against EF Core for this type of
behavior it is my suspicion that I am doing something wrong here that is resulting in EFCore not aliasing column names
for this type of query. I was thinking it could be the bit of trickery I have to do to ensure my entity implements it's interface
(this is something I can't due to constraints in the codebase and other integrations) but the more I look at it the less likely that
seems to me since we are directly dealing with the "mapped" property in EF related code and it's completely unaware of the interface.
My questions are - can anyone see something in my implementation that would cause this? Could anyone
suggest a better workaround than what I have here? Any advice here would be appreciated. Thanks much.
This is an old Entity framework bug with the Oracle company products bug including the MySQL database and Oracle database (12.1 and older).
I see the
ORA-00918: column ambiguously defined
error mostly when:
Selecting one entity with including parent entity.
Selecting one entity with value object own one command
This error appears when using Find, First, FirstOrDefault, Last, Single and all single entity selector commands.
I tested many solutions and check generated sql statement to find out a very unique way without any performance overhead:
// This the way of getting one entity from oracle 12.1 without throwing Oracle exception => ORA-00918: column ambiguously defined without any extra overhead
var entities = await dbSet.Where(x => x.Id == id).Take(1).ToListAsync();
var entity = entities.FirstOrDefault();
Another Sample:
var entities = await dbSet.OrderByDescending(x => x.Id).Take(1).ToListAsync();
var entity = entities.FirstOrDefault();
At the end of your IQueryable Linq add Take(1) and get all with .ToList() or .ToListAsync() to execute the statement and fetch a list with one record. Then use Enumerable Single Entity Selector to change the list to an entity.
That’s all.
I use EF Database first, because I like to design my databases in SQL Management Studio, and it's quite frankly very easy to make Visual Studio create all the entities directly from the database, without having to do any code.
How ever, I noticed on SQL Profiler, that EF is eagerly loading all the related entities of an object, whenever I call ToList() on the result collection.
Let's say I have an entity like this:
public class SomeEntity
{
public string Name { get; set; }
public IEnumerable<SomeOtherEntity> ListOfOtherEntites { get; set; }
}
I could then have a query, getting a list of these entities:
public IEnumerable<SomeEntity> GetAllOfTheSomeEntities(Guid customerId){
return dbContex.SomeEntity.Where(x => x.CustomerId == customerId);
}
At a later point in the code, I want to do stuff with this list (for instance in a controller in MVC), and I would call ToList() on the query:
var list = repository.GetAllOfTheSomeEntities(customerId).ToList();
Even though I never use the property "ListOfOtherEntites" on the Entity, it's still being loaded into my server's memory.
I know that I don't have to worry about this in Code first, because I can control the loading with the "virtual/non-virtual" properties and Include - but how am I to go about this in Database first?
I could alter my entities after I load them from the database; but they're just gonna get recreated if I update my model at a later point.
You need to mark your navigation properties as virtual to enable lazy loading in entity framework.
public class SomeEntity
{
public string Name { get; set; }
public virtual IEnumerable<SomeOtherEntity> ListOfOtherEntites { get; set; }
}
Doesn't simply modifying the generated code fix the problem?
If you're experiencing issues with lazy loading then I might recommend turning lazy loading off within your EF context.
public class YourContext : DbContext
{
public YourContext()
{
this.Configuration.LazyLoadingEnabled = false;
}
}
From here you can then include the related entities using LINQ's .Include() method like so:
var posts = context.Blogs.Include(b => b.Posts).ToList();
I have the following two model objects which have a many-to-many relationship:
public class StaffMember
{
public Guid StaffMemberKey {get; set;}
// lots of other properties that aren't relevant
public ICollection<Case> Cases {get; set;}
}
public class Case
{
public int CaseKey {get; set;}
// lots of other properties that aren't relevant
public ICollection<StaffMember> Staff {get; set;}
}
The mapping for the many-to-many relationship is handled in the configuration for the Case entity:
public class CaseMapping : EntityTypeConfiguration<Case>
{
public CaseMapping()
{
// other property and relationship mappings
// Many-to-Many mapping with Staff Members
HasMany(c => c.Staff)
.WithMany(staffMember => staffMember.Cases)
.Map(m =>
{
m.ToTable("Cases_StaffMembers", "dbo");
m.MapLeftKey("CaseKey");
m.MapRightKey("StaffMemberKey");
});
}
}
Everything is working great in terms of being able to query against this relationship, add, delete, etc. However, when trying to explicitly load and filter staff members for a case, as described here, no data is being loaded in to the appropriate collection of related entities.
Here is an example of what I'm attempting do:
var staffMemberKey = Guid.Parse("...");
var caseKey = 5;
using (var context = new CodeFirstContext())
{
var selectedCase = context.Cases.Find(caseKey);
context.Entry(selectedCase).Collection(c => c.Staff).Query().Where(sm => sm.StaffMemberKey == staffMemberKey).Load();
}
I would expect that selectedCase.Staff would contain the staff member that was loaded, but it remains null. If I call ToList() instead of Load when querying for the related data, the resulting list does contain the correct staff member entity. If I simply call context.Entry(selectedCase).Collection(c => c.Staff).Load();, then the data is loaded as expected. Is there something I'm missing? What gives?
As a final note, I have lazy loading and proxy creation disabled for my context, in case that makes any difference in this scenario.
When you call Query(), it returns an IQueryable that gives the entities that would be in that property -- it is not designed to be used to update the property. It is basically a "shortcut" for:
ctx.Staff.Where(staff => staff.Case.Id == caseKey);
Load() will load entities into your context, as if you had called ToList() but without returning anything. It works on any IQueryable, and does not capture anything related to the Entry().