WebApi
Let's have a Person class with many one-to-many relations to classes like Car, Pet, Children ...
and PersonRepository where i have to include all relations and their relations or just disable lazy-loading
public Person getPerson()
{
...
using (db = new DbContext())
{
// var person = linq-query
// return person;
}
...
}
So in my controller i have fully loaded Person with relations and their relations.
If there is array of objects with many nested relations it is inefficient and memory demanding.
How i can lazy-load properties outside of repository or what is general solution to this problem?
How i can lazy-load properties outside of repository or what is general solution to this problem?
If you mean to know how to load navigational properties after disabling Lazy Loading on your context, then you have the following solutions:
Eager Loading by using Include methode of your DbSet like this:
db.Persons.Include(p => p.Cars).Include(p => p.Pets).Include(p => p.Children).Where(p => p.Id == personId);
Explicit Loading by using the change tracker and Load method on your entry like this : db.entry(person).Collection(p => p.Cars).Load();
The solution that I see a lot with EF and Repository Pattern is to use a collection lambda expression as a parameter of your method like this :
public Person GetPerson(params Expression<Func<Person, object>>[] includes)
{
using (var db = new DbContext())
{
IQueryable<Person> query = db.Posts;
Array.ForEach(includes, expression => query = query.Include(expression));
return query.FirstOrDefault();
}
}
You use it like this : GetPerson(p => P.Cars, p => p.Pets, p => p.Children) With this solution you only load the navigational properties you need by specifing them as parameters of your method.
You can use ajax to populate data on demand.
Like this:
https://cmatskas.com/update-an-mvc-partial-view-with-ajax/
Happy to help you!
As far as I know there is not the single general solution.
So you can try these cases:
Don't close read-only DbContext (just don't forget disable tracking). In some scenarios like web request you can create DbContext at the start of request, and dispose it and the end.
Forget for memory demanding. If your plan to read 2Mb data from SQL, the something wrong with your design. 5Kb, 30Kb are not the big data.
Think about cache. Let the necessary data will available always.
Refactor your interfaces to load necessary data only. F.e. append methods LoadPersonsWithAddresses, LoadPersonsWithFamily, and so on.
Explicitly call load method on your collection. This forces lazy load.
Related
I would like to keep the lifespan of my DbContext as short as possible, so I take advantage of the using statement.
However, since Entity Framework tracks the entities, I cannot do something like this :
public IEnumerable<Person> GetPersons()
{
using (var db = new AppContext())
{
Logging.Log(Information, "Requested getPersons from service");
return db.Persons;
}
}
Because as soon as I use the list of Persons in my ViewModel, I will receive an InvalidOperationException (the context has been disposed before I use the object).
To bypass that, I convert the result of the query to a concrete List, by calling ToList()
public IEnumerable<Person> GetPersons()
{
using (var db = new AppContext())
{
Logging.Log(Information, "Requested getPersons from service");
return db.Persons.ToList();
}
}
Is this an okay thing to do ? AsNoTracking() doesn't have any effects because it still returns some entity that gets dropped if the context is disposed.
When it comes to Entities you have two options. One is simple, and the other looks even simpler but is a whole lot more complex.
The simple option: An entity should never be referenced beyond the scope of it's DBContext. This means in something like an MVC app, entities don't get sent to a view. They don't get serialized, they are consumed solely within the scope of their DbContext.
The key to this approach is using POCO view models and projection, (Select and Automapper's ProjectTo) and when dealing with various common layers and such, adopting something like a Unit of Work pattern to manage the lifetime scope of a DbContext.
The deceptively simple option: Detached entities. Simply ensuring the DbContext is alive when returning entities can be enough to solve your problem. A Unit of Work pattern can be leveraged for this approach as well. While it looks simple enough, it is loaded with pitfalls. Serializing an entity for instance to pass back to a view will trigger lazy load calls as the serializer traverses the entity(ies). Passing entities back to be persisted is also problematic as you have to be cautious of trying to re-attach entities where the Context may already be tracking leading to situational runtime errors, overwriting data with stale copies, and exposing your system to unintended tampering if overly trusting the passed in entity. From a performance standpoint, this is also the worst option as you will be serializing and transmitting entire entity graphs back and forth rather than just the data a view needs.
Fixing your problem with option 2 is generally as easy as Eager Loading anything the calling code might touch. For example, if an Person has a reference to an Address:
public IEnumerable<Person> GetPersons()
{
using (var db = new AppContext())
{
Logging.Log(Information, "Requested getPersons from service");
return db.Persons
.Include(x => x.Address)
.ToList();
}
}
However, here is where the complexity creeps in. This requires the method to know about how the Persons might be consumed. Person might have references to five or more other objects/sets, and those objects may have references to more. (I.e. Address having a reference to a Country entity, or AddressType, etc.) The catch-all fix ends up being to eager load everything, which results in a lot of unnecessary data and is prone to bugs as entities evolve and code grows to rely on different bits. Then when performance becomes a problem you start diving down a rabbit hole in trying to support the caller telling this method what to eager load. As the system grows and you want to support pagination, sorting, etc. the method becomes more and more complex or it becomes slower and slower.
The solution I advocate for is to ensure that entities never cross the boundary of their DbContext, and leveraging Linq/EF's IQueryable implementation in combination with a unit of work so that consumers of these repository or service methods can be responsible for the scope of the DbContext. For example:
private AppDbContext Context
{
get { return AmbientDbContextLocator.Get<AppDbContext>(); }
}
public IQueryable<Person> GetPersons()
{
Logging.Log(Information, "Requested getPersons from service");
return Context.Persons.AsQueryable();
}
Then in calling code:
var mapperConfig = new MapperConfiguration(cfg =>
{
cfg.CreateMap<Person, PersonViewModel>(); // Can include any relevant mapping to flatten needed fields...
});
using( var contextScope = ContextScopeFactory.Create())
{
var viewModels = PersonRepository.GetPersons()
.ProjectTo<PersonViewModel>(mapperConfig)
.ToList();
return View(viewModel);
}
Often you will have low-level rules such as defaulting to only returning Active rows. .AsQueryable() is only needed if you have no rules. Anything that results in a .Where or such inside the repository/service method returns IQueryable so it could return:
return Context.Persons.Where(x => x.IsActive);
Repositories can enforce low level filtering rules like IsActive, authorization checks, etc. leaving more variable filtering, sorting, etc. up to the consumers.
The advantage of this approach is that the consumer (Controller, etc.) has full control over how the data is consumed without introducing any complexity/business logic into the repository/service. If I want to support sorting and pagination:
var viewModels = PersonRepository.GetPersons()
.OrderBy(x => x.Age)
.ProjectTo<PersonViewModel>(mapperConfig)
.Skip(pageNumber * pageSize)
.Take(pageSize)
.ToList();
The key point is that the entity (Person) doesn't leave the scope of the unit of work / DbContext, it is projected into a ViewModel which holds no references to entities, only data. That model can be safely serialized and represents only the data that the view needs. For projection I've used Automappper's ProjectTo as an example. You can use Linq's Select as a more manual option. The unit of work pattern I use and have outlined above is Mehdime's DbContextScope.
If you just return db.Persons which is an IEnumerable<Person> to some caller and then dispose the db context before that caller can iterate the result, you will get the exception, due to the lazy nature by default of entity framework. ToList gets around this by forcing iteration immediately before the context is disposed, but still has overhead for object tracking, so combining ToList with AsNoTracking gets you the best performance.
AsNoTracking removes some internal tracking code inside the db context, it does not allow you to pass out the IEnumerable<Person> after disposing the context, you still need ToList for that.
Use the extension method AsNoTracking and then call ToList:
db.Persons.AsNoTracking().ToList();
Or you can disable change tracking on the change tracker property of the db context to make the whole thing read only, which avoids having to call AsNoTracking everywhere. You still need ToList with this approach.
/// <summary>
/// Disable all change tracking - place in db context sub class
/// </summary>
public void DisableChangeTracking()
{
ChangeTracker.AutoDetectChangesEnabled = false;
ChangeTracker.LazyLoadingEnabled = false;
ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
}
It is acceptable to do so. The problem with IEnumerable is that Entity Framework supports deferred execution, so they won't be populated until you fetch them to a concrete collection (i.e. .ToList()), which could happen after it's been disposed of.
Your 2 possibilities are:
Executing the work on the deferred object by fetching it to a concrete collection (.ToList()).
Using dependency injection with a scoped Lifetime to let it live until the request completes.
Both are correct, and which one to prefer depends on your use-case.
I've seen this question asked a million times, but every single suggestion I've encountered I already seem to have covered.
I have entity framework in my MVC solution, and I have a 'repository' which attempts to retrieve a collection of MyObjects:
public static List<MyObject> GetMyObjects()
{
using (var context = new MyEntities())
{
return context.MyObjects.Include("Contact").OrderByDescending(o => o.Date).ToList();
}
}
I call this method as part of a controller action which attempts to serialize it:
public JsonResult All()
{
return Json(MyRepository.GetMyObjects(), JsonRequestBehavior.AllowGet);
}
And I get the following error:
The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.
I don't know where to turn on this one, I appreciate entity framework 'lazy-loads' relational data sets if and when they're needed, and I appreciate attempting to serialize the whole collection would indeed attempt to load these (outside of the using statement), but I have the 'Include' there.
What I've Tried
I've tried the 'include', I've also ensured no other associations are part of the MyObject class (i.e. I have no other Include()s to write).
To avoid this you have some options.Don't declare your navigation properties as virtual or disable Lazy Loading behavior on your context. Lazy loading is enable by default and is achieved by creating instances of derived proxy types and then overriding virtual properties to add the loading hook. So, if you want to work with a serializer I recommend you turn off lazy loading:
public class YourContext : DbContext
{
public YourContext()
{
this.Configuration.LazyLoadingEnabled = false;
}
}
These links can help you to understand better what I explain in my answer:
Loading Related Entities
Requirements for Creating POCO Proxies
If you remove the virtual keyword from your navigation properties, the POCO entity not meet the requirements described in the second link, so, EF won't create a proxy class to lazy load your navigation properties. But if you disabled lazy loading, even when your navigation properties are virtual, they won't be loaded in any entity. It's good idea disable lazy loading when you are using a serializer. Most serializers work by accessing each property on an instance of a type.
As a third option you could use the JsonIgnore attribute on your navigation properties that you don't want to be serialized as part of your entity, but as I said before, the best option is disable lazy loading.
When you use Include and using Lazy loading, and wrap the dbContext in a Using statement, then once it tries to get the linked objects, the dbContext is already disposed.
You can try eager loading of the navigation property like this:
IQueryable<MyObjects> query = db.MyObjects.Include(m => m.Contact);
Or you could take out the Using statement, as it is limiting your lazy loading...
I had the same problem and solved like below;
I created a new object and put the values I am gonna use after getting object from db.
Example Code:
var listFromDb= db.XTable.Where(x => x.Id > 5).ToList();
var list = new List<Package>();
foreach (var item in listFromDb)
{
var a = new Package()
{
AccountNo = item.AccountNo,
CreateDate = item.CreateDate,
IsResultCreated = item.IsResultCreated,
};
list.Add(a);
}
For a few days now, I have been struggling with retrieving my entities from a repository (DbContext).
I am trying to save all the entities in an atomic action. Thus, different entities together represent something of value to me. If all the entities are 'valid', then I can save them all to the database. Entity 'a' is already stored in my repository, and needs to be retrieved to 'validate' entity 'b'.
That's where the problem arises. My repository relies on the DbSet<TEntity> class which works great with Linq2Sql (Include() navigation properties e.g.). But, the DbSet<TEntity> does not contain entities that are in the 'added' state.
So I have (as far as I know) two options:
Use the ChangeTracker to see which entities are available and query them into a set based on their EntityState.
Use the DbSet<TEntity>.Local property.
The ChangeTracker seems to involve some extra hard work to get it working in a way such that I can use Linq2Sql to Include() navigation properties e.g.
The DbSet<TEntity>.Local seems a bit weird to me. It might just be the name. I just read something that it is not performing very well (slower than DbSet<> itself). Not sure if that is a false statement.
Could somebody with significant EntityFramework experience shine some light on this? What's the 'wise' path to follow? Or am I seeing ghosts and should I always use the .Local property?
Update with code examples:
An example of what goes wrong
public void AddAndRetrieveUncommittedTenant()
{
_tenantRepository = new TenantRepository(new TenantApplicationTestContext());
const string tenantName = "testtenant";
// Create the tenant, but not call `SaveChanges` yet until all entities are validated
_tenantRepository.Create(tenantName);
//
// Some other code
//
var tenant = _tenantRepository.GetTenants().FirstOrDefault(entity => entity.Name.Equals(tenantName));
// The tenant will be null, because I did not call save changes yet,
// and the implementation of the Repository uses a DbSet<TEntity>
// instead of the DbSet<TEntity>.Local.
Assert.IsNotNull(tenant);
// Can I safely use DbSet<TEntity>.Local ? Or should I play
// around with DbContext.ChangeTracker instead?
}
An example of how I want to use my Repository
In my Repository I have this method:
public IQueryable<TEntity> GetAll()
{
return Context.Set<TEntity>().AsQueryable();
}
Which I use in business code in this fashion:
public List<Case> GetCasesForUser(User user)
{
return _repository.GetAll().
Where(#case => #case.Owner.EmailAddress.Equals(user.EmailAddress)).
Include(#case => #case.Type).
Include(#case => #case.Owner).
ToList();
}
That is mainly the reason why I prefer to stick to DbSet like variables. I need the flexibility to Include navigation properties. If I use the ChangeTracker I retrieve the entities in a List, which does not allow me to lazy load related entities at a later point in time.
If this is close to incomprehensible bullsh*t, then please let me know so that I can improve the question. I desperately need an answer.
Thx a lot in advance!
If you want to be able to 'easily' issue a query against the DbSet and have it find newly created items, then you will need to call SaveChanges() after each entity is created. If you are using a 'unit of work' style approach to working with persistent entities, this is actually not problematic because you can have the unit of work wrap all actions within the UoW as a DB transaction (i.e. create a new TransactionScope when the UoW is created, and call Commit() on it when the UoW completed). With this structure, the changes are sent to the DB, and will be visible to DbSet, but not visible to other UoWs (modulo whatever isolation level you use).
If you don't want the overhead of this, then you need to modify your code to make use of Local at appropriate times (which may involve looking at Local, and then issuing a query against the DbSet if you didn't find what you were looking for). The Find() method on DbSet can also be quite helpful in these situations. It will find an entity by primary key in either Local or the DB. So if you only need to locate items by primary key, this is pretty convenient (and has performance advantages as well).
As mentioned by Terry Coatta, the best approach if you don't want to save the records first would be checking both sources.
For example:
public Person LookupPerson(string emailAddress, DateTime effectiveDate)
{
Expression<Func<Person, bool>> criteria =
p =>
p.EmailAddress == emailAddress &&
p.EffectiveDate == effectiveDate;
return LookupPerson(_context.ObjectSet<Person>.Local.AsQueryable(), criteria) ?? // Search local
LookupPerson(_context.ObjectSet<Person>.AsQueryable(), criteria); // Search database
}
private Person LookupPerson(IQueryable<Person> source, Expression<Func<Person, bool>> predicate)
{
return source.FirstOrDefault(predicate);
}
For those who come after, I ran into some similar issues and decided to give the .Concat method a try. I have not done extensive performance testing so someone with more knowledge than I should feel free to chime in.
Essentially, in order to properly break up functionality into smaller chunks, I ended up with a situation in which I had a method that didn't know about consecutive or previous calls to that same method in the current UoW. So I did this:
var context = new MyDbContextClass();
var emp = context.Employees.Concat(context.Employees.Local).FirstOrDefault(e => e.Name.Contains("some name"));
This may only apply to EF Core, but every time you reference .Local of a DbSet, you're silently triggering change detection on the context, which can be a performance hit, depending on how complex your model is, and how many entries are currently being tracked.
If this is a concern, you'll want to use (fore EFCore) dbContext.ChangeTracker.Entries<T>() to get the locally tracked entities, which will not trigger change detection, but does require manual filtering of the DB state, as it will include deleted and detached entities.
There's a similar version of this in EF6, but in EFCore the Entries is a list of EntityEntries which you'll have to select out the entry.Entity to get out the same data the DbSet would give you.
I'm still fairly new to EF (v4.1), so correct me if I'm wrong, but if I have an InverseProperty as follows:
public virtual ICollection<ItemComment> Comments { get; set; }
This will be lazy loaded when the property is accessed. However, if I wish to filter this list - for example, to get only active comments, I could just add another property as follows:
public IEnumerable<ItemComment> ActiveComments {
get { return Comments.Where(x => x.IsActive); }
}
However, this will load the entire Comments collection first, and then filter right? So not using IQueryable? For performance, ideally I'd like to get the list using IQueryable.
So my question is, can this be done using a property of an entity like this? Or am I going to have to do a where on the ItemComments directly:
var comments = itemCommentRepository.QueryAll()
.Where(x => x.IsActive && x.ItemId == XX).
This will obviously work... but going forward I wonder if there's a better solution?
Update: It seems the entire result set IS loaded, and any filtering would be done on the whole dataset client-side. Aside from hacks, or changing the entity to pass the context in (yuck!), there doesn't appear to be an in-built way to do so. Have marked #Slauma's response as the answer.
this will load the entire Comments collection first, and then filter
right?
Yes.
can this be done using a property of an entity
In theory, by injecting the repository or even a context into the entity constructor. But you would have a dependency of your POCO entities on a data access layer. I would not like this solution.
Your proposed solution is a way. You could alternatively use explicit loading:
itemRepository.LoadActiveComments(item);
Implemented like this:
void LoadActiveComments(Item item)
{
context.Entry(item).Collection(i => i.Comments).Query()
.Where(c => c.IsActive).Load();
}
I wonder if there is a possibility to eager load related entities for certain subclass of given class.
Class structure is below
Order has relation to many base suborder classes (SuborderBase). MySubOrder class inherits from SuborderBase. I want to specify path for Include() to load MySubOrder related entities (Customer) when loading Order, but I got an error claiming that there is no relation between SuborderBase and Customer. But relation exists between MySubOrder and Customer.
Below is query that fails
Context.Orders.Include("SubOrderBases").Include("SubOrderBases.Customers")
How can I specify that explicitly?
Update. Entity scheme is below
This is a solution which requires only a single roundtrip:
var orders = Context.Orders
.Select(o => new
{
Order = o,
SubOrderBases = o.SubOrderBases.Where(s => !(s is MyOrder)),
MyOrdersWithCustomers = o.SubOrderBases.OfType<MyOrder>()
.Select(m => new
{
MyOrder = m,
Customers = m.Customers
})
})
.ToList() // <- query is executed here, the rest happens in memory
.Select(a =>
{
a.Order.SubOrderBases = new List<SubOrderBase>(
a.SubOrderBases.Concat(
a.MyOrdersWithCustomers.Select(m =>
{
m.MyOrder.Customers = m.Customers;
return m.MyOrder;
})));
return a.Order;
})
.ToList();
It is basically a projection into an anonymous type collection. Afterwards the query result is transformed into entities and navigation properties in memory. (It also works with disabled tracking.)
If you don't need entities you can omit the whole part after the first ToList() and work directly with the result in the anonymous objects.
If you must modify this object graph and need change tracking, I am not sure if this approach is safe because the navigation properties are not completely set when the data are loaded - for example MyOrder.Customers is null after the projection and then setting relationship properties in memory could be detected as a modification which it isn't and cause trouble when you call SaveChanges.
Projections are made for readonly scenarios, not for modifications. If you need change tracking the probably safer way is to load full entities in multiple roundtrips as there is no way to use Include in a single roundtrip to load the whole object graph in your situation.
Suppose u loaded the orders list as lstOrders, try this:
foreach (Orders order in lstOrders)
order.SubOrderBases.Load();
and the same for the customers..