I'm still fairly new to EF (v4.1), so correct me if I'm wrong, but if I have an InverseProperty as follows:
public virtual ICollection<ItemComment> Comments { get; set; }
This will be lazy loaded when the property is accessed. However, if I wish to filter this list - for example, to get only active comments, I could just add another property as follows:
public IEnumerable<ItemComment> ActiveComments {
get { return Comments.Where(x => x.IsActive); }
}
However, this will load the entire Comments collection first, and then filter right? So not using IQueryable? For performance, ideally I'd like to get the list using IQueryable.
So my question is, can this be done using a property of an entity like this? Or am I going to have to do a where on the ItemComments directly:
var comments = itemCommentRepository.QueryAll()
.Where(x => x.IsActive && x.ItemId == XX).
This will obviously work... but going forward I wonder if there's a better solution?
Update: It seems the entire result set IS loaded, and any filtering would be done on the whole dataset client-side. Aside from hacks, or changing the entity to pass the context in (yuck!), there doesn't appear to be an in-built way to do so. Have marked #Slauma's response as the answer.
this will load the entire Comments collection first, and then filter
right?
Yes.
can this be done using a property of an entity
In theory, by injecting the repository or even a context into the entity constructor. But you would have a dependency of your POCO entities on a data access layer. I would not like this solution.
Your proposed solution is a way. You could alternatively use explicit loading:
itemRepository.LoadActiveComments(item);
Implemented like this:
void LoadActiveComments(Item item)
{
context.Entry(item).Collection(i => i.Comments).Query()
.Where(c => c.IsActive).Load();
}
Related
Currently I am using this code:
opportunity.Contacts.Where(x => x.IsDeleted = false).IsNullOrEmpty()
in each entity to check if any collection exists in the entity (example Opportunity).
public bool Delete(int companyId, int opportunityId)
{
var opportunity = _opportunityRepository.FindOne(companyId: companyId, opportunityId: opportunityId).FirstOrDefault();
if (!opportunity.Contacts.Where(x => x.IsDeleted = false).IsNullOrEmpty())
{
throw new UserFriendlyException(ErrorMessages.UserFriendly.UnableToDeleteEntityHasRelatedData);
}
opportunity.IsDeleted = true;
_opportunityRepository.Edit(opportunity);
CurrentUnitOfWork.Commit();
return true;
}
This method is repetitive and time consuming to put in hundreds of places.
How can we make it a generic function which can check entity type and use reflection or another way to check all its properties which implement ICollection<T> and execute query to check their count?
[ForeignKey("DepartmentId")]
public virtual ICollection<DepartmentLocation> DepartmentLocations { get; set; }
[ForeignKey("DepartmentId")]
public virtual ICollection<EmployeePosition> EmployeePositions { get; set; }
Even if you did want to go through and resolve collections by reflection to perform a check like this, I would not recommend it. The problem domain you are looking at looks like that you want to enforce business logic around soft deletion that essentially says "I can only be marked as deleted if all of my children are first marked as deleted." The issues you would quickly face with trying to do this through reflection include:
Hits to Lazy Loading
Complex and slow code to inspect every object on delete.
The lazy loading risk and trying to escape it would be my first warning of dragons. I'd be looking at leveraging IQueryable from the repository rather than returning a single domain object. From there you have the flexibility to drill down into your model to determine if an object has any active children:
var activeState = _opportunityRepository.GetById(companyId, opportunityId) // return IQueryable<Opportunity>
.Select(o=> new {o.IsDeleted, HasActiveContact = o.Contacts.Any(c=> !c.IsDeleted)})
.SingleOrDefault();
From there you can check properties on the anon. type. The query sent to the server should remain pretty optimal in terms of performance with a single hit and the code is a simple, extendable structure that you can expand upon easily without worrying about lazy loads and such. You can expand it to return active children for instance if you want to extend a message about which children are still active to deal with before marking the parent as deleted.
I have a problem with EF6 when trying to optimize the queries. Consider this class with one collection:
public class Client
{
... a lot of properties
public virtual List<Country> Countries { get; set; }
}
As you might know, with Lazy Loading I have this n+1 problem, when EF tries to get all the Countries, for each client.
I tried to use Linq projections; for example:
return _dbContext.Clients
.Select(client => new
{
client,
client.Countries
}).ToList().Select(data =>
{
data.client.Countries = data.Countries; // Here is the problem
return data.client;
}).ToList();
Here I'm using two selects: the first for the Linq projection, so EF can create the SQL, and the second to map the result to a Client class. The reason for that is because I'm using a repository interface, which returns List<Client>.
Despite the query is generated with the Countries in it, EF still is using Lazy Loading when I try to render the whole information (the same n+1 problem). The only way to avoid this, is to remove the virtual accessor:
public class Client
{
... a lot of properties
public List<Country> Countries { get; set; }
}
The issue I have with this solution is that we still want to have this property as virtual. This optimization is only necessary for a particular part of the application, whilst on the other sections we want to have this Lazy Loading feature.
I don't know how to "inform" EF about this property, that has been already lazy-loaded via this Linq projection. Is that possible? If not, do we have any other options? The n+1 problems makes the application to take several seconds to load like 1000 rows.
Edit
Thanks for the responses. I know I can use the Include() extension to get the collections, but my problem is with some additional optimizations I need to add (I'm sorry for not posting the complete example, I thought with the Collection issue would be enough):
public class Client
{
... a lot of properties
public virtual List<Country> Countries { get; set; }
public virtual List<Action> Actions { get; set; }
public virtual List<Investment> Investments { get; set; }
public User LastUpdatedBy {
get {
if(Actions != null) {
return Actions.Last();
}
}
}
}
If I need to render the clients, the information about the last update and the number of investments (Count()), with the Include() I practically need to bring all the information from the database. However, if I use the projection like
return _dbContext.Clients
.Select(client => new
{
client,
client.Countries,
NumberOfInvestments = client.Investments.Count() // this is translated to an SQL query
LastUpdatedBy = client.Audits.OrderByDescending(m => m.Id).FirstOrDefault(),
}).ToList().Select(data =>
{
// here I map back the data
return data.client;
}).ToList();
I can reduce the query, getting only the required information (in the case of LastUpdatedBy I need to change the property to a getter/setter one, which is not a big issue, as its only used for this particular part of the application).
If I use the Select() with this approach (projection and then mapping), the Include() section is not considered by EF.
If i understand correctly you can try this
_dbContext.LazyLoading = false;
var clientWithCountres = _dbContext.Clients
.Include(c=>c.Countries)
.ToList();
This will fetch Client and only including it Countries. If you disable lazy-loading the no other collection will load from the query. Unless you are specifying a include or projection.
FYI : Projection and Include() doesn't work together see this answer
If you are projection it will bypass the include.
https://stackoverflow.com/a/7168225/1876572
don't know what you want to do, you are using lambda expression not linq, and your second select it's unnecessary.
data.client is client, data.Countries is client.Countries, so data.client.Countries = data.Countries alway true.
if you don't want lazy load Countries, use _dbContext.Clients.Include("Countries").Where() or select ().
In order to force eager loading of virtual properties you are supposed to use Include extension method.
Here is a link to MSDN https://msdn.microsoft.com/en-us/library/jj574232(v=vs.113).aspx.
So something like this should work:
return _dbContext.Clients.Include(c=>c.Countries).ToList();
Im not 100% sure but I think your issue is that you are still maintaining a queryable for your inner collection through to the end of the query.
This queryable is lazy (because in the model it was lazy), and you havent done anything to explain that this should not be the case, you have simply projected that same lazy queryable into the result set.
I cant tell you off the top of my head what the right answer here is but I would try things around the following:
1 use a projection on the inner queriable also eg
return _dbContext.Clients
.Select(client => new
{
client,
Countries = client.Countries.Select(c=>c)// or a new Country
})
2 Put the include at the end of the query (Im pretty sure include applies to the result not the input. It definitally doesnt work if you put it before a projection) eg:
_dbContext.Clients
.Select(client => new
{
client,
client.Countries
}.Include(c=>c.Countries)`
3 Try specifying the enumeration inside the projection eg:
_dbContext.Clients
.Select(client => new
{
client,
Countries = client.Countries.AsEnumerable() //perhaps tolist if it works
}`
I do want to caviat this by saying that I havent tried any of the above but I think this will set you on the right path.
A note on lazy loading
IMO there are very few good use cases for lazy loading. It almost always causes too many queries to be generated, unless your user is following a lazy path directly on the model. Use it only with extreme caution and IMO not at all in request response (eg web) apps.
WebApi
Let's have a Person class with many one-to-many relations to classes like Car, Pet, Children ...
and PersonRepository where i have to include all relations and their relations or just disable lazy-loading
public Person getPerson()
{
...
using (db = new DbContext())
{
// var person = linq-query
// return person;
}
...
}
So in my controller i have fully loaded Person with relations and their relations.
If there is array of objects with many nested relations it is inefficient and memory demanding.
How i can lazy-load properties outside of repository or what is general solution to this problem?
How i can lazy-load properties outside of repository or what is general solution to this problem?
If you mean to know how to load navigational properties after disabling Lazy Loading on your context, then you have the following solutions:
Eager Loading by using Include methode of your DbSet like this:
db.Persons.Include(p => p.Cars).Include(p => p.Pets).Include(p => p.Children).Where(p => p.Id == personId);
Explicit Loading by using the change tracker and Load method on your entry like this : db.entry(person).Collection(p => p.Cars).Load();
The solution that I see a lot with EF and Repository Pattern is to use a collection lambda expression as a parameter of your method like this :
public Person GetPerson(params Expression<Func<Person, object>>[] includes)
{
using (var db = new DbContext())
{
IQueryable<Person> query = db.Posts;
Array.ForEach(includes, expression => query = query.Include(expression));
return query.FirstOrDefault();
}
}
You use it like this : GetPerson(p => P.Cars, p => p.Pets, p => p.Children) With this solution you only load the navigational properties you need by specifing them as parameters of your method.
You can use ajax to populate data on demand.
Like this:
https://cmatskas.com/update-an-mvc-partial-view-with-ajax/
Happy to help you!
As far as I know there is not the single general solution.
So you can try these cases:
Don't close read-only DbContext (just don't forget disable tracking). In some scenarios like web request you can create DbContext at the start of request, and dispose it and the end.
Forget for memory demanding. If your plan to read 2Mb data from SQL, the something wrong with your design. 5Kb, 30Kb are not the big data.
Think about cache. Let the necessary data will available always.
Refactor your interfaces to load necessary data only. F.e. append methods LoadPersonsWithAddresses, LoadPersonsWithFamily, and so on.
Explicitly call load method on your collection. This forces lazy load.
I've seen this question asked a million times, but every single suggestion I've encountered I already seem to have covered.
I have entity framework in my MVC solution, and I have a 'repository' which attempts to retrieve a collection of MyObjects:
public static List<MyObject> GetMyObjects()
{
using (var context = new MyEntities())
{
return context.MyObjects.Include("Contact").OrderByDescending(o => o.Date).ToList();
}
}
I call this method as part of a controller action which attempts to serialize it:
public JsonResult All()
{
return Json(MyRepository.GetMyObjects(), JsonRequestBehavior.AllowGet);
}
And I get the following error:
The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.
I don't know where to turn on this one, I appreciate entity framework 'lazy-loads' relational data sets if and when they're needed, and I appreciate attempting to serialize the whole collection would indeed attempt to load these (outside of the using statement), but I have the 'Include' there.
What I've Tried
I've tried the 'include', I've also ensured no other associations are part of the MyObject class (i.e. I have no other Include()s to write).
To avoid this you have some options.Don't declare your navigation properties as virtual or disable Lazy Loading behavior on your context. Lazy loading is enable by default and is achieved by creating instances of derived proxy types and then overriding virtual properties to add the loading hook. So, if you want to work with a serializer I recommend you turn off lazy loading:
public class YourContext : DbContext
{
public YourContext()
{
this.Configuration.LazyLoadingEnabled = false;
}
}
These links can help you to understand better what I explain in my answer:
Loading Related Entities
Requirements for Creating POCO Proxies
If you remove the virtual keyword from your navigation properties, the POCO entity not meet the requirements described in the second link, so, EF won't create a proxy class to lazy load your navigation properties. But if you disabled lazy loading, even when your navigation properties are virtual, they won't be loaded in any entity. It's good idea disable lazy loading when you are using a serializer. Most serializers work by accessing each property on an instance of a type.
As a third option you could use the JsonIgnore attribute on your navigation properties that you don't want to be serialized as part of your entity, but as I said before, the best option is disable lazy loading.
When you use Include and using Lazy loading, and wrap the dbContext in a Using statement, then once it tries to get the linked objects, the dbContext is already disposed.
You can try eager loading of the navigation property like this:
IQueryable<MyObjects> query = db.MyObjects.Include(m => m.Contact);
Or you could take out the Using statement, as it is limiting your lazy loading...
I had the same problem and solved like below;
I created a new object and put the values I am gonna use after getting object from db.
Example Code:
var listFromDb= db.XTable.Where(x => x.Id > 5).ToList();
var list = new List<Package>();
foreach (var item in listFromDb)
{
var a = new Package()
{
AccountNo = item.AccountNo,
CreateDate = item.CreateDate,
IsResultCreated = item.IsResultCreated,
};
list.Add(a);
}
I am currently working with Entity Framework 4 on a project that is using Table Per Hierarchy to represent one set of classes using a single table. The way this works is that this table represents states and different states are associated with different other classes.
So you might imagine it to look like this, ignoring the common fields all states share:
InactiveState
has a -> StateStore
ActiveState
has a -> ActivityLocation
CompletedState
has a -> CompletionDate
Each state has a collection of all the InventoryItems that belong to it.
Now each item in my inventory has many states, where the last one in the history is the current state. To save on lists I have a shortcut field that points to the current state of my Inventory:
public class InventoryItem : Entity
{
// whole bunch of other properties
public virtual ICollection<InventoryState> StateHistory { get; set; }
public virtual InventoryState LastState { get; set; }
}
The first problem I am having is when I want to find, for example, all the InventoryItems which are in the Active state.
It turns out Linq-To-Entities doesn't support GetType() so I can't use a statement like InventoryRepository.Get().Where( x => x.LastState.GetType() == someType ). I can use the is operator, but that requires a fixed type so rather than being able to have a method like this:
public ICollection<InventoryItem> GetInventoryItemsByState( Type state )
{
return inventoryRepository.Get().Where( x => x.LastState is state );
}
I have to run some kind of if statement based on the type before I make the Linq query, which feels wrong. The InventoryItem list is likely to get large, so I need to do this at the EF level, I can't pull the whole list into memory and use GetType(), for example.
I have two questions in this situation, connected closely enough that I think they can be combined as they probably reflect a lack of understanding on my part:
Is it possible to find a list of items that share a child table type using Linq To Entities?
Given that I am not using Lazy Loading, is it possible to Include related items for child table types using TPH so that, for example, if I have an InactiveState as the child of my InventoryItem I can preload the StateStore for that InactiveState?
Is it possible to find a list of items that share a child table type
using Linq To Entities?
I don't think it's possible in another way than using an if/switch that checks for the type and builds a filter expression using is T or OfType<T>. You could encapsulate this logic into an extension method for example to have a single place to maintain and a reusable method:
public static class Extensions
{
public static IQueryable<InventoryItem> WhereIsLastState(
this IQueryable<InventoryItem> query, Type state)
{
if (state == typeof(InactiveState))
return query.Where(i => i.LastState is InactiveState);
if (state == typeof(ActiveState))
return query.Where(i => i.LastState is ActiveState);
if (state == typeof(CompletedState))
return query.Where(i => i.LastState is CompletedState);
throw new InvalidOperationException("Unsupported type...");
}
}
To be used like this:
public ICollection<InventoryItem> GetInventoryItemsByState(Type state)
{
return inventoryRepository.Get().WhereIsLastState(state).ToList();
}
I don't know if it would be possible to build the i => i.LastState is XXX expression manually using the .NET Expression API and based on the Type passed into the method. (Would interest me too, to be honest, but I have almost no clue about expression manipulation to answer that myself.)
Given that I am not using Lazy Loading, is it possible to Include
related items for child table types using TPH so that, for example, if
I have an InactiveState as the child of my InventoryItem I can preload
the StateStore for that InactiveState?
I am not sure if I understand that correctly but generally eager loading with Include does not support any filtering or additional operations on specific included children.
One way to circumvent this limitation and still get the result in a single database roundtrip is using a projection which would look like this:
var result = context.InventoryItems
.Select(i => new
{
InventoryItem = i,
LastState = i.LastState,
StateStore = (i.LastState is InactiveState)
? (i.LastState as InactiveState).StateStore
: null
})
.AsEnumerable()
.Select(x => x.InventoryItem)
.ToList();
If the query is a tracked query (which it is in the example above) and the relationships are not many-to-many (they are not in your example) the context will fixup the relationships when the entities are loaded into the context, that is InventoryItem.LastState and InventoryItem.LastState.StateStore (if LastState is of type InactiveState) will be set to the loaded entities (as if they had been loaded with eager loading).