Most of the examples I see on the internet show the navigation properties as either ICollection or straight List implementation. They are usually virtual, to enable lazy-loading.
However, when you access such property, it will load the entire collection in memory and if you have a subquery after it (i.e. object.MyListProperty.Where(...)) I have noticed that an SQL query will be issued for each item in the MyListProperty.
How do I avoid this? I want the where clause after the list property to execute on the SQL server, if possible. Can I use an IQueryable navigation property? Is there any best-practice for such case?
My advice for best practise is to disable Lazy loading altogether. Instead force the caller to eagerly load navigation properties through include statements or by using projections.
There are 3rd party products that support include with filters, as described in this post: How to filter include entities in entity framework, but in my experience this further complicates down-stream processing of the objects that are retrieved. If the entity object is loaded outside of method X, because method X can't know for sure if the navigation properties have been loaded with the correct filters, method X starts off by re-querying for the precise rows that it knows it needs.
using (var context = new MyDbContext())
{
context.Configuration.LazyLoadingEnabled = false;
// TODO: load your data
...
}
In this way the records will only be loaded when they are explicitly requested.
When you want access to an IQueryable so you can defer the loading of the data, then make those queries against the DbContext instance and not from the object.
In this example assume that a Customer has many thousands of transactions, so we don't want them to be eagerly or lazy loaded at all.
using (var context = new MyDbContext())
{
context.Configuration.LazyLoadingEnabled = false;
var customer = context.Customers.First(x => x.Id == 123);
...
// count the transactions in the last 30 days for this customer
int customerId = customer.Id;
DateTime dateFrom = DateTime.Today.AddDays(-30)
// different variations on the same query
int transactionCount1 = context.Customers.Where(x => x.Id == customerId)
.SelectMany(x => x.Transactions.Where(x => x.TransactionDate >= dateFrom))
.Count();
int transactionCount2 = context.Customers.Where(x => x.Id == customerId)
.SelectMany(x => x.Transactions)
.Where(x => x.TransactionDate >= dateFrom)
.Count();
int transactionCount3 = context.Transactions.Where(x => x.CustomerId == customerId)
.Where(x => x.TransactionDate >= dateFrom)
.Count();
}
It is good that you have identified that you want to use an IQueryable<T> we access them from the DbContext directly, not from the instances that were previously retrieved.
Related
There is a list of managers (in fact a history), where current is marked as property with TillDate == null and IsIsCurrentManager = true.
db.MyManagers.Remove(db.MyManagers
.Where(e => e.ProjectId == projectId
&& e.MyManagerId == Id).First());
var newCurrentManager = db.MyManagers
.Where(e => e.ProjectId == projectId)
.OrderByDescending(i => i.FromDate)
.FirstOrDefault();
newCurrentManager.TillDate = null;
newCurrentManager.IsCurrentManager = true;
db.SaveChanges();
The problem is that Remove() method will only mark entity as deleted, but, as far as i can see, it still will be added to the list. How can deleted 1 record and than build a list of managers without it, without using SaveChanges() 2 times or hardcoding MyManagerList[2]
It's important to remember that Entity Framework is just an interface into your data store and all you are usually doing is building queries, queries that don't actually run until you materialise the results of a select (i.e. by enumerating through the results or calling something like ToList() or Single()) or, in the case of an update/delete operation, when you call SaveChanges().
So while you can do something like checking that the entity is tracked in the context as a deleted item, you are probably much better off sending the results to the database by calling SaveChanges() twice.
The secondary benefit of that is that another query that doesn't use the same context object (e.g. another web request) will be able to see that the item has been deleted while the original query is trying to pull the list of managers.
You can query the Entity State to exclude it from the second list.
I also have updated the linq request with some cleaning.
Also there is no chek for null, your db query can return null and should be managed as such.
db.MyManagers.Remove(db.MyManagers
.FirstOrDefault(e => e.ProjectId == projectId
&& e.MyManagerId == Id));
var newCurrentManager = db.MyManagers
.ToList()
.Where(e => e.ProjectId == projectId
&& db.Entry(e).State != EntityState.Deleted)//This will query the entity traker
.OrderByDescending(i => i.FromDate)
.FirstOrDefault();
newCurrentManager.TillDate = null;
newCurrentManager.IsCurrentManager = true;
db.SaveChanges();
The current setup of EF in my application is lazy loading, which is great for the most part. However, I am lost trying work out how to load a list of related entities based on their IsEnabled bit attribute.
In this example I am just returning a list of entities.
return Context.Entities.ToList()
Let's say the Entities object contains a list of ChildEntities like so:
public class Entities
{
private string EntityName;
private List<ChildEntities> ChildEntities;
}
public class ChildEntites
{
private string ChildEntityName;
private bool IsEnabled;
}
I want to only want to get out the ChildEntities based on their IsEnabled flag when loading the list of Entities.
You can use the Include() method to load all child entities and then select only those which are enabled like
Context.Entities.Include("ChildEntites").Select(c => e.IsEnabled == true)
Another way would be to get the filter entities and then run the query as shown in this post
var data = from e in Context.Entities
select new
{
Entities = e,
Childs = e.ChildEntites.Where(c => c.IsEnabled == true)
};
var Results = data.ToArray().Select(x => x.Entities);
I think there is no way to filter when you load related entities in case that you use lazy loading or eager loading unless you project your query to an anonymous type or a DTO, but if you have an entity instance,you can load related entities based on a condition using explicit loading:
var entity=context.Entities.FirstOrDefault();
context.Entry(entity)
.Collection(b => b.ChildEntities)
.Query()
.Where(ce => ce.IsEnabled == true)
.Load();
If that doesn't satisfy what you are trying to achieve because you need to load the entire entity collection, then, as I said before, you should project your query to a custom class or an anonymous type:
var query= from e in Context.Entities.Include(c=>c.ChildEntities)
select new EntityDTO
{
EntityName= e.EntityName,
ChildEntites= e.ChildEntites.Where(c => c.IsEnabled == true)
};
Using a projection
var entities = context.Entities
.Select(x => new {x, x.ChildEntities.Where(y => y.IsEnabled))
.ToList() // resolve from database before selecting the main entity
.Select(x => x.x);
Using a third party library
EF+ Query IncludeFilter allow you to easily filter related entities
var entities = context.Entities.IncludeFilter(x => x.ChildEntities.Where(y => y.IsEnabled))
.ToList();
You can find the documentation here
Disclaimer: I'm the owner of the project EF+.
A couple ways I would recommend this approach. I would either lazy load where IsEnabled = false and eager load in a separate call where IsEnabled = true OR once you have the lazy loaded collection, in a separate call, get the children where IsEnabled = true. I don't believe you will be able to do this in one call. The other option would be a stored procedure. I hope this helps.
I had a similar problem. I solved it in the following manner.
Create a new method in your model Entities. Lets call it ChildEntitiesEnabled
public ICollection<ChildEntity> ChildEntitiesEnabled()
{
//First I get the full list using the lazy loading...
var allChildEntities=ChildEntities.ToList();
//do further processing if there is data
if(allChildEntities!=null && allChildEntities.Count()>0)
{
var childEntitiesEnabled = ChildEntities.Where(x=>x.Enabled==true).ToList();
return childEntitiesEnabled;
}
return null; //or you can return an empty list...
}
I like this method because you can use it anywhere the model is available without complicated code strewn all over the place. Also, you dont lose all the ChildEntities data... that is available too from the original call.
I have the following model:
A User has a collection of Photos. In the Photo model, there is a property called IsProfilePhoto.
When I do the following, the results are not as expected.
var user = dbContext.Users.SingleOrDefault(u => u.Id == 1);
var profilePhoto = user.Photos.SingleOrDefault(p => p.IsProfilePhoto);
With lazy loading on, this performs two queries.
The first one gets the user by id as expected.
The second one however, gets the collection of photos by user id and then in memory does the match on IsProfilePhoto.
I was hoping that with lazy loading on it would add the SingleOrDefault to the query as well.
Is this just not possible and I must always do the inverse? E.g.
var profilePhoto = dbContext.Photos.SingleOrDefault(p => p.UserId == 1 && p.IsProfilePhoto);
var user = profilePhoto.User;
I get the reasoning, there are just certain reasons why it's more convenient to go from the User to get the profile photo.
You can get the result with a single database query by using a projection:
var userWithProfilePhoto = dbContext.Users
.Where(u => u.Id == 1)
.Select(u => new
{
User = u,
ProfilePhoto = u.Photos.Where(p => p.IsProfilePhoto).FirstOrDefault()
})
.SingleOrDefault();
userWithProfilePhoto.User and userWithProfilePhoto.ProfilePhoto are the two entities you are looking for.
You have to use Eagerly loading to load multiple levels. Lazy load, loads the level when you access this.
var user = dbContext.Users.Include(u => u.Photos).SingleOrDefault(u => u.Id == 1);
var profilePhoto = user.Photos.SingleOrDefault(p => p.IsProfilePhoto);
This is the subtle difference in LINQ methods.
You can do the filtering as part of the query as:
var profilePhoto = user.Photos.Where(p => p.IsProfilePhoto).SingleOrDefault();
Due to this behavior in LINQ to Entities, I try to always use the Where method for the condition and the parameterless overloads for First, Single, FirstOrDefault, SingleOrDefault, Any, and Count.
Edit:
My bad, MSDN mentions it directly, but any reference to a navigation property (when lazy loading is enabled) loads all of the related records.
My best suggestion then is to a) accept extra database access, or b) query as you did in your alternative example, with the first table being the 'many' of the 'one-to-many' relationship.
First, and FirstOrDefault are lazy loaded, Single, and SingleOrDefault eagerly loaded. If you don't need a an exception thrown in case of several items returned by the query, you can change it to FirstOrDefault.
So I have this query in my repository (also using Unit of Work pattern) which uses eager loading to make one hit to the database:
from g in _context.Games.Include(pg => pg.PreviousGame).Include(go => go.GameObjects)
where EntityFunctions.DiffMilliseconds(DateTime.Now, g.EndDate) > 0
&& g.GameTypeId == (int)GameTypes.Lottery
&& g.GameStatusId == (int)GameStatues.Open
select new LotteryModel
{
EndDate = g.EndDate,
GameId = g.Id,
PreviousGameEndDate = g.PreviousGame.EndDate,
PreviousGameId = g.PreviousGameId.HasValue ? g.PreviousGameId.Value : 0,
PreviousGameStartDate = g.PreviousGame.StartDate,
PreviousWinningObjectCount = g.PreviousGame.GameObjects.Select(go => go.Object.Count).FirstOrDefault(),
PreviousWinningObjectExternalVideoId = g.PreviousGame.GameObjects.Select(go => go.Object.Video.ExternalVideoId).FirstOrDefault(),
PreviousWinningObjectName = g.PreviousGame.GameObjects.Select(go => go.Object.Video.Name).FirstOrDefault(),
StartDate = g.StartDate,
WinningObjectCount = g.GameObjects.Select(go => go.Object.Count).FirstOrDefault(),
WinningObjectExternalVideoId = g.GameObjects.Select(go => go.Object.Video.ExternalVideoId).FirstOrDefault(),
WinningObjectName = g.GameObjects.Select(go => go.Object.Video.Name).FirstOrDefault()
};
However I'm reluctant to use this because I now have to create a separate LotteryModel object to return up throughout my other layers.
I would like to be able to return an entity of type "Game" which has all of the navigational methods to all of my other data (PreviousGame, GameObjects, etc) and then map the needed properties to my flat view model, but when I do this it seems to only lazy load the objects and then I have the additional hits to the DB.
Or do I have this wrong and whenever I need to return heirarchical data I should return it through my LINQ query in the select portion?
My basic goal is to reduce the hits to the DB.
I don't really understand the problem. You return your Games object and you can access the properties and subobjects off it. Your use of the Include() method tells it to load what you need, and not lazy load it.
Make sure you return a single object via a .First, .FirstOrDefault, .Single, .SingleOrDefault, or similar methods.
I ended up with this query (FYI I'm using the System.Data.Objects namespace for the Include extension):
(from g in _context.Games.Include(pg => pg.PreviousGame.GameObjects.Select(o => o.Object.Video))
.Include(go => go.GameObjects.Select(o => o.Object.Video))
where EntityFunctions.DiffMilliseconds(DateTime.Now, g.EndDate) > 0
&& g.GameTypeId == (int)GameTypes.Lottery
&& g.GameStatusId == (int)GameStatues.Open
select g).FirstOrDefault();
I guess I just needed to include more of the heirarchy and didn't know I could use Select() in the Include() function!
I would think this is very basic stuff, but I'm just not getting it to work.
I try to get a list of objects using lambda expressions like this :
List<LocalizationGlobalText> list = _entities.LocalizationGlobalTexts.Where(l => l.Language.Id == _currentlanguage).ToList<LocalizationGlobalText>();
The list is fetched, but the foreign key objects are all null.
I also tried using LINQ to entities but this results in the same problem :
IEnumerable<LocalizationGlobalText> bla = (from lgt in _entities.LocalizationGlobalTexts
join lg in _entities.LocalizationGlobals on lgt.IdLocalizationGlobal equals lg.Id
where lgt.IdLanguage == _currentlanguage
select lgt);
By default, Entity Framework only brings in the collection that you specify, without any foreign objects. If you have lazy loading enabled, accessing the foreign properties will cause them to be lazily initialized. If not, you'll need to tell entity framework to eagerly load the properties you want with the first batch.
There are two ways to do this. The first is the "official" way, but I don't like it because it uses magic strings:
var list = _entities.LocalizationGlobalTexts.Include("ForeignProp")
.Where(l => l.Language.Id == _currentlanguage)
.ToList<LocalizationGlobalText>();
(Replace "ForeignProp" with the name of the property you want it to eagerly load)
The second way is to set up your selector so that it will be forced to pull this extra data in:
var list = _entities.LocalizationGlobalTexts
.Where(l => l.Language.Id == _currentlanguage)
.Select(l => new {l, l.ForeignProp})
.ToList();
foreach(var item in list)
{
Console.WriteLine(item.l.Name + item.ForeignProp.Title);
}
Since Entity Framework is smart enough to have made the appropriate connections, you could throw on one more selector and avoid using the anonymous type afterward:
var list = _entities.LocalizationGlobalTexts
.Where(l => l.Language.Id == _currentlanguage)
.Select(l => new {l, l.ForeignProp})
.AsEnumerable() // tells EF to load now. The rest is LINQ to Objects
.Select(i => i.l)
.ToList();
foreach(var localization in list)
{
Console.WriteLine(localization.Name + localization.ForeignProp.Title);
}