Entity framework: How to reduce database hits? - c#

So I have this query in my repository (also using Unit of Work pattern) which uses eager loading to make one hit to the database:
from g in _context.Games.Include(pg => pg.PreviousGame).Include(go => go.GameObjects)
where EntityFunctions.DiffMilliseconds(DateTime.Now, g.EndDate) > 0
&& g.GameTypeId == (int)GameTypes.Lottery
&& g.GameStatusId == (int)GameStatues.Open
select new LotteryModel
{
EndDate = g.EndDate,
GameId = g.Id,
PreviousGameEndDate = g.PreviousGame.EndDate,
PreviousGameId = g.PreviousGameId.HasValue ? g.PreviousGameId.Value : 0,
PreviousGameStartDate = g.PreviousGame.StartDate,
PreviousWinningObjectCount = g.PreviousGame.GameObjects.Select(go => go.Object.Count).FirstOrDefault(),
PreviousWinningObjectExternalVideoId = g.PreviousGame.GameObjects.Select(go => go.Object.Video.ExternalVideoId).FirstOrDefault(),
PreviousWinningObjectName = g.PreviousGame.GameObjects.Select(go => go.Object.Video.Name).FirstOrDefault(),
StartDate = g.StartDate,
WinningObjectCount = g.GameObjects.Select(go => go.Object.Count).FirstOrDefault(),
WinningObjectExternalVideoId = g.GameObjects.Select(go => go.Object.Video.ExternalVideoId).FirstOrDefault(),
WinningObjectName = g.GameObjects.Select(go => go.Object.Video.Name).FirstOrDefault()
};
However I'm reluctant to use this because I now have to create a separate LotteryModel object to return up throughout my other layers.
I would like to be able to return an entity of type "Game" which has all of the navigational methods to all of my other data (PreviousGame, GameObjects, etc) and then map the needed properties to my flat view model, but when I do this it seems to only lazy load the objects and then I have the additional hits to the DB.
Or do I have this wrong and whenever I need to return heirarchical data I should return it through my LINQ query in the select portion?
My basic goal is to reduce the hits to the DB.

I don't really understand the problem. You return your Games object and you can access the properties and subobjects off it. Your use of the Include() method tells it to load what you need, and not lazy load it.
Make sure you return a single object via a .First, .FirstOrDefault, .Single, .SingleOrDefault, or similar methods.

I ended up with this query (FYI I'm using the System.Data.Objects namespace for the Include extension):
(from g in _context.Games.Include(pg => pg.PreviousGame.GameObjects.Select(o => o.Object.Video))
.Include(go => go.GameObjects.Select(o => o.Object.Video))
where EntityFunctions.DiffMilliseconds(DateTime.Now, g.EndDate) > 0
&& g.GameTypeId == (int)GameTypes.Lottery
&& g.GameStatusId == (int)GameStatues.Open
select g).FirstOrDefault();
I guess I just needed to include more of the heirarchy and didn't know I could use Select() in the Include() function!

Related

ASP.NET Entity FrameWork remove() + ToList()

There is a list of managers (in fact a history), where current is marked as property with TillDate == null and IsIsCurrentManager = true.
db.MyManagers.Remove(db.MyManagers
.Where(e => e.ProjectId == projectId
&& e.MyManagerId == Id).First());
var newCurrentManager = db.MyManagers
.Where(e => e.ProjectId == projectId)
.OrderByDescending(i => i.FromDate)
.FirstOrDefault();
newCurrentManager.TillDate = null;
newCurrentManager.IsCurrentManager = true;
db.SaveChanges();
The problem is that Remove() method will only mark entity as deleted, but, as far as i can see, it still will be added to the list. How can deleted 1 record and than build a list of managers without it, without using SaveChanges() 2 times or hardcoding MyManagerList[2]
It's important to remember that Entity Framework is just an interface into your data store and all you are usually doing is building queries, queries that don't actually run until you materialise the results of a select (i.e. by enumerating through the results or calling something like ToList() or Single()) or, in the case of an update/delete operation, when you call SaveChanges().
So while you can do something like checking that the entity is tracked in the context as a deleted item, you are probably much better off sending the results to the database by calling SaveChanges() twice.
The secondary benefit of that is that another query that doesn't use the same context object (e.g. another web request) will be able to see that the item has been deleted while the original query is trying to pull the list of managers.
You can query the Entity State to exclude it from the second list.
I also have updated the linq request with some cleaning.
Also there is no chek for null, your db query can return null and should be managed as such.
db.MyManagers.Remove(db.MyManagers
.FirstOrDefault(e => e.ProjectId == projectId
&& e.MyManagerId == Id));
var newCurrentManager = db.MyManagers
.ToList()
.Where(e => e.ProjectId == projectId
&& db.Entry(e).State != EntityState.Deleted)//This will query the entity traker
.OrderByDescending(i => i.FromDate)
.FirstOrDefault();
newCurrentManager.TillDate = null;
newCurrentManager.IsCurrentManager = true;
db.SaveChanges();

Entity Framework code-first IQueryable navigation property

Most of the examples I see on the internet show the navigation properties as either ICollection or straight List implementation. They are usually virtual, to enable lazy-loading.
However, when you access such property, it will load the entire collection in memory and if you have a subquery after it (i.e. object.MyListProperty.Where(...)) I have noticed that an SQL query will be issued for each item in the MyListProperty.
How do I avoid this? I want the where clause after the list property to execute on the SQL server, if possible. Can I use an IQueryable navigation property? Is there any best-practice for such case?
My advice for best practise is to disable Lazy loading altogether. Instead force the caller to eagerly load navigation properties through include statements or by using projections.
There are 3rd party products that support include with filters, as described in this post: How to filter include entities in entity framework, but in my experience this further complicates down-stream processing of the objects that are retrieved. If the entity object is loaded outside of method X, because method X can't know for sure if the navigation properties have been loaded with the correct filters, method X starts off by re-querying for the precise rows that it knows it needs.
using (var context = new MyDbContext())
{
context.Configuration.LazyLoadingEnabled = false;
// TODO: load your data
...
}
In this way the records will only be loaded when they are explicitly requested.
When you want access to an IQueryable so you can defer the loading of the data, then make those queries against the DbContext instance and not from the object.
In this example assume that a Customer has many thousands of transactions, so we don't want them to be eagerly or lazy loaded at all.
using (var context = new MyDbContext())
{
context.Configuration.LazyLoadingEnabled = false;
var customer = context.Customers.First(x => x.Id == 123);
...
// count the transactions in the last 30 days for this customer
int customerId = customer.Id;
DateTime dateFrom = DateTime.Today.AddDays(-30)
// different variations on the same query
int transactionCount1 = context.Customers.Where(x => x.Id == customerId)
.SelectMany(x => x.Transactions.Where(x => x.TransactionDate >= dateFrom))
.Count();
int transactionCount2 = context.Customers.Where(x => x.Id == customerId)
.SelectMany(x => x.Transactions)
.Where(x => x.TransactionDate >= dateFrom)
.Count();
int transactionCount3 = context.Transactions.Where(x => x.CustomerId == customerId)
.Where(x => x.TransactionDate >= dateFrom)
.Count();
}
It is good that you have identified that you want to use an IQueryable<T> we access them from the DbContext directly, not from the instances that were previously retrieved.

Entity Framework eager loading based on a specific attribute

The current setup of EF in my application is lazy loading, which is great for the most part. However, I am lost trying work out how to load a list of related entities based on their IsEnabled bit attribute.
In this example I am just returning a list of entities.
return Context.Entities.ToList()
Let's say the Entities object contains a list of ChildEntities like so:
public class Entities
{
private string EntityName;
private List<ChildEntities> ChildEntities;
}
public class ChildEntites
{
private string ChildEntityName;
private bool IsEnabled;
}
I want to only want to get out the ChildEntities based on their IsEnabled flag when loading the list of Entities.
You can use the Include() method to load all child entities and then select only those which are enabled like
Context.Entities.Include("ChildEntites").Select(c => e.IsEnabled == true)
Another way would be to get the filter entities and then run the query as shown in this post
var data = from e in Context.Entities
select new
{
Entities = e,
Childs = e.ChildEntites.Where(c => c.IsEnabled == true)
};
var Results = data.ToArray().Select(x => x.Entities);
I think there is no way to filter when you load related entities in case that you use lazy loading or eager loading unless you project your query to an anonymous type or a DTO, but if you have an entity instance,you can load related entities based on a condition using explicit loading:
var entity=context.Entities.FirstOrDefault();
context.Entry(entity)
.Collection(b => b.ChildEntities)
.Query()
.Where(ce => ce.IsEnabled == true)
.Load();
If that doesn't satisfy what you are trying to achieve because you need to load the entire entity collection, then, as I said before, you should project your query to a custom class or an anonymous type:
var query= from e in Context.Entities.Include(c=>c.ChildEntities)
select new EntityDTO
{
EntityName= e.EntityName,
ChildEntites= e.ChildEntites.Where(c => c.IsEnabled == true)
};
Using a projection
var entities = context.Entities
.Select(x => new {x, x.ChildEntities.Where(y => y.IsEnabled))
.ToList() // resolve from database before selecting the main entity
.Select(x => x.x);
Using a third party library
EF+ Query IncludeFilter allow you to easily filter related entities
var entities = context.Entities.IncludeFilter(x => x.ChildEntities.Where(y => y.IsEnabled))
.ToList();
You can find the documentation here
Disclaimer: I'm the owner of the project EF+.
A couple ways I would recommend this approach. I would either lazy load where IsEnabled = false and eager load in a separate call where IsEnabled = true OR once you have the lazy loaded collection, in a separate call, get the children where IsEnabled = true. I don't believe you will be able to do this in one call. The other option would be a stored procedure. I hope this helps.
I had a similar problem. I solved it in the following manner.
Create a new method in your model Entities. Lets call it ChildEntitiesEnabled
public ICollection<ChildEntity> ChildEntitiesEnabled()
{
//First I get the full list using the lazy loading...
var allChildEntities=ChildEntities.ToList();
//do further processing if there is data
if(allChildEntities!=null && allChildEntities.Count()>0)
{
var childEntitiesEnabled = ChildEntities.Where(x=>x.Enabled==true).ToList();
return childEntitiesEnabled;
}
return null; //or you can return an empty list...
}
I like this method because you can use it anywhere the model is available without complicated code strewn all over the place. Also, you dont lose all the ChildEntities data... that is available too from the original call.

Linq to sql expression tree execution zone issue

I have got a bit of an issue and was wondering if there is a way to have my cake and eat it.
Currently I have a Repository and Query style pattern for how I am using Linq2Sql, however I have got one issue and I cannot see a nice way to solve it. Here is an example of the problem:
var someDataMapper = new SomeDataMapper();
var someDataQuery = new GetSomeDataQuery();
var results = SomeRepository.HybridQuery(someDataQuery)
.Where(x => x.SomeColumn == 1 || x.SomeColumn == 2)
.OrderByDescending(x => x.SomeOtherColumn)
.Select(x => someDataMapper.Map(x));
return results.Where(x => x.SomeMappedColumn == "SomeType");
The main bits to pay attention to here are Mapper, Query, Repository and then the final where clause. I am doing this as part of a larger refactor, and we found that there were ALOT of similar queries which were getting slightly different result sets back but then mapping them the same way to a domain specific model. So take for example getting back a tbl_car and then mapping it to a Car object. So a mapper basically takes one type and spits out another, so exactly the same as what would normally happen in the select:
// Non mapped version
select(x => new Car
{
Id = x.Id,
Name = x.Name,
Owner = x.FirstName + x.Surname
});
// Mapped version
select(x => carMapper.Map(x));
So the car mapper is more re-usable on all areas which do similar queries returning same end results but doing different bits along the way. However I keep getting the error saying that Map is not able to be converted to SQL, which is fine as I dont want it to be, however I understand that as it is in an expression tree it would try to convert it.
{"Method 'SomeData Map(SomeTable)' has no supported translation to SQL."}
Finally the object that is returned and mapped is passed further up the stack for other objects to use, which make use of Linq to SQL's composition abilities to add additional criteria to the query then finally ToList() or itterate on the data returned, however they filter based on the mapped model, not the original table model, which I believe is perfectly fine as answered in a previous question:
Linq2Sql point of retrieving data
So to sum it up, can I use my mapping pattern as shown without it trying to convert that single part to SQL?
Yes, you can. Put AsEnumerable() before the last Select:
var results = SomeRepository.HybridQuery(someDataQuery)
.Where(x => x.SomeColumn == 1 || x.SomeColumn == 2)
.OrderByDescending(x => x.SomeOtherColumn)
.AsEnumerable()
.Select(x => someDataMapper.Map(x));
Please note, however, that the second Where - the one that operates on SomeMappedColumn - will now be executed in memory and not by the database. If this last where clause significantly reduces the result set this could be a problem.
An alternate approach would be to create a method that returns the expression tree of that mapping. Something like the following should work, as long as everything happening in the mapping is convertible to SQL.
Expression<Func<EntityType, Car>> GetCarMappingExpression()
{
return new Expression<Func<EntityType, Car>>(x => new Car
{
Id = x.Id,
Name = x.Name,
Owner = x.FirstName + x.Surname
});
}
Usage would be like this:
var results = SomeRepository.HybridQuery(someDataQuery)
.Where(x => x.SomeColumn == 1 || x.SomeColumn == 2)
.OrderByDescending(x => x.SomeOtherColumn)
.Select(GetCarMappingExpression());

Entity Framework include Where

If I have a query that looks like this:
var forms = repo.GetForms().Where(f => f.SubForms.Any(sf => sf.Classes.Any(c => c.TermId == termId)));
From this you can see my schema is as follows:
SubForm has many Class which has many Term.
What I want:
All SubForms with their Classes In a particular Term.
What is happening now is that I get all the SubForm that has any Class in a particular Term. That means that SubForm comes back with ALL child Class and not just the ones related to the Term.
For eg. I have 2 terms, a subform with 2 classes in each term. This query brings back 4 classes instead of the 2 in that particular term.
Is there any Include('Expression') that I can use to say that I only want to include all classes based on a condition? Or is my query wrong?
Use this:
var subForms = repo.GetSubForms.Select(sf = new {
SubForm = sf,
Classes = sf.Classes.Where(c => c.TermId == termId)
}).ToList()
.Select(t => t.SubForm)
.ToList();
UPDATE: based on #Slauma's comment:
If you want to load SubForms that they have any Class that has a Term by termId, you can go from end to begin; like this:
var subForms = repo.Terms.Where(t => t.Id == termId).Select(t => new {
Term = t,
Class = t.Class,
SubForm = t.Class.SubForm
}).ToList()
.Select(t => t.SubForm).ToList();
OR in a easiest way, you can use Include on your Term, see:
var subForms = repo.Terms.Include("Class.SubForm").Where(t => t.Id == termId)
.Select(t => t.Class.SubForm).ToList();
NOTE: As I can understand from your question, you have a relationship like this:
SubForm has_many Class has_many Term
But, your provided code is showing a relationship like this one:
SubForm has_many Class
Term has_many Class
If you can, put your entities in question, or explain their relationship more please. Thank you.
An Include(Where Expression) does not exist. If you use eager loading with Include you will always load all the elements.
There is a way around this by using projections. The basic idea is you will select a new anonymous type with the property you want and another property with the filtered navigational items. EF will link those together and as a result you will fake a Include(Where ... )
Check this for an example.
You know sometimes I start getting lost in fancy LINQ extension methods and attempting to figure out how to eagerly load exactly what I want and resort to a very simple "join" concept.
var result =
(from f in SubForums
from c in Classes
from t in Term
where t.TermId = 1
select new { SubForum = f, Class = c, Term = t }).ToList();
This is a simple join that uses predefined navigation properties (hence you don't have to specify the join condition). You return an anonymous type with everything that you need. The beauty of this is that Entity Framework will do the auto-fixup for you, therefor you're free to return the SubForum only from your method if you wish, it will automatically contain the Class and subsequent Term references.
I don't know the exact names of the relations, but it should be something along the lines of
repo.Terms
.Include("Classes")
.Include("Classes.SubForms")
.SingleOrDefault(x => x.TermId = termId);
// or
repo.GetSubForms
.Include("Classes")
.Where(sf => sf.Classes.Where(c => c.TermId == termId));
This seems to be a common request, it was difficult to find a solution to it when I was looking earlier this year. I ended up using the solution included in the link below (I'm not sure this is the exact solution I found, but it's the same idea). Hope this helps!
Filter the "Includes" table on Entity Framework query
//Found this method to filter our child objects instead of using .include()
var Results = (from res in
(from u in DataContext.User
where u.Type.ToUpper() != "ADMIN"
&& u.StartDate <= DateTime.Now
&& (u.EndDate == null || u.EndDate >= DateTime.Now)
select new
{
User = u,
Access = u.Access.Where(a => a.StartDate <= DateTime.Now
&& (a.EndDate == null || a.EndDate >= DateTime.Now))
}
)
select res);
//The ToArray is neccesary otherwise the Access is not populated in the Users
ReturnValue = Results.ToArray().Select(x => x.User).ToList();

Categories

Resources