I am currently using EF Core 6 (w/ lazy loading) to access my DB.
When I access my desired data like this:
var depCount = admin.Departments.Count(d => !d.Deleted)
SQL Server Profiler shows me this:
exec sp_executesql N'SELECT [t].[DepId], [t].[BaseDepId], [t].[CreatedTimeStamp], [t].[CustCode], [t].[Deleted], [t].[DeletedTimeStamp], [t].[DepName], [t].[ForSingleSurvey], [t].[Level], [t].[SendInvitationMails], [a].[AdminId], [t].[AdministratorsAdminId], [t].[DepartmentsDepId], [t0].[AdministratorsAdminId], [t0].[DepartmentsDepId], [t0].[AdminId], [t0].[AdminEmail], [t0].[AdminPwdHash], [t0].[AuthToken], [t0].[CreatedTimeStamp], [t0].[CustCode], [t0].[CycleId], [t0].[EmailConfirmed], [t0].[FirstName], [t0].[LastName], [t0].[LastTokenGenerated], [t0].[OnlyManaging]
FROM [Administrators] AS [a]
INNER JOIN (
SELECT [d].[DepId], [d].[BaseDepId], [d].[CreatedTimeStamp], [d].[CustCode], [d].[Deleted], [d].[DeletedTimeStamp], [d].[DepName], [d].[ForSingleSurvey], [d].[Level], [d].[SendInvitationMails], [a0].[AdministratorsAdminId], [a0].[DepartmentsDepId]
FROM [AdministratorDepartment] AS [a0]
INNER JOIN [Departments] AS [d] ON [a0].[DepartmentsDepId] = [d].[DepId]
) AS [t] ON [a].[AdminId] = [t].[AdministratorsAdminId]
LEFT JOIN (
SELECT [a1].[AdministratorsAdminId], [a1].[DepartmentsDepId], [a2].[AdminId], [a2].[AdminEmail], [a2].[AdminPwdHash], [a2].[AuthToken], [a2].[CreatedTimeStamp], [a2].[CustCode], [a2].[CycleId], [a2].[EmailConfirmed], [a2].[FirstName], [a2].[LastName], [a2].[LastTokenGenerated], [a2].[OnlyManaging]
FROM [AdministratorDepartment] AS [a1]
INNER JOIN [Administrators] AS [a2] ON [a1].[AdministratorsAdminId] = [a2].[AdminId]
WHERE [a2].[AdminId] = #__p_0
) AS [t0] ON [t].[DepId] = [t0].[DepartmentsDepId]
WHERE [a].[AdminId] = #__p_0
ORDER BY [a].[AdminId], [t].[AdministratorsAdminId], [t].[DepartmentsDepId], [t].[DepId], [t0].[AdministratorsAdminId], [t0].[DepartmentsDepId]',N'#__p_0 int',#__p_0=122
which obviously is very inefficient and way too much overhead.
However, when I access my desired data like this:
var depCount = await context.Departments.CountAsync(d => d.Admins.Any(a => a.AdminId == admin.AdminId) && !d.Deleted)
the Profiler shows me the following statement:
exec sp_executesql N'SELECT COUNT(*)
FROM [Departments] AS [d]
WHERE EXISTS (
SELECT 1
FROM [AdministratorDepartment] AS [a]
INNER JOIN [Administrators] AS [a0] ON [a].[AdministratorsAdminId] = [a0].[AdminId]
WHERE ([d].[DepId] = [a].[DepartmentsDepId]) AND ([a0].[AdminId] = #__a_AdminId_0)) AND ([d].[Deleted] = CAST(0 AS bit))',N'#__a_AdminId_0 int',#__a_AdminId_0=113
which is what I would want.
Does anybody know if I can produce that behavior with the first accessing method (via the Navigation Property)?
Since this would be way easier to code...
Thank you in advance!
The first example will access the collection if eager loaded, otherwise if you have lazy loading enabled then the proxy will result in a DB hit to load the related Departments. Based on that you're seeing the query run then that indicates lazy loading is enabled and running. Lazy loading incurs a performance hit, but serves as a safety net to ensure that if data is available, it can be loaded when accessed.
When loading entities you need to plan ahead a bit and either ensure data you need will be eager loaded with the entity(ies) or that you are projecting the data down to include all of the details you will need.
For example, eager loading:
var admin = await context.Admins
.Include(x => x.Departments)
.SingleOrDefaultAsync(x => x.AdminId == adminId);
// later...
var departmentCount = admin.Departments.Count();
This avoids the extra DB hit as the departments would have been loaded when the Admin was. The cost to consider with eager loading is that you are still loading a lot of data (everything about admin and all of their departments and any other related data you eager load) which may not be necessary. In the above case we just want a count. The issue I often find with teams that insist on loading entities and want to avoid lazy loading is that they default to eager loading everything. Still, when loading single entities, eager loading related data is generally Ok where it is useful for that data to be loaded.
When reading data especially, it can be much better to consider using projection solutions to fetch the data you want to consume. For instance take a situation where you want to search for Admins and include information like their department count. Using eager loading you'd have something like:
var admins = await context.Admins
.Include(x => x.Departments)
.Where(x => *some condition*)
.Skip(pageNumber * pageSize)
.Take(pageSize)
.ToListAsync();
Even best case that we use server-side pagination to control the amount of data included, this still loads a page of admins and all departments for each. If there are other related entities this can balloon out the amount of data loaded fairly quickly. EF can end up generating things like Cartesian products between related tables which it will chew through to produce the entity graph, but this will still take up memory and time.
With projection we could create something like:
[Serializable]
public class AdminSummaryViewModel
{
public int AdminId { get; set; }
public string Name { get; set; }
// Additional details we want to show....
public int DepartmentCount { get; set; }
}
Then when we go to get the data:
var admins = await context.Admins
.Where(x => *some condition*)
.Select(x => new AdminSummaryViewModel
{
AdminId = x.AdminId,
Name = x.Name,
// ...
DepartmentCount = x.Departments.Count()
})
.Skip(pageNumber * pageSize)
.Take(pageSize)
.ToListAsync();
This generates an SQL statement that will only query the details we need from the relevant tables, including condensing down the requested department count. This reduces the total amount of data passed over the wire and time needed for EF to produce the view models desired. Automapper has a ProjectTo extension method for IQueryable which can populate a ViewModel without having to write out that .Select() block.
admin.Departments.Count(d => !d.Deleted) is lazy loading the entire Departments collection, then calling Enumerable.Count to give the answer.
Without using any helper methods, you can get the answer this way;
public int DepartmentCount (DbContext context, Admin admin)
{
var navigation = context
.Entry(admin)
.Navigation(nameof(Admin.Departments));
// if it's already in memory, just use that
if (navigation.IsLoaded)
return admin.Departments.Count(d => !d.Deleted);
// otherwise query the database
// EF Core can generate the query for you, equivalent to supplying FK parameters;
// context.Departments.Where(d => d.AdminId == admin.Id)
var query = (IQueryable<Department>)navigation.Query();
return query.Count(d => !d.Deleted);
}
I have the following code. It brings back the Show and its related Slides. However Slide also has related Items but I have no idea how to make the query include those (There is a foreign key in the db).
Show sh = (from s in _context.Shows
where s.ShowId == id
select new Show()
{
ShowId=s.ShowId,
ShowName=s.ShowName,
Slides=s.Slides
}).FirstOrDefault();
How do I modify this to make it also fetch the list of Items for each Slide?
I am using .Net5
If you defined your DbContext correctly the LINQ would be
Show sh = _context.Shows
.Where(s => s.ShowId == id)
.Include(s => s.Slides)
.ThenInclude(sl => sl.Items)
.FirstOrDefault();
In a .NET Core Project I'm using EntityFramework and I have a DbContext (shopContext) injected in my class repository. I have the next query:
var res = shopContext.Orders.Where(x => x.Status == 1).Sum(p => p.Total);
Occasionally, the Orders table is doing maintenance tasks and the table is locked. For this query, I can't wait to the maintenance tasks and I need access to the table with the IsolationLevel.ReadUncommited option in the transaction:
using (var transaction = mutuaContext.Database.BeginTransaction(IsolationLevel.ReadUncommitted))
{
var res = shopContext.Orders.Where(x => x.Status == 1).Sum(p => p.Total);
}
The problem is that I only want that the context execute the query with this IsolationLevel configuration in these query, but the next queries continue executing although the table is locked yet.
Why are the following queries not waiting for the Table to be unlocked?
Example of my code:
using (var transaction = mutuaContext.Database.BeginTransaction(IsolationLevel.ReadUncommitted))
{
var res = shopContext.Orders.Where(x => x.Status == 1).Sum(p => p.Total); // this code would be executed
}
var total = shopContext.Orders.Where(x => x.Status == 0).Sum(p => p.Total); // this code would NOT be executed but is executed
I don't understand how the context get the transaction configuration. I would like that someone explain it to me.
I tried call to transaction.Commit() after fist query, but still not working.
use
yourContext.Database.BeginTransaction(IsolationLevel.ReadUncommitted)
// your normal queries via yourContext goes here
// do not forget to end the transaction
You can use raw SQL query (there is a similar SqlQuery() method for EF6, as well) and specify with (nolock) table hint. Something like this:
var res = shopContext.Orders.FromSqlRaw("select sum(Total) from dbo.Orders with (nolock) where Status = 1").ToList();
However, once you will deploy this into production environment and put your code under a decent concurrent load, most probably you will not like the outcome.
UPD: For EF Core 2.2, the syntax is a bit different:
var res = shopContext.Orders.FromSql("select * from Orders with(nolock)")
.Where(x => x.Status == 1).Sum(p => p.Total);
I want to use Take() in projection but I dont want to produce N+1 query, in addition without Take() in projection I'm facing with performance issue.
I used Take() with EF6 but I faced with N+1 issue on EF Core.
example projection:
source.Select(post => new PostProject
{
PostDisableCoins = post.PostDisableCoins
.OrderBy(x=>x.CoinAmount)
.Take(3)
.ToList(),
WarStartTime = post.WarStartTime,
WarEndTime = post.WarEndTime,
WarWinner = post.WarWinner,
WarDeclarer = post.WarDeclarer
});
I want to have Take(3) whithout N+1, any suggestion?!?
This is EF Core 2.1 implementation defect. Following is the workaround, but use it only if you really have performance issues, because it requires breaking the navigation property join abstraction and using manual join, which I always say should not be used with EF (Core). Also might not work if used to project more than one collection, or as part of a more complex query.
It requires replacing the usage of the collection navigation property post.PostDisableCoins with SelectMany using lateral join and hiding the OrderBy / Take operators (update with proper types and PK/FK names):
var postDisableCoinsQuery = source.SelectMany(p =>
db.Set<PostDisableCoin>()
.Where(c => c.PostId == p.Id)
.OrderByDescending(c => c.CoinAmount)
.Take(3)
);
Then do GroupJoin to it:
var query =
from p in source
join c in postDisableCoinsQuery on p.Id equals c.PostId into postDisableCoins
select new PostProject
{
PostDisableCoins = postDisableCoins.ToList(),
WarStartTime = p.WarStartTime,
WarEndTime = p.WarEndTime,
WarWinner = p.WarWinner,
WarDeclarer = post.WarDeclarer
};
When executed, the above will produce the desired result with single SQL query.
Please note the doc regarding EF core 2.1 new features:
We have improved our query translation to avoid executing "N + 1" SQL
queries in many common scenarios in which the usage of a navigation
property in the projection leads to joining data from the root query
with data from a correlated subquery. The optimization requires
buffering the results from the subquery, and we require that you
modify the query to opt-in the new behavior.
for example:
var query = context.Customers.Select(
c => c.Orders.Where(o => o.Amount > 100).Select(o => o.Amount).ToList());
Notice where the .ToList() is included.
You need to modify your projection query accordingly, in order to enable the optimization feature.
In your case it might be:
source.Select(post => new PostProject
{
PostDisableCoins = post.PostDisableCoins
.Select(x => x.OrderBy(x=>x.CoinAmount))
.Select(x => x)
.Take(3)
.ToList(),
WarStartTime = post.WarStartTime,
WarEndTime = post.WarEndTime,
WarWinner = post.WarWinner,
WarDeclarer = post.WarDeclarer
});
I'm executing following linq query in ASP.NET MVC project:
Version baseVersion = new Version("1.0.0.0");
var v = (from a in db.table where new Version(a.version) > baseVersion select new { a.id, a.version, a.name}).ToList();
While executing this I'm getting this error:
Only parameterless constructors and initializers are supported in LINQ to Entities
When I removed version check and used this query:
var v = (from a in db.table select new { a.id, a.version, a.name}).ToList();
it executed successfully. How can I make the linq query with version condition to work?
If you want to use the Version class to do the comparison, then I see no other option than importing the whole table into memory, and do the comparison there. You can use IQueryable<T>.AsEnumerable() for to specify you want to do anything after in memory.
Version baseVersion = new Version("1.0.0.0");
var v = db.Table
.Select(x => new { x.id, x.version, x.name })
.AsEnumerable()
.Where(x => new Version(x.version) > baseVersion)
.ToList();
Note: I've moved the .Select(...) to immediately after the db.Table to only select the columns you need, which minimizes the memory footprint.
The alternative is that you recreate the Version comparison in SQL. But then you are reinventing the wheel.