Develop Reference

Finding common items is evaluated locally

Using Entity Framework Core 2.2 I have the following query:
IQueryable<User> users = _context.Users.AsNoTracking();
User user = await users
.Include(x => x.UserSkills)
.ThenInclude(x => x.Skill)
.FirstOrDefaultAsync(x => x.Id == 1);
var userSkills = user.UserSkills.ToList();
IQueryable<Lesson> lessons = _context.Lessons.AsNoTracking();
var test = lessons
.Where(x => x.IsEnabled)
.Where(x => x.LessonSkills.All(y => userSkills.Any(z => y.SkillId == z.SkillId)))
.ToList();
I am looking to get User Skills contains all Lesson Skills.
When I run this query I get the following error:
Exception thrown: 'System.InvalidOperationException' in System.Private.CoreLib.dll:
'Error generated for warning 'Microsoft.EntityFrameworkCore.Query.QueryClientEvaluationWarning:
The LINQ expression 'where ([y].SkillId == [z].SkillId)' could not be translated and will be evaluated locally.'.
How to change the query to solve this problem?
Update
I need to extend this query with an extra option (y.SkillLevelId <= z.SkillLevelId):
var test = lessons
.Where(x => x.IsEnabled)
.Where(x => x.LessonSkills.All(y => userSkills.Any(z =>
y.SkillId == z.SkillId
&&
y.SkillLevelId <= z.SkillLevelId)))
.ToList();

userSkills is in-memory collection, and from my experience with EF6 and EF Core so far I can say that the only reliable translatable construct with in-memory collections is Enumerable.Contains method on primitive type in-memory collection.
So the following solves the problem is question.
First (should be outside the query expression tree):
var userSkillIds = user.UserSkills.Select(x => x.SkillId);
Then instead of
.Where(x => x.LessonSkills.All(y => userSkills.Any(z => y.SkillId == z.SkillId)))
use the equivalent (but translatable):
.Where(x => x.LessonSkills.All(y => userSkillIds.Contains(y.SkillId)))
Update: If you can't use Contains, the options you have until EF Core starts supporting it are (1) EntityFrameworkCore.MemoryJoin package (I personally haven't tested it, but the idea is interesting), (2) manually building Or based predicate with Expression class (hard and works for small memory collections) and (3) replace the memory collection with real IQueryable<>, for instance
var userSkills = users
.Where(x => x.Id == 1)
.SelectMany(x => x.UserSkills);
and use the original query.

How can I order in Linq Select Distinct if ordered by an Included entity?

I know that in Linq I have to do the OrderBy after doing a Select - Distinct, but I'm trying to order by an Included entity property that get lost after the Select.
For example:
var accounts = _context.AccountUser
.Include(o => o.Account)
.Where(o => o.UserId == userId || o.Account.OwnerId == userId)
.OrderByDescending(o => o.LastAccessed)
.Select(o => o.Account)
.Distinct();
As I'm doing the Where by an or of two different parameters, there is a good chance to obtain duplicated results. That's why I'm using the Distinct.
The problem here is that after I do the Select, I don't have the LastAccessed property anymore because it doesn't belong to the selected entity.
I thing the structure of the AccountUser and Account can be inferred from the query itself.

If you have the bi-directional navigation properties set up:
var accountsQuery = _context.AccountUser
.Where(o => o.UserId == userId || o.Account.OwnerId == userId)
.Select(o => o.Account)
.Distinct()
.OrderByDescending(a => a.AccountUser.LastAccessed);
When Selecting the Account you do not need .Include() Keep in mind that any related entities that you access off the Account will be lazy-loaded. I recommend using a .Select() to extract either a flattened view model or a view model hierarchy so that the SQL loads all needed fields rather than either eager-loading everything or tripping lazy-load calls.

Since LINQ doesn't implement DistinctBy and LINQ to SQL doesn't implement Distinct that takes an IEqualityComparer, you must substiture GroupBy+Select instead:
var accounts = _context.AccountUser
.Include(o => o.Account)
.Where(o => o.UserId == userId || o.Account.OwnerId == userId)
.GroupBy(o => o.Account).Select(og => og.First())
.OrderByDescending(o => o.LastAccessed)
.Select(o => o.Account);

Linq: additional parameter

I have the following line:
orderBaseData.Single(o => o.Id == order.Id).OrderMessages.Count;
I want to filter this a bit more. Each OrderMessage has a variable called hideFromUser, and I want to get the count of the OrderMessages where this is set to FALSE only.
Thanks in advance,
Bob

Use Where on OrderMessages
orderBaseData
.Single(o => o.Id == order.Id)
.OrderMessages
.Where(x => !x.hideFromUser)
.Count();

In order to retrieve the data efficiently in a single query, instead of relying on LazyLoading the OrderMessages afterwards, you could use the following:
orderBaseData
.Where(o => o.Id == order.Id)
.Select(o => o.OrderMessages.Where(x => !x.hideFromUser).Count())
.Single();
This approach is mostly interesting when orderBaseData is some IQueryable targeting a database. If all the data are already in memory, then it is not really better or worse than the other approaches.

WHERE is not being included in the LINQ-query

Today I ran into a problem with Entity Framework. I'm not sure if this is a weird bug or that i'm doing something wrong. I've already looked all over the forum for any possible solutions, but none I found worked for me.
I have the following LINQ query:
return (from sp in context.ServiceProviders.DefaultIfEmpty()
join pl in context.Platforms on sp.Id equals pl.ServiceProviderId into innerPl
from pl in innerPl.DefaultIfEmpty()
join pp in context.Participants on pl.Id equals pp.PlatformId into innerPp
from pp in innerPp.DefaultIfEmpty()
join ps in context.Paymentsettlements on pp.Id equals ps.ParticipantId into innerPs
from ps in innerPs.Where(ps => ps.ConfirmedOn.HasValue && ps.ExportDate.HasValue && !ps.StatisticsDate.HasValue).DefaultIfEmpty()
select sp).Include(sp => sp.Environment)
.Include(sp => sp.Platforms.Select(pl => pl.Participants.Select(pp => pp.Paymentsettlements.Select(ps => ps.Requester))))
.Include(sp => sp.Platforms.Select(pl => pl.Participants.Select(pp => pp.Paymentsettlements.Select(ps => ps.Payer))))
.ToList();
The result i'm looking for is that i always get the ServiceProvider no matter if there are objects inside the ServiceProvider. I am getting this result at the moment, but the where I've put in the query does not get taken into account. The following where does not make any difference:
innerPs.Where(ps => ps.ConfirmedOn.HasValue && ps.ExportDate.HasValue && !ps.StatisticsDate.HasValue).DefaultIfEmpty()
If the StatisticsDate has a value, those Paymentsettlements also are given in the output.
I've already tried to put the WHERE statement on the context.Paymentsettlements object.
I hope anyone can help me with this problem.
Kind regards,
Rob H

Actually you are doing left join and then selecting ServiceProviders. Here you are getting all providers. Then you are including all child elements: select sp).Include(sp => sp.Environment). This won't work. It will include all rows.
What you can really do is select to anonymous type like
select new {sp, ps }
Unfortunately there is no way of filtering in included objects. Include is something like all or nothing.
You can read about it:
How to filter nested collection Entity Framework objects?
EF Query With Conditional Include

I've finally made another (hacky) solution. Here is my final code:
using (var context = new BetaalplatformContext())
{
var dienstverleners = context.Dienstverleners.Include(dv => dv.Omgeving)
.Include(dv => dv.Platformen)
.Include(dv => dv.Platformen.Select(pl => pl.Deelnemers))
.Include(dv => dv.Platformen.Select(pl => pl.Deelnemers.Select(dn => dn.Betaalregelingen)))
.Include(dv => dv.Platformen.Select(pl => pl.Deelnemers.Select(dn => dn.Betaalregelingen.Select(br => br.Aanvrager))))
.Include(dv => dv.Platformen.Select(pl => pl.Deelnemers.Select(dn => dn.Betaalregelingen.Select(br => br.Betaler))))
.ToList();
dienstverleners.ForEach(
dv => dv.Platformen.ForEach(
pl => pl.Deelnemers.ForEach(
dn => dn.Betaalregelingen = dn.Betaalregelingen
.Where(br2 => br2.BevestigdOp.HasValue && br2.ExportDatum.HasValue && !br2.StatistiekDatum.HasValue)
.ToList()
)
)
);
return dienstverleners;
}
This way i am abled to keep my models intact (I don't like to use anonymous objects).

Visiting Entity Framework's Include method

I'm trying to visit the Entity Framework's Include method using QueryResultCache class which is motioned here. It's a very popular article and a lot of query caching libraries are using it.
When I try an expression like:
var exp1 = context.Products.Include(x => x.Tags)
.Where(x => x.Tags.Any(y => y.Name.Contains("Test")))
.Select(x => new {x.ProductId}).Expression;
with it, it produces this string:
value(System.Data.Entity.Core.Objects.ObjectQuery`1
[EfSecondLevelCaching.Test.Models.Product]).MergeAs(AppendOnly).IncludeSpan
(value(System.Data.Entity.Core.Objects.Span))
.Where(x => x.Tags.Any(y => y.Name.Contains("Test")))
.Select(x => new <>f__AnonymousType5`1(ProductId = x.ProductId))
As you can see, the result doesn't contain the parameters of Include method (x => x.Tags). So most of the linq caching libraries on the net can't create a valid unique query key for the EF queries. How can I fix this?
Edit:
If I remove the select method, it will produce:
value(System.Data.Entity.Core.Objects.ObjectQuery`1
[EfSecondLevelCaching.Test.Models.Product])
.MergeAs(AppendOnly)
.IncludeSpan(value(System.Data.Entity.Core.Objects.Span))
.Where(x => x.Tags.Any(y => y.Name.Contains("Test")))
So here there is no difference between Include(x=>x.Tags) and Include(x=>x.Users).

The query will only return what is in your Select expression. In this case Select(x => new {x.ProductId}) means that only a single field ProductId will be returned.
Your Include would have made a difference if you were returning Products as they contain Tags, but makes no difference if you just have ProductId.
See this MSDN article for more information on eager loading (Include ensures eager loading)