Develop Reference

Suggestions on why this linq ef core query wont execute

Good morning,
I have the following query in EF Core which works as required if I do not add into the the where clause a condition for the createdtimestamp. If I add other conditions clauses to filter down the data as shown below the query will execute without issue.
var q = (
from trckhead in DbContext.TrackingBatchHeader
from userdets in DbContext.UserDetails
.Where(u => u.UserId == trckhead.ClosedByUserId).DefaultIfEmpty()
from trckkeys in DbContext.TrackingBatchesItemKey
.Where(t => t.TrackingNo == trckhead.TrackingNo).DefaultIfEmpty()
from trcklink1 in DbContext.TrackingBatchesLink
.Where(x => x.TrackingNo == trckhead.TrackingNo && x.TrackingType == "I").DefaultIfEmpty()
from trcklink2 in DbContext.TrackingBatchesLink
.Where(x => x.TrackingNo == trckhead.TrackingNo && x.TrackingType == "T").DefaultIfEmpty()
join trckref in DbContext.TrackingBatchesReference on trckhead.TrackingBatchType equals trckref
.TrackingBatchType
join mthsite in DbContext.SiteDetail on trckhead.SiteCode equals mthsite.SiteCode
join userdets2 in DbContext.UserDetails on trckhead.CreatedByUserId equals userdets2.UserId
where
(string.IsNullOrEmpty(searchRequest.Status) || trckhead.Status == searchRequest.Status) &&
trckhead.CreatedTimestamp >= DateTime.Now.AddMonths(-11)
select new TrackingBatchSearchResultDto
{
TrackingBatchId = trckhead.TrackingNo,
SiteCode = trckhead.SiteCode,
TrackingBatchType = trckhead.TrackingBatchType,
Status = trckhead.Status,
Created = trckhead.CreatedTimestamp,
CreatedById = trckhead.CreatedByUserId,
ClosedById = trckhead.ClosedByUserId,
UserDescription = trckhead.UsersDescription,
TrackingBatchDescription = trckref.TrackingBatchTypeDescription,
SiteName = mthsite.Title,
CreatedByName = userdets2.FullUserName ?? string.Empty,
ClosedByName = userdets.FullUserName ?? string.Empty,
Link1 = trcklink1.TrackingData ?? string.Empty,
Link2 = trcklink2.TrackingData ?? string.Empty
}).Distinct();
return await q.ToListAsync();
If I keep the date clause in the query the exception that is raised shows the following. I would have thought the server evaluation wouldnt be happy if I was passing a function into the query but checking against a simple date has thrown me. I could filter the results by date after obtaining the whole data set but the number of records returned can be massive so I would rather filter it down from a database query. Any help would be appreciated.
The LINQ expression 'DbSet<TrackingBatchHeader>
.SelectMany(
source: t => DbSet<UserDetail>
.Where(u => u.UserId == t.ClosedByUserId)
.DefaultIfEmpty(),
collectionSelector: (t, c) => new TransparentIdentifier<TrackingBatchHeader, UserDetail>(
Outer = t,
Inner = c
))
.SelectMany(
source: ti => DbSet<TrackingBatchesItemKey>
.Where(t0 => t0.TrackingNo == ti.Outer.TrackingNo)
.DefaultIfEmpty(),
collectionSelector: (ti, c) => new TransparentIdentifier<TransparentIdentifier<TrackingBatchHeader, UserDetail>, TrackingBatchesItemKey>(
Outer = ti,
Inner = c
))
.SelectMany(
source: ti0 => DbSet<TrackingBatchesLink>
.Where(t1 => t1.TrackingNo == ti0.Outer.Outer.TrackingNo && t1.TrackingType == "I")
.DefaultIfEmpty(),
collectionSelector: (ti0, c) => new TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TrackingBatchHeader, UserDetail>, TrackingBatchesItemKey>, TrackingBatchesLink>(
Outer = ti0,
Inner = c
))
.SelectMany(
source: ti1 => DbSet<TrackingBatchesLink>
.Where(t2 => t2.TrackingNo == ti1.Outer.Outer.Outer.TrackingNo && t2.TrackingType == "T")
.DefaultIfEmpty(),
collectionSelector: (ti1, c) => new TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TrackingBatchHeader, UserDetail>, TrackingBatchesItemKey>, TrackingBatchesLink>, TrackingBatchesLink>(
Outer = ti1,
Inner = c
))
.Join(
outer: DbSet<TrackingBatchesReference>,
inner: ti2 => ti2.Outer.Outer.Outer.Outer.TrackingBatchType,
outerKeySelector: t3 => t3.TrackingBatchType,
innerKeySelector: (ti2, t3) => new TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TrackingBatchHeader, UserDetail>, TrackingBatchesItemKey>, TrackingBatchesLink>, TrackingBatchesLink>, TrackingBatchesReference>(
Outer = ti2,
Inner = t3
))
.Join(
outer: DbSet<SiteDetail>,
inner: ti3 => ti3.Outer.Outer.Outer.Outer.Outer.SiteCode,
outerKeySelector: s => s.SiteCode,
innerKeySelector: (ti3, s) => new TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TrackingBatchHeader, UserDetail>, TrackingBatchesItemKey>, TrackingBatchesLink>, TrackingBatchesLink>, TrackingBatchesReference>, SiteDetail>(
Outer = ti3,
Inner = s
))
.Join(
outer: DbSet<UserDetail>,
inner: ti4 => ti4.Outer.Outer.Outer.Outer.Outer.Outer.CreatedByUserId,
outerKeySelector: u0 => u0.UserId,
innerKeySelector: (ti4, u0) => new TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TransparentIdentifier<TrackingBatchHeader, UserDetail>, TrackingBatchesItemKey>, TrackingBatchesLink>, TrackingBatchesLink>, TrackingBatchesReference>, SiteDetail>, UserDetail>(
Outer = ti4,
Inner = u0
))
.Where(ti5 => ti5.Outer.Outer.Outer.Outer.Outer.Outer.Outer.CreatedTimestamp >= (Nullable<DateTime>)DateTime.Now.AddMonths(-11))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

Plainly speaking, neglecting either establish navigation properties for the relationships between entities and instead using manual outer joins using "from" plus then adding the ultimate insult to poorly formed queries with Distinct is trying to use EF like a hammer to pound a square peg through a round hole.
With proper relationships, that query should be trivial.
Your datetime condition error message does not match your query.
** Edit: Removed details on Date comparison - This does not appear to be directly related as I did verify that the Nullable Date/DateTime comparisons do work with .AddMonth() in EF Core.
Expanding out conditional logic can help simplify the queries being executed.
For example:
var query = (/*build your base query*/);
if (!string.IsNullOrEmpty(searchRequest.Status))
query = query.Where(x => x.Status == searchRequest.Status);
if (searchRequest.DateFrom.HasValue)
{
var fromDate = searchRequest.DateFrom.Value.AddMonths(-11);
query = query.Where(x => x.CreatedTimestamp >= fromDate);
}
var results = query.Select(...).Distinct().ToList();
EF can perform some magic to build queries provided you give it enough information to do it properly. Seeing it build something like ti5.Outer.Outer.Outer.Outer.Outer.Outer.Outer.CreatedTimestamp should be sending off alarm bells. There may be something deep in your query lurking that is tripping a Client-side evaluation, or it is simply a case that a combination of joins and such has tipped over a complexity limit or exposed a bug in EF Core. The starting point would be to look to simplify that query expression by leveraging navigation properties rather than joins, and externalizing the conditional logic where you can.
If you have no recourse but to try and query across relationships that you are not prepared to map out properly with navigation properties, I would recommend exploring building your query in SQL as a Stored Procedure and then defining an Entity that can map to the resulting record from that Sproc.

Problem in Search From One List in Another List in Ef .Net Core

List<string> groupId = request.GroupId.Split(',').ToList();
ENTITIES.ProductGroup
.Where(p => p.IsDisplay)
.Where(p => p.FK_GroupNavigation.IsDisplay)
.Where(p => groupId.Any(g => g == (p.FK_Group ?? 0) + "")
.ToList();
The value in request.GroupId is "12,15" and the same values are in the table, but give the following error.
In Ef Core I want to search for some value in another list but it gives the following error What is the problem?
TargetFramework=5.0
The LINQ expression 'DbSet()
.Where(p => p.IsDisplay)
.LeftJoin(
inner: DbSet(),
outerKeySelector: p => EF.Property<Nullable>(p, "FK_Group"),
innerKeySelector: g => EF.Property<Nullable>(g, "PK_Group"),
resultSelector: (o, i) => new TransparentIdentifier<ProductGroup, Group>(
Outer = o,
Inner = i
))
.Where(p => p.Inner.IsDisplay)
.Count(p => __groupId_0
.Any(g => (g ?? "").Equals((object)(p.Outer.FK_Group ?? 0) + "")))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.

Any with local collection is not translatable to SQL, use Contains instead. Also compare integers by integer values, query will use indexes if they are exists.
List<int> groupId = request.GroupId.Split(',').Slect(s => int.Parse(s)).ToList();
SGP_PRODUCT.ProductGroup
.Where(p => p.IsDisplay)
.Where(p => p.FK_GroupNavigation.IsDisplay)
.Where(p => groupId.Contains(p.FK_Group))
.ToList();

Entity Framework Core Where All

I can do this fine in EF.Net, but not in EFCore
List<string> keyList = string.IsNullOrEmpty(keywords) ? new List<string>() : keywords.Split(' ').ToList();
collections = await db.ProductCollections
.Where(m => m.Children.Count == 0 && (!keyList.Any() ? true : keyList.All(x => m.Name.Contains(x))))
.ToListAsync();
I changed it into:
collections = await db.ProductCollections
.Where(m => m.Children.Count == 0 && (keyList.Count == 0 || keyList.All(x => m.Name.Contains(x))))
.ToListAsync();
So I guess the problem is in keyList.All(). How can I achieve this in EFCore?
The error message:
The LINQ expression 'DbSet()
.Where(p => DbSet()
.Where(p0 => EF.Property<Nullable>(p, "ID") != null && object.Equals(
objA: (object)EF.Property<Nullable>(p, "ID"),
objB: (object)EF.Property<Nullable>(p0, "ParentId")))
.Count() == 0 && False || __keyList_0
.All(x => p.Name.Contains(x)))' could not be translated. Either rewrite the query in a form that can be translated, or switch
to client evaluation explicitly by inserting a call to 'AsEnumerable',
'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See
https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

Alternative approach to build query based on key list, before executing it.
var keys = string.IsNullOrEmpty(keywords) ? Array.Empty<string>() : keywords.Split(' ');
var query = db.ProductCollections.Where(p => p.Children.Any() == false);
foreach (var key in keys)
{
query = query.Where(p => p.Name.Contains(key));
}
var collections = await query.ToListAsync();

LINQ expression error could not be translated

I'm getting this error with trying to execute my query.
System.InvalidOperationException:
.Where(ti => (int)ti.Inner.OptionType == 1 && ti.Inner.QualifierId == null && ti.Inner.CreditingMethod != "xxx" && __ToList_0
.Any(e => e.Vdate== (Nullable)ti.Outer.ADate))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'
The object that I'm populating is
public class OCList
{
public DateTime? Vdate{ get; set; }
public Double? VWeight{ get; set; }
}
/// example of adding to the list
List<OCList> ocList= new List<OCList>();
if (request.Date1 != null && request.Weight1 != null)
{
ocList.Add(new OCList{ Vdate = request.Date1.Value, VWeight = request.Weight1.Value });
}
it error's here:
&& ocList.ToList().Any(e => e.Vdate == x.Tb2.ADate))
Linq expression:
var results = _context.Tb1
.Join(_context.Tb2, oc => oc.OptionId, o => o.OptionId, (oc, o) => new { OptionCost = oc, Option = o })
.Where(x => x.Tb2.Val1 == 1
&& x.Tb2.Val2 == null
&& ocList.ToList().Any(e => e.Vdate == x.Tb2.ADate))
////.........

Check client and server side evaluation here: https://learn.microsoft.com/en-us/ef/core/querying/client-eval
EF Core supports partial client evaluation in the top-level projection (essentially, the last call to Select()). If the top-level projection in the query can't be translated to the server, EF Core will fetch any required data from the server and evaluate remaining parts of the query on the client. If EF Core detects an expression, in any place other than the top-level projection, which can't be translated to the server, then it throws a runtime exception.
Your ToList and then Any can't be translated to sql and thus you get the error.
This would work
var results = _context.Tb1
.Join(_context.Tb2, oc => oc.OptionId, o => o.OptionId, (oc, o) => new { OptionCost = oc, Option = o })
.AsEnumerable()
.Where(x => x.Tb2.Val1 == 1
&& x.Tb2.Val2 == null
&& ocList.ToList().Any(e => e.Vdate == x.Tb2.ADate))
BUT it would first fetch everything from the server and then apply the where clause, resulting to poor performance.
You could make it a bit better like this
var results = _context.Tb1
.Join(_context.Tb2, oc => oc.OptionId, o => o.OptionId, (oc, o) => new { OptionCost = oc, Option = o })
.Where(x => x.Tb2.Val1 == 1
&& x.Tb2.Val2 == null)
.AsEnumerable()
.Where(ocList.ToList().Any(e => e.Vdate == x.Tb2.ADate))

How to make LINQ query with Unions more efficient

I inherited the LINQ query below and I feel that the query can be refactored for efficiency. The query currently takes about 6-8 seconds of processing time to return one record to the user on the front-end of the application. LINQ is not my strong suite, so any help would be greatly appreciated.
The query should ultimately produce a distinct list of CA_TASK_VW objects that are tied to a list of distinct CA_OBJECT_ID's obtained from the CA_OBJECT, CA_PEOPLE, and CA_CONTRACTOR tables.
var data = (from a in _db.CA_TASK_VW
where a.TASK_TYPE == "INSPECTION" && a.TASK_AVAILABLE_FLAG == "Y" && a.TARGET_END_DATE == null
select a).AsQueryable();
data = data.Join(_db.CA_OBJECT.Where(o => o.ENTERED_BY == _userId),
o => o.CA_OBJECT_ID, p => p.CA_OBJECT_ID,
(t, p) => t)
.Union(data.Join(_db.CA_PEOPLE.Where(p => p.EMAIL == _email),
t => t.CA_OBJECT_ID, p => p.CA_OBJECT_ID,
(t, p) => t))
.Union(data.Join(_db.CA_CONTRACTOR.Where(c => c.CONTRACTOR.EMAIL == _email),
t => t.CA_OBJECT_ID, c => c.CA_OBJECT_ID,
(t, c) => t));

The code seems to be using Join/Union to execute basically a where predicate on the list of CA_TASK_VW, filtering it step by step to the final result, so what happens if you just specify the where condition directly?
var data = from a in _db.CA_TASK_VW
where a.TASK_TYPE == "INSPECTION" && a.TASK_AVAILABLE_FLAG == "Y" && a.TARGET_END_DATE == null
select a;
data = data.Where(t => _db.CA_OBJECT.Where(o => o.ENTERED_BY == _userId).Select(o => o.CA_OBJECT_ID).Contains(t.CA_OBJECT_ID) ||
_db.CA_PEOPLE.Where(p => p.EMAIL == _email).Select(p => p.CA_OBJECT_ID).Contains(t.CA_OBJECT_ID) ||
_db.CA_CONTRACTOR.Where(c => c.CONTRACTOR.EMAIL == _email).Select(c => c.CA_OBJECT_ID).Contains(t.CA_OBJECT_ID));

You could try using UNION ALL if you don`t really care about duplicates in your query results as it works much faster than UNION