I have a simple table structure of services with each a number of facilities. In the database, this is a Service table and a Facility table, where the Facility table has a reference to a row in the Service table.
In our application, we have the following LINQ working:
Services
.Where(s => s.Facilities.Any(f => f.Name == "Sample"))
.GroupBy(s => s.Type)
.Select(g => new { Type = g.Key, Count = g.Count() })
But for reasons beyond my control, the source set is projected to a non-entity object before the Where call, in this way:
Services
.Select(s => new { Id = s.Id, Type = s.Type, Facilities = s.Facilities })
.Where(s => s.Facilities.Any(f => f.Name == "Sample"))
.GroupBy(s => s.Type)
.Select(g => new { Type = g.Key, Count = g.Count() })
But this raises the following exception, with no inner exception:
EntityCommandCompilationException: The nested query is not supported. Operation1='GroupBy' Operation2='MultiStreamNest'
Removing the Where, however, makes it work, which makes me believe it's only in this specific combination of method calls:
Services
.Select(s => new { Id = s.Id, Type = s.Type, Facilities = s.Facilities })
//.Where(s => s.Facilities.Any(f => f.Name == "Sample"))
.GroupBy(s => s.Type)
.Select(g => new { Type = g.Key, Count = g.Count() })
Is there a way to make the above work: select to an non-entity object, and then use Where and GroupBy on the resulting queryable? Adding ToList after the Select works, but the large source set makes this unfeasible (it would execute the query on the database and then do grouping logic in C#).
This exception originates from this piece of code in the EF source...
// <summary>
// Not Supported common processing
// For all those cases where we don't intend to support
// a nest operation as a child, we have this routine to
// do the work.
// </summary>
private Node NestingNotSupported(Op op, Node n)
{
// First, visit my children
VisitChildren(n);
m_varRemapper.RemapNode(n);
// Make sure we don't have a child that is a nest op.
foreach (var chi in n.Children)
{
if (IsNestOpNode(chi))
{
throw new NotSupportedException(Strings.ADP_NestingNotSupported(op.OpType.ToString(), chi.Op.OpType.ToString()));
}
}
return n;
}
I have to admit: it's not obvious what happens here and there's no technical design document disclosing all of EF's query building strategies. But this piece of code...
// We can only pull the nest over a Join/Apply if it has keys, so
// we can order things; if it doesn't have keys, we throw a NotSupported
// exception.
foreach (var chi in n.Children)
{
if (op.OpType != OpType.MultiStreamNest
&& chi.Op.IsRelOp)
{
var keys = Command.PullupKeys(chi);
if (null == keys
|| keys.NoKeys)
{
throw new NotSupportedException(Strings.ADP_KeysRequiredForJoinOverNest(op.OpType.ToString()));
}
}
}
Gives a little peek behind the curtains. I just tried an OrderBy in a case of my own that exactly reproduced yours, and it worked. So I'm pretty sure that if you do...
Services
.Select(s => new { Id = s.Id, Type = s.Type, Facilities = s.Facilities })
.OrderBy(x => x.Id)
.Where(s => s.Facilities.Any(f => f.Name == "Sample"))
.GroupBy(s => s.Type)
.Select(g => new { Type = g.Key, Count = g.Count() })
the exception will be gone.
Related
return await result.Select(student => new MarkSheetsStudentByIdDto
{
Id = student.RegId,
FullName = student.FullName,
AnnualMarkSheets = student.TermOne
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
}).Union(student.TermTwo
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
})).OrderBy(c => c.Rank).ToList()
}).ToList();
For the above example code snippet, I am getting the following error at runtime.
42601: syntax error at or near \"SELECT\"\r\n\r\nPOSITION: 5680
I used ToList() method otherwise I am getting the following error.
Collections in the final projection must be an 'IEnumerable' type
such as 'List'. Consider using 'ToList' or some other mechanism to
convert the 'IQueryable' or 'IOrderedEnumerable' into an
'IEnumerable'.
Can anyone please guide me on how to address this scenario?
Try to load data via Include and then do projection on the client-side:
var rawResult = await result
.Include(x => x.Comments)
.Include(x => x.TermOne)
.Include(x => x.TermTwo)
.Take(1)
.ToListAsync(cancellationToken);
return rawResult.Select(student => new MarkSheetsStudentByIdDto
{
Id = student.RegId,
FullName = student.FullName,
AnnualMarkSheets = student.TermOne
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
}).Union(student.TermTwo
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
})).OrderBy(c => c.Rank).ToList()
}).ToList();
Includes can be replaced with Select later when you experienced that not needed data is requested from database.
I have a problem trying to reuse some subqueries. I have the following situation:
var rooms = dbContext.Rooms.Select(r => new
{
RoomId = r.Id,
Zones = r.Zones.Select(zr => zr.Zone),
Name = r.Name,
Levels = r.Levels.Select(lr => lr.Level),
IdealSetpoint = (double?)r.Group.Setpoints.First(sp => sp.ClimaticZoneId == dbContext.ClimaticZonesLogs.OrderByDescending(cz => cz.Timestamp).First().ClimaticZoneId).Setpoint??int.MinValue,
Devices = r.Devices.Select(rd => rd.Device)
}).ToList();
var tagsTypes = rooms.Select(r => r.Devices.Select(d => GetSetpointTagTypeId(d.DeviceTypeId))).ToList().SelectMany(x => x).Distinct().ToList();
predicate = predicate.And(pv => tagsTypes.Contains(pv.TagSettings.TagTypeId) &&
pv.ClimaticZoneId == dbContext.ClimaticZonesLogs.OrderByDescending(cz => cz.Timestamp).First().ClimaticZoneId);
var setpoints = valuesSubquery.Include(t=>t.TagSettings).Where(predicate).ToList();
This works fine, and generates the exact queries as wanted. The problem is that I want to have this subquery dbContext.ClimaticZonesLogs.OrderByDescending(cz => cz.Timestamp).First().ClimaticZoneId to be taken from a method and not repeat it every time I need it.
I've tested it with the database, where I have values in the corresponding tables, and I've tested the query with the database without any data in the corresponding tables. It works fine with no problems or exceptions.
But when I try to extract the repeating subquery in a separate method and execute it against empty database tables (no data) the .First() statement throws error. Here is the code:
protected long GetClimaticZoneId()
{
return dbContext.ClimaticZonesLogs.OrderByDescending(cz => cz.Timestamp).First().ClimaticZoneId;
}
and the query generation:
var rooms = dbContext.Rooms.Select(r => new
{
RoomId = r.Id,
Zones = r.Zones.Select(zr => zr.Zone),
Name = r.Name,
Levels = r.Levels.Select(lr => lr.Level),
IdealSetpoint = (double?)r.Group.Setpoints.First(sp => sp.ClimaticZoneId == GetClimaticZoneId()).Setpoint??int.MinValue,
Devices = r.Devices.Select(rd => rd.Device)
}).ToList();
var tagsTypes = rooms.Select(r => r.Devices.Select(d => GetSetpointTagTypeId(d.DeviceTypeId))).ToList().SelectMany(x => x).Distinct().ToList();
predicate = predicate.And(pv => tagsTypes.Contains(pv.TagSettings.TagTypeId) &&
pv.ClimaticZoneId == GetClimaticZoneId());
var setpoints = valuesSubquery.Include(t=>t.TagSettings).Where(predicate).ToList();
After execution I get InvalidOperationException "Sequence do not contain any elements" exception in the GetClimaticZoneId method:
I'm sure that I'm not doing something right.
Please help!
Regards,
Julian
As #Gert Arnold suggested, I used the GetClimaticZoneId() method to make a separate call to the database, get the Id and use it in the other queries. I gust modified the query to not generate exception when there is no data in the corresponding table:
protected long GetClimaticZoneId()
{
return dbContext.ClimaticZonesLogs.OrderByDescending(cz => cz.Timestamp).FirstOrDefault()?.ClimaticZoneId??0;
}
I have 2 tables Outlet and Order with below schemas:
Outlet Order
------ -------------------
Id Id
Name Name
OrderCompletedTime
NextOrderDueTime
OutletIds
Earlier when I wanted to get the NextOrderDueTime for each outlet using entity framework core, I did:
return _dbAccessor.RequestContext.MyDbContext.Order
.Where(i => i.OutletId == _dbAccessor.RequestContext.OutletId &&
!i.IsRemoved && i.NextOrderDueTime.HasValue)
.GroupBy(i => i.OutletId)
.Select(g => new { OutletId = g.Key, NextOrderDueTime = g.Min(x => x.NextOrderDueTime) })
.ToDictionary(i => i.OutletId, i => i.NextOrderDueTime);
Now on the UI we need to make this due time as link and wants user to get navigated to that Order details page based on order id
How can I change the above query to also return OrderId along with time?
My thoughts:
Change return type of method from Dictionary<int, DateTimeOffset?> to Dictionary<int, Tuple<int,DateTimeOffset?>>
I tried changing the Linq query to :
return _dbAccessor.RequestContext.MyDbContext.Order
.Where(i => i.OutletId == _dbAccessor.RequestContext.OutletId &&
!i.IsRemoved && i.NextOrderDueTime.HasValue)
.GroupBy(i => i.OutletId)
.Select(g =>
new
{
OutletId = g.Key,
NextOrderDueTime = g.FirstOrDefault(x => x.NextOrderDueTime == g.Min(y => y.NextOrderDueTime)).NextOrderDueTime,
NextOrderId = g.FirstOrDefault(x => x.NextOrderDueTime == g.Min(y => y.NextOrderDueTime)).OrderId
})
.ToDictionary(i => i.OutletId, i => new Tuple<int, DateTimeOffset?>(i.NextOrderId, i.NextOrderDueTime));
But this throws exception at runtime?
Please help to let me know what I am doing wrong here.
You could just take the entire order along with the outletid when you return:
.Select(g => new {
OutletId = g.Key,
NextOrder = g.OrderBy(x => x.NextOrderDueTime).FirstOrDefault()
})
You could select on this to take multiple properties from the order:
.Select(g => new {
OutletId = g.Key,
NextOrder = g.OrderBy(x => x.NextOrderDueTime).FirstOrDefault()
})
.Select(s => new {
s.OutletId,
NextOrderId = NextOrder.Id,
NextOrder.NextOrderDueTime,
NextOrderName = NextOrder.Name
})
etc..
The main thing to appreciate is that grouping gives you an object that has a key, but itself is a list of all things that have that key, so if you order the list by something like the DUeDat and take the first thing then you have an entire object with the lowest duedate from which you can take various things
The .GroupBy(...).Select(...).ToDictionary(...); cannot be converted to SQL since EF Core 3.0.
Due to the breaking change in EF Core 3.0. https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-3.0/breaking-changes , EF Core 3.0 will throw exception to make sure you know that all records in Order will be fetched from database before grouping and map to Dictionary.
I was able to get my query working as below:
return _dbAccessor.RequestContext.MyDbContext.Order
.Where(i => i.OutletId == _dbAccessor.RequestContext.OutletId &&
!i.IsRemoved && i.NextOrderDueTime.HasValue).AsEnumerable()
.GroupBy(i => i.OutletId)
.Select(g =>
new
{
OutletId = g.Key,
NextOrderDueTime = g.FirstOrDefault(x => x.NextOrderDueTime == g.Min(y => y.NextOrderDueTime)).NextOrderDueTime,
NextOrderId = g.FirstOrDefault(x => x.NextOrderDueTime == g.Min(y => y.NextOrderDueTime)).OrderId
})
.ToDictionary(i => i.OutletId, i => new Tuple<int, DateTimeOffset?>(i.NextOrderId, i.NextOrderDueTime));
The same can be done as shown in another answer by just adding AsEnumerable() before GroupBy:
_dbAccessor.RequestContext.MyDbContext.Order
.Where(i => i.OutletId == _dbAccessor.RequestContext.OutletId &&
!i.IsRemoved && i.NextOrderDueTime.HasValue).AsEnumerable()
.GroupBy(i => i.OutletId)
.Select(g => new {
OutletId = g.Key,
NextOrder = g.OrderBy(x => x.NextOrderDueTime).FirstOrDefault()
})
.Select(s => new {
s.OutletId,
NextOrderId = NextOrder.Id,
NextOrder.NextOrderDueTime,
NextOrderName = NextOrder.Name
})`enter code here`;
Consider the following LINQ statements:
var model = getModel();
// apptId is passed in, not the order, so get the related order id
var order = (model.getMyData
.Where(x => x.ApptId == apptId)
.Select(y => y.OrderId));
var orderId = 0;
var orderId = order.LastOrDefault();
// see if more than one appt is associated to the order
var apptOrders = (model.getMyData
.Where(x => x.OrderId == orderId)
.Select(y => new { y.OrderId, y.AppointmentsId }));
This code works as expected, but I could not help but think that there is a more efficient way to accomplish the goal ( one call to the db ).
Is there a way to combine the two LINQ statements above into one? For this question please assume I need to use LINQ.
You can use GroupBy method to group all orders by OrderId. After applying LastOrDefault and ToList will give you the same result which you get from above code.
Here is a sample code:
var apptOrders = model.getMyData
.Where(x => x.ApptId == apptId)
.GroupBy(s => s.OrderId)
.LastOrDefault().ToList();
Entity Framework can't translate LastOrDefault, but it can handle Contains with sub-queries, so lookup the OrderId as a query and filter the orders by that:
// apptId is passed in, not the order, so get the related order id
var orderId = model.getMyData
.Where(x => x.ApptId == apptId)
.Select(y => y.OrderId);
// see if more than one appt is associated to the order
var apptOrders = model.getMyData
.Where(a => orderId.Contains(a.OrderId))
.Select(a => a.ApptId);
It seems like this is all you need:
var apptOrders =
model
.getMyData
.Where(x => x.ApptId == apptId)
.Select(y => new { y.OrderId, y.AppointmentsId });
I have the following code where I get error when loading Peers:
Value cannot be null. Parameter name: source
I am using FirstOrDefault and DefaultIfEmpty methods, and inside the select statement I am also checking if the object is empty m => m == null ?. But, I cannot avoid the error. Any ideas?
ReviewRoundDTO_student results = _context.ReviewRounds
.Include(rr => rr.ReviewTasks).ThenInclude(rt => rt.ReviewTaskStatuses)
.Include(rr => rr.Submissions).ThenInclude(s => s.PeerGroup.PeerGroupMemberships).ThenInclude(m => m.User)
.Include(rr => rr.Rubric)
.Where(rr => rr.Id == reviewRoundId)
.Select(rr => new ReviewRoundDTO_student
{
Id = rr.Id,
SubmissionId = rr.Submissions.FirstOrDefault(s => s.StudentId == currentUser.Id).Id,
Peers = rr.Submissions.FirstOrDefault(s => s.StudentId == currentUser.Id)
.PeerGroup.PeerGroupMemberships.DefaultIfEmpty()
.Select(m => m == null ? new ApplicationUserDto { } : new ApplicationUserDto
{
//FullName = m.User.FullName,
//Id = new Guid(m.UserId)
}),
}).FirstOrDefault();
Try avoiding FirstOrDefault().Something construct - expression trees do not support ?. operator which you'd normally use in similar LINQ to Objects query, and EF Core currently has issues translating it correctly - if you look at the exception stack trace, most likely the exception is coming deeply from EF Core infrastructure with no user code involved.
I would recommend rewriting the LINQ query w/o such constructs, for instance something like this:
var results = _context.ReviewRounds
.Where(rr => rr.Id == reviewRoundId)
.Select(rr => new ReviewRoundDTO_student
{
Id = rr.Id,
SubmissionId = rr.Submissions
.Where(s => s.StudentId == currentUser.Id)
.Select(s => s.Id)
.FirstOrDefault(),
Peers = rr.Submissions
.Where(s => s.StudentId == currentUser.Id)
.Take(1)
.SelectMany(s => s.PeerGroup.PeerGroupMemberships)
.Select(m => new ApplicationUserDto
{
FullName = m.User.FullName,
Id = m.UserId
})
.ToList(),
})
.FirstOrDefault();
Note that Include / ThenInclude are not needed in projection queries, because they are ignored.