I have pretty simple LINQ expression
IQueryable<FreeBetDTO> records = UnitOfWork.FreeBets
.Include(f => f.FreeBetCategories)
.Include(f => f.FreeBetCards)
.Where(f => f.FreeBetCards.Any(cards => cards.UserId == request.UserId))
.Select(f => new FreeBetDTO
{
FreeBetId = f.FreeBetId
LineCategories = f.FreeBetCategories
.GroupBy(g => new { g.LineCategoryID, g.Title })
.Select(c =>
new LineCategoryDTO
{
LineCategoryID = c.Key.LineCategoryID,
Title = c.Key.Title
}).AsEnumerable()
});
When I am executing it I catch the error:
System.InvalidOperationException: Unable to translate collection subquery in projection since it uses 'Distinct' or 'Group By' operations and doesn't project key columns of all of it's tables which are required to generate results on client side. Missing column: t.ID. Either add column(s) to the projection or rewrite query to not use 'GroupBy'/'Distinct' operation.
at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.
....
The problem is here .GroupBy(g => new { g.LineCategoryID, g.Title }). If I don't group records, the error disappears.
I was trying a lot of cases with GroupBy() and Distinct(). But can't understand why this is happening. Because I just need grouping like this.
Error message says that you have to include Id column in projection. But you can't do that with GroupBy. So rewrite query into two steps (removed not needed includes):
var rawRecords = UnitOfWork.FreeBets
.Where(f => f.FreeBetCards.Any(cards => cards.UserId == request.UserId))
.Select(f => new
{
FreeBetId = f.FreeBetId
LineCategories = f.FreeBetCategories.Select(c => new { c.Id, c.LineCategoryID, c.Title })
.ToList()
})
.AsEnumerable();
var records = rawRecords
.Select(f => new FreeBetDTO
{
FreeBetId = f.FreeBetId
LineCategories = f.LineCategories.GroupBy(g => new { g.LineCategoryID, g.Title })
.Select(c =>
new LineCategoryDTO
{
LineCategoryID = c.Key.LineCategoryID,
Title = c.Key.Title
})
});
Similar query, but more optimal:
var query =
from f in UnitOfWork.FreeBets
from c in f.FreeBetCards
where f.FreeBetCards.Any(cards => cards.UserId == request.UserId)
select new { f.FreeBetId, c.LineCategoryID, c.Title };
query = query.Distinct();
var records = query.AsEnumerable()
.GroupBy(f => f.FreeBetId)
.Select(g => new FreeBetDTO
{
FreeBetId = g.Key
LineCategories = g.Select(c =>
new LineCategoryDTO
{
LineCategoryID = c.LineCategoryID,
Title = c.Title
})
.AsEnumerable()
});
Related
we are migrating reports from SQL to EF. In the process I just stumbled over the following query:
SELECT
pl.clientid,
pl.loginname,
COUNT(*)
FROM
(
SELECT
DISTINCT
s.clientid,
s.doctype
FROM
Specifications s
) A
INNER JOIN portal_logins pl ON A.clientid = pl.clientid
WHERE
A.clientid != '0'
GROUP BY
A.clientid,
A.doctype
My current solution is the following
return await _nfContext.Specifications
.Select( x => new
{
Clientid = x.Clientid,
Doctype = x.Doctype
})
.Distinct()
.Join(
_nfContext.PortalLogins,
s => s.Clientid,
p => p.Clientid,
(spec, login) => new
{
loginName = login.Loginname,
clientId = spec.Clientid,
doctype = spec.Doctype
}
)
.Where(x => x.clientId != "0")
.GroupBy(c => new
{
c.clientId,
c.doctype,
c.loginName
})
.Select(result => new TotalActiveConnections
{
ActiveConnections = result.Count(),
ClientId = result.Key.clientId,
LoginName = result.Key.loginName
})
.OrderByDescending(x => x.ActiveConnections)
.ToListAsync()
.ConfigureAwait(false);
My problem is, I need all three columns as described in the Group by, but I don't want to group by login name. Does anyone have an idea what I can do? Thanks in advance for the help....
You should just use Min or Max to get any random login value
.GroupBy(c => new
{
c.clientId,
c.doctype
})
.Select(result => new TotalActiveConnections
{
ActiveConnections = result.Count(),
ClientId = result.Key.clientId,
LoginName = result.Min(g => g.loginName),
})
Presumably it worked in SQL because you were using MySQL/MariaDB and did not have ONLY_FULL_GROUP_BY turned on.
Since:
"Eager loading a collection navigation in a single query may cause
performance issues."
see: Source
And it is advise to use split queries with include. I wonder if instead of include in the query bellow:
var task = await context.Tasks
.Include(x => x.TaskDependencies)
.Select(x => new TaskBaseModel
{
Id = x.Id,
Name = x.Name,
Description = x.Description,
TaskDependencies= x.TaskDependencies.ToArray()
})
.SingleOrDefaultAsync(x => x.Id == _id);
I should do this:
var task = await context.Tasks
.Select(x => new TaskBaseModel
{
Id = x.Id,
Name = x.Name,
Description = x.Description,
TaskDependencies= context.TaskDependencies
.Where(y => y.TaskId == x.Id).ToArray()
})
.SingleOrDefaultAsync(x => x.Id == _id);
Anyone as any info regarding this? about performance, etc..
Regards
Both queries should have the same performance and SQL. Note that Include followed by Select is ignored by EF Core.
So, most comfortable query is:
var task = await context.Tasks
.Select(x => new TaskBaseModel
{
Id = x.Id,
Name = x.Name,
Description = x.Description,
TaskDependencies = x.TaskDependencies.ToArray()
})
.SingleOrDefaultAsync(x => x.Id == _id);
This query is used to select distinct value from a table in the database
public List<SubDepartmentViewModel> GetSubDepartments()
{
var query = _db.SendersReceivers.Select(a => new SubDepartmentViewModel
{
SubDepartmentName = a.SubDepartment //column in the database
})
.Distinct().ToList();
return query;
}
var query = _db.SendersReceivers
.Where(a => !string.IsNullOrEmpty(a.SubDepartment))
.Select(a => new SubDepartmentViewModel
{
SubDepartmentName = a.SubDepartment
})
.Distinct().ToList();
This should do the trick.
Why am I getting only one entry in DownTimeDetails list even though in Data we have 3 entries.
VehicleEventDetails Res = dbEntity.DownTimeHeaders
.Join(dbEntity.DownTimeDetails, dth => dth.DownTimeHeaderID, dtd => dtd.DownTimeHeaderID, (dth, dtd) => new { dth, dtd })
.Where(x => x.dth.DownTimeHeaderID == 42)
.GroupBy(gx => gx.dtd.DownTimeDetailID)
.Select(t => new VehicleEventDetails()
{
BookingId = t.Select(a => a.dth.BookingId).FirstOrDefault(),
DownTimeDetails = t.Select(ab => new DownTimeDetails
{
LocalDTStartTime = (DateTime)ab.dtd.LocalDTStartTime,
LocalDTEndTime = (DateTime)ab.dtd.LocalDTEndTime,
CalculatedEventDTReason = ab.dtd.CalculatedEventDTReason,
CalculatedEventDTInMinutes = (int)ab.dtd.CalculatedEventDT,
}).ToList()
}).FirstOrDefault();
You are looking for something like this:
VehicleEventDetails Res = dbEntity.DownTimeHeaders
.Where(x => x.DownTimeHeaderID == 42)
.Select(x => new VehicleEventDetails
{
BookingId = x.BookingId,
DownTimeDetails = x.DownTimeDetails
.Select(dtd=> new DownTimeDetails
{
LocalDTStartTime = (DateTime)dtd.LocalDTStartTime,
LocalDTEndTime = (DateTime)dtd.LocalDTEndTime,
CalculatedEventDTReason = dtd.CalculatedEventDTReason,
CalculatedEventDTInMinutes = (int)dtd.CalculatedEventDT,
})
.ToList()
})
.FirstOrDefault();
Notes:
Using .Join is an anti-Entity Framework pattern. Always try to use navigation properties, they exist for a reason.
Don't use .GroupBy unless you actually need a group. You don't want any grouping in this query.
As a general note, try not to make the expression variable names so confusing.
I'm reading a CSV file splitting it into cols, then grouping into a new class.
It looks clunky just wondering is there is a more simple method for instance like not selecting them into the class first:
EDIT: so to clarify I'm trying to get the TimesheetHours grouped by all the other columns.
var rowList = csvFile.Rows.Select(row => row.Split(','))
.Select(cols => new UtilisationRow {
UploadId = savedUpload.Id,
FullName = cols[0],
TimesheetWorkDateMonthYear = Convert.ToDateTime(cols[1]),
TimesheetTaskJobnumber = cols[2],
TimesheetWorktype = cols[3],
TimesheetHours = Convert.ToDouble(cols[4]),
TimesheetOverhead = cols[5]
})
.GroupBy(d => new {
d.FullName,
d.TimesheetWorkDateMonthYear,
d.TimesheetTaskJobnumber,
d.TimesheetWorktype,
d.TimesheetOverhead
})
.Select(g => new UtilisationRow {
FullName = g.First().FullName,
TimesheetWorkDateMonthYear = g.First().TimesheetWorkDateMonthYear,
TimesheetTaskJobnumber = g.First().TimesheetTaskJobnumber,
TimesheetWorktype = g.First().TimesheetWorktype,
TimesheetHours = g.Sum(s => s.TimesheetHours),
TimesheetOverhead = g.First().TimesheetOverhead
})
.ToList();
Many thanks,
Lee.
The two problems in your code are that you call First() repeatedly on a group, while you should retrieve that same data from group's key, and that you are using UtilisationRow in the first Select, which should use an anonymous type instead:
var rowList = csvFile.Rows.Select(row => row.Split(','))
.Select(cols => new {
UploadId = savedUpload.Id,
FullName = cols[0],
TimesheetWorkDateMonthYear = Convert.ToDateTime(cols[1]),
TimesheetTaskJobnumber = cols[2],
TimesheetWorktype = cols[3],
TimesheetHours = Convert.ToDouble(cols[4]),
TimesheetOverhead = cols[5]
})
.GroupBy(d => new {
d.FullName,
d.TimesheetWorkDateMonthYear,
d.TimesheetTaskJobnumber,
d.TimesheetWorktype,
d.TimesheetOverhead
})
.Select(g => new UtilisationRow {
FullName = g.Key.FullName,
TimesheetWorkDateMonthYear = g.Key.TimesheetWorkDateMonthYear,
TimesheetTaskJobnumber = g.Key.TimesheetTaskJobnumber,
TimesheetWorktype = g.Key.TimesheetWorktype,
TimesheetHours = g.Sum(s => s.TimesheetHours),
TimesheetOverhead = g.Key.TimesheetOverhead
})
.ToList();
Now the "pipeline" of your method looks pretty clean:
The first Select does the initial parsing into a temporary record
GroupBy bundles matching records into a group
The final Select produces records of the required type.