Develop Reference

Select all distinct last records of column1 and order by column2 in Linq (Not Working Accordingly)

So I have a table like this:
Now I want distinct ShortCode order by the ID descending. In other words, the distinct last records. Like this:
So I tried GroupBy like:
var data = db.ShortCodes.GroupBy(x => x.ShortCode).Select(x => x.FirstOrDefault()).OrderByDescending(s=> s.ID);
This gave me distinct records but not the last ones, nor ordered by ID descending:
Now I also tried like suggested here
var data = db.ShortCodeManager
.GroupBy(s => s. ShortCode)
.Select(g => g.First())
.OrderByDescending(s => s.ID);
This gave me the error The method 'First' can only be used as a final query operation. Consider using the method 'FirstOrDefault' in this instance instead.
So I modified to FirstOrDefault() like:
var data = db.ShortCodeManager
.GroupBy(s => s. ShortCode)
.Select(g => g.FirstOrDefault())
.OrderByDescending(s => s.ID);
This also gave me distinct records but not the last records:
So finally I tried like suggested here:
var data = db.ShortCodeManager.Where(a => a.ID > 0).GroupBy(x => x.ShortCode).OrderByDescending(grp => grp.Max(g => g.ID)).Select(a => a.FirstOrDefault());
Again, this gave me distinct records but not the last ones, nor ordered by ID descending:
So how am I to write the query to get the result I want in Linq? Also note, I need more of the distinct last records than ordering by ID descending. If anyone also knows how to write it in raw SQL it might be useful as well.

This LINQ query should work for your case:
var result = db.ShortCodeManager
.GroupBy(x => x.ShortCode)
.Select(gr => new { Id = gr.Max(g => g.Id), ShortCode = gr.Key})
.ToList();
EDIT:
Based on your comment it looks like you need to cast anonymous object result to ShortCodeManagerModel type and then pass it to your view. So, somethin like this:
var result = db.ShortCodeManager
.GroupBy(x => x.ShortCode)
.Select(gr => new { Id = gr.Max(g => g.Id), ShortCode = gr.Key})
.ToList();
var model = result
.Select(x => new ShortCodeManagerModel { Id = x.Id, ShortCode = x.ShortCode })
.ToList();
And then pass model to you view.

EF Core "Group By could not be translated and will be evaluated locally."

I am running a simple query against an Sql Server database using Entity Framework Core 2.2.6 however the GroupBy is not being executed on the server, instead it is being executed locally.
Is there something i'm missing that will force the group by onto the server?
The 2 variations of EF query i have tried:
public class Holiday
{
public int Id {get;set;}
public DateTime Date {get;set;}
public string Username {get;set;}
public string Approver {get;set;}
}
//version 1
await _dbContext.Holidays
.GroupBy(h => new { h.Date})
.ToDictionaryAsync(x => x.Key.Date, x => x.Select(x1 => x1.Username).ToList());
//version 2
await _dbContext.Holidays
.GroupBy(h => h.Date)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
Both variations produces the following SQL:
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY [h].[Date]
warning produced:
warnwarn: Microsoft.EntityFrameworkCore.Query[20500]
The LINQ expression 'GroupBy([h].Date, [h])' could not be translated and will be evaluated locally.
Suggestions from comments:
//group by string
await _dbContext.Holidays
.GroupBy(h => h.Username)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
//group by part of date
await _dbContext.Holidays
.GroupBy(h => h.Date.Year)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
--group by string
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY [h].[Username]
--group by part of date
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY DATEPART(year, [h].[Date])

The problem is that when you're trying to group in the database, you don't really have the means to materialize values inside a group. You only get to SELECT grouped columns or aggregated values (via SUM, etc.) of non-grouped columns.
For example:
SELECT [h].[Date], [h].[Username]
FROM [Holidays] AS [h]
This query would produce result set of which every row would have two columns, date and name.
Let's try grouping though:
SELECT [h].[Date], [h].[Username]
FROM [Holidays] AS [h]
GROUP BY [h.Date]
This SQL query wouldn't be evaluated at all because it's invalid from SQL server perspective. Error message would be
Column 'Holidays.Username' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
Summing all this up, you can either do what #Ercan Tirman has suggested, or, load all the usernames and dates and group them in-memory:
var dateAndUsername = await _dbContext.Holidays
.Select(x => new {x.Date, x.Username})
.ToArrayAsync();
Dictionary<DateTime, List<string>> grouped = dateAndUsername
.GroupBy(x => x.Date)
.ToDictionary(x => x.Key, x => x.Select(y => y.Username).ToList());

It's because there is no SQL query like that.
Think like SQL. If you want to get Usernames by group of Dates, you need both of those.
Basically :
await _dbContext.Holidays
.GroupBy(h => new { h.Date, h.Username})
.Select(g => new
{
g.Key.Date,
g.Key.Username
});
This will produce a SQL query like this.
SELECT [h].[Date],[h].[Username]
FROM [Holidays] AS [h]
GROUP BY [h].[Date],[h].[Username]
After that you can use the data to create the structure of your dictionary however you want.

What can I do to improve the speed of this query?

I have a linq query that returns the last page a user looked at based on a table of page hits. The fields are simply TimeStamp, UserID and URL which are logged from user activity. The query looks like this:
public static IQueryable GetUserStatus()
{
var ctx = new AppEntities();
var currentPageHits = ctx.Pagehits
.GroupBy(x => x.UserID)
.Select(x => x.Where(y => y.TimeStamp == x.Max(z => z.TimeStamp)))
.SelectMany(x => x);
return currentPageHits.OrderByDescending(o => o.TimeStamp);
}
The query works perfectly but runs slowly. Our DBA assures us that the table has indexes in all the right places and that the trouble must be with the query.
Is there anything inherently wrong or BAD with this, or is there a more efficient way of getting the same results?

You could try:
var currentPageHits2 = ctx.Pagehits
.GroupBy(x => x.UserID)
.Select(x => x.OrderByDescending(y => y.TimeStamp).First())
.OrderByDescending(x => x.TimeStamp);
But the speed should be the same.
Note that there is a subtle difference between this query and yours... With yours, if a UserId has two "max TimeStamp" PageHits with the same TimeStamp, two "rows" will be returned, with this one only one will be returned.

So you try to implement DENSE_RANK() OVER (PARTITION BY UserID ORDER BY TimeStamp DESC) with LINQ? So all latest records per user-group according to the Timestamp. You could try:
public static IQueryable GetUserStatus()
{
var ctx = new AppEntities();
var currentPageHits = ctx.Pagehits
.GroupBy(x => x.UserID)
.SelectMany(x => x.GroupBy(y => y.TimeStamp).OrderByDescending(g=> g.Key).FirstOrDefault())
.OrderByDescending(x => x.TimeStamp);
return currentPageHits;
}
So it's grouping the user-group by TimeStamp, then it takes the latest group(one or more records in case of ties). The SelectMany flattens the goups to records. I think this is more efficient than your query.

How Can I Generate Nhibernate GROUP BY without SELECT the property

I'd like to resolve that problem :
SELECT Max(Date)
FROM Table
GROUP BY SubId
(Then pass it as a SubQuery to mid-action so I can get the Id of the item in Table)
SELECT Id
FROM Table
WHERE Date in
[[[ previous request ]]]
(Then Get the full Table Item with other table join)
SELECT *
FROM Table
LEFT JOIN...
WHERE Id in
[[[ previous request ]]]
I tried this kind of request :
var subquery = QueryOver.Of<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id)
);
var filter = QueryOver.Of<Table>().WithSubquery.
WhereExists(subquery)
.Select(p => p.Id);
var result = Session.QueryOver<Table>().WithSubquery.WhereProperty(p => p.Id).In(filter).Left.JoinQueryOver(p => p.Sub).List();
But the problem is that I can't get the first request right with only the date out of my request.
Is there a better way to do it than that kind of subqueries ? And is there a possibility in NHibernate to Groupy By a Property without selecting it ?
Thanks !

Finally did it that way and it generated the SQL i wanted. But it wasn't 3 subqueries exactly it was 3 queries looking in a set of datas (The arrays subquery and CorrespondingIds).
var subquery = Session.QueryOver<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id))
.List<object[]>().Select(p => p[0]).ToArray();
var CorrespondingIds = Session.QueryOver<Table>(() => x)
.WhereRestrictionOn(() => x.Date).IsIn(subquery)
.Select(p => p.Id).List<int>().ToArray();
var result = Session.QueryOver<Table>(() => x).WhereRestrictionOn(() => x.Id).IsIn(CorrespondingIds).Left.JoinQueryOver(p => p.Sub).List();

Join several queries to optimise QueryOver query

I am using NHibernate and while traversing my code I came upon two functions that are called in sequence. They are probably a school example of 1) extra database round trip and 2) in-memory processing at the application side. The code involved is:
// 1) Getting the surveys in advance
var surveys = DatabaseServices.Session.QueryOver<Survey>()
.Where(x => x.AboutCompany.IsIn(companyAccounts.ToList()))
// Actual query that can be optimized
var unverifiedSurveys = DatabaseServices.Session.QueryOver<QuestionInSurvey>()
.Where(x => x.Survey.IsIn(surveys.ToList()))
.And(x => x.ApprovalStatus == status)
.List();
// 2) In-memory processing
return unverifiedSurveys.Select(x => x.Survey).Distinct()
.OrderByDescending(m => m.CreatedOn).ToList();
I have read that the actual Distinct() operation with the QueryOver API can be done using 1 .TransformUsing(Transformers.DistinctRootEntity)
Could anyone give an example how the queries can be combined thus having one round trip to the database and no application-side processing?

The most suitable way in this scenario is to use Subselect. We will firstly create the detached query (which will be executed as a part of main query)
Survey survey = null;
QueryOver<Survey> surveys = QueryOver.Of<Survey>(() => survey)
.Where(() => survey.AboutCompany.IsIn(companyAccounts.ToList()))
.Select(Projections.Distinct(Projections.Property(() => survey.ID)));
So, what we have now is a statement, which will return the inner select. Now the main query:
QuestionInSurvey question = null;
var query = session.QueryOver<QuestionInSurvey>(() => question)
.WithSubquery
.WhereProperty(() => qeustion.Survey.ID)
.In(subQuery) // here we will fitler the results
.And(() => question.ApprovalStatus == status)
.List();
And what we get is the:
SELECT ...
FROM QuestionInSurvey
WHERE SurveyId IN (SELECT SurveyID FROM Survey ...)
So, in one trip to DB we will recieve all the data, which will be completely filtered on DB side... so we are provided with "distinct" set of values, which could be even paged (Take(), Skip())

This might be something like this, which requires a distinct projection of all properties of Survey. I guess there is a better solution, but can not get to it ;-) Hope this will help anyway.
Survey surveyAlias = null;
var result =
session.QueryOver<QuestionInSurvey>()
.JoinAlias(x => x.Survey, () => surveyAlias)
.WhereRestrictionOn(() => surveyAlias.AboutCompany).IsIn(companyAccounts.ToList())
.And(x => x.ApprovalStatus == status)
.Select(
Projections.Distinct(
Projections.ProjectionList()
.Add(Projections.Property(() => surveyAlias.Id))
.Add(Projections.Property(() => surveyAlias.AboutCompany))
.Add(Projections.Property(() => surveyAlias.CreatedOn))
)
)
.OrderBy(Projections.Property(() => surveyAlias.CreatedOn)).Desc
.TransformUsing(Transformers.AliasToBean<Survey>())
.List<Survey>();