I'd like to resolve that problem :
SELECT Max(Date)
FROM Table
GROUP BY SubId
(Then pass it as a SubQuery to mid-action so I can get the Id of the item in Table)
SELECT Id
FROM Table
WHERE Date in
[[[ previous request ]]]
(Then Get the full Table Item with other table join)
SELECT *
FROM Table
LEFT JOIN...
WHERE Id in
[[[ previous request ]]]
I tried this kind of request :
var subquery = QueryOver.Of<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id)
);
var filter = QueryOver.Of<Table>().WithSubquery.
WhereExists(subquery)
.Select(p => p.Id);
var result = Session.QueryOver<Table>().WithSubquery.WhereProperty(p => p.Id).In(filter).Left.JoinQueryOver(p => p.Sub).List();
But the problem is that I can't get the first request right with only the date out of my request.
Is there a better way to do it than that kind of subqueries ? And is there a possibility in NHibernate to Groupy By a Property without selecting it ?
Thanks !
Finally did it that way and it generated the SQL i wanted. But it wasn't 3 subqueries exactly it was 3 queries looking in a set of datas (The arrays subquery and CorrespondingIds).
var subquery = Session.QueryOver<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id))
.List<object[]>().Select(p => p[0]).ToArray();
var CorrespondingIds = Session.QueryOver<Table>(() => x)
.WhereRestrictionOn(() => x.Date).IsIn(subquery)
.Select(p => p.Id).List<int>().ToArray();
var result = Session.QueryOver<Table>(() => x).WhereRestrictionOn(() => x.Id).IsIn(CorrespondingIds).Left.JoinQueryOver(p => p.Sub).List();
Related
I have a table with the following structure (and sample data):
Identifier
UseDate
PartId
a123
05/01/2000
237
a123
05/01/2000
4656
a123
01/01/2000
2134
a124
04/01/2000
5234
a124
01/01/2000
2890
I need to get the most recent entry of every (non-unique) identifier, but at most one per identifier.
The SQL-Query (MariaDB) that seems to fulfill my problem is the following:
SELECT a.Identifier, a.MaxDate, b.PartId, b.UseDate
FROM
(SELECT Identifier, MAX(UseDate) AS MaxDate FROM MyTable GROUP BY Identifier) a
LEFT JOIN MyTable b ON a.Identifier = b.Identifier
WHERE a.MaxDate = b.UseDate GROUP BY a.Identifier;
However I need this to work with C# and EF Core (Pomelo.EntitiFrameworkCore.MySql 5.0.3), my attempts have been:
var q1 = db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => new { Identifier = t.Key, MaxDate = t.Max(x => x.UseDate) });
return new ObjectResult(db.MyTable
.Join(
q1,
t1 => t1.Identifier,
t2 => t2.Identifier,
(t1, t2) => new { Identifier = t2.Identifier, PartId = t1.PartId, MaxDate = t1.MaxDate, UseDate = t1.UseDate })
.Where(t => t.UseDate == q1.First(x => x.Identifier == t.Identifier).MaxDate)
.GroupBy(t => t.Identifier)
.ToList()
);
and
return new ObjectResult(db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => t.OrderByDescending(x => x.UseDate).FirstOrDefault())
.ToList()
);
The first one throws this error:
System.InvalidOperationException: "Unable to translate the given 'GroupBy' pattern. Call 'AsEnumerable' before 'GroupBy' to evaluate it client-side."
The second one essentially yields the same, just complaining about the LINQ expression instead of the GroupBy.
I want to avoid using raw SQL, but how do I correctly (and hopefully efficiently) implement this?
There are many ways to write such query in LINQ, with most of them being able to be translated by EF Core 5/6+.
The straightforward approach once you have defined a subquery for the necessary grouping and aggregates is to join it to the data table, but not with join operator - instead, use row limiting correlated subquery (SelectMany with Where and Take), e.g.
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => new { Identifier = t.Key, MaxDate = t.Max(x => x.UseDate) })
.SelectMany(g => db.MyTable
.Where(t => t.Identifier == g.Identifier && t.UseDate == g.MaxDate)
.Take(1));
If the ordering field is unique per each other key value (i.e. in your case if UseDate is unique per each unique Identifier value), you can use directly Join operator (since lo limiting is needed), e.g.
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(t => new { Identifier = t.Key, MaxDate = t.Max(x => x.UseDate) });
.Join(db.MyTable,
g => new { g.Identifier, UseDate = g.MaxDate },
t => new { t.Identifier, t.UseDate },
(g, t) => t);
or directly apply Max based Where condition to the data table:
var query = db.MyTable
.Where(t => t.UseDate == db.MyTable
.Where(t2 => t2.Identifier == t.Identifier)
.Max(t2 => t2.UseDate)
);
Finally, the "standard" LINQ way of getting top 1 item per group.
For EF Core 6.0+:
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(g => g
.OrderByDescending(t => t.UseDate)
.First());
For EF Core 5.0 the grouping result set inside the query must be emulated:
var query = db.MyTable
.GroupBy(t => t.Identifier)
.Select(g => db.MyTable
.Where(t => t.Identifier == g.Key)
.OrderByDescending(t => t.UseDate)
.First());
So I have a table like this:
Now I want distinct ShortCode order by the ID descending. In other words, the distinct last records. Like this:
So I tried GroupBy like:
var data = db.ShortCodes.GroupBy(x => x.ShortCode).Select(x => x.FirstOrDefault()).OrderByDescending(s=> s.ID);
This gave me distinct records but not the last ones, nor ordered by ID descending:
Now I also tried like suggested here
var data = db.ShortCodeManager
.GroupBy(s => s. ShortCode)
.Select(g => g.First())
.OrderByDescending(s => s.ID);
This gave me the error The method 'First' can only be used as a final query operation. Consider using the method 'FirstOrDefault' in this instance instead.
So I modified to FirstOrDefault() like:
var data = db.ShortCodeManager
.GroupBy(s => s. ShortCode)
.Select(g => g.FirstOrDefault())
.OrderByDescending(s => s.ID);
This also gave me distinct records but not the last records:
So finally I tried like suggested here:
var data = db.ShortCodeManager.Where(a => a.ID > 0).GroupBy(x => x.ShortCode).OrderByDescending(grp => grp.Max(g => g.ID)).Select(a => a.FirstOrDefault());
Again, this gave me distinct records but not the last ones, nor ordered by ID descending:
So how am I to write the query to get the result I want in Linq? Also note, I need more of the distinct last records than ordering by ID descending. If anyone also knows how to write it in raw SQL it might be useful as well.
This LINQ query should work for your case:
var result = db.ShortCodeManager
.GroupBy(x => x.ShortCode)
.Select(gr => new { Id = gr.Max(g => g.Id), ShortCode = gr.Key})
.ToList();
EDIT:
Based on your comment it looks like you need to cast anonymous object result to ShortCodeManagerModel type and then pass it to your view. So, somethin like this:
var result = db.ShortCodeManager
.GroupBy(x => x.ShortCode)
.Select(gr => new { Id = gr.Max(g => g.Id), ShortCode = gr.Key})
.ToList();
var model = result
.Select(x => new ShortCodeManagerModel { Id = x.Id, ShortCode = x.ShortCode })
.ToList();
And then pass model to you view.
I am running a simple query against an Sql Server database using Entity Framework Core 2.2.6 however the GroupBy is not being executed on the server, instead it is being executed locally.
Is there something i'm missing that will force the group by onto the server?
The 2 variations of EF query i have tried:
public class Holiday
{
public int Id {get;set;}
public DateTime Date {get;set;}
public string Username {get;set;}
public string Approver {get;set;}
}
//version 1
await _dbContext.Holidays
.GroupBy(h => new { h.Date})
.ToDictionaryAsync(x => x.Key.Date, x => x.Select(x1 => x1.Username).ToList());
//version 2
await _dbContext.Holidays
.GroupBy(h => h.Date)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
Both variations produces the following SQL:
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY [h].[Date]
warning produced:
warnwarn: Microsoft.EntityFrameworkCore.Query[20500]
The LINQ expression 'GroupBy([h].Date, [h])' could not be translated and will be evaluated locally.
Suggestions from comments:
//group by string
await _dbContext.Holidays
.GroupBy(h => h.Username)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
//group by part of date
await _dbContext.Holidays
.GroupBy(h => h.Date.Year)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
--group by string
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY [h].[Username]
--group by part of date
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY DATEPART(year, [h].[Date])
The problem is that when you're trying to group in the database, you don't really have the means to materialize values inside a group. You only get to SELECT grouped columns or aggregated values (via SUM, etc.) of non-grouped columns.
For example:
SELECT [h].[Date], [h].[Username]
FROM [Holidays] AS [h]
This query would produce result set of which every row would have two columns, date and name.
Let's try grouping though:
SELECT [h].[Date], [h].[Username]
FROM [Holidays] AS [h]
GROUP BY [h.Date]
This SQL query wouldn't be evaluated at all because it's invalid from SQL server perspective. Error message would be
Column 'Holidays.Username' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
Summing all this up, you can either do what #Ercan Tirman has suggested, or, load all the usernames and dates and group them in-memory:
var dateAndUsername = await _dbContext.Holidays
.Select(x => new {x.Date, x.Username})
.ToArrayAsync();
Dictionary<DateTime, List<string>> grouped = dateAndUsername
.GroupBy(x => x.Date)
.ToDictionary(x => x.Key, x => x.Select(y => y.Username).ToList());
It's because there is no SQL query like that.
Think like SQL. If you want to get Usernames by group of Dates, you need both of those.
Basically :
await _dbContext.Holidays
.GroupBy(h => new { h.Date, h.Username})
.Select(g => new
{
g.Key.Date,
g.Key.Username
});
This will produce a SQL query like this.
SELECT [h].[Date],[h].[Username]
FROM [Holidays] AS [h]
GROUP BY [h].[Date],[h].[Username]
After that you can use the data to create the structure of your dictionary however you want.
I'm working on a feed system where I need to show a certain number of feeds and then showing more by clicking a "Load more" button.
So I've defined a number of rows to fetch from the database (take) and everytime I push the Load More button I send the skip number (the number of feeds already shown). The problem is that I'm getting duplicated rows with my query.
I want to use something like DISTINCT to skip those duplicates, so I've read I should use TransformUsing(Transformers.DistinctRootEntity), but that function works code-side and not db-side, so, when I try to apply skip and take, the number of feeds is wrong. Let's say I have 5 feeds, but I got 2 duplicates so when I apply the take function, the result just have 4 feeds.
This is my code, I appreciate any help
public IList<FeedBO> GetFeedsByUserSkills(int companyId, List<SkillBO> userSkills, int? skip = null, int? take = null)
{
FeedSkillBO feedSkillAlias = null;
var query = session.QueryOver<FeedBO>()
.JoinAlias(x => x.FeedSkills, () => feedSkillAlias)
.Where(Restrictions.Disjunction()
.Add(Restrictions.Conjunction()
.Add(Restrictions.On(() => feedSkillAlias.Skill).IsIn(userSkills))
.Add(Restrictions.Eq("Company.Id", companyId)))
.Add(Restrictions.Eq("Scope", Constants.FEED_SCOPE_GLOBAL)))
.TransformUsing(Transformers.DistinctRootEntity);
if (skip.HasValue) query.Skip(skip.Value);
if (take.HasValue) query.Take(take.Value);
var sql = GetGeneratedSql(query);
return query.OrderBy(NHibernate.Criterion.Projections.Property("CreationDate")).Desc.List();
}
UPDATE
I gave up with Transform and I'm now trying to do a group by:
var query = session.QueryOver<FeedBO>()
.JoinAlias(x => x.FeedSkills, () => feedSkillAlias)
.Where(Restrictions.Disjunction()
.Add(Restrictions.Conjunction()
.Add(Restrictions.On(() => feedSkillAlias.Skill).IsIn(userSkills))
.Add(Restrictions.Eq("Company.Id", companyId)))
.Add(Restrictions.Eq("Scope", Constants.FEED_SCOPE_GLOBAL)))
.Select(
Projections.Distinct(
Projections.ProjectionList()
.Add(Projections.Property<FeedBO>(x => x.Id).As("Id"))
.Add(Projections.Property<FeedBO>(x => x.Uuid).As("Uuid"))
.Add(Projections.Property<FeedBO>(x => x.Title).As("Title"))
.Add(Projections.Property<FeedBO>(x => x.Content).As("Content"))
.Add(Projections.Property<FeedBO>(x => x.Link).As("Link"))
.Add(Projections.Property<FeedBO>(x => x.ImagePreview).As("ImagePreview"))
.Add(Projections.Property<FeedBO>(x => x.Scope).As("Scope"))
.Add(Projections.Property<FeedBO>(x => x.EventDate).As("EventDate"))
.Add(Projections.Property<FeedBO>(x => x.CreationDate).As("CreationDate"))
.Add(Projections.Property<FeedBO>(x => x.CreationUser).As("CreationUser"))
.Add(Projections.Property<FeedBO>(x => x.Active).As("Active"))
.Add(Projections.Property<FeedBO>(x => x.Company).As("Company"))
.Add(Projections.Property<FeedBO>(x => x.FeedSkills).As("FeedSkills"))
.Add(Projections.Property<FeedBO>(x => x.Likes).As("Likes"))
)
).TransformUsing(Transformers.AliasToBeanConstructor(typeof(FeedBO).GetConstructors().First()))
.OrderBy(NHibernate.Criterion.Projections.Property("CreationDate")).Desc;
if (skip.HasValue) query.Skip(skip.Value);
if (take.HasValue) query.Take(take.Value);
And this generates the following SQL:
SELECT distinct this_.id as y0_, this_.uuid as y1_, this_.title as y2_, this_.content as y3_, this_.link as y4_, this_.image_preview as y5_, this_.scope as y6_, this_.event_date as y7_, this_.creation_date as y8_, this_.creation_user_id as y9_, this_.active as y10_, this_.company_id as y11_, this_.id as y12_, this_.id as y12_
FROM dbo.FEED this_ inner join dbo.FEED_SKILL feedskilla1_ on this_.id=feedskilla1_.feed_id
WHERE ((feedskilla1_.skill_id in (2, 1, 24) and this_.company_id = 1) or this_.scope = 'global')
ORDER BY this_.creation_date desc OFFSET 0 ROWS FETCH FIRST 5 ROWS ONLY
Which works if I run it directly in the db, but when I run the program, it throws this exception
index was outside the bounds of the array
I don't know what to do, help!!!!
Import nHibernate.Linq
Replace QueryOver<> with Query<> which implements System.Linq and simplify your query.
You then would simply admin .Distinct() to your linq. No need to get into the unnecessary complexity!
Note that you can only join relations that you have defined in mapping.
I have a linq query that returns the last page a user looked at based on a table of page hits. The fields are simply TimeStamp, UserID and URL which are logged from user activity. The query looks like this:
public static IQueryable GetUserStatus()
{
var ctx = new AppEntities();
var currentPageHits = ctx.Pagehits
.GroupBy(x => x.UserID)
.Select(x => x.Where(y => y.TimeStamp == x.Max(z => z.TimeStamp)))
.SelectMany(x => x);
return currentPageHits.OrderByDescending(o => o.TimeStamp);
}
The query works perfectly but runs slowly. Our DBA assures us that the table has indexes in all the right places and that the trouble must be with the query.
Is there anything inherently wrong or BAD with this, or is there a more efficient way of getting the same results?
You could try:
var currentPageHits2 = ctx.Pagehits
.GroupBy(x => x.UserID)
.Select(x => x.OrderByDescending(y => y.TimeStamp).First())
.OrderByDescending(x => x.TimeStamp);
But the speed should be the same.
Note that there is a subtle difference between this query and yours... With yours, if a UserId has two "max TimeStamp" PageHits with the same TimeStamp, two "rows" will be returned, with this one only one will be returned.
So you try to implement DENSE_RANK() OVER (PARTITION BY UserID ORDER BY TimeStamp DESC) with LINQ? So all latest records per user-group according to the Timestamp. You could try:
public static IQueryable GetUserStatus()
{
var ctx = new AppEntities();
var currentPageHits = ctx.Pagehits
.GroupBy(x => x.UserID)
.SelectMany(x => x.GroupBy(y => y.TimeStamp).OrderByDescending(g=> g.Key).FirstOrDefault())
.OrderByDescending(x => x.TimeStamp);
return currentPageHits;
}
So it's grouping the user-group by TimeStamp, then it takes the latest group(one or more records in case of ties). The SelectMany flattens the goups to records. I think this is more efficient than your query.