I have a linq query that returns the last page a user looked at based on a table of page hits. The fields are simply TimeStamp, UserID and URL which are logged from user activity. The query looks like this:
public static IQueryable GetUserStatus()
{
var ctx = new AppEntities();
var currentPageHits = ctx.Pagehits
.GroupBy(x => x.UserID)
.Select(x => x.Where(y => y.TimeStamp == x.Max(z => z.TimeStamp)))
.SelectMany(x => x);
return currentPageHits.OrderByDescending(o => o.TimeStamp);
}
The query works perfectly but runs slowly. Our DBA assures us that the table has indexes in all the right places and that the trouble must be with the query.
Is there anything inherently wrong or BAD with this, or is there a more efficient way of getting the same results?
You could try:
var currentPageHits2 = ctx.Pagehits
.GroupBy(x => x.UserID)
.Select(x => x.OrderByDescending(y => y.TimeStamp).First())
.OrderByDescending(x => x.TimeStamp);
But the speed should be the same.
Note that there is a subtle difference between this query and yours... With yours, if a UserId has two "max TimeStamp" PageHits with the same TimeStamp, two "rows" will be returned, with this one only one will be returned.
So you try to implement DENSE_RANK() OVER (PARTITION BY UserID ORDER BY TimeStamp DESC) with LINQ? So all latest records per user-group according to the Timestamp. You could try:
public static IQueryable GetUserStatus()
{
var ctx = new AppEntities();
var currentPageHits = ctx.Pagehits
.GroupBy(x => x.UserID)
.SelectMany(x => x.GroupBy(y => y.TimeStamp).OrderByDescending(g=> g.Key).FirstOrDefault())
.OrderByDescending(x => x.TimeStamp);
return currentPageHits;
}
So it's grouping the user-group by TimeStamp, then it takes the latest group(one or more records in case of ties). The SelectMany flattens the goups to records. I think this is more efficient than your query.
Related
So I have a table like this:
Now I want distinct ShortCode order by the ID descending. In other words, the distinct last records. Like this:
So I tried GroupBy like:
var data = db.ShortCodes.GroupBy(x => x.ShortCode).Select(x => x.FirstOrDefault()).OrderByDescending(s=> s.ID);
This gave me distinct records but not the last ones, nor ordered by ID descending:
Now I also tried like suggested here
var data = db.ShortCodeManager
.GroupBy(s => s. ShortCode)
.Select(g => g.First())
.OrderByDescending(s => s.ID);
This gave me the error The method 'First' can only be used as a final query operation. Consider using the method 'FirstOrDefault' in this instance instead.
So I modified to FirstOrDefault() like:
var data = db.ShortCodeManager
.GroupBy(s => s. ShortCode)
.Select(g => g.FirstOrDefault())
.OrderByDescending(s => s.ID);
This also gave me distinct records but not the last records:
So finally I tried like suggested here:
var data = db.ShortCodeManager.Where(a => a.ID > 0).GroupBy(x => x.ShortCode).OrderByDescending(grp => grp.Max(g => g.ID)).Select(a => a.FirstOrDefault());
Again, this gave me distinct records but not the last ones, nor ordered by ID descending:
So how am I to write the query to get the result I want in Linq? Also note, I need more of the distinct last records than ordering by ID descending. If anyone also knows how to write it in raw SQL it might be useful as well.
This LINQ query should work for your case:
var result = db.ShortCodeManager
.GroupBy(x => x.ShortCode)
.Select(gr => new { Id = gr.Max(g => g.Id), ShortCode = gr.Key})
.ToList();
EDIT:
Based on your comment it looks like you need to cast anonymous object result to ShortCodeManagerModel type and then pass it to your view. So, somethin like this:
var result = db.ShortCodeManager
.GroupBy(x => x.ShortCode)
.Select(gr => new { Id = gr.Max(g => g.Id), ShortCode = gr.Key})
.ToList();
var model = result
.Select(x => new ShortCodeManagerModel { Id = x.Id, ShortCode = x.ShortCode })
.ToList();
And then pass model to you view.
I am running a simple query against an Sql Server database using Entity Framework Core 2.2.6 however the GroupBy is not being executed on the server, instead it is being executed locally.
Is there something i'm missing that will force the group by onto the server?
The 2 variations of EF query i have tried:
public class Holiday
{
public int Id {get;set;}
public DateTime Date {get;set;}
public string Username {get;set;}
public string Approver {get;set;}
}
//version 1
await _dbContext.Holidays
.GroupBy(h => new { h.Date})
.ToDictionaryAsync(x => x.Key.Date, x => x.Select(x1 => x1.Username).ToList());
//version 2
await _dbContext.Holidays
.GroupBy(h => h.Date)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
Both variations produces the following SQL:
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY [h].[Date]
warning produced:
warnwarn: Microsoft.EntityFrameworkCore.Query[20500]
The LINQ expression 'GroupBy([h].Date, [h])' could not be translated and will be evaluated locally.
Suggestions from comments:
//group by string
await _dbContext.Holidays
.GroupBy(h => h.Username)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
//group by part of date
await _dbContext.Holidays
.GroupBy(h => h.Date.Year)
.ToDictionaryAsync(x => x.Key, x => x.Select(x1 => x1.Username).ToList());
--group by string
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY [h].[Username]
--group by part of date
SELECT [h].[Id], [h].[Approver], [h].[Date], [h].[HolidayTypeId], [h].[OwningRequestId], [h].[HolidayStatusId], [h].[Username]
FROM [Holidays] AS [h]
ORDER BY DATEPART(year, [h].[Date])
The problem is that when you're trying to group in the database, you don't really have the means to materialize values inside a group. You only get to SELECT grouped columns or aggregated values (via SUM, etc.) of non-grouped columns.
For example:
SELECT [h].[Date], [h].[Username]
FROM [Holidays] AS [h]
This query would produce result set of which every row would have two columns, date and name.
Let's try grouping though:
SELECT [h].[Date], [h].[Username]
FROM [Holidays] AS [h]
GROUP BY [h.Date]
This SQL query wouldn't be evaluated at all because it's invalid from SQL server perspective. Error message would be
Column 'Holidays.Username' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
Summing all this up, you can either do what #Ercan Tirman has suggested, or, load all the usernames and dates and group them in-memory:
var dateAndUsername = await _dbContext.Holidays
.Select(x => new {x.Date, x.Username})
.ToArrayAsync();
Dictionary<DateTime, List<string>> grouped = dateAndUsername
.GroupBy(x => x.Date)
.ToDictionary(x => x.Key, x => x.Select(y => y.Username).ToList());
It's because there is no SQL query like that.
Think like SQL. If you want to get Usernames by group of Dates, you need both of those.
Basically :
await _dbContext.Holidays
.GroupBy(h => new { h.Date, h.Username})
.Select(g => new
{
g.Key.Date,
g.Key.Username
});
This will produce a SQL query like this.
SELECT [h].[Date],[h].[Username]
FROM [Holidays] AS [h]
GROUP BY [h].[Date],[h].[Username]
After that you can use the data to create the structure of your dictionary however you want.
I am trying to convert the result of the query into IDictionary
Here string will contain orderId and the int will contain the TradedQuantity
The query below should join three objects Order, OrderRevision and OrderEvent.
1 Order can have many orderRevisions
1 OrderRevision can have many orderEvents
What the query is trying to do is to inner join three objects and get all order objects whose order id matches the list of orderids supplied to it. Then it does a group by based on orderId and gets the latest TradedQuantity from orderEvents object. LatestTradedQuantity will be the TradedQuantityFrom latest OrderEvent. For now the latest orderevent can be regarded as the one that has highest OrderEventId value.
OrderRevision revisionAlias = null;
Order orderAlias = null;
var query =
Session.QueryOver<OrderEvent>()
.JoinAlias(oe => oe.OrderRevision,() => revisionAlias)
.JoinAlias(oe => oe.OrderRevision.Order,() => orderAlias)
.Where(x => x.OrderRevision.Order.SourceSystem.Name.ToLower() == sourceSystem.ToLower())
.WhereRestrictionOn(x => x.OrderRevision.Order.Id).IsIn(orderIds.ToList())
.SelectList(list => list.SelectGroup(x => x.OrderRevision.Order.SourceOrderIdentifier)
.SelectMax(x => x.Id).Select(x => x.TradedQuantity))
.Select(x => new KeyValuePair<string, int?>(x.OrderRevision.Order.SourceOrderIdentifier, x.TradedQuantity)
);
As this query does not do what is supposed to. Could you please help and let me know how the result can be cast into IDictionary?
You have tagged your question with linq-to-nhibernate, so I guess using it instead of queryover would suit you. With Linq, use a sub-query for selecting the "max" order events ids for each order, then query them and project them to a dictionary.
using System.Linq;
using NHibernate.Linq;
...
var orderEventsIdsQuery = Session.Query<OrderEvent>()
.Where(oe => orderIds.Contains(oe.OrderRevision.Order.Id))
.GroupBy(oe => oe.OrderRevision.Order.SourceOrderIdentifier,
(soi, oes) => oes.Max(oe => oe.Id));
var result = Session.Query<OrderEvent>()
.Where(oe => orderEventsIdsQuery.Contains(oe.Id))
.ToDictionary(oe => oe.OrderRevision.Order.SourceOrderIdentifier,
oe => oe.TradedQuantity);
This should do the job. I do not use QueryOver and I will not try to give an answer for doing it with QueryOver.
I have Personne (person) entity with sub collections Polices and Procedures. I want to load a batch of Personnes with those collections loaded as well, using the power of Future queries. But! I was asked not to duplicate the columns of Personne in the queries getting the Polices and Procedures, which seems wise to optimize the amount of data traveling from database to server.
Here is what I ended up doing.
public IList<Personne> GetForDTO(IList<int> ids)
{
IEnumerable<Personne> query = NHibernateSession.Current.Query<Personne>()
.Fetch(x => x.Adresse)
.Where(x => ids.Contains(x.Id))
.ToFuture();
var queryWithPolices = NHibernateSession.Current.Query<Personne>()
.FetchMany(x => x.Polices)
.Where(x => ids.Contains(x.Id))
.Select(x => new
{
x.Id,
x.Polices
})
.ToFuture();
var queryWithProcedures = NHibernateSession.Current.Query<Personne>()
.FetchMany(x => x.Procedures)
.Where(x => ids.Contains(x.Id))
.Select(x => new
{
x.Id,
x.Procedures
})
.ToFuture();
return query.ToList();
}
This query doesn't work, with error:
Collections.IList' cannot be used for parameter of type 'System.Collections.Generic.IEnumerable`1[System.Object]'
It does work if I remove the Select() line call, but of course it results in the Personne columns being fetched as well as the Polices and Procedures columns.
Any idea of to fix this?
change return type 'IList' to 'IEnumerable'
Change your return type to IEnumerable:
public IIEnumerable<Personne> GetForDTO(IList<int> ids)
I'd like to resolve that problem :
SELECT Max(Date)
FROM Table
GROUP BY SubId
(Then pass it as a SubQuery to mid-action so I can get the Id of the item in Table)
SELECT Id
FROM Table
WHERE Date in
[[[ previous request ]]]
(Then Get the full Table Item with other table join)
SELECT *
FROM Table
LEFT JOIN...
WHERE Id in
[[[ previous request ]]]
I tried this kind of request :
var subquery = QueryOver.Of<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id)
);
var filter = QueryOver.Of<Table>().WithSubquery.
WhereExists(subquery)
.Select(p => p.Id);
var result = Session.QueryOver<Table>().WithSubquery.WhereProperty(p => p.Id).In(filter).Left.JoinQueryOver(p => p.Sub).List();
But the problem is that I can't get the first request right with only the date out of my request.
Is there a better way to do it than that kind of subqueries ? And is there a possibility in NHibernate to Groupy By a Property without selecting it ?
Thanks !
Finally did it that way and it generated the SQL i wanted. But it wasn't 3 subqueries exactly it was 3 queries looking in a set of datas (The arrays subquery and CorrespondingIds).
var subquery = Session.QueryOver<Table>(() => x)
.SelectList(list => list
.SelectMax(() => x.Date)
.SelectGroup(() => x.Sub.Id))
.List<object[]>().Select(p => p[0]).ToArray();
var CorrespondingIds = Session.QueryOver<Table>(() => x)
.WhereRestrictionOn(() => x.Date).IsIn(subquery)
.Select(p => p.Id).List<int>().ToArray();
var result = Session.QueryOver<Table>(() => x).WhereRestrictionOn(() => x.Id).IsIn(CorrespondingIds).Left.JoinQueryOver(p => p.Sub).List();