I have a table that shows a list of sync's from our mobile users back to our database. This means that each user could have thousands of sync records.
I have written a query that uses the ROW_NUMBER() function to pull the most recent sync for every user and only active users, as I don't want to see sync'd data from terminated employees. (i.e. User A sync'd yesterday at noon, User A sync'd today at noon but I only want to see the sync from today).
SELECT * FROM
(
SELECT *, ROW_NUMBER() OVER (PARTITION BY [SerialNumber] ORDER BY SyncDate DESC )as RN
FROM [TSCH].[dbo].[SYNCREPORT]
) as T
Where RN = 1 and WorkerStatus = 'ACTIVE' and SerialNumber = ######;
What would the best approach for writing this using LINQ in c# for my .net web application? Thanks for the help!
Could be something like this
var result=yourtable.OrderBy(x=>x.SyncDate).GroupBy(x=>x.SerialNumber)
.Where(x=>x.WorkerStatus=="Active" && x.SerialNumber=="####")
.Select(g => new {g, count= g.Count()})
.SelectMany(t => t.g.Select(b => b)
.Zip(Enumerable.Range(1,t.count), (c,i) => new {c.value1, c,value2, rn = i}));
Related
I'm trying implement the follow query in LINQ, but I don't find solution:
SQL:
SELECT COUNT(*) AS AmountMonths
FROM (SELECT SUBSTRING(CONVERT(NVARCHAR(12), pay_date, 112), 1, 6) AS Month
FROM #tmp
GROUP BY SUBSTRING(CONVERT(NVARCHAR(12), pay_date, 112), 1, 6)) AS AmountMonths
What I need is get the amounts of months in which the clients made payments, with the condition that there may be months in which no payments have been made.
In C# I tried the following:
int amountMonths = payDetail.GroupBy(x => Convert.ToDateTime(x.PayDate)).Count();
and
int amountMonths = payDetail.GroupBy(x => Convert.ToDateTime(x.PayDate).Month).Count();
But I am not getting the expected result.
(Assuming you're using EF Core)
You're almost there. You could do:
var amountMonths = context.AmountMonths.GroupBy(c => new { c.PayDate.Year, c.PayDate.Month }).Count();
This will translate to something like:
SELECT COUNT(*)
FROM (
SELECT DATEPART(year, [a].[PayDate]) AS [a]
FROM [AmountMonths] AS [a]
GROUP BY DATEPART(year, [a].[PayDate]), DATEPART(month, [a].[Pay_Date])
) AS [t]
which I'd find preferable over creating a string and chopping it up. EOMONTH isn't a standard mapped function, alas, otherwise it can be used to convert a date to month level granularity
I have a Blazor Web Application that has been working and in the field for a few months. I want to extend the DB querying to the group of similar "Detections".
It was written starting with .NET 5, and just today was updated to .NET 6 trying and get this working.
I would like to know how to get the results ordered by TimeStamp (a DateTime property). I have a working example with an in-memory DB, but production will be in SQL Server. I am not that great in SQL, but I have played around with it for a while in Management Studio with no luck.
Commenting out the OrderByDescending() groups things properly, but the results are not in the correct order. It seems the EF translation process is completely removing that line, it makes no difference in the generated query or the result set.
var results = context.Detections
//Line below makes no change ignored by SQL Server. Works when using in memory DB.
//.OrderByDescending(det => det.TimeStamp)
.GroupBy(det => new
{
Year = det.TimeStamp.Year,
Month = det.TimeStamp.Month,
Day = det.TimeStamp.Day,
Hour = det.TimeStamp.Hour,
})
.Select(grp => new
{
Count = grp.Count(),
Detection = grp.OrderByDescending(det => det.TimeStamp).First(),
})
//The following line will not translate
//.OrderByDescending(det => det.Detection.TimeStamp)
.ToList();
If any of this matters:
Visual Studio 2022 (4.8.04084)
.Net 6.0
SQL Server 2019 (15.0.2080.9)
*All NuGet packages related to EF have been updated to 6.0
Edit for clarification
The above code segment produces the following SQL query.
SELECT [t].[c], [t0].[Id], [t0].[TimeStamp]
FROM (
SELECT COUNT(*) AS [c], DATEPART(year, [d].[TimeStamp]) AS [c0], DATEPART(month, [d].[TimeStamp]) AS [c1], DATEPART(day, [d].[TimeStamp]) AS [c2], DATEPART(hour, [d].[TimeStamp]) AS [c3]
FROM [Detections] AS [d]
WHERE [d].[TimeStamp] > DATEADD(day, CAST(-16.0E0 AS int), GETUTCDATE())
GROUP BY DATEPART(year, [d].[TimeStamp]), DATEPART(month, [d].[TimeStamp]), DATEPART(day, [d].[TimeStamp]), DATEPART(hour, [d].[TimeStamp])
) AS [t]
OUTER APPLY (
SELECT TOP(1) [d0].[Id], [d0].[TimeStamp]
FROM [Detections] AS [d0]
WHERE ([d0].[TimeStamp] > DATEADD(day, CAST(-30.0E0 AS int), GETUTCDATE())) AND (((([t].[c0] = DATEPART(year, [d0].[TimeStamp])) AND ([t].[c1] = DATEPART(month, [d0].[TimeStamp]))) AND ([t].[c2] = DATEPART(day, [d0].[TimeStamp]))) AND ([t].[c3] = DATEPART(hour, [d0].[TimeStamp])))
ORDER BY [d0].[TimeStamp] DESC
) AS [t0]
It produces results similar to the following. Notice not sorted by time.
1 628591 2021-11-02 14:34:06.0442966
10 628601 2021-11-12 05:43:27.7015291
150 628821 2021-11-12 21:59:27.6444236
20 628621 2021-11-12 06:17:13.7798282
50 628671 2021-11-12 15:17:23.8893856
If I add ORDER BY [t0].TimeStamp DESC at the end of that SQL query in Management Studio I get the results I am looking for (see below). I just need to know how to write that in LINQ.
150 628821 2021-11-12 21:59:27.6444236
50 628671 2021-11-12 15:17:23.8893856
20 628621 2021-11-12 06:17:13.7798282
10 628601 2021-11-12 05:43:27.7015291
1 628591 2021-11-02 14:34:06.0442966
Adding .OrderByDescending(det => det.Detection.TimeStamp) at the end before ToList() was my first thought, but that "could not be translated". I will need to do some pagination with these results so I would really like to do the sorting in SQL.
GroupBy has to do its own Ordering so that 'ignores' is not totally unexpected.
Move it to below the grouping:
var results = context.Detections
//.OrderByDescending(det => det.TimeStamp)
.GroupBy(det => new
{
Year = det.TimeStamp.Year,
Month = det.TimeStamp.Month,
Day = det.TimeStamp.Day,
Hour = det.TimeStamp.Hour,
})
// .OrderByDescending(grp => grp.Key) // may have to split into y/m/d/h again
.OrderByDescending(grp => grp.Key.Year)
.ThenByDescending( grp => grp.Key.Month)
.ThenByDescending( grp => grp.Key.Day)
.ThenByDescending( grp => grp.Key.Hour)
.Select(grp => new
{
Count = grp.Count(),
Detection = grp.OrderByDescending(det => det.TimeStamp).First(),
})
.ToList();
When EF supports it, the Ordering and Grouping might become a little easier with
.GroupBy(det => new
{
Date = det.TimeStamp.Date,
Hour = det.TimeStamp.Hour,
})
For anyone looking at this in the future.
I was able to make this work by declaring and populating a TimeStamp property and using the OrderByDescending() at the end. I am not sure if this is the best solution, but it did solve my problem.
var results = context.Detections
.GroupBy(det => new
{
Year = det.TimeStamp.Year,
Month = det.TimeStamp.Month,
Day = det.TimeStamp.Day,
Hour = det.TimeStamp.Hour,
})
.Select(grp => new
{
Count = grp.Count(),
TimeStamp = grp.OrderByDescending(det => det.TimeStamp).First().TimeStamp,
Detection = grp.OrderByDescending(det => det.TimeStamp).First(),
})
.OrderByDescending(det => det.TimeStamp)
.ToList();
My problem is that I want to start a database query which should give me the last (maxDate) entry of every Serial number.
I am working with a Microsoft SQL Server database.
The first picture shows all entries in the database:
After I have run the following code I get this output:
string aQuery = #" SELECT *
FROM (
SELECT SerialNumber, MAX(Date) as MaxDate
FROM eBox_Deploy
GROUP BY SerialNumber
) r
INNER JOIN eBox_Deploy t
ON t.SerialNumber = r.SerialNumber AND t.Date = r.MaxDate";
using (var db = new eBoxDataContext())
{
list.AddRange(db.ExecuteQuery<eBox_Deploy>(bQuery));
}
After picture:
Now my problem is that I have duplicates because they already exists in the database. Distinct doesn't work well because these all have different Id´s.
How can I get them away?
You could use windowed functions:
SELECT *
FROM (SELECT *,
rn = ROW_NUMBER() OVER(PARTITION BY [SerialNumber] ORDER BY [Date] DESC)
FROM eBox_Deploy) AS sub
WHERE rn = 1;
If your [Date] is not unique within SerialNumber group use RANK() to get ties.
UPDATE
thanks to #usr I have got this down to ~3 seconds simply by changing
.Select(
log => log.OrderByDescending(
d => d.DateTimeUTC
).FirstOrDefault()
)
to
.Select(
log => log.OrderByDescending(
d => d.Id
).FirstOrDefault()
)
I have a database with two tables - Logs and Collectors - which I am using Entity Framework to read. There are 86 collector records and each one has 50000+ corresponding Log records.
I want to get the most recent log record for each collector which is easily done with this SQL
SELECT CollectorLogModels_1.Status, CollectorLogModels_1.NumericValue,
CollectorLogModels_1.StringValue, CollectorLogModels_1.DateTimeUTC,
CollectorSettingsModels.Target, CollectorSettingsModels.TypeName
FROM
(SELECT CollectorId, MAX(Id) AS Id
FROM CollectorLogModels GROUP BY CollectorId) AS RecentLogs
INNER JOIN CollectorLogModels AS CollectorLogModels_1
ON RecentLogs.Id = CollectorLogModels_1.Id
INNER JOIN CollectorSettingsModels
ON CollectorLogModels_1.CollectorId = CollectorSettingsModels.Id
This takes ~2 seconds to execute.
the closest I have been able to get with LINQ is the following
var logs = context.Logs.Include(co => co.Collector)
.GroupBy(
log => log.CollectorId, log => log
)
.Select(
log => log.OrderByDescending(
d => d.DateTimeUtc
).FirstOrDefault()
)
.Join(
context.Collectors,
(l => l.CollectorId),
(c => c.Id),
(l, c) => new
{
c.Target,
DateTimeUTC = l.DateTimeUtc,
l.Status,
l.StringValue,
CollectorName = c.TypeName
}
).OrderBy(
o => o.Target
).ThenBy(
o => o.CollectorName
)
;
This produces the results I want but takes ~35 seconds to execute.
This becomes the following SQL
SELECT
[Distinct1].[CollectorId] AS [CollectorId],
[Extent3].[Target] AS [Target],
[Limit1].[DateTimeUtc] AS [DateTimeUtc],
[Limit1].[Status] AS [Status],
[Limit1].[StringValue] AS [StringValue],
[Extent3].[TypeName] AS [TypeName]
FROM (SELECT DISTINCT
[Extent1].[CollectorId] AS [CollectorId]
FROM [dbo].[CollectorLogModels] AS [Extent1] ) AS [Distinct1]
OUTER APPLY (SELECT TOP (1) [Project2].[Status] AS [Status], [Project2].[StringValue] AS [StringValue], [Project2].[DateTimeUtc] AS [DateTimeUtc], [Project2].[CollectorId] AS [CollectorId]
FROM ( SELECT
[Extent2].[Status] AS [Status],
[Extent2].[StringValue] AS [StringValue],
[Extent2].[DateTimeUtc] AS [DateTimeUtc],
[Extent2].[CollectorId] AS [CollectorId]
FROM [dbo].[CollectorLogModels] AS [Extent2]
WHERE [Distinct1].[CollectorId] = [Extent2].[CollectorId]
) AS [Project2]
ORDER BY [Project2].[DateTimeUtc] DESC ) AS [Limit1]
INNER JOIN [dbo].[CollectorSettingsModels] AS [Extent3] ON [Limit1].[CollectorId] = [Extent3].[Id]
ORDER BY [Extent3].[Target] ASC, [Extent3].[TypeName] ASC
How can I get performance closer to what is achievable with SQL alone?
In your original SQL you can select a collection DateTimeUTC from a different row than the MAX(ID). That's probably a bug. The EF does not have that problem. It's not semantically identical, it is a harder query.
If you rewrite the EF query to be structurally the same as the SQL query you'll get identical performance. I see nothing here that EF would not support.
Compute the max(id) with EF as well and join on that.
I had the exact same issue, i solved it by adding indexes.
A query of mine would take 45 seconds to complete, i managed to get it completing in less than a second.
I have an SQL query which I want to call from LINQ to SQL in asp.net application.
SELECT TOP 5 *
FROM (SELECT SongId,
DateInserted,
ROW_NUMBER()
OVER(
PARTITION BY SongId
ORDER BY DateInserted DESC) rn
FROM DownloadHistory) t
WHERE t.rn = 1
ORDER BY DateInserted DESC
I don't know whether its possible or not through linq to sql, if not then please provide any other way around.
I think you'd have to change the SQL partition to a Linq group-by. (Effectively all the partition does is group by song, and select the newest row for each group.) So something like this:
IEnumerable<DownloadHistory> top5Results = DownloadHistory
// group by SongId
.GroupBy(row => row.SongId)
// for each group, select the newest row
.Select(grp =>
grp.OrderByDescending(historyItem => historyItem.DateInserted)
.FirstOrDefault()
)
// get the newest 5 from the results of the newest-1-per-song partition
.OrderByDescending(historyItem => historyItem.DateInserted)
.Take(5);
Although McGarnagle answer solves the problem, but when i see the execution plan for the two queries, it was really amazing to see that linq to sql was really too slow as compare to native sql queries. See the generated query for the above linq to sql:
--It took 99% of the two execution
SELECT TOP (5) [t3].[SongId], [t3].[DateInserted]
FROM (
SELECT [t0].[SongId]
FROM [dbo].[DownloadHistory] AS [t0]
GROUP BY [t0].[SongId]
) AS [t1]
OUTER APPLY (
SELECT TOP (1) [t2].[SongId], [t2].[DateInserted]
FROM [dbo].[DownloadHistory] AS [t2]
WHERE [t1].[SongId] = [t2].[SongId]
ORDER BY [t2].[DateInserted] DESC
) AS [t3]
ORDER BY [t3].[DateInserted] DESC
--It took 1% of the two execution
SELECT TOP 5 t.SongId,t.DateInserted
FROM (SELECT SongId,
DateInserted,
ROW_NUMBER()
OVER(
PARTITION BY SongId
ORDER BY DateInserted DESC) rn
FROM DownloadHistory) t
WHERE t.rn = 1
ORDER BY DateInserted DESC