Develop Reference

SQL Unions with table counts using EntityFramework LINQ query

I am trying replicate the SQL below using LINQ and Entity Framework and cannot figure out how this should be written.
My simplistic LINQ version does a query per table
public IActionResult Index()
{
dynamic view = new ExpandoObject();
view.AppUsers = Context.AppUsers.Count();
view.CustomerShops = Context.CustomerShops.Count();
view.FavouriteOrders = Context.FavouriteOrders.Count();
view.Items = Context.Items.Count();
view.ItemVariations = Context.ItemVariations.Count();
view.MenuCategories = Context.MenuCategories.Count();
view.MenuCategoryProducts = Context.MenuCategoryProducts.Count();
view.Orders = Context.Orders.Count();
view.Products = Context.Products.Count();
view.ProductVariations = Context.ProductVariations.Count();
view.Shops = Context.Shops.Count();
view.Staffs = Context.Staffs.Count();
return View(view);
}
I use this pattern from time to time to for reporting on my column counts and thought this should be easy to do in LINQ, but no luck so far.
This pure SQL UNION would only generate 1 SQL request, instead of a request per table.
select * from (
select 'asp_net_roles' as type, count(*) from asp_net_roles
union
select 'asp_net_user_roles' as type, count(*) from asp_net_user_roles
union
select 'asp_net_users' as type, count(*) from asp_net_users
union
select 'app_users' as type, count(*) from app_users
union
select 'shops' as type, count(*) from shops
union
select 'staffs' as type, count(*) from shops
union
select 'items' as type, count(*) from items
union
select 'item_variations' as type, count(*) from item_variations
union
select 'products' as type, count(*) from products
union
select 'product_variations' as type, count(*) from product_variations
union
select 'menu_categories' as type, count(*) from menu_categories
) as counters
order by 1;
I saw a partial implementation [linq-group-by-multiple-tables] (https://stackoverflow.com/a/3435503/473923) but this is based of grouping data.
FYI: I'm new to C#/Linq, so sorry if this seams obvious.

Use the this code from my answer
And fill ExpandoObject with result:
var tablesinfo = Context.GetTablesInfo();
var expando = new ExpandoObject();
if (tablesinfo != null)
{
var dic = (IDictionary<string, object>)expando;
foreach(var info in tablesinfo)
{
dic.Add(info.TableName, info.RecordCount);
}
}
Idea is that you can UNION counts if you group entities by constant.
Schematically function builds the following IQueryable Expression:
var tablesinfo =
Context.AppUsers.GroupBy(x => 1).Select(g => new TableInfo{ TableName = "asp_net_roles", RecordCount = g.Count() })
.Concat(Context.MenuCategories.GroupBy(x => 1).Select(g => new TableInfo{ TableName = "menu_categories", RecordCount = g.Count() }))
.Concat(Context.Items.GroupBy(x => 1).Select(g => new TableInfo{ TableName = "items", RecordCount = g.Count() }))
....

There is nothing wrong with your LINQ query. It's very acceptable approach. However it's not the most efficient.
There is no need to fetch count from individual tables one by one. You can get the counts from all the tables at once using the System tables Sys.Objects and Sys.Partitions. Just try running this query in your database.
SELECT A.Name AS TableName, SUM(B.rows) AS RecordCount
FROM sys.objects A INNER JOIN sys.partitions B
ON A.object_id = B.object_id
WHERE A.type = 'U' AND B.index_id IN (0, 1)
GROUP BY A.Name
For quick response and cleaner code, you can store this SQL query in a string variable, and run the LINQ
var result = dataContext.ExecuteQuery<YOUR_MODEL_CLASS>
(your_string_query);

I would put something like this:
Dictionary<string, int> view = new() {
new() {'asp_net_roles', Context.AppUsers.Count() },
...
}
return View(view);
maybe not the most pure way, but does the job (unless I misunderstood what you try to accomplish)

SQL to Linq conversion returning default value in subquery

I have a ticketing system that I am trying to get some basic information of the last ticket note added, based on what tickets are still open.
Here are the tables summarized, including extra columns that I'm not querying:
Ticket table
TicketId
CustomerId
DateIn
CallNature
OpenClosed
TicketDetails table
TicketDetailsId
TicketId
TicketNote
DateLogged
Here is the SQL query:
SELECT
t.TicketId,
t.CustomerId,
t.DateIn,
(SELECT TOP 1 td.DateLogged
FROM TicketDetails td
WHERE td.TicketId = t.TicketId
ORDER BY td.DateLogged DESC) DateLogged
FROM
Tickets t
WHERE
t.OpenClosed = 1
Here is my current Linq query:
var result = from t in ef.Tickets
where t.OpenClosed == true
select new
{
TicketId = t.TicketId,
CustomerId = t.CustomerId,
DateIn = t.DateIn,
DateLogged = (from td in ef.TicketDetails
where td.TicketId == t.TicketId
orderby td.DateLogged descending
select td.DateLogged).Take(1)
};
Here is a result sample of the Linq query:
TicketId = 11000
CustomerId = 4622
DateIn = 2018-01-25T00:00:00
DateLogged = 0001-01-01T00:00:00
should be:
TicketId = 11000
CustomerId = 4622
DateIn = 2018-01-25T00:00:00
DateLogged = 2018-12-12T13:32:42
I don't have all the fields in the example but they are irrelevant to the question.
When I run the SQL query, the results are as expected. When I run the Linq query, all fields are populated except for the DateLogged, it keeps returning default value (0001-01-01).
I have confirmed that each Ticket has at least 1 TicketDetail to return.

Final solution thanks to Neil's reference to deferred execution is:
var result = (from t in ef.Tickets
where t.OpenClosed == true
select new {
TicketId = t.TicketId,
CustomerId = t.CustomerId,
DateIn = t.DateIn,
DateLogged = (from td in ef.TicketDetails
where td.TicketId == t.TicketId
orderby td.DateLogged descending
select td.DateLogged).Take(1).FirstOrDefault()
}).ToList();
And a link to a post explaining difference between Linq query execution.
https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/query-execution

Inefficient LINQ - Know What SQL Should Be - Can't get there

I know the SQL I want to produce:
SELECT qdt, pbn, cid, pid, SUM(Amount) AS TotalAmount
FROM SomeDb.SomeTable
WHERE Status = 'Open'
GROUP BY cid, pid, qdt, pbn
ORDER BY qdt
I have LINQ, that I was hoping would produce something as clean as the above SQL:
var query = (
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new { someTable.cid, someTable.pid } into grouped
select new
{
lid = grouped.FirstOrDefault().lid,
qdt = grouped.FirstOrDefault().qdt,
pbn = grouped.FirstOrDefault().pbn,
cid = grouped.FirstOrDefault().cid,
cn = grouped.FirstOrDefault().cn,
pid = grouped.FirstOrDefault().pid,
amount = grouped.Sum(o => o.Amount),
Status = grouped.FirstOrDefault().Status
});
But that produces many lines of nasty SQL. Each grouped value ends up getting its own SELECT statement within the query, like this, for qdt:
SELECT [t5].[qdt]
FROM (
SELECT TOP (1) [t4].[qdt]
FROM [SomeDb].[SomeTable] AS [t4]
WHERE ([t1].[cid] = [t4].[cid]) AND ([t1].[pid] = [t4].[pid]) AND ([t4].[Status] = #p0)
) AS [t5]
) AS [qdt]
Is there a way to change the LINQ to produce the simpler SQL?

There are two issues I see with your LINQ attempt. For starters, you are not grouping by the same fields in the SQL query as in your LINQ. Then, you are using FirstOrDefault on the results of the group instead of selecting the group's key data.
Your query should look like this:
var query =
(
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new
{
someTable.lid,
someTable.qdt,
someTable.pbn,
someTable.cid,
someTable.cn,
someTable.pid,
someTable.Status,
} into grouped
select new
{
lid = grouped.Key.lid,
qdt = grouped.Key.qdt,
pbn = grouped.Key.pbn,
cid = grouped.Key.cid,
cn = grouped.Key.cn,
pid = grouped.Key.pid,
amount = grouped.Sum(o => o.Amount),
Status = grouped.Key.Status
}
);
I cannot test right now whether it will generate the exact same SQL though.

You don't seem to have all the grouped by columns in the desired sql in your linq;
var query = (
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new { someTable.cid, someTable.pid, someTable.qdt,someTable.pbn, someTable.Status } into grouped
select new
{
qdt = grouped.FirstOrDefault().qdt,
pbn = grouped.FirstOrDefault().pbn,
cid = grouped.FirstOrDefault().cid,
pid = grouped.FirstOrDefault().pid,
Status = grouped.FirstOrDefault().Status,
amount = grouped.Sum(o => o.Amount)
});
I can't really test it but that looks a little more like the sql you are trying to replicate in linq.

Linq join and group by together on multiple fields and truncate time in date

I have following Sql query which joins two table and group by date and user Id
User Table
UserName UserId
LookupScannedHistory
HistoryId UserId ScannedDate ScannedCount
I have written following sql query to join table and group by on userId and Scanned Date removing Time varaint
SELECT
l.[UserID]
,CAST([ScannedDate] AS DATE)
,Sum([ScannedCount])
,[UserName]
FROM [dbo].[LookupScannedHistory] l
Join [dbo].[UserMaster] u
on l.UserID = u.UserId
group by l.UserId, u.UserName, CAST([ScannedDate] AS DATE)
I want to convert this into linq.
I have tried this
(from log in dataContext.LookupScannedHistories
join user in dataContext.UserMasters
on log.UserID equals user.UserId
where log.ScannedDate >= fiveDayPriorDate
orderby log.ScannedDate
group log by new
{
ScannedDateOnly = EntityFunctions.TruncateTime(log.ScannedDate),
log.UserID
} into dateClickedHistory
select dateClickedHistory
).ToList();
I have done this this gives what I want but I am unable to include username by combining join and group by

I found the solution.
var usersReportGroupList = (from log in dataContext.LookupScannedHistories
join user in dataContext.UserMasters
on log.UserID equals user.UserId
where log.ScannedDate >= fiveDayPriorDate
orderby log.ScannedDate
group new { user.UserName, log.ScannedCount,
log.ScannedDate}
by new
{
ScannedDateOnly =
EntityFunctions.TruncateTime(log.ScannedDate),
log.UserID,
user.UserName
}
into dateClickedHistory
select dateClickedHistory
).ToList();
var usersReportList = new List<LookupScannedHistoryDetails>();
foreach (var group in usersReportGroupList)
{
usersReportList.Add(new LookupScannedHistoryDetails()
{
ScannedCount = group.Sum(x => x.ScannedCount.Value),
ScannedDate = group.Key.ScannedDateOnly.Value,
UserId = group.Key.UserID.Value,
UniqueCount = group.Count(),
UserName = group.Key.UserName
});
}

Complicated SQL Server query

I am trying to write an SQL (Server) query which will return all events on a current day, and for all events where the column recurring= 1, I want it to return this event on the day it is being held and for the subsequent 52 weeks following the event.
My tables are structured as followed :
Event
{
event_id (PK)
title,
description,
event_start DATETIME,
event_end DATETIME,
group_id,
recurring
}
Users
{
UserID (PK)
Username
}
Groups
{
GroupID (PK)
GroupName
}
Membership
{
UserID (FK)
GroupID (FK)
}
The code I have thus far is as follows :
var db = Database.Open("mPlan");
string username = HttpContext.Current.Request.Cookies.Get("mpUsername").Value;
var listOfGroups = db.Query("SELECT GroupID FROM Membership WHERE UserID = (SELECT UserID from Users WHERE Username = #0 )", username);
foreach(var groupID in listOfGroups)
{
int newGroupID = groupID.GroupID;
var result = db.Query(
#"SELECT e.event_id, e.title, e.description, e.event_start, e.event_end, e.group_id, e.recurring
FROM event e
JOIN Membership m ON m.GroupID = e.group_id
WHERE e.recurring = 0
AND m.GroupID = #0
AND e.event_start >= #1
AND e.event_end <= #2
UNION ALL
SELECT e.event_id, e.title, e.description, DATEADD(week, w.weeks, e.event_start), DATEADD(week, w.weeks, e.event_end), e.group_id, e.recurring
FROM event e
JOIN Membership m ON m.GroupID = e.group_id
CROSS JOIN
( SELECT row_number() OVER (ORDER BY Object_ID) AS weeks
FROM SYS.OBJECTS
) AS w
WHERE e.recurring = 1
AND m.GroupID = #3
AND DATEADD(WEEK, w.Weeks, e.event_start) >= #4
AND DATEADD(WEEK, w.Weeks, e.event_end) <= #5", newGroupID, start, end, newGroupID, start, end
);
This results in when one queries for the date of the event stored in the database, this event and 52 weeks of events are returned. When one queries for the event the week after this one, nothing is returned.

The simplest solution would be to alter the following 2 lines
AND e.event_start >= #4
AND e.event_end <= #5"
to
AND DATEADD(WEEK, w.Weeks, e.event_start) >= #4
AND DATEADD(WEEK, w.Weeks, e.event_end) <= #5"
However, I'd advise putting all this SQL into a stored procedure, SQL-Server will cache the execution plans and it will result in (slightly) better performance.
CREATE PROCEDURE dbo.GetEvents #UserName VARCHAR(50), #StartDate DATETIME, #EndDate DATETIME
AS
BEGIN
-- DEFINE A CTE TO GET ALL GROUPS ASSOCIATED WITH THE CURRENT USER
;WITH Groups AS
( SELECT GroupID
FROM Membership m
INNER JOIN Users u
ON m.UserID = u.UserID
WHERE Username = #UserName
GROUP BY GroupID
),
-- DEFINE A CTE TO GET ALL EVENTS FOR THE GROUPS DEFINED ABOVE
AllEvents AS
( SELECT e.*
FROM event e
INNER JOIN Groups m
ON m.GroupID = e.group_id
UNION ALL
SELECT e.event_id, e.title, e.description, DATEADD(WEEK, w.weeks, e.event_start), DATEADD(WEEK, w.weeks, e.event_end), e.group_id, e.recurring
FROM event e
INNER JOIN Groups m
ON m.GroupID = e.group_id
CROSS JOIN
( SELECT ROW_NUMBER() OVER (ORDER BY Object_ID) AS weeks
FROM SYS.OBJECTS
) AS w
WHERE e.recurring = 1
)
-- GET ALL EVENTS WHERE THE EVENTS FALL IN THE PERIOD DEFINED
SELECT *
FROM AllEvents
WHERE Event_Start >= #StartDate
AND Event_End <= #EndDate
END
Then you can call this with
var result = db.Query("EXEC dbo.GetEvents #0, #1, #2", username, start, end);
This elimates the need to iterate over groups in your code behind. If this is actually a requirement then you could modify the stored procedure to take #GroupID as a parameter, and change the select statements/where clauses as necessary.
I have assumed knowledge of Common Table Expressions. They are not required to make the query work, they just make things slightly more legible in my opinion. I can rewrite this without them if required.

I would check my parameters one at a time against some trivial SQL, just to rule them out as possible culprits. Something like this:
var result = db.Query("select r=cast(#0 as varchar(80))",username);
var result = db.Query("select r=cast(#0 as int)",newGroupID);
var result = db.Query("select r=cast(#0 as datetime)",start);
var result = db.Query("select r=cast(#0 as datetime)",end);