Develop Reference

Linq equivalent of aggregate function on multiple tables in one database trip

I have a table function which returns table names and number of entries within that table :
CREATE FUNCTION [dbo].[ufnGetLookups] ()
RETURNS
#lookupsWithItemCounts TABLE
(
[Name] VARCHAR(100),
[EntryCount] INT
)
AS
BEGIN
INSERT INTO #lookupsWithItemCounts([Name],[EntryCount])
VALUES
('Table1', (SELECT COUNT(*) FROM Table1)),
('Table2', (SELECT COUNT(*) FROM Table2)),
('Table3', (SELECT COUNT(*) FROM Table))
RETURN;
END
What would be the Linq equivalent of above simple function? Notice that I want to get the result in one single shot and the speed of the operation is quite important for me. If I realise that the converted linq to sql results in a massive bulky sql with performance hit, I would rather stick to my existing user defined function and forget about the linq equivilant.

You can do that with a UNION query. EG
var q = db.Books.GroupBy(g => "Books").Select(g => new { Name = g.Key, EntryCount = g.Count() })
.Union(db.Authors.GroupBy(g => "Authors").Select(g => new { Name = g.Key, EntryCount = g.Count() }));
var r = q.ToList();

Not an EF guy, and not sure if this would be more performant.
Select TableName = o.name
,RowCnt = sum(p.Rows)
From sys.objects as o
Join sys.partitions as p on o.object_id = p.object_id
Where o.type = 'U'
and o.is_ms_shipped = 0x0
and index_id < 2 -- 0:Heap, 1:Clustered
--and o.name in ('Table1','Table2','Table3' ) -- Include (or not) your own filter
Group By o.schema_id,o.name
Note: Wish I could recall the source of this, but I've used it in my discovery process.

String or binary data would be truncated WHERE IN

I am getting SQL Exception:
String or binary data would be truncated
in SELECT. I read few questions with same title but they were all about inserting. I am SELECTING.
Code is following:
List<CategoryName> navigation = await db.Query<CategoryName>().FromSql(
$"WITH NestedCategories AS (
SELECT *
FROM Categories
WHERE Id IN (
{string.Join(",", products.Select(x =>
x.Categories.First().CategoryId).Distinct().Select(x => $"'{x}'"))}
)
UNION ALL
SELECT t.*
FROM Categories t
INNER JOIN NestedCategories c On c.ParentId = t.Id
)
SELECT DISTINCT c.Id, c.Name, c.ParentId
FROM NestedCategories c")
.AsNoTracking()
.ToListAsync();
If I generate string.Join to console and then put SQL command into query window in Management Studio I dont get any error. I get proper results. Issue is obviously in EF CORE that I am passing too many category Ids. Command is to get nesting categories based on Product-Category Id.
EDIT:
public class CategoryName
{
public int Id { get; set; }
public string Name { get; set; }
public int? ParentId { get; set; }
}
EDIT 2 - Solution
string inClause = string.Join(",", products.Select(x => x.Categories.First().CategoryId).Distinct().Select(x => $"'{x}'"));
List<CategoryName> navigation = new List<CategoryName>();
using (DbCommand command = db.Database.GetDbConnection().CreateCommand())
{
command.CommandText = $"WITH NestedCategories AS (SELECT * FROM Categories WHERE Id IN ({inClause}) UNION ALL SELECT t.* FROM Categories t INNER JOIN NestedCategories c On c.ParentId = t.Id) SELECT DISTINCT c.Id, c.Name, c.ParentId FROM NestedCategories c";
await db.Database.GetDbConnection().OpenAsync();
DbDataReader reader = await command.ExecuteReaderAsync();
while (await reader.ReadAsync())
navigation.Add(new CategoryName() { Id = reader.GetInt32(0), Name = reader.GetString(1), ParentId = await reader.IsDBNullAsync(2) ? null : await reader.GetFieldValueAsync<int?>(2) });
}

You should be very careful when using FromSql method with an inline interpolated SQL string.
Normally interpolated strings are resolved to string type, but FromSql method has overload with FormattableString parameter, which allows it to find the placeholders inside the interpolated string and bind a command parameter for each of them.
This normally is a functionality. But in your case you want just the joined string with ids to be embedded in the SQL string, while EF creates a parameter for it, so even if there was not a truncation error, the query won't return correct results because it would contain something like WHERE IN (#param) and #param will contain the comma separated text which will never match.
The simplest fix is to force the other FromSql method overload by either putting the SQL in a variable:
var sql = $"...";
List<CategoryName> navigation = await db.Query<CategoryName>().FromSql(sql)
// ...
or use cast operator:
List<CategoryName> navigation = await db.Query<CategoryName>().FromSql((string)
$"...")
// ...
A better approach would to create placeholders ({0}, {1}, ...) inside the SQL string and pass values through params object[] parameters argument. This way EF Core will bind parameter for each value (e.g. WHERE IN (#p0, #p1, ...)) rather than embedded constants:
var parameters = products.Select(x => x.Categories.First().CategoryId).Distinct()
.Cast<object>().ToArray(); // need object[]
var placeholders = string.Join(",", Enumerable.Range(0, parameters.Length)
.Select(i = "{" + i + "}"));
var sql =
$"WITH NestedCategories AS (
SELECT *
FROM Categories
WHERE Id IN ({placeholders})
UNION ALL
SELECT t.*
FROM Categories t
INNER JOIN NestedCategories c On c.ParentId = t.Id
)
SELECT DISTINCT c.Id, c.Name, c.ParentId
FROM NestedCategories c";
var query = db.Query<CategoryName>().FromSql(sql, parameters);

Inefficient LINQ - Know What SQL Should Be - Can't get there

I know the SQL I want to produce:
SELECT qdt, pbn, cid, pid, SUM(Amount) AS TotalAmount
FROM SomeDb.SomeTable
WHERE Status = 'Open'
GROUP BY cid, pid, qdt, pbn
ORDER BY qdt
I have LINQ, that I was hoping would produce something as clean as the above SQL:
var query = (
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new { someTable.cid, someTable.pid } into grouped
select new
{
lid = grouped.FirstOrDefault().lid,
qdt = grouped.FirstOrDefault().qdt,
pbn = grouped.FirstOrDefault().pbn,
cid = grouped.FirstOrDefault().cid,
cn = grouped.FirstOrDefault().cn,
pid = grouped.FirstOrDefault().pid,
amount = grouped.Sum(o => o.Amount),
Status = grouped.FirstOrDefault().Status
});
But that produces many lines of nasty SQL. Each grouped value ends up getting its own SELECT statement within the query, like this, for qdt:
SELECT [t5].[qdt]
FROM (
SELECT TOP (1) [t4].[qdt]
FROM [SomeDb].[SomeTable] AS [t4]
WHERE ([t1].[cid] = [t4].[cid]) AND ([t1].[pid] = [t4].[pid]) AND ([t4].[Status] = #p0)
) AS [t5]
) AS [qdt]
Is there a way to change the LINQ to produce the simpler SQL?

There are two issues I see with your LINQ attempt. For starters, you are not grouping by the same fields in the SQL query as in your LINQ. Then, you are using FirstOrDefault on the results of the group instead of selecting the group's key data.
Your query should look like this:
var query =
(
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new
{
someTable.lid,
someTable.qdt,
someTable.pbn,
someTable.cid,
someTable.cn,
someTable.pid,
someTable.Status,
} into grouped
select new
{
lid = grouped.Key.lid,
qdt = grouped.Key.qdt,
pbn = grouped.Key.pbn,
cid = grouped.Key.cid,
cn = grouped.Key.cn,
pid = grouped.Key.pid,
amount = grouped.Sum(o => o.Amount),
Status = grouped.Key.Status
}
);
I cannot test right now whether it will generate the exact same SQL though.

You don't seem to have all the grouped by columns in the desired sql in your linq;
var query = (
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new { someTable.cid, someTable.pid, someTable.qdt,someTable.pbn, someTable.Status } into grouped
select new
{
qdt = grouped.FirstOrDefault().qdt,
pbn = grouped.FirstOrDefault().pbn,
cid = grouped.FirstOrDefault().cid,
pid = grouped.FirstOrDefault().pid,
Status = grouped.FirstOrDefault().Status,
amount = grouped.Sum(o => o.Amount)
});
I can't really test it but that looks a little more like the sql you are trying to replicate in linq.

Get total row count in Entity Framework

I'm using Entity Framework to get the total row count for a table. I simply want the row count, no where clause or anything like that. The following query works, but is slow. It took about 7 seconds to return the count of 4475.
My guess here is that it's iterating through the entire table, just like how IEnumerable.Count() extension method works.
Is there a way I can get the total row count "quickly"? is there a better way?
public int GetLogCount()
{
using (var context = new my_db_entities(connection_string))
{
return context.Logs.Count();
}
}

You can even fire Raw SQL query using entity framework as below:
var sql = "SELECT COUNT(*) FROM dbo.Logs";
var total = context.Database.SqlQuery<int>(sql).Single();

That is the way to get your row count using Entity Framework. You will probably see faster performance on the second+ queries as there is an initialization cost the first time that you run it. (And it should be generating a Select Count() query here, not iterating through each row).
If you are interested in a faster way to get the raw row count in a table, then you might want to try using a mini ORM like Dapper or OrmLite.
You should also make sure that your table is properly defined (at the very least, that it has a Primary Key), as failure to do this can also affect the time to count rows in the table.

If you have access to do so, it would be much quicker to query the sys tables to pull this information.
E.g.
public Int64 GetLogCount()
{
var tableNameParam = new SqlParameter("TableName", "Logs");
var schemaNameParam = new SqlParameter("SchemaName", "dbo");
using (var context = new my_db_entities(connection_string))
{
var query = #"
SELECT ISNULL([RowCount],0)
FROM (
SELECT SchemaName,
TableName,
Sum(I.rowcnt) [RowCount]
FROM sysindexes I
JOIN sysobjects O (nolock) ON I.id = o.id AND o.type = 'U'
JOIN (
SELECT so.object_id,
ss.name as SchemaName,
so.name as TableName
FROM sys.objects SO (nolock)
JOIN sys.schemas SS (nolock) ON ss.schema_id = so.schema_id
) SN
ON SN.object_id = o.id
WHERE I.indid IN ( 0, 1 )
AND TableName = #TableName AND SchemaName = #SchemaName
GROUP BY
SchemaName, TableName
) A
";
return context.ExecuteStoreQuery<Int64>(query, tableNameParam, schemaNameParam).First();
}
}

Select category tree in Entity Framework

I have a Category table with a tree structure (Id,MasterId)
I'd like to select all products that belong to a Category and all Child Categories.
Today I use this SQL Query which works, but I'd like to add pagination and that would be easier with a pure LINQ query. I use Entity Framework 4.
#Count int = 100,
#CategoryId int
with mq as
(
select c.Id as parent, c.Id as child
from dbo.Categories c
where c.Id = #CategoryId
union all
select q.child, c.Id
from mq q
inner join dbo.Categories c on q.child = c.MasterId
)
select top (#Count) P.* from Products P
inner join ProductToCategory PC ON(PC.ProductId = P.Id)
where PC.CategoryId in (
select child from mq
)
and P.PublishStatus = 1
order by P.PublishedDate DESC;
Any ideas how to get a nice LINQ query on this with pagination (current page, number of products per page, total product count)?

This is recursive / hiearchical query with table expression. EF does not provide support for such queries. If you want to receive data by single roundtrip to DB you must wrap it in stored procedure and import that procedure to your entity framework model.
Paging in SQL is also possible when using table expressions and ROW_NUMBER().

there is an idea. i haven't tested it, so dont blame if it doesn't work :P
var ids = context.TreeItems.Where(x => x.Id == parentId).Select(x => (int?)x.Id);
var tmp = ids;
while (true)
{
IQueryable<int?> localIds = tmp;
var subIds = context.TreeItems.Where(x => ids.Contains(x.ParentId)).Select(x => (int?)x.Id);
if (subIds.Any())
{
tmp = subIds;
ids = ids.Union(subIds);
}
else
break;
}
var allSubItems = context.TreeItems.Where(x => ids.Contains(x.Id));