I am getting SQL Exception:
String or binary data would be truncated
in SELECT. I read few questions with same title but they were all about inserting. I am SELECTING.
Code is following:
List<CategoryName> navigation = await db.Query<CategoryName>().FromSql(
$"WITH NestedCategories AS (
SELECT *
FROM Categories
WHERE Id IN (
{string.Join(",", products.Select(x =>
x.Categories.First().CategoryId).Distinct().Select(x => $"'{x}'"))}
)
UNION ALL
SELECT t.*
FROM Categories t
INNER JOIN NestedCategories c On c.ParentId = t.Id
)
SELECT DISTINCT c.Id, c.Name, c.ParentId
FROM NestedCategories c")
.AsNoTracking()
.ToListAsync();
If I generate string.Join to console and then put SQL command into query window in Management Studio I dont get any error. I get proper results. Issue is obviously in EF CORE that I am passing too many category Ids. Command is to get nesting categories based on Product-Category Id.
EDIT:
public class CategoryName
{
public int Id { get; set; }
public string Name { get; set; }
public int? ParentId { get; set; }
}
EDIT 2 - Solution
string inClause = string.Join(",", products.Select(x => x.Categories.First().CategoryId).Distinct().Select(x => $"'{x}'"));
List<CategoryName> navigation = new List<CategoryName>();
using (DbCommand command = db.Database.GetDbConnection().CreateCommand())
{
command.CommandText = $"WITH NestedCategories AS (SELECT * FROM Categories WHERE Id IN ({inClause}) UNION ALL SELECT t.* FROM Categories t INNER JOIN NestedCategories c On c.ParentId = t.Id) SELECT DISTINCT c.Id, c.Name, c.ParentId FROM NestedCategories c";
await db.Database.GetDbConnection().OpenAsync();
DbDataReader reader = await command.ExecuteReaderAsync();
while (await reader.ReadAsync())
navigation.Add(new CategoryName() { Id = reader.GetInt32(0), Name = reader.GetString(1), ParentId = await reader.IsDBNullAsync(2) ? null : await reader.GetFieldValueAsync<int?>(2) });
}
You should be very careful when using FromSql method with an inline interpolated SQL string.
Normally interpolated strings are resolved to string type, but FromSql method has overload with FormattableString parameter, which allows it to find the placeholders inside the interpolated string and bind a command parameter for each of them.
This normally is a functionality. But in your case you want just the joined string with ids to be embedded in the SQL string, while EF creates a parameter for it, so even if there was not a truncation error, the query won't return correct results because it would contain something like WHERE IN (#param) and #param will contain the comma separated text which will never match.
The simplest fix is to force the other FromSql method overload by either putting the SQL in a variable:
var sql = $"...";
List<CategoryName> navigation = await db.Query<CategoryName>().FromSql(sql)
// ...
or use cast operator:
List<CategoryName> navigation = await db.Query<CategoryName>().FromSql((string)
$"...")
// ...
A better approach would to create placeholders ({0}, {1}, ...) inside the SQL string and pass values through params object[] parameters argument. This way EF Core will bind parameter for each value (e.g. WHERE IN (#p0, #p1, ...)) rather than embedded constants:
var parameters = products.Select(x => x.Categories.First().CategoryId).Distinct()
.Cast<object>().ToArray(); // need object[]
var placeholders = string.Join(",", Enumerable.Range(0, parameters.Length)
.Select(i = "{" + i + "}"));
var sql =
$"WITH NestedCategories AS (
SELECT *
FROM Categories
WHERE Id IN ({placeholders})
UNION ALL
SELECT t.*
FROM Categories t
INNER JOIN NestedCategories c On c.ParentId = t.Id
)
SELECT DISTINCT c.Id, c.Name, c.ParentId
FROM NestedCategories c";
var query = db.Query<CategoryName>().FromSql(sql, parameters);
Related
I am trying replicate the SQL below using LINQ and Entity Framework and cannot figure out how this should be written.
My simplistic LINQ version does a query per table
public IActionResult Index()
{
dynamic view = new ExpandoObject();
view.AppUsers = Context.AppUsers.Count();
view.CustomerShops = Context.CustomerShops.Count();
view.FavouriteOrders = Context.FavouriteOrders.Count();
view.Items = Context.Items.Count();
view.ItemVariations = Context.ItemVariations.Count();
view.MenuCategories = Context.MenuCategories.Count();
view.MenuCategoryProducts = Context.MenuCategoryProducts.Count();
view.Orders = Context.Orders.Count();
view.Products = Context.Products.Count();
view.ProductVariations = Context.ProductVariations.Count();
view.Shops = Context.Shops.Count();
view.Staffs = Context.Staffs.Count();
return View(view);
}
I use this pattern from time to time to for reporting on my column counts and thought this should be easy to do in LINQ, but no luck so far.
This pure SQL UNION would only generate 1 SQL request, instead of a request per table.
select * from (
select 'asp_net_roles' as type, count(*) from asp_net_roles
union
select 'asp_net_user_roles' as type, count(*) from asp_net_user_roles
union
select 'asp_net_users' as type, count(*) from asp_net_users
union
select 'app_users' as type, count(*) from app_users
union
select 'shops' as type, count(*) from shops
union
select 'staffs' as type, count(*) from shops
union
select 'items' as type, count(*) from items
union
select 'item_variations' as type, count(*) from item_variations
union
select 'products' as type, count(*) from products
union
select 'product_variations' as type, count(*) from product_variations
union
select 'menu_categories' as type, count(*) from menu_categories
) as counters
order by 1;
I saw a partial implementation [linq-group-by-multiple-tables] (https://stackoverflow.com/a/3435503/473923) but this is based of grouping data.
FYI: I'm new to C#/Linq, so sorry if this seams obvious.
Use the this code from my answer
And fill ExpandoObject with result:
var tablesinfo = Context.GetTablesInfo();
var expando = new ExpandoObject();
if (tablesinfo != null)
{
var dic = (IDictionary<string, object>)expando;
foreach(var info in tablesinfo)
{
dic.Add(info.TableName, info.RecordCount);
}
}
Idea is that you can UNION counts if you group entities by constant.
Schematically function builds the following IQueryable Expression:
var tablesinfo =
Context.AppUsers.GroupBy(x => 1).Select(g => new TableInfo{ TableName = "asp_net_roles", RecordCount = g.Count() })
.Concat(Context.MenuCategories.GroupBy(x => 1).Select(g => new TableInfo{ TableName = "menu_categories", RecordCount = g.Count() }))
.Concat(Context.Items.GroupBy(x => 1).Select(g => new TableInfo{ TableName = "items", RecordCount = g.Count() }))
....
There is nothing wrong with your LINQ query. It's very acceptable approach. However it's not the most efficient.
There is no need to fetch count from individual tables one by one. You can get the counts from all the tables at once using the System tables Sys.Objects and Sys.Partitions. Just try running this query in your database.
SELECT A.Name AS TableName, SUM(B.rows) AS RecordCount
FROM sys.objects A INNER JOIN sys.partitions B
ON A.object_id = B.object_id
WHERE A.type = 'U' AND B.index_id IN (0, 1)
GROUP BY A.Name
For quick response and cleaner code, you can store this SQL query in a string variable, and run the LINQ
var result = dataContext.ExecuteQuery<YOUR_MODEL_CLASS>
(your_string_query);
I would put something like this:
Dictionary<string, int> view = new() {
new() {'asp_net_roles', Context.AppUsers.Count() },
...
}
return View(view);
maybe not the most pure way, but does the job (unless I misunderstood what you try to accomplish)
I have a table function which returns table names and number of entries within that table :
CREATE FUNCTION [dbo].[ufnGetLookups] ()
RETURNS
#lookupsWithItemCounts TABLE
(
[Name] VARCHAR(100),
[EntryCount] INT
)
AS
BEGIN
INSERT INTO #lookupsWithItemCounts([Name],[EntryCount])
VALUES
('Table1', (SELECT COUNT(*) FROM Table1)),
('Table2', (SELECT COUNT(*) FROM Table2)),
('Table3', (SELECT COUNT(*) FROM Table))
RETURN;
END
What would be the Linq equivalent of above simple function? Notice that I want to get the result in one single shot and the speed of the operation is quite important for me. If I realise that the converted linq to sql results in a massive bulky sql with performance hit, I would rather stick to my existing user defined function and forget about the linq equivilant.
You can do that with a UNION query. EG
var q = db.Books.GroupBy(g => "Books").Select(g => new { Name = g.Key, EntryCount = g.Count() })
.Union(db.Authors.GroupBy(g => "Authors").Select(g => new { Name = g.Key, EntryCount = g.Count() }));
var r = q.ToList();
Not an EF guy, and not sure if this would be more performant.
Select TableName = o.name
,RowCnt = sum(p.Rows)
From sys.objects as o
Join sys.partitions as p on o.object_id = p.object_id
Where o.type = 'U'
and o.is_ms_shipped = 0x0
and index_id < 2 -- 0:Heap, 1:Clustered
--and o.name in ('Table1','Table2','Table3' ) -- Include (or not) your own filter
Group By o.schema_id,o.name
Note: Wish I could recall the source of this, but I've used it in my discovery process.
The example given in the blog has the following
from e in s.StudentCourseEnrollments where courseIDs.Contains(e.Course.CourseID) select e
The contains logic will not work when we are looking for an exact match. If a student has enrolled for 6 courses (ex : 1,2,3,4,5,6) and the requested list contains 5 (ex: 1,2,3,4,5) the query will return a match when it should not. The other way works well when the student has enrolled in a subset of the requested list.
Below solution works but need help to convert the below sql to LINQ (EF Core 3.0) ?
Create TABLE dbo.Enrollments (StudentId INT NOT NULL, CourseId INT NOT NULL)
insert into dbo.Enrollments values (1,1)
insert into dbo.Enrollments values (1,2)
insert into dbo.Enrollments values (1,3)
insert into dbo.Enrollments values (1,4)
insert into dbo.Enrollments values (1,5)
insert into dbo.Enrollments values (1,6)
DECLARE #TempCourses TABLE
(
CourseId INT
);
INSERT INTO #TempCourses (CourseId) VALUES (1), (2), (3),(4),(5);
SELECT t.StudentId
FROM
(
SELECT StudentId, cnt=COUNT(*)
FROM dbo.Enrollments
GROUP BY StudentId
) kc
INNER JOIN
(
SELECT cnt=COUNT(*)
FROM #TempCourses
) nc ON nc.cnt = kc.cnt
JOIN dbo.Enrollments t ON t.StudentId = kc.StudentId
JOIN #TempCourses n ON n.CourseId = t.CourseId
GROUP BY t.StudentId
HAVING COUNT(*) = MIN(nc.cnt);
drop table dbo.Enrollments
db<>Fiddle
I don't know about the SQL query, but the EF Core 3.0 LINQ query for the same task is something like this:
var matchIds = new[] { 1, 2, 3, 4, 5 }.AsEnumerable();
var query = dbContext.Students
.Where(s => s.Enrollments.All(e => matchIds.Contains(e.CourseId))
&& s.Enrollments.Count() == matchIds.Count());
The main matching job is done with All subquery. Unfortunately that's not enough for the case when related link records are more than the matching ids, so additional counts comparison solves that.
You can achieve it with a simple way like this, live demo here
Let's say that you've got the list of enrollments by this way
var enrollments = from s in dc.Students
from c in s.Courses
select new { StudentID = s.StudentID, CourseID = c.CourseID };
Then get the result by this way
var groupedEnrollment = enrollments.GroupBy(p => p.StudentId)
.Select(g => new
{
StudentId = g.Key,
Courses = g.Select(p => p.CourseId).ToArray()
});
var result = groupedEnrollment.Where(g =>
g.Courses.Length == courses.Length &&
g.Courses.Intersect(courses).Count() == courses.Length);
I know the SQL I want to produce:
SELECT qdt, pbn, cid, pid, SUM(Amount) AS TotalAmount
FROM SomeDb.SomeTable
WHERE Status = 'Open'
GROUP BY cid, pid, qdt, pbn
ORDER BY qdt
I have LINQ, that I was hoping would produce something as clean as the above SQL:
var query = (
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new { someTable.cid, someTable.pid } into grouped
select new
{
lid = grouped.FirstOrDefault().lid,
qdt = grouped.FirstOrDefault().qdt,
pbn = grouped.FirstOrDefault().pbn,
cid = grouped.FirstOrDefault().cid,
cn = grouped.FirstOrDefault().cn,
pid = grouped.FirstOrDefault().pid,
amount = grouped.Sum(o => o.Amount),
Status = grouped.FirstOrDefault().Status
});
But that produces many lines of nasty SQL. Each grouped value ends up getting its own SELECT statement within the query, like this, for qdt:
SELECT [t5].[qdt]
FROM (
SELECT TOP (1) [t4].[qdt]
FROM [SomeDb].[SomeTable] AS [t4]
WHERE ([t1].[cid] = [t4].[cid]) AND ([t1].[pid] = [t4].[pid]) AND ([t4].[Status] = #p0)
) AS [t5]
) AS [qdt]
Is there a way to change the LINQ to produce the simpler SQL?
There are two issues I see with your LINQ attempt. For starters, you are not grouping by the same fields in the SQL query as in your LINQ. Then, you are using FirstOrDefault on the results of the group instead of selecting the group's key data.
Your query should look like this:
var query =
(
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new
{
someTable.lid,
someTable.qdt,
someTable.pbn,
someTable.cid,
someTable.cn,
someTable.pid,
someTable.Status,
} into grouped
select new
{
lid = grouped.Key.lid,
qdt = grouped.Key.qdt,
pbn = grouped.Key.pbn,
cid = grouped.Key.cid,
cn = grouped.Key.cn,
pid = grouped.Key.pid,
amount = grouped.Sum(o => o.Amount),
Status = grouped.Key.Status
}
);
I cannot test right now whether it will generate the exact same SQL though.
You don't seem to have all the grouped by columns in the desired sql in your linq;
var query = (
from someTable in SomeTable
where someTable.Status == "Open"
group someTable by new { someTable.cid, someTable.pid, someTable.qdt,someTable.pbn, someTable.Status } into grouped
select new
{
qdt = grouped.FirstOrDefault().qdt,
pbn = grouped.FirstOrDefault().pbn,
cid = grouped.FirstOrDefault().cid,
pid = grouped.FirstOrDefault().pid,
Status = grouped.FirstOrDefault().Status,
amount = grouped.Sum(o => o.Amount)
});
I can't really test it but that looks a little more like the sql you are trying to replicate in linq.
I have a Category table with a tree structure (Id,MasterId)
I'd like to select all products that belong to a Category and all Child Categories.
Today I use this SQL Query which works, but I'd like to add pagination and that would be easier with a pure LINQ query. I use Entity Framework 4.
#Count int = 100,
#CategoryId int
with mq as
(
select c.Id as parent, c.Id as child
from dbo.Categories c
where c.Id = #CategoryId
union all
select q.child, c.Id
from mq q
inner join dbo.Categories c on q.child = c.MasterId
)
select top (#Count) P.* from Products P
inner join ProductToCategory PC ON(PC.ProductId = P.Id)
where PC.CategoryId in (
select child from mq
)
and P.PublishStatus = 1
order by P.PublishedDate DESC;
Any ideas how to get a nice LINQ query on this with pagination (current page, number of products per page, total product count)?
This is recursive / hiearchical query with table expression. EF does not provide support for such queries. If you want to receive data by single roundtrip to DB you must wrap it in stored procedure and import that procedure to your entity framework model.
Paging in SQL is also possible when using table expressions and ROW_NUMBER().
there is an idea. i haven't tested it, so dont blame if it doesn't work :P
var ids = context.TreeItems.Where(x => x.Id == parentId).Select(x => (int?)x.Id);
var tmp = ids;
while (true)
{
IQueryable<int?> localIds = tmp;
var subIds = context.TreeItems.Where(x => ids.Contains(x.ParentId)).Select(x => (int?)x.Id);
if (subIds.Any())
{
tmp = subIds;
ids = ids.Union(subIds);
}
else
break;
}
var allSubItems = context.TreeItems.Where(x => ids.Contains(x.Id));