Develop Reference

EF Core filtering + paging with child entities

I want to perform 1 query using EF Core where I have to filter on a child entity and also apply paging. See my (sample) data model below:
I want to retrieve all data (customers, orders order details and products). I have to apply a filter on Order.OrderState and I only want the 1st 10 records (customers)
This is the LINQ query I tried:
var customers = await _ctx.Customer
.Include(c => c.Order.Where(o => o.OrderState == 0))
.ThenInclude(o => o.OrderDetail)
.ThenInclude(d => d.Product)
.Skip(0).Take(10)
.ToListAsync();
When executing this query I get the following error: InvalidOperationException: The property expression 'c => {from Order o in c.Order where ([o].OrderState == 0) select [o]}' is not valid. The expression should represent a property access: 't => t.MyProperty'. For more information on including related data
So I tried another query:
var qry = from c in _ctx.Customer
join o in _ctx.Order on c.Id equals o.CustomerId
join d in _ctx.OrderDetail on o.Id equals d.OrderId
join p in _ctx.Product on d.ProductId equals p.Id
where o.OrderState == 0
select new { Customer = c, Order = o, OrderDetail = d, Product = p };
var customers = await qry.Skip(0).Take(10).ToListAsync();
Now the query does not produce an error, but the result is not what I want. Because of the 1-n relationships this query returns customers multiple times in the result, so I do not get the 1st 10 customers.
Does anybody have a better query to get the results that I want?

I think you can use ".Distinct()" while building the query. Hope this will solve the problem.
var qry = (from c in _ctx.Customer
join o in _ctx.Order on c.Id equals o.CustomerId
join d in _ctx.OrderDetail on o.Id equals d.OrderId
join p in _ctx.Product on d.ProductId equals p.Id
where o.OrderState == 0
select new { Customer = c, Order = o, OrderDetail = d,
Product = p }).Distinct();
var customers = await qry.Skip(0).Take(10).ToListAsync();

Is there any way to optimize this LINQ to Entities query?

I was asked to produce a report that is driven by a fairly complex SQL query against a SQL Server database. Since the site of the report was already using Entity Framework 4.1, I thought I would attempt to write the query using EF and LINQ:
var q = from r in ctx.Responses
.Where(x => ctx.Responses.Where(u => u.UserId == x.UserId).Count() >= VALID_RESPONSES)
.GroupBy(x => new { x.User.AwardCity, x.Category.Label, x.ResponseText })
orderby r.FirstOrDefault().User.AwardCity, r.FirstOrDefault().Category.Label, r.Count() descending
select new
{
City = r.FirstOrDefault().User.AwardCity,
Category = r.FirstOrDefault().Category.Label,
Response = r.FirstOrDefault().ResponseText,
Votes = r.Count()
};
This query tallies votes, but only from users who have submitted a certain number of required minimum votes.
This approach was a complete disaster from a performance perspective, so we switched to ADO.NET and the query ran very quickly. I did look at the LINQ generated SQL using the SQL Profiler, and although it looked atrocious as usual I didn't see any clues as to how to optimize the LINQ statement to make it more efficient.
Here's the straight TSQL version:
WITH ValidUsers(UserId)
AS
(
SELECT UserId
FROM Responses
GROUP BY UserId
HAVING COUNT(*) >= 103
)
SELECT d.AwardCity
, c.Label
, r.ResponseText
, COUNT(*) AS Votes
FROM ValidUsers u
JOIN Responses r ON r.UserId = u.UserId
JOIN Categories c ON r.CategoryId = c.CategoryId
JOIN Demographics d ON r.UserId = d.Id
GROUP BY d.AwardCity, c.Label, r.ResponseText
ORDER BY d.AwardCity, s.SectionName, COUNT(*) DESC
What I'm wondering is: is this query just too complex for EF and LINQ to handle efficiently or have I missed a trick?

Using a let to reduce the number of r.First()'s will probably improve performance. It's probably not enough yet.
var q = from r in ctx.Responses
.Where()
.GroupBy()
let response = r.First()
orderby response.User.AwardCity, response.Category.Label, r.Count() descending
select new
{
City = response.User.AwardCity,
Category = response.Category.Label,
Response = response.ResponseText,
Votes = r.Count()
};

Maybe this change improve the performance, removing the resulting nested sql select in the where clause
First get the votes of each user and put them in a Dictionary
var userVotes = ctx.Responses.GroupBy(x => x.UserId )
.ToDictionary(a => a.Key.UserId, b => b.Count());
var cityQuery = ctx.Responses.ToList().Where(x => userVotes[x.UserId] >= VALID_RESPONSES)
.GroupBy(x => new { x.User.AwardCity, x.Category.Label, x.ResponseText })
.Select(r => new
{
City = r.First().User.AwardCity,
Category = r.First().Category.Label,
Response = r.First().ResponseText,
Votes = r.Count()
})
.OrderByDescending(r => r.City, r.Category, r.Votes());

how to convert T-SQL Query to linq

i have 4 table in SQL: DocumentType,ClearanceDocument,Request, RequestDocument.
i want when page load and user select one request, show all Document Based on clearanceType in RequestTable and check in RequestDocument and when exist set is_exist=true
I have written this query with SqlServer Query Editor for get result this Scenario but i can't convert this Query to Linq
select *,
is_Orginal=
(select is_orginal from CLEARANCE_REQUEST_DOCUMENT
where
DOCUMENT_ID=a.DOCUMENT_ID and REQUEST_ID=3)
from
DOCUMENT_TYPES a
where
DOCUMENT_ID in
(select DOCUMENT_ID from CLEARANCE_DOCUMENTS dt
where
dt.CLEARANCE_ID=
(SELECT R.CLEARANCE_TYPE FROM CLEARANCE_REQUEST R
WHERE
R.REQUEST_ID=3))
i write this Query in linq but not work
var list = (from r in context.CLEARANCE_REQUEST
where r.REQUEST_ID == 3
join cd in context.CLEARANCE_DOCUMENTS on r.CLEARANCE_TYPE equals cd.CLEARANCE_ID
join dt in context.DOCUMENT_TYPES on cd.DOCUMENT_ID equals dt.DOCUMENT_ID into outer
from t in outer.DefaultIfEmpty()
select new
{
r.REQUEST_ID,
cd.CLEARANCE_ID,
t.DOCUMENT_ID,
t.DOCUMENT_NAME,
is_set=(from b in context.CLEARANCE_REQUEST_DOCUMENT where
b.REQUEST_ID==r.REQUEST_ID && b.DOCUMENT_ID==t.DOCUMENT_ID
select new{b.IS_ORGINAL})
}
).ToList();
I want convert this Query to LINQ. Please help me. Thanks.

There is no need to manually join objects returned from an Entity Framework context.
See Why use LINQ Join on a simple one-many relationship?
If you use the framework as intended your job will be much easier.
var result = var clearanceTypes = context.CLEARANCE_REQUEST
.Single(r => r.REQUEST_ID == 3)
.CLEARANCE_DOCUMENTS
.SelectMany(dt => dt.DOCUMENT_TYPES)
.Select(a => new
{
DocumentType = a,
IsOriginal = a.CLEARANCE_REQUEST_DOCUMENT.is_original
});

Since your query won't be executed untill you iterate over the data, you can split your query in several subqueries to help you obtain the results like this:
var clearanceIds = context.CLEARANCE_REQUEST
.Where(r => r.REQUEST_ID == 3)
.Select(r => r.CLEARANCE_TYPE);
var documentIds = context.CLEARANCE_DOCUMENTS
.Where(dt => clearanceIds.Contains(dt.CLEARANCE_ID))
.Select(dt => dt.DOCUMENT_ID);
var result = context.DOCUMENT_TYPES
.Where(a => documentIds.Contains(a.DOCUMENT_ID))
.Select(a => new
{
// Populate properties here
IsOriginal = context.CLEARANCE_REQUEST_DOCUMENT
.Single(item => item.DOCUMENT_ID == a.DOCUMENT_ID &&
item.REQUEST_ID == 3)
.IS_ORIGINAL
})
.ToList();

How can I write a linq query for this?

I need to write following query in Linq to SQL but not sure what is the best way of doing, given it has two derived tables. Any suggestions.
SELECT A.ID
FROM
(
SELECT *
FROM Orders
WHERE ProductID = 5
) A
JOIN
(
SELECT CustomerID, MAX(Price) Price
FROM Orders
WHERE ProductID = 5
GROUP BY CustomerID
) B
ON A.CustomerID = B.CustomerID and A.Price = B.Price

var b = (
from o in db.Orders
where o.ProductID == 5
group o by o.CustomerID into og
select new {
CustomerID = og.Key
Price = Max(og.Price)
}
);
var a = (
from o in db.Orders
join p in b on new {a.CustomerID, a.Price} equals
new {b.CustomerID, b.Price}
where o.ProductID == 5
select a.ID
);
var r = a.ToString();
These two links are invaluable when forming things like this:
http://code.msdn.microsoft.com/101-LINQ-Samples-3fb9811b
http://msdn.microsoft.com/en-us/vstudio/bb688085

Can you simplify this with LINQ, especially if you use method syntax instead of query syntax.
orders.Where(o => o.ProductID == 5)
.GroupBy(o => o.CustomerID)
.SelectMany(g => g.Where(o => o.Price == g.Max(m => m.Price)));
My advice when writing LINQ, do not simply attempt to convert a SQL statement exactly. Think about the desired result and develop a solution designed for LINQ.

Something along these lines:
var result = from a in context.Orders
join b in (context.Orders.Where(o => o.ProductID == 5).GroupBy(o => o.CustomerID).Select(g => new { CustomerID = g.Key, Price = g.Max(o => o.Price)))
on new {a.CustomerID, a.Price} equals new {b.CustomerID, b.Price}
where a.ProductID == 5
select a.ID;

Linq can these sub queries be optimised?

This query takes a group of comments, then counts their upvotes and downvotes in the tblCommentVotes table.
At the moment, it counts these via the select new statement, in the form of a subquery. Would this be more efficient if it was in some sort of group by in the main query? Also if it would, could anyone show me how to do this, as I can't work out how you would do this.
// Get comments
var q = (
from C in db.tblComments
where
C.CategoryID == Category &&
C.IdentifierID == Identifier
join A in db.tblForumAuthors on C.UserID equals A.Author_ID
orderby C.PostDate descending
select new
{
C,
A.Username,
UpVotes = (from V in db.tblCommentVotes where V.CommentID == C.ID && V.UpVote == true select new { V.ID }).Count(),
DownVotes = (from V in db.tblCommentVotes where V.CommentID == C.ID && V.UpVote == false select new { V.ID }).Count()
}
)
.Skip(ToSkip > 0 ? ToSkip : 0)
.Take(ToTake > 0 ? ToTake : int.MaxValue);

What you need to do is to do an left outer join of the db.tblCommentVotes in the query expression, cause probably there might be no commentVotes?
When you have that, you should be able to perform ONE query in order to get your result.
It might look like this:
var q = (
from C in db.tblComments
where
C.CategoryID == Category &&
C.IdentifierID == Identifier
join A in db.tblForumAuthors on C.UserID equals A.Author_ID
// the following two lines are the left outer join thing.
join voteTemp in db.tblCommentVotes on voteTemp.CommentID equals C.ID into voteJoin
from vote in voteJoin.DefaultIfEmpty()
orderby C.PostDate descending
group C by new { Comment = C, Username = A.Username } into g
select new
{
g.Key.Comment,
g.Key.Username,
UpVotes = g.Count(x => x.UpVote),
DownVotes = g.Count(x => !x.UpVote)
}
)
.Skip(ToSkip > 0 ? ToSkip : 0)
.Take(ToTake > 0 ? ToTake : int.MaxValue);
This is untested and might not even compile, but I think it should be something like this.

db.tblComments.Where(c => c.CategoryID == Category && c.IdentifierID == Identifier)
.Join(db.tblForumAuthors, c => c.UserID, a => a.Author_ID,
(c, a) =>
new
{
CommentID = c,
AuthorName = a.UserName,
UpVotes = c.Join(db.tblCommentVotes, c => c.CommentID
v => v.CommentID,
(c, v) => v).Count(v => v.UpVote)
DownVotes = c.Join(db.tblCommentVotes, c => c.CommentID
v => v.CommentID,
(c, v) => v).Count(v => v.DownVote)
});

To optimise it's best first to measure.
Try, using something like LinqPad to view the generated SQL
Then use SQL Server Management Studio to see the query plan for that SQL
or:
Try running the code and seeing what SQL trace tells you is happening
Without the DB, it's quite hard (but fun) to guess whether that Linq will result in a single query or in multiple queries for working out the UpVotes and DownVotes. My guess is that calculating the UpVotes and DownVotes this way could be quite expensive - it may result in 2 additional queries per comment.

http://www.thereforesystems.com/view-query-generate-by-linq-to-sql/
without analyzing whats being output this question is impossible to answer.. however the link provided above should give you the tools necessary to perform this analysis yourself.