Develop Reference

How to improve LINQ statement to use INNER JOIN in resulting SQL statement?

Assuming the following code that applies filtering logic to a passed on collection.
private IQueryable<Customer> ApplyCustomerFilter(CustomerFilter filter, IQueryable<Customer> customers)
{
...
if (filter.HasProductInBackOrder == true)
{
customers = customers.Where(c => c.Orders.Any(o => o.Products.Any(p => p.Status == ProductStatus.BackOrder)))
}
....
return customers;
}
Results in this SQL statement:
SELECT [Extent1].[CustomerId] AS [CustomerId],
[Extent1].[Status] AS [Status]
FROM [Customers] AS [Extent1]
WHERE
(
EXISTS
(
SELECT 1 AS [C1]
FROM
(
SELECT [Extent3].[OrderId] AS [OrderId]
FROM [Orders] AS [Extent3]
WHERE [Extent1].[CustomerId] = [Extent3].[CustomerId]
) AS [Project1]
WHERE EXISTS
(
SELECT 1 AS [C1]
FROM [Products] AS [Extent4]
WHERE ([Project1].[OrderId] = [Extent4].[OrderId])
AND ([Extent4].[Status] = #p__linq__6)
)
)
)
However, I would like to optimize this by forcing to use INNER JOINS so that the result will be similar to this:
SELECT [Extent1].[CustomerId] AS [CustomerId],
[Extent1].[Status] AS [Status]
FROM [Customers] AS [Extent1]
INNER JOIN [Orders] AS [Extent2] ON [Extent1].[CustomerId] = [Extent2].[CustomerId]
INNER JOIN [Products] AS [Extent3] ON [Extent2].[OrderId] = [Extent3].[OrderId]
WHERE [Extent3].[Status] = #p__linq__6
I've tried multiple approaches, but I was unable to accomplish the desired result. Any suggestions on how to force the correct joins and avoiding subselects?

Entity Framework 6: Skip() & Take() do not generate SQL, instead the result set is filtered after loading into memory. Or am I doing something wrong?

I have the following code that should gets some book, and retrieve the first 2 tags (Tag entities) from that book (Book entity).
So Tags is a navigation property of the Book entity.
using (var context = new FakeEndavaBookLibraryEntities())
{
Book firstBook = context.Set<Book>().Take(1).First();
var firstTwoTags = firstBook.Tags.OrderBy(tag => tag.Id).Skip(0).Take(2).ToList();
}
I expect obtaining the following SQL query that has to be generated by EF.
SELECT TOP(2)
[Extent2].[Id] AS [Id],
[Extent2].[Version] AS [Version],
[Extent2].[Name] AS [Name]
FROM [Literature].[BookTagRelation] AS [Extent1]
INNER JOIN [Literature].[Tag] AS [Extent2]
ON [Extent1].[TagId] = [Extent2].[Id]
WHERE [Extent1].[BookId] = 1 /* #EntityKeyValue1 - [BookId] */
Instead, the EF Profiler shows me that the EF is generating unbounded result set (like SELECT * FROM ...)
SELECT [Extent2].[Id] AS [Id],
[Extent2].[Version] AS [Version],
[Extent2].[Name] AS [Name]
FROM [Literature].[BookTagRelation] AS [Extent1]
INNER JOIN [Literature].[Tag] AS [Extent2]
ON [Extent1].[TagId] = [Extent2].[Id]
WHERE [Extent1].[BookId] = 1 /* #EntityKeyValue1 - [BookId] */
Here is a scheme fragment if you need it
I also tried to append the .AsQueryable() to firstBook.Tags property and/or remove .Skip(0) method as is shown below, but this didn't help as well.
var firstTwoTags = firstBook.Tags.AsQueryable().OrderBy(tag => tag.Id).Skip(0).Take(2).ToList();
The same undesired behavior:
SELECT [Extent2].[Id] AS [Id],
[Extent2].[Version] AS [Version],
[Extent2].[Name] AS [Name]
FROM [Literature].[BookTagRelation] AS [Extent1]
INNER JOIN [Literature].[Tag] AS [Extent2]
ON [Extent1].[TagId] = [Extent2].[Id]
WHERE [Extent1].[BookId] = 1 /* #EntityKeyValue1 - [BookId] */
Have you ever encountered the same problem when working with Entity Framework 6?
Are there any workarounds to overcome this problem or I've designed the query in a wrong way...?
Thanks for any tip!

firstBook.Tags is a lazily-loaded IEnumerable<Tag>. On the first access, all tags are loaded, and subsequent attempts to turn it into an IQueryable<Tag> do not work, since you did not start from something from which you could sensibly query.
Instead, start from a known good IQueryable<Tag>. Something along the lines of
Tag firstTag = context.Set<Tag>()
.Where(tag => tag.Books.Contains(firstBook))
.OrderBy(tag => tag.Id).Skip(0).Take(1).SingleOrDefault();
should work. You might need minor tweaking to turn the filter condition into something EF understands.

As #hvd pointed out, I had to work with IQueryable<Tag>, whereas firstBook.Tags navigation property is just a lazy-loaded IEnumerable<Tag>.
So here is the solution of my problem, based on the #hvd's answer.
Tag firstTag = context.Set<Tag>() // or even context.Tags
.Where(tag => tag.Books.Any(book => book.Id == firstBook.Id))
.OrderBy(tag => tag.Id)
.Skip(0).Take(1)
.SingleOrDefault();
So the minor changes of #hvd's solution are: replacing the
.Where(tag => tag.Books.Contains(firstBook)) with
Something that EF understands
1) .Where(tag => tag.Books.Any(book => book.Id == firstBook.Id)).
or
2) .Where(tag => tag.Books.Select(book => book.Id).Contains(firstBook.Id))
Any sequence of code (1) or (2) generates the following SQL query, which is definitely no longer an unbounded result set.
SELECT [Project2].[Id] AS [Id],
[Project2].[Version] AS [Version],
[Project2].[Name] AS [Name]
FROM (SELECT [Extent1].[Id] AS [Id],
[Extent1].[Version] AS [Version],
[Extent1].[Name] AS [Name]
FROM [Literature].[Tag] AS [Extent1]
WHERE EXISTS (SELECT 1 AS [C1]
FROM [Literature].[BookTagRelation] AS [Extent2]
WHERE ([Extent1].[Id] = [Extent2].[TagId])
AND ([Extent2].[BookId] = 1 /* #p__linq__0 */))) AS [Project2]
ORDER BY [Project2].[Id] ASC
OFFSET 0 ROWS
FETCH NEXT 1 ROWS ONLY

Entity Framework Include returning different values for Single and Where

My application can be used to send messages to a combination of single users and teams containing sets of users. The structure is such, that a Message entity has one Recipients entity, which in turn has a list of User entities and a list of Team entities.
I am trying to get a list of Message entities using EF Code First and Linq-to-Entities, and I want to Include the Recipients and Teams to avoid large amounts of lazy loading requests later on.
The strange thing is, the Teams list is always empty if I use the Include clause. After some experimenting, it boils down to this:
var messages = GetAll()
.Include(m => m.Recipients.Teams)
.Where(m => m.Id == 123)
.ToList();
returns a list with one message, where the Teams list is empty. (GetAll() just returns an IQueryable<Message>.) But if I do
var message = GetAll()
.Include(m => m.Recipients.Teams)
.Single(m => m.Id == 123);
then I get the single message, with the Teams correctly populated.
Any ideas why this is happening?
Edit: Here's the generated SQL (taken from Entity Framework Profiler)
Where statement
SELECT *
FROM (SELECT [Extent1].[Id] AS [Id],
[Extent1].[ParentRelation] AS [ParentRelation],
[Extent1].[CreatedUtc] AS [CreatedUtc],
[Extent1].[Subject] AS [Subject],
[Extent1].[Introduction] AS [Introduction],
[Extent1].[Body] AS [Body],
[Extent1].[GlobalId] AS [GlobalId],
[Extent1].[Team_Id] AS [Team_Id],
[Extent1].[Creator_Id] AS [Creator_Id],
[Extent1].[Parent_Id] AS [Parent_Id],
[Extent1].[ReplyTo_Id] AS [ReplyTo_Id],
[Join1].[Id1] AS [Id1],
[Join1].[ToSupervisors] AS [ToSupervisors],
[Join1].[Organisation_Id] AS [Organisation_Id],
[Join1].[Id2] AS [Id2],
[Join4].[Id3] AS [Id3],
[Join4].[Name] AS [Name],
[Join4].[CreatedUtc] AS [CreatedUtc1],
[Join4].[Description] AS [Description],
[Join4].[Color] AS [Color],
[Join4].[Status] AS [Status],
[Join4].[Organisation_Id] AS [Organisation_Id1],
CASE
WHEN ([Join4].[Recipients_Id1] IS NULL) THEN CAST(NULL AS int)
ELSE 1
END AS [C1]
FROM [dbo].[Messages] AS [Extent1]
INNER JOIN (SELECT [Extent2].[Id] AS [Id1],
[Extent2].[ToSupervisors] AS [ToSupervisors],
[Extent2].[Organisation_Id] AS [Organisation_Id],
[Extent3].[Id] AS [Id2]
FROM [dbo].[Recipients] AS [Extent2]
LEFT OUTER JOIN [dbo].[MessageExtensions] AS [Extent3]
ON [Extent2].[Id] = [Extent3].[Recipients_Id]) AS [Join1]
ON [Extent1].[Recipients_Id] = [Join1].[Id1]
LEFT OUTER JOIN (SELECT [Extent4].[Recipients_Id] AS [Recipients_Id1],
[Extent5].[Id] AS [Id3],
[Extent5].[Name] AS [Name],
[Extent5].[CreatedUtc] AS [CreatedUtc],
[Extent5].[Description] AS [Description],
[Extent5].[Color] AS [Color],
[Extent5].[Status] AS [Status],
[Extent5].[Organisation_Id] AS [Organisation_Id],
[Extent6].[Recipients_Id] AS [Recipients_Id2]
FROM [dbo].[RecipientsTeams] AS [Extent4]
INNER JOIN [dbo].[Teams] AS [Extent5]
ON [Extent4].[Team_Id] = [Extent5].[Id]
INNER JOIN [dbo].[MessageExtensions] AS [Extent6]
ON 1 = 1) AS [Join4]
ON ([Extent1].[Recipients_Id] = [Join4].[Recipients_Id2])
AND ([Extent1].[Recipients_Id] = [Join4].[Recipients_Id1])
WHERE 11021 = [Extent1].[Id]) AS [Project1]
ORDER BY [Project1].[Id] ASC,
[Project1].[Id1] ASC,
[Project1].[Id2] ASC,
[Project1].[C1] ASC
Single statement
SELECT *
FROM (SELECT [Limit1].[Id1] AS [Id],
[Limit1].[ParentRelation] AS [ParentRelation],
[Limit1].[CreatedUtc] AS [CreatedUtc],
[Limit1].[Subject] AS [Subject],
[Limit1].[Introduction] AS [Introduction],
[Limit1].[Body1] AS [Body],
[Limit1].[GlobalId1] AS [GlobalId],
[Limit1].[Team_Id] AS [Team_Id],
[Limit1].[Creator_Id] AS [Creator_Id],
[Limit1].[Parent_Id] AS [Parent_Id],
[Limit1].[ReplyTo_Id] AS [ReplyTo_Id],
[Limit1].[Id2] AS [Id1],
[Limit1].[ToSupervisors] AS [ToSupervisors],
[Limit1].[Organisation_Id] AS [Organisation_Id],
[Limit1].[Id3] AS [Id2],
[Join5].[Id4] AS [Id3],
[Join5].[Name] AS [Name],
[Join5].[CreatedUtc1] AS [CreatedUtc1],
[Join5].[Description] AS [Description],
[Join5].[Color] AS [Color],
[Join5].[Status] AS [Status],
[Join5].[Organisation_Id] AS [Organisation_Id1],
CASE
WHEN ([Join5].[Recipients_Id1] IS NULL) THEN CAST(NULL AS int)
ELSE 1
END AS [C1]
FROM (SELECT TOP (2) [Extent1].[Id] AS [Id1],
[Extent1].[ParentRelation] AS [ParentRelation],
[Extent1].[CreatedUtc] AS [CreatedUtc],
[Extent1].[Subject] AS [Subject],
[Extent1].[Introduction] AS [Introduction],
[Extent1].[Body] AS [Body1],
[Extent1].[GlobalId] AS [GlobalId1],
[Extent1].[Team_Id] AS [Team_Id],
[Extent1].[Creator_Id] AS [Creator_Id],
[Extent1].[Parent_Id] AS [Parent_Id],
[Extent1].[ReplyTo_Id] AS [ReplyTo_Id],
[Join1].[Id2],
[Join1].[ToSupervisors],
[Join1].[Organisation_Id],
[Join1].[Id3]
FROM [dbo].[Messages] AS [Extent1]
INNER JOIN (SELECT [Extent2].[Id] AS [Id2],
[Extent2].[ToSupervisors] AS [ToSupervisors],
[Extent2].[Organisation_Id] AS [Organisation_Id],
[Extent3].[Id] AS [Id3]
FROM [dbo].[Recipients] AS [Extent2]
LEFT OUTER JOIN [dbo].[MessageExtensions] AS [Extent3]
ON [Extent2].[Id] = [Extent3].[Recipients_Id]) AS [Join1]
ON [Extent1].[Recipients_Id] = [Join1].[Id2]
WHERE 11021 = [Extent1].[Id]) AS [Limit1]
LEFT OUTER JOIN (SELECT [Extent4].[Recipients_Id] AS [Recipients_Id1],
[Extent5].[Id] AS [Id4],
[Extent5].[Name] AS [Name],
[Extent5].[CreatedUtc] AS [CreatedUtc1],
[Extent5].[Description] AS [Description],
[Extent5].[Color] AS [Color],
[Extent5].[Status] AS [Status],
[Extent5].[Organisation_Id] AS [Organisation_Id],
[Join4].[Id5],
[Join4].[Recipients_Id2]
FROM [dbo].[RecipientsTeams] AS [Extent4]
INNER JOIN [dbo].[Teams] AS [Extent5]
ON [Extent4].[Team_Id] = [Extent5].[Id]
INNER JOIN (SELECT [Extent6].[Id] AS [Id5],
[Extent6].[Recipients_Id] AS [Recipients_Id2]
FROM [dbo].[Messages] AS [Extent6]
LEFT OUTER JOIN [dbo].[MessageExtensions] AS [Extent7]
ON [Extent6].[Recipients_Id] = [Extent7].[Recipients_Id]) AS [Join4]
ON [Extent4].[Recipients_Id] = [Join4].[Recipients_Id2]) AS [Join5]
ON [Limit1].[Id1] = [Join5].[Id5]) AS [Project1]
ORDER BY [Project1].[Id] ASC,
[Project1].[Id1] ASC,
[Project1].[Id2] ASC,
[Project1].[C1] ASC
When I run these queries by hand, I have the same result. For the Where, the Team related properties are all NULL, while for the Single, they are populated.
Edit 2 The GetAll method is a repository method
public virtual IQueryable<T> GetAll()
{
return Context.Set<T>();
}
where T is Message

Can you try this?
var messages = GetAll().Include(m => m.Recipients.Teams) .Where(m => m.Id == 123).Select(m=>m);

Entity Framework query with 2 many-to-many joins

I'm working on a project which is using EF Code first and has the following model relationships:
Item (Id, Name, virtual List<Category>, virtual List<Tag>)
Category (Id, Name, virtual List<Item>)
Tag (Id, Name, virtual List<Item>)
I'm running a search where I would like to get all items where the item name = searchTerm, the category id is contained in a list of ints and where the tag name exists in a list of tags.
public IEnumerable<Item> Search(string searchTerm, IEnumerable<int> categoryIds, IEnumerable<string> tags)
{
var query = (
from i in context.Items
from c in context.Categories
from t in context.Tags
where i.Name.Contains(searchTerm)
&& categoryIds.Contains(c.Id)
&& tags.Contains(t.Name)
select i);
return query.ToList();
}
In SQL the query would look like the following:
SELECT I.* FROM Items I
INNER JOIN ItemItemCategories IIC ON IIC.Item_Id = I.Id
INNER JOIN ItemCategories C ON C.Id = IIC.ItemCategory_Id
INNER JOIN ItemItemTags IIT ON IIT.Item_Id = I.Id
INNER JOIN ItemTags T On T.Id = IIT.ItemTag_Id
WHERE I.Question like '%sample%' -- searchTerm
AND C.Id in (1,2) -- categoryIds
AND (T.Text like '%Difficult%' OR T.Text like '%Technical%') -- tags
My question is how can I form my code to return the query above. This is the most efficient way to perform the query from my knowledge. Currently the following query is being run from code:
SELECT
[Filter1].[Id1] AS [Id],
[Filter1].[Name] AS [Name]
FROM (
SELECT
[Extent1].[Id] AS [Id1],
[Extent1].[Name] AS [Name]
FROM [dbo].[Items] AS [Extent1]
CROSS JOIN [dbo].[Categories] AS [Extent2]
WHERE [Extent2].[Id] IN (1, 2) ) AS [Filter1]
CROSS JOIN [dbo].[Tags] AS [Extent3]
WHERE ([Filter1].[Name] LIKE #p__linq__0 ESCAPE N'~') AND ([Extent3].[Name] IN (N'Difficult', N'Technical')) AND ([Extent3].[Name] IS NOT NULL)

Try this:
var query = ( from i in context.Items
from c in i.Categories
from t in i.Tags
where i.Name.Contains(searchTerm)
&& categoryIds.Contains(c.Id)
&& tags.Contains(t.Name)
select i).ToList();
You do not have to search through all the categories and tags elements, only those who are related with you Item.
About the query you want, IMHO I don't think there's a more efficient query in Linq to Entities to get the result you are expecting that the query I propose above. Look the sql code that is generated:
SELECT
[Filter1].[Id] AS [Id],
[Filter1].[Name] AS [Name]
FROM (SELECT [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name]
FROM [dbo].[Items] AS [Extent1]
INNER JOIN [dbo].[ItemCategories] AS [Extent2] ON [Extent1].[Id] = [Extent2].[Item_Id]
WHERE [Extent2].[Category_Id] IN (1, 2) ) AS [Filter1]
INNER JOIN (SELECT [Extent3].[Item_Id] AS [Item_Id]
FROM [dbo].[TagItems] AS [Extent3]
INNER JOIN [dbo].[Tags] AS [Extent4] ON [Extent4].[Id] = [Extent3].[Tag_Id]
WHERE ([Extent4].[Name] IN (N'Difficult', N'Technical')) AND ([Extent4].[Name] IS NOT NULL) ) AS [Filter2] ON [Filter1].[Id] = [Filter2].[Item_Id]
WHERE [Filter1].[Name] LIKE #p__linq__0 ESCAPE N'~'
As you can see it is quite similar with the query that you expect.

EF returns incorrect rows

I have a simple query:
IEnumerable<AmountChangeDocumentItemExtended> GetItemsByHeaderId(Guid headerId){
var results = (from item in _context.AmountChangeDocumentItems
where item.HeaderId == headerId
select new AmountChangeDocumentItemExtended
{.....}).ToArray();
return results;
}
It returns 0 rows, but when i get rows using plain query (ExecuteStoreQuery<>()) i receive actual data.
_context.ExecuteStoreQuery<AmountChangeDocumentItem>(#"SELECT *
FROM
dbo.AmountChangeDocumentItems i
WHERE i.HeaderId = '201dae45-64a6-4b49-b801-c8fb20165ee5'");
i don't understand, why the first method not work?
For any another guid both methods returns correct rows...
In first case i don't see any income queries in my SQL Server profiler...
UPD:
Trace string:
SELECT
[Extent1].[ProductCode] AS [ProductCode],
[Extent1].[Id] AS [Id],
[Extent1].[HeaderId] AS [HeaderId],
[Extent1].[ProductVersion] AS [ProductVersion],
[Extent1].[Amount] AS [Amount],
[Extent1].[PriceCalculatorVersion] AS [PriceCalculatorVersion],
[Extent1].[FirstSupplierPrice] AS [FirstSupplierPrice],
[Extent1].[NonTaxablePart] AS [NonTaxablePart],
[Extent1].[SupplierTaxRate] AS [SupplierTaxRate],
[Extent1].[SupplierInterestRate] AS [SupplierInterestRate],
[Extent1].[RetailTaxRate] AS [RetailTaxRate],
[Extent1].[RetailInterestRate] AS [RetailInterestRate],
[Extent1].[RetailPrice] AS [RetailPrice],
[Extent2].[AgentCode] AS [AgentCode],
[Extent3].[Name] AS [Name],
[Extent1].[ItemOrder] AS [ItemOrder],
[Extent1].[Discount] AS [Discount],
[Extent1].[ExpectedRetailFinalPriceTotal] AS [ExpectedRetailFinalPriceTotal],
[Extent4].[BarCodeString] AS [BarCodeString],
[Extent4].[Name] AS [Name1],
[Extent4].[CountryOfOrigin] AS [CountryOfOrigin],
[Extent4].[UnitOfMeasureCode] AS [UnitOfMeasureCode],
[Extent5].[ShortName] AS [ShortName],
[Extent6].[IsFractional] AS [IsFractional],
[Extent1].[CheckId] AS [CheckId],
CASE WHEN ([Extent1].[CheckId] IS NOT NULL) THEN [Extent7].[Number] END AS [C1],
CASE WHEN ([Extent1].[CheckId] IS NOT NULL) THEN [Extent7].[Sum] END AS [C2],
[Extent1].[AlternateBarCode] AS [AlternateBarCode],
[Extent1].[AlternateBarCodeCoeff] AS [AlternateBarCodeCoeff],
[Extent1].[PalmInitialAmount] AS [PalmInitialAmount],
[Extent1].[PalmInitialRetailPrice] AS [PalmInitialRetailPrice],
[Extent1].[PalmOutAmount] AS [PalmOutAmount],
[Extent1].[PalmOutRetailPrice] AS [PalmOutRetailPrice]
FROM [dbo].[AmountChangeDocumentItems] AS [Extent1]
INNER JOIN [dbo].[CurrentSuppliers] AS [Extent2] ON ([Extent1].[ProductCode] = [Extent2].[ProductCode]) AND ([Extent1].[ProductVersion] = [Extent2].[ProductVersion])
LEFT OUTER JOIN [dbo].[Agents] AS [Extent3] ON [Extent2].[AgentCode] = [Extent3].[Code]
LEFT OUTER JOIN [dbo].[Products] AS [Extent4] ON ([Extent1].[ProductCode] = [Extent4].[InternalCode]) AND ([Extent1].[ProductVersion] = [Extent4].[Version])
LEFT OUTER JOIN [dbo].[UnitsOfMeasure] AS [Extent5] ON [Extent4].[UnitOfMeasureCode] = [Extent5].[Code]
LEFT OUTER JOIN [dbo].[UnitsOfMeasure] AS [Extent6] ON [Extent4].[UnitOfMeasureCode] = [Extent6].[Code]
LEFT OUTER JOIN [dbo].[CheckItems] AS [Extent7] ON [Extent1].[CheckId] = [Extent7].[Id]
WHERE [Extent1].[HeaderId] = #p__linq__0

The problem was in my edmx model: i have incorrect association multiplier (1 - 1 insted of 1 - 0..1), so EF generate INNER JOIN insted of LEFT JOIN.