I have a list of StudentGrades which contains TermNumber, I want to group them by TermNumber. However, when I look at the results, the ones with null TermNumbers are not returned/grouped.
IQueryable<StudentGradeDm> query = GetListQuery().Where(m => m.StudentId == studentId);
IQueryable<StudentGradeDto> groupedQuery = query.GroupBy(m => m.TermNumber)
.Select(m => new StudentGradeDto
{
TermNumber = m.Key,
StudentGrades = m.ToList()
});
return groupedQuery;
As you can see from the lovely screenshot i've taken here .. there are 3 groups, however, the first group is null because their TermNumber is null. But theoretically, who cares if its null? The StudentGrades with null TermNumber should still be a group.
I understand that one fix to this would be to call
query.ToList().GroupBy ......
But this is not an option for me, as the streamlined application will take a query such as the one above, and feed it into a generic pagination and sort function, in which only a subset of records will be fetched from the database to improve performance.
Any expert inputs would be greatly appreciated!
Update: Here is the generated SQL
{SELECT
[Project2].[C2] AS [C1],
[Project2].[C1] AS [C2],
[Project2].[C3] AS [C3],
[Project2].[Id] AS [Id],
[Project2].[StudentId] AS [StudentId],
[Project2].[Year] AS [Year],
[Project2].[TermTypeId] AS [TermTypeId],
[Project2].[TermNumber] AS [TermNumber],
[Project2].[ClassId] AS [ClassId],
[Project2].[GradeId] AS [GradeId],
[Project2].[FileId] AS [FileId],
[Project2].[CreatedById] AS [CreatedById],
[Project2].[CreatedDate] AS [CreatedDate],
[Project2].[ModifiedById] AS [ModifiedById],
[Project2].[ModifiedDate] AS [ModifiedDate],
[Project2].[Deleted] AS [Deleted]
FROM ( SELECT
[Distinct1].[C1] AS [C1],
1 AS [C2],
[Extent2].[Id] AS [Id],
[Extent2].[StudentId] AS [StudentId],
[Extent2].[Year] AS [Year],
[Extent2].[TermTypeId] AS [TermTypeId],
[Extent2].[TermNumber] AS [TermNumber],
[Extent2].[ClassId] AS [ClassId],
[Extent2].[GradeId] AS [GradeId],
[Extent2].[FileId] AS [FileId],
[Extent2].[CreatedById] AS [CreatedById],
[Extent2].[CreatedDate] AS [CreatedDate],
[Extent2].[ModifiedById] AS [ModifiedById],
[Extent2].[ModifiedDate] AS [ModifiedDate],
[Extent2].[Deleted] AS [Deleted],
CASE WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C3]
FROM (SELECT DISTINCT
[Extent1].[TermNumber] AS [C1]
FROM [dbo].[StudentGrade] AS [Extent1]
WHERE ([Extent1].[Deleted] <> 1) AND ([Extent1].[StudentId] = #p__linq__0) ) AS [Distinct1]
LEFT OUTER JOIN [dbo].[StudentGrade] AS [Extent2] ON ([Extent2].[Deleted] <> 1) AND ([Extent2].[StudentId] = #p__linq__0) AND ([Distinct1].[C1] = [Extent2].[TermNumber])
) AS [Project2]
ORDER BY [Project2].[C1] ASC, [Project2].[C3] ASC}
Related
Assuming the following code that applies filtering logic to a passed on collection.
private IQueryable<Customer> ApplyCustomerFilter(CustomerFilter filter, IQueryable<Customer> customers)
{
...
if (filter.HasProductInBackOrder == true)
{
customers = customers.Where(c => c.Orders.Any(o => o.Products.Any(p => p.Status == ProductStatus.BackOrder)))
}
....
return customers;
}
Results in this SQL statement:
SELECT [Extent1].[CustomerId] AS [CustomerId],
[Extent1].[Status] AS [Status]
FROM [Customers] AS [Extent1]
WHERE
(
EXISTS
(
SELECT 1 AS [C1]
FROM
(
SELECT [Extent3].[OrderId] AS [OrderId]
FROM [Orders] AS [Extent3]
WHERE [Extent1].[CustomerId] = [Extent3].[CustomerId]
) AS [Project1]
WHERE EXISTS
(
SELECT 1 AS [C1]
FROM [Products] AS [Extent4]
WHERE ([Project1].[OrderId] = [Extent4].[OrderId])
AND ([Extent4].[Status] = #p__linq__6)
)
)
)
However, I would like to optimize this by forcing to use INNER JOINS so that the result will be similar to this:
SELECT [Extent1].[CustomerId] AS [CustomerId],
[Extent1].[Status] AS [Status]
FROM [Customers] AS [Extent1]
INNER JOIN [Orders] AS [Extent2] ON [Extent1].[CustomerId] = [Extent2].[CustomerId]
INNER JOIN [Products] AS [Extent3] ON [Extent2].[OrderId] = [Extent3].[OrderId]
WHERE [Extent3].[Status] = #p__linq__6
I've tried multiple approaches, but I was unable to accomplish the desired result. Any suggestions on how to force the correct joins and avoiding subselects?
I have a problem with an unwanted ORDER BY on Entity Frameworks GroupJoin.
But at first my query. Its a simple query chain with a JOIN and ORDER.
First step - Simple condition query:
var dataQuery = model.InverterData
.Where(d => d.Timestamp >= from && d.Timestamp < till)
.Select(d => new
{
d.InverterID,
d.Timestamp
});
Second step - OUTER JOIN query:
var dayDataQuery = model.InverterDayData
.GroupJoin(dataQuery
, outer => outer.InverterID
, inner => inner.InverterID
, (outer, inner) => new
{
outer.InverterID,
outer.Date,
outer.DayYield,
InverterData = inner.Select(d => new
{
d.Timestamp
})
});
Third step - ORDER BY:
var orderedQuery = dayDataQuery
.OrderBy(d => d.DayYield)
.ThenBy(d => d.InverterID);
My Problem now is that the GroupJoin builds up a query with an unwanted ORDER BY at the end. That made my order complete senseless.
But in details...
This is the generated SQL after Step 1:
SELECT
1 AS [C1],
[Extent1].[InverterID] AS [InverterID],
[Extent1].[Timestamp] AS [Timestamp]
FROM [data].[InverterData] AS [Extent1]
WHERE ([Extent1].[Timestamp] >= #p__linq__0) AND ([Extent1].[Timestamp] < #p__linq__1)
Looks good!
Generated SQL after Step 2:
SELECT
[Project1].[C1] AS [C1],
[Project1].[InverterID] AS [InverterID],
[Project1].[Date] AS [Date],
[Project1].[DayYield] AS [DayYield],
[Project1].[C2] AS [C2],
[Project1].[Timestamp] AS [Timestamp]
FROM ( SELECT
[Extent1].[InverterID] AS [InverterID],
[Extent1].[Date] AS [Date],
[Extent1].[DayYield] AS [DayYield],
1 AS [C1],
[Extent2].[Timestamp] AS [Timestamp],
CASE WHEN ([Extent2].[InverterID] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C2]
FROM [data].[InverterDayData] AS [Extent1]
LEFT OUTER JOIN [data].[InverterData] AS [Extent2] ON ([Extent2].[Timestamp] >= #p__linq__0) AND ([Extent2].[Timestamp] < #p__linq__1) AND ([Extent1].[InverterID] = [Extent2].[InverterID])
) AS [Project1]
ORDER BY [Project1].[InverterID] ASC, [Project1].[Date] ASC, [Project1].[C2] ASC
An unwanted ORDER BY is added ORDER BY [Project1].[InverterID] ASC, [Project1].[Date] ASC, [Project1].[C2] ASC!
Generated SQL after Step 3:
SELECT
[Project1].[InverterID] AS [InverterID],
[Project1].[C1] AS [C1],
[Project1].[Date] AS [Date],
[Project1].[DayYield] AS [DayYield],
[Project1].[C2] AS [C2],
[Project1].[Timestamp] AS [Timestamp]
FROM ( SELECT
[Extent1].[InverterID] AS [InverterID],
[Extent1].[Date] AS [Date],
[Extent1].[DayYield] AS [DayYield],
1 AS [C1],
[Extent2].[Timestamp] AS [Timestamp],
CASE WHEN ([Extent2].[InverterID] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C2]
FROM [data].[InverterDayData] AS [Extent1]
LEFT OUTER JOIN [data].[InverterData] AS [Extent2] ON ([Extent2].[Timestamp] >= #p__linq__0) AND ([Extent2].[Timestamp] < #p__linq__1) AND ([Extent1].[InverterID] = [Extent2].[InverterID])
) AS [Project1]
ORDER BY [Project1].[DayYield] ASC, [Project1].[InverterID] ASC, [Project1].[Date] ASC, [Project1].[C2] ASC
My order (by DayYield and InverterID) will not work because Date and C2 is now also include.
Question:
Why and/or how can I make it work correct?
On-server order is need because on-server paging.
Answer related to #IvanStoev
How do you mean this? Its made an huge different on the resulting order.
I'm wondering why the generated SQL is checking for nulls on a column that is not nullable (column Value which is a not-null float),
var query = _context.Events
.Select(e => new
{
eventId = e.EventId,
data = e.Data.Take(10).Select(x => new
{
name = x.Name,
value = Math.Round(x.Value,1),
})
});
Generates the following SQL code:
SELECT
[Project2].[EventId] AS [EventId],
[Project2].[DeviceId] AS [DeviceId],
[Project2].[TimeEnd] AS [TimeEnd],
[Project2].[C2] AS [C1],
[Project2].[EventId1] AS [EventId1],
[Project2].[Name] AS [Name],
[Project2].[C1] AS [C2]
FROM ( SELECT
[Limit1].[EventId] AS [EventId],
[Limit1].[TimeEnd] AS [TimeEnd],
[Limit1].[DeviceId] AS [DeviceId],
[Extent2].[Name] AS [Name],
[Extent2].[EventId] AS [EventId1],
CASE WHEN ([Extent2].[Value] IS NULL) THEN CAST(NULL AS float) ELSE ROUND([Extent2].[Value], 1) END AS [C1],
CASE WHEN ([Extent2].[Value] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C2]
FROM (SELECT [Project1].[EventId] AS [EventId], [Project1].[TimeEnd] AS [TimeEnd], [Project1].[DeviceId] AS [DeviceId]
FROM ( SELECT
[Extent1].[EventId] AS [EventId],
[Extent1].[TimeEnd] AS [TimeEnd],
[Extent1].[DeviceId] AS [DeviceId]
FROM [dbo].[Events] AS [Extent1]
WHERE ([Extent1].[DeviceId] = 1)
) AS [Project1]
ORDER BY [Project1].[TimeEnd] DESC ) AS [Limit1]
LEFT OUTER JOIN [dbo].[Octaves] AS [Extent2] ON [Limit1].[EventId] = [Extent2].[EventId]
) AS [Project2]
ORDER BY [Project2].[TimeEnd] DESC, [Project2].[EventId] ASC, [Project2].[C2] ASC
If I remove the two CASE WHEN ([Extent2].[Value] IS NULL) and just leave ROUND([Limit2].[Value], 1) AS [C1] and also remove the ORDER BY Project2].[C2] ASC in SQL Server Management the query speeds up.
There is a LEFT OUTER JOIN, so the values (in the query) can definitely be NULL (even if the underlying table definition is not nullable).
See http://docs.oracle.com/javadb/10.8.3.0/ref/rrefsqlj18922.html (for Oracle - but the same basic pattern applies for all the main SQL vendors).
In this case, it is a bit pointless since ROUND(NULL, 1) would return NULL anyway (well, it would on SQL Server). But there isn't much you can do about that, given you can't control the SQL it generates.
My application can be used to send messages to a combination of single users and teams containing sets of users. The structure is such, that a Message entity has one Recipients entity, which in turn has a list of User entities and a list of Team entities.
I am trying to get a list of Message entities using EF Code First and Linq-to-Entities, and I want to Include the Recipients and Teams to avoid large amounts of lazy loading requests later on.
The strange thing is, the Teams list is always empty if I use the Include clause. After some experimenting, it boils down to this:
var messages = GetAll()
.Include(m => m.Recipients.Teams)
.Where(m => m.Id == 123)
.ToList();
returns a list with one message, where the Teams list is empty. (GetAll() just returns an IQueryable<Message>.) But if I do
var message = GetAll()
.Include(m => m.Recipients.Teams)
.Single(m => m.Id == 123);
then I get the single message, with the Teams correctly populated.
Any ideas why this is happening?
Edit: Here's the generated SQL (taken from Entity Framework Profiler)
Where statement
SELECT *
FROM (SELECT [Extent1].[Id] AS [Id],
[Extent1].[ParentRelation] AS [ParentRelation],
[Extent1].[CreatedUtc] AS [CreatedUtc],
[Extent1].[Subject] AS [Subject],
[Extent1].[Introduction] AS [Introduction],
[Extent1].[Body] AS [Body],
[Extent1].[GlobalId] AS [GlobalId],
[Extent1].[Team_Id] AS [Team_Id],
[Extent1].[Creator_Id] AS [Creator_Id],
[Extent1].[Parent_Id] AS [Parent_Id],
[Extent1].[ReplyTo_Id] AS [ReplyTo_Id],
[Join1].[Id1] AS [Id1],
[Join1].[ToSupervisors] AS [ToSupervisors],
[Join1].[Organisation_Id] AS [Organisation_Id],
[Join1].[Id2] AS [Id2],
[Join4].[Id3] AS [Id3],
[Join4].[Name] AS [Name],
[Join4].[CreatedUtc] AS [CreatedUtc1],
[Join4].[Description] AS [Description],
[Join4].[Color] AS [Color],
[Join4].[Status] AS [Status],
[Join4].[Organisation_Id] AS [Organisation_Id1],
CASE
WHEN ([Join4].[Recipients_Id1] IS NULL) THEN CAST(NULL AS int)
ELSE 1
END AS [C1]
FROM [dbo].[Messages] AS [Extent1]
INNER JOIN (SELECT [Extent2].[Id] AS [Id1],
[Extent2].[ToSupervisors] AS [ToSupervisors],
[Extent2].[Organisation_Id] AS [Organisation_Id],
[Extent3].[Id] AS [Id2]
FROM [dbo].[Recipients] AS [Extent2]
LEFT OUTER JOIN [dbo].[MessageExtensions] AS [Extent3]
ON [Extent2].[Id] = [Extent3].[Recipients_Id]) AS [Join1]
ON [Extent1].[Recipients_Id] = [Join1].[Id1]
LEFT OUTER JOIN (SELECT [Extent4].[Recipients_Id] AS [Recipients_Id1],
[Extent5].[Id] AS [Id3],
[Extent5].[Name] AS [Name],
[Extent5].[CreatedUtc] AS [CreatedUtc],
[Extent5].[Description] AS [Description],
[Extent5].[Color] AS [Color],
[Extent5].[Status] AS [Status],
[Extent5].[Organisation_Id] AS [Organisation_Id],
[Extent6].[Recipients_Id] AS [Recipients_Id2]
FROM [dbo].[RecipientsTeams] AS [Extent4]
INNER JOIN [dbo].[Teams] AS [Extent5]
ON [Extent4].[Team_Id] = [Extent5].[Id]
INNER JOIN [dbo].[MessageExtensions] AS [Extent6]
ON 1 = 1) AS [Join4]
ON ([Extent1].[Recipients_Id] = [Join4].[Recipients_Id2])
AND ([Extent1].[Recipients_Id] = [Join4].[Recipients_Id1])
WHERE 11021 = [Extent1].[Id]) AS [Project1]
ORDER BY [Project1].[Id] ASC,
[Project1].[Id1] ASC,
[Project1].[Id2] ASC,
[Project1].[C1] ASC
Single statement
SELECT *
FROM (SELECT [Limit1].[Id1] AS [Id],
[Limit1].[ParentRelation] AS [ParentRelation],
[Limit1].[CreatedUtc] AS [CreatedUtc],
[Limit1].[Subject] AS [Subject],
[Limit1].[Introduction] AS [Introduction],
[Limit1].[Body1] AS [Body],
[Limit1].[GlobalId1] AS [GlobalId],
[Limit1].[Team_Id] AS [Team_Id],
[Limit1].[Creator_Id] AS [Creator_Id],
[Limit1].[Parent_Id] AS [Parent_Id],
[Limit1].[ReplyTo_Id] AS [ReplyTo_Id],
[Limit1].[Id2] AS [Id1],
[Limit1].[ToSupervisors] AS [ToSupervisors],
[Limit1].[Organisation_Id] AS [Organisation_Id],
[Limit1].[Id3] AS [Id2],
[Join5].[Id4] AS [Id3],
[Join5].[Name] AS [Name],
[Join5].[CreatedUtc1] AS [CreatedUtc1],
[Join5].[Description] AS [Description],
[Join5].[Color] AS [Color],
[Join5].[Status] AS [Status],
[Join5].[Organisation_Id] AS [Organisation_Id1],
CASE
WHEN ([Join5].[Recipients_Id1] IS NULL) THEN CAST(NULL AS int)
ELSE 1
END AS [C1]
FROM (SELECT TOP (2) [Extent1].[Id] AS [Id1],
[Extent1].[ParentRelation] AS [ParentRelation],
[Extent1].[CreatedUtc] AS [CreatedUtc],
[Extent1].[Subject] AS [Subject],
[Extent1].[Introduction] AS [Introduction],
[Extent1].[Body] AS [Body1],
[Extent1].[GlobalId] AS [GlobalId1],
[Extent1].[Team_Id] AS [Team_Id],
[Extent1].[Creator_Id] AS [Creator_Id],
[Extent1].[Parent_Id] AS [Parent_Id],
[Extent1].[ReplyTo_Id] AS [ReplyTo_Id],
[Join1].[Id2],
[Join1].[ToSupervisors],
[Join1].[Organisation_Id],
[Join1].[Id3]
FROM [dbo].[Messages] AS [Extent1]
INNER JOIN (SELECT [Extent2].[Id] AS [Id2],
[Extent2].[ToSupervisors] AS [ToSupervisors],
[Extent2].[Organisation_Id] AS [Organisation_Id],
[Extent3].[Id] AS [Id3]
FROM [dbo].[Recipients] AS [Extent2]
LEFT OUTER JOIN [dbo].[MessageExtensions] AS [Extent3]
ON [Extent2].[Id] = [Extent3].[Recipients_Id]) AS [Join1]
ON [Extent1].[Recipients_Id] = [Join1].[Id2]
WHERE 11021 = [Extent1].[Id]) AS [Limit1]
LEFT OUTER JOIN (SELECT [Extent4].[Recipients_Id] AS [Recipients_Id1],
[Extent5].[Id] AS [Id4],
[Extent5].[Name] AS [Name],
[Extent5].[CreatedUtc] AS [CreatedUtc1],
[Extent5].[Description] AS [Description],
[Extent5].[Color] AS [Color],
[Extent5].[Status] AS [Status],
[Extent5].[Organisation_Id] AS [Organisation_Id],
[Join4].[Id5],
[Join4].[Recipients_Id2]
FROM [dbo].[RecipientsTeams] AS [Extent4]
INNER JOIN [dbo].[Teams] AS [Extent5]
ON [Extent4].[Team_Id] = [Extent5].[Id]
INNER JOIN (SELECT [Extent6].[Id] AS [Id5],
[Extent6].[Recipients_Id] AS [Recipients_Id2]
FROM [dbo].[Messages] AS [Extent6]
LEFT OUTER JOIN [dbo].[MessageExtensions] AS [Extent7]
ON [Extent6].[Recipients_Id] = [Extent7].[Recipients_Id]) AS [Join4]
ON [Extent4].[Recipients_Id] = [Join4].[Recipients_Id2]) AS [Join5]
ON [Limit1].[Id1] = [Join5].[Id5]) AS [Project1]
ORDER BY [Project1].[Id] ASC,
[Project1].[Id1] ASC,
[Project1].[Id2] ASC,
[Project1].[C1] ASC
When I run these queries by hand, I have the same result. For the Where, the Team related properties are all NULL, while for the Single, they are populated.
Edit 2 The GetAll method is a repository method
public virtual IQueryable<T> GetAll()
{
return Context.Set<T>();
}
where T is Message
Can you try this?
var messages = GetAll().Include(m => m.Recipients.Teams) .Where(m => m.Id == 123).Select(m=>m);
I need to query multiple tables with one query, and I need to limit the results from each table individually.
An example ...
I have a ContentItem, Retailer, and Product table.
ContentItem has a Type (int) field that corresponds to an enum of content types like "Retailer" and "Product." I am filtering ContentItem using this field for each sub-subquery.
ContentItem has an Id (pkey) field.
Retailer and Product have an Id (pkey) field. Id is also an FK to ContentItem.Id.
I can select from all three tables with a LEFT JOIN query. From there, I can then limit the total number of rows returned, let's say 6 rows total.
What I want to do is limit the number of rows returned from Retailer and Product individually. This way, I will have 12 rows (max) total: 6 from Retailer, and 6 from Product.
I can already accomplish this with SQL, but I am having a difficult time getting LINQ-to-Entities to "do the right thing."
Here's my SQL
SELECT * From
(
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 0 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Retailers)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 1 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Brands)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 2 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Products)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 3 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Certifications)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 4 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Claims)
) as ContentItem
LEFT JOIN [dbo].[Retailer] ON (Retailer.Id = ContentItem.Id)
LEFT JOIN [dbo].[Brand] ON (Brand.Id = ContentItem.Id)
LEFT JOIN [dbo].[Product] ON (Product.Id = ContentItem.Id)
LEFT JOIN [dbo].[Certification] ON (Certification.Id = ContentItem.Id)
LEFT JOIN [dbo].[Claim] ON (Claim.Id = ContentItem.Id);
Here's one of my many iterations of LINQ queries (which is not returning the desired result).
var queryRetailers = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Retailer)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryBrands = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Brand)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryProducts = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Product)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryCertifications = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Certification)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryClaims = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Claim)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var query = from contentItem in
queryRetailers
.Concat(queryBrands)
.Concat(queryProducts)
.Concat(queryCertifications)
.Concat(queryClaims)
join item in context.Retailer on contentItem.Id equals item.Id into retailerGroup
from retailer in retailerGroup.DefaultIfEmpty(null)
join item in context.Brand on contentItem.Id equals item.Id into brandGroup
from brand in brandGroup.DefaultIfEmpty(null)
join item in context.Product on contentItem.Id equals item.Id into productGroup
from product in productGroup.DefaultIfEmpty(null)
join item in context.Certification on contentItem.Id equals item.Id into certificationGroup
from certification in certificationGroup.DefaultIfEmpty(null)
join item in context.Claim on contentItem.Id equals item.Id into claimGroup
from claim in claimGroup.DefaultIfEmpty(null)
select new
{
contentItem,
retailer,
brand,
product,
certification,
claim
};
var results = query.ToList();
This query returns SQL that essentially "nests" my UNION ALL statements, and the server returns all rows from the database.
SELECT
[Distinct4].[C1] AS [C1],
[Distinct4].[C2] AS [C2],
[Extent6].[Id] AS [Id],
[Extent6].[RowVersion] AS [RowVersion],
[Extent6].[ctime] AS [ctime],
[Extent6].[mtime] AS [mtime],
[Extent7].[Id] AS [Id1],
[Extent7].[Recommended] AS [Recommended],
[Extent7].[RowVersion] AS [RowVersion1],
[Extent7].[ctime] AS [ctime1],
[Extent7].[mtime] AS [mtime1],
[Extent8].[Id] AS [Id2],
[Extent8].[OverrideGrade] AS [OverrideGrade],
[Extent8].[PlantBased] AS [PlantBased],
[Extent8].[Recommended] AS [Recommended1],
[Extent8].[RowVersion] AS [RowVersion2],
[Extent8].[ctime] AS [ctime2],
[Extent8].[mtime] AS [mtime2],
[Extent8].[Brand_Id] AS [Brand_Id],
[Extent8].[Grade_Name] AS [Grade_Name],
[Extent8].[Grade_Value] AS [Grade_Value],
[Extent9].[Id] AS [Id3],
[Extent9].[RowVersion] AS [RowVersion3],
[Extent9].[ctime] AS [ctime3],
[Extent9].[mtime] AS [mtime3],
[Extent9].[Grade_Name] AS [Grade_Name1],
[Extent9].[Grade_Value] AS [Grade_Value1],
[Extent10].[Id] AS [Id4],
[Extent10].[RowVersion] AS [RowVersion4],
[Extent10].[ctime] AS [ctime4],
[Extent10].[mtime] AS [mtime4],
[Extent10].[Grade_Name] AS [Grade_Name2],
[Extent10].[Grade_Value] AS [Grade_Value2]
FROM (SELECT DISTINCT
[UnionAll4].[C1] AS [C1],
[UnionAll4].[C2] AS [C2]
FROM (SELECT
[Distinct3].[C1] AS [C1],
[Distinct3].[C2] AS [C2]
FROM ( SELECT DISTINCT
[UnionAll3].[C1] AS [C1],
[UnionAll3].[C2] AS [C2]
FROM (SELECT
[Distinct2].[C1] AS [C1],
[Distinct2].[C2] AS [C2]
FROM ( SELECT DISTINCT
[UnionAll2].[C1] AS [C1],
[UnionAll2].[C2] AS [C2]
FROM (SELECT
[Distinct1].[C1] AS [C1],
[Distinct1].[C2] AS [C2]
FROM ( SELECT DISTINCT
[UnionAll1].[C1] AS [C1],
[UnionAll1].[Id] AS [C2]
FROM (SELECT TOP (1000)
[Project1].[C1] AS [C1],
[Project1].[Id] AS [Id]
FROM ( SELECT [Project1].[Id] AS [Id], [Project1].[mtime] AS [mtime], [Project1].[C1] AS [C1], row_number() OVER (ORDER BY [Project1].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent1]
WHERE 0 = CAST( [Extent1].[Type] AS int)
) AS [Project1]
) AS [Project1]
WHERE [Project1].[row_number] > 0
ORDER BY [Project1].[mtime] DESC
UNION ALL
SELECT TOP (1000)
[Project3].[C1] AS [C1],
[Project3].[Id] AS [Id]
FROM ( SELECT [Project3].[Id] AS [Id], [Project3].[mtime] AS [mtime], [Project3].[C1] AS [C1], row_number() OVER (ORDER BY [Project3].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent2].[Id] AS [Id],
[Extent2].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent2]
WHERE 1 = CAST( [Extent2].[Type] AS int)
) AS [Project3]
) AS [Project3]
WHERE [Project3].[row_number] > 0
ORDER BY [Project3].[mtime] DESC) AS [UnionAll1]
) AS [Distinct1]
UNION ALL
SELECT TOP (1000)
[Project7].[C1] AS [C1],
[Project7].[Id] AS [Id]
FROM ( SELECT [Project7].[Id] AS [Id], [Project7].[mtime] AS [mtime], [Project7].[C1] AS [C1], row_number() OVER (ORDER BY [Project7].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent3].[Id] AS [Id],
[Extent3].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent3]
WHERE 2 = CAST( [Extent3].[Type] AS int)
) AS [Project7]
) AS [Project7]
WHERE [Project7].[row_number] > 0
ORDER BY [Project7].[mtime] DESC) AS [UnionAll2]
) AS [Distinct2]
UNION ALL
SELECT TOP (1000)
[Project11].[C1] AS [C1],
[Project11].[Id] AS [Id]
FROM ( SELECT [Project11].[Id] AS [Id], [Project11].[mtime] AS [mtime], [Project11].[C1] AS [C1], row_number() OVER (ORDER BY [Project11].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent4].[Id] AS [Id],
[Extent4].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent4]
WHERE 3 = CAST( [Extent4].[Type] AS int)
) AS [Project11]
) AS [Project11]
WHERE [Project11].[row_number] > 0
ORDER BY [Project11].[mtime] DESC) AS [UnionAll3]
) AS [Distinct3]
UNION ALL
SELECT TOP (1000)
[Project15].[C1] AS [C1],
[Project15].[Id] AS [Id]
FROM ( SELECT [Project15].[Id] AS [Id], [Project15].[mtime] AS [mtime], [Project15].[C1] AS [C1], row_number() OVER (ORDER BY [Project15].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent5].[Id] AS [Id],
[Extent5].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent5]
WHERE 4 = CAST( [Extent5].[Type] AS int)
) AS [Project15]
) AS [Project15]
WHERE [Project15].[row_number] > 0
ORDER BY [Project15].[mtime] DESC) AS [UnionAll4] ) AS [Distinct4]
LEFT OUTER JOIN [dbo].[Retailer] AS [Extent6] ON [Distinct4].[C2] = [Extent6].[Id]
LEFT OUTER JOIN [dbo].[Brand] AS [Extent7] ON [Distinct4].[C2] = [Extent7].[Id]
LEFT OUTER JOIN [dbo].[Product] AS [Extent8] ON [Distinct4].[C2] = [Extent8].[Id]
LEFT OUTER JOIN [dbo].[Certification] AS [Extent9] ON [Distinct4].[C2] = [Extent9].[Id]
LEFT OUTER JOIN [dbo].[Claim] AS [Extent10] ON [Distinct4].[C2] = [Extent10].[Id]
So my overall questions are:
1) Is there a simpler SQL query I can execute to get the same results? I know that T-SQL doesn't support offsets per table in a subquery, hence the subquery wrapping.
2) If there isn't, what am I doing wrong in my LINQ query? Is this even possible with LINQ?
I wanted to add the SQL from #radar here all nice and formatted. It at least appears to be an elegant solution to avoid the sub-subqueries, and still accomplishes the offset/fetch.
SELECT *
FROM (SELECT
[ContentItem].*,
row_number() OVER ( PARTITION BY Type ORDER BY ContentItem.mtime ) as rn
FROM [dbo].[ContentItem]
LEFT JOIN [dbo].[Retailer] ON (Retailer.Id = ContentItem.Id)
LEFT JOIN [dbo].[Brand] ON (Brand.Id = ContentItem.Id)
LEFT JOIN [dbo].[Product] ON (Product.Id = ContentItem.Id)
LEFT JOIN [dbo].[Certification] ON (Certification.Id = ContentItem.Id)
LEFT JOIN [dbo].[Claim] ON (Claim.Id = ContentItem.Id)
) as x
WHERE x.rn >= a AND x.rn <= b;
a is the lower threshold (offset) and b is the upper threshold (fetch-ish). The only catch is that b now equals fetch + a instead of just fetch. The first set of results would be WHERE x.rn >= 0 AND x.rn <= 6, the second set WHERE x.rn >= 6 AND x.rn <= 12, third WHERE x.rn >= 12 AND x.rn <= 18, and so on.
As you are looking simpler SQL, you can use row_number analytic function, which would be faster
You need to try and see as there are many left joins and also proper index need to exists in these tables.
select *
from (
select *, row_number() over ( partition by Type order by ContentItem.mtime ) as rn
from [dbo].[ContentItem]
LEFT JOIN [dbo].[Retailer] ON (Retailer.Id = ContentItem.Id)
LEFT JOIN [dbo].[Brand] ON (Brand.Id = ContentItem.Id)
LEFT JOIN [dbo].[Product] ON (Product.Id = ContentItem.Id)
LEFT JOIN [dbo].[Certification] ON (Certification.Id = ContentItem.Id)
LEFT JOIN [dbo].[Claim] ON (Claim.Id = ContentItem.Id);
)
where rn <= 6
Well, it appears that I'm an idiot. That TOP(1000) call should have tipped me off. I assumed that my take variable was set to 6 but it was, in fact, set to 1000. Turns out my giant LINQ query works as expected, but the nested UNION ALL statements threw me off.
Still, I'm going to investigate #radar's answer further. It's hard to argue with better performance.