Develop Reference

EF Core difficulty with INNER JOINS and LEFT JOINS

I am trying to retrieve some data from a very big database. In order to retrieve it I need to get some data from within 6 inner joins. The following options is what I tried already:
Option 1 Using EF Core en ThenInclude
var enquiry = await _context.Enquiries
.Where(e => e.Id == id)
.Include(e => e.EnqPartJobs.Where(ej => ej.RecordStateId == 0))
.ThenInclude(ej => ej.AnswerGrpBatch)
.ThenInclude(agb => agb.AnswerGrps)
.ThenInclude(ag => ag.EnqPropResults)
.ThenInclude(epr => epr.EnqPropResultDevs)
.ThenInclude(epr => epr.Deviation)
.ToListAsync();
The advantage of this method is that it will map nicely in a Enquiry class with a list of AnswerGrpBatches. Btw I did not name these classes this is just how the database looks.
When looking at the query it is the following:
SELECT [e].[SID], [e].[FK_CRMPartner_SID], [e].[FK_Certificate_SID], [e].[createDate], [e].[displayDate], [e].[extID], [e].[finishDate], [e].[fixedGenId], [e].[aName], [t2].[SID], [t2].[FK_Enquiry_SID], [t2].[aName], [t2].[FK_RecordState_SID], [t2].[remark], [t2].[validFrom], [t2].[validTo], [t2].[SID0], [t2].[FK_EnqPartJob_SID], [t2].[FK_RecordState_SID0], [t2].[SID1], [t2].[FK_AnswerGrpBatch_SID], [t2].[FK_RecordState_SID1], [t2].[SID00], [t2].[FK_AnswerGrp_SID], [t2].[FK_EnqPropertyForm_SID], [t2].[FK_PossResult_SID], [t2].[FK_RecordState_SID00], [t2].[value], [t2].[SID000], [t2].[FK_Deviation_SID], [t2].[FK_EnqPropResult_SID], [t2].[SID0000], [t2].[aDescription], [t2].[aName0]
FROM [Enquiry] AS [e]
LEFT JOIN (
SELECT [e0].[SID], [e0].[FK_Enquiry_SID], [e0].[aName], [e0].[FK_RecordState_SID], [e0].[remark], [e0].[validFrom], [e0].[validTo], [a].[SID] AS [SID0], [a].[FK_EnqPartJob_SID], [a].[FK_RecordState_SID] AS [FK_RecordState_SID0], [t1].[SID] AS [SID1], [t1].[FK_AnswerGrpBatch_SID], [t1].[FK_RecordState_SID] AS [FK_RecordState_SID1], [t1].[SID0] AS [SID00], [t1].[FK_AnswerGrp_SID], [t1].[FK_EnqPropertyForm_SID], [t1].[FK_PossResult_SID], [t1].[FK_RecordState_SID0] AS [FK_RecordState_SID00], [t1].[value], [t1].[SID00] AS [SID000], [t1].[FK_Deviation_SID], [t1].[FK_EnqPropResult_SID], [t1].[SID000] AS [SID0000], [t1].[aDescription], [t1].[aName] AS [aName0]
FROM [EnqPartJob] AS [e0]
LEFT JOIN [AnswerGrpBatch] AS [a] ON [e0].[SID] = [a].[FK_EnqPartJob_SID]
LEFT JOIN (
SELECT [a0].[SID], [a0].[FK_AnswerGrpBatch_SID], [a0].[FK_RecordState_SID], [t0].[SID] AS [SID0], [t0].[FK_AnswerGrp_SID], [t0].[FK_EnqPropertyForm_SID], [t0].[FK_PossResult_SID], [t0].[FK_RecordState_SID] AS [FK_RecordState_SID0], [t0].[value], [t0].[SID0] AS [SID00], [t0].[FK_Deviation_SID], [t0].[FK_EnqPropResult_SID], [t0].[SID00] AS [SID000], [t0].[aDescription], [t0].[aName]
FROM [AnswerGrp] AS [a0]
LEFT JOIN (
SELECT [e1].[SID], [e1].[FK_AnswerGrp_SID], [e1].[FK_EnqPropertyForm_SID], [e1].[FK_PossResult_SID], [e1].[FK_RecordState_SID], [e1].[value], [t].[SID] AS [SID0], [t].[FK_Deviation_SID], [t].[FK_EnqPropResult_SID], [t].[SID0] AS [SID00], [t].[aDescription], [t].[aName]
FROM [EnqPropResult] AS [e1]
LEFT JOIN (
SELECT [e2].[SID], [e2].[FK_Deviation_SID], [e2].[FK_EnqPropResult_SID], [d].[SID] AS [SID0], [d].[aDescription], [d].[aName]
FROM [EnqPropResultDev] AS [e2]
INNER JOIN [Deviation] AS [d] ON [e2].[FK_Deviation_SID] = [d].[SID]
) AS [t] ON [e1].[SID] = [t].[FK_EnqPropResult_SID]
) AS [t0] ON [a0].[SID] = [t0].[FK_AnswerGrp_SID]
) AS [t1] ON [a].[SID] = [t1].[FK_AnswerGrpBatch_SID]
WHERE [e0].[FK_RecordState_SID] = 0
) AS [t2] ON [e].[SID] = [t2].[FK_Enquiry_SID]
WHERE [e].[SID] = 1790797
But it needs to be one of these 2 options:
using a IS NOT NULL statement but I cannot figure out how to make this one beceause it is deep inside different classes.
SELECT [e].[SID], [e].[FK_CRMPartner_SID], [e].[FK_Certificate_SID], [e].[createDate], [e].[displayDate], [e].[extID], [e].[finishDate], [e].[fixedGenId], [e].[aName], [t2].[SID], [t2].[FK_Enquiry_SID], [t2].[aName], [t2].[FK_RecordState_SID], [t2].[remark], [t2].[validFrom], [t2].[validTo], [t2].[SID0], [t2].[FK_EnqPartJob_SID], [t2].[FK_RecordState_SID0], [t2].[SID1], [t2].[FK_AnswerGrpBatch_SID], [t2].[FK_RecordState_SID1], [t2].[SID00], [t2].[FK_AnswerGrp_SID], [t2].[FK_EnqPropertyForm_SID], [t2].[FK_PossResult_SID], [t2].[FK_RecordState_SID00], [t2].[value], [t2].[SID000], [t2].[FK_Deviation_SID], [t2].[FK_EnqPropResult_SID], [t2].[SID0000], [t2].[aDescription], [t2].[aName0]
FROM [Enquiry] AS [e]
LEFT JOIN (
SELECT [e0].[SID], [e0].[FK_Enquiry_SID], [e0].[aName], [e0].[FK_RecordState_SID], [e0].[remark], [e0].[validFrom], [e0].[validTo], [a].[SID] AS [SID0], [a].[FK_EnqPartJob_SID], [a].[FK_RecordState_SID] AS [FK_RecordState_SID0], [t1].[SID] AS [SID1], [t1].[FK_AnswerGrpBatch_SID], [t1].[FK_RecordState_SID] AS [FK_RecordState_SID1], [t1].[SID0] AS [SID00], [t1].[FK_AnswerGrp_SID], [t1].[FK_EnqPropertyForm_SID], [t1].[FK_PossResult_SID], [t1].[FK_RecordState_SID0] AS [FK_RecordState_SID00], [t1].[value], [t1].[SID00] AS [SID000], [t1].[FK_Deviation_SID], [t1].[FK_EnqPropResult_SID], [t1].[SID000] AS [SID0000], [t1].[aDescription], [t1].[aName] AS [aName0]
FROM [EnqPartJob] AS [e0]
LEFT JOIN [AnswerGrpBatch] AS [a] ON [e0].[SID] = [a].[FK_EnqPartJob_SID]
LEFT JOIN (
SELECT [a0].[SID], [a0].[FK_AnswerGrpBatch_SID], [a0].[FK_RecordState_SID], [t0].[SID] AS [SID0], [t0].[FK_AnswerGrp_SID], [t0].[FK_EnqPropertyForm_SID], [t0].[FK_PossResult_SID], [t0].[FK_RecordState_SID] AS [FK_RecordState_SID0], [t0].[value], [t0].[SID0] AS [SID00], [t0].[FK_Deviation_SID], [t0].[FK_EnqPropResult_SID], [t0].[SID00] AS [SID000], [t0].[aDescription], [t0].[aName]
FROM [AnswerGrp] AS [a0]
LEFT JOIN (
SELECT [e1].[SID], [e1].[FK_AnswerGrp_SID], [e1].[FK_EnqPropertyForm_SID], [e1].[FK_PossResult_SID], [e1].[FK_RecordState_SID], [e1].[value], [t].[SID] AS [SID0], [t].[FK_Deviation_SID], [t].[FK_EnqPropResult_SID], [t].[SID0] AS [SID00], [t].[aDescription], [t].[aName]
FROM [EnqPropResult] AS [e1]
LEFT JOIN (
SELECT [e2].[SID], [e2].[FK_Deviation_SID], [e2].[FK_EnqPropResult_SID], [d].[SID] AS [SID0], [d].[aDescription], [d].[aName]
FROM [EnqPropResultDev] AS [e2]
INNER JOIN [Deviation] AS [d] ON [e2].[FK_Deviation_SID] = [d].[SID]
) AS [t] ON [e1].[SID] = [t].[FK_EnqPropResult_SID]
) AS [t0] ON [a0].[SID] = [t0].[FK_AnswerGrp_SID]
) AS [t1] ON [a].[SID] = [t1].[FK_AnswerGrpBatch_SID]
WHERE [e0].[FK_RecordState_SID] = 0
) AS [t2] ON [e].[SID] = [t2].[FK_Enquiry_SID]
WHERE [e].[SID] = 1790797 AND [t2].[SID000] IS NOT NULL
Forcing INNER JOINS:
SELECT [e].[SID], [e].[FK_CRMPartner_SID], [e].[FK_Certificate_SID], [e].[createDate], [e].[displayDate], [e].[extID], [e].[finishDate], [e].[fixedGenId], [e].[aName], [t2].[SID], [t2].[FK_Enquiry_SID], [t2].[aName], [t2].[FK_RecordState_SID], [t2].[remark], [t2].[validFrom], [t2].[validTo], [t2].[SID0], [t2].[FK_EnqPartJob_SID], [t2].[FK_RecordState_SID0], [t2].[SID1], [t2].[FK_AnswerGrpBatch_SID], [t2].[FK_RecordState_SID1], [t2].[SID00], [t2].[FK_AnswerGrp_SID], [t2].[FK_EnqPropertyForm_SID], [t2].[FK_PossResult_SID], [t2].[FK_RecordState_SID00], [t2].[value], [t2].[SID000], [t2].[FK_Deviation_SID], [t2].[FK_EnqPropResult_SID], [t2].[SID0000], [t2].[aDescription], [t2].[aName0]
FROM [Enquiry] AS [e]
LEFT JOIN (
SELECT [e0].[SID], [e0].[FK_Enquiry_SID], [e0].[aName], [e0].[FK_RecordState_SID], [e0].[remark], [e0].[validFrom], [e0].[validTo], [a].[SID] AS [SID0], [a].[FK_EnqPartJob_SID], [a].[FK_RecordState_SID] AS [FK_RecordState_SID0], [t1].[SID] AS [SID1], [t1].[FK_AnswerGrpBatch_SID], [t1].[FK_RecordState_SID] AS [FK_RecordState_SID1], [t1].[SID0] AS [SID00], [t1].[FK_AnswerGrp_SID], [t1].[FK_EnqPropertyForm_SID], [t1].[FK_PossResult_SID], [t1].[FK_RecordState_SID0] AS [FK_RecordState_SID00], [t1].[value], [t1].[SID00] AS [SID000], [t1].[FK_Deviation_SID], [t1].[FK_EnqPropResult_SID], [t1].[SID000] AS [SID0000], [t1].[aDescription], [t1].[aName] AS [aName0]
FROM [EnqPartJob] AS [e0]
LEFT JOIN [AnswerGrpBatch] AS [a] ON [e0].[SID] = [a].[FK_EnqPartJob_SID]
LEFT JOIN (
SELECT [a0].[SID], [a0].[FK_AnswerGrpBatch_SID], [a0].[FK_RecordState_SID], [t0].[SID] AS [SID0], [t0].[FK_AnswerGrp_SID], [t0].[FK_EnqPropertyForm_SID], [t0].[FK_PossResult_SID], [t0].[FK_RecordState_SID] AS [FK_RecordState_SID0], [t0].[value], [t0].[SID0] AS [SID00], [t0].[FK_Deviation_SID], [t0].[FK_EnqPropResult_SID], [t0].[SID00] AS [SID000], [t0].[aDescription], [t0].[aName]
FROM [AnswerGrp] AS [a0]
INNER JOIN (
SELECT [e1].[SID], [e1].[FK_AnswerGrp_SID], [e1].[FK_EnqPropertyForm_SID], [e1].[FK_PossResult_SID], [e1].[FK_RecordState_SID], [e1].[value], [t].[SID] AS [SID0], [t].[FK_Deviation_SID], [t].[FK_EnqPropResult_SID], [t].[SID0] AS [SID00], [t].[aDescription], [t].[aName]
FROM [EnqPropResult] AS [e1]
INNER JOIN (
SELECT [e2].[SID], [e2].[FK_Deviation_SID], [e2].[FK_EnqPropResult_SID], [d].[SID] AS [SID0], [d].[aDescription], [d].[aName]
FROM [EnqPropResultDev] AS [e2]
INNER JOIN [Deviation] AS [d] ON [e2].[FK_Deviation_SID] = [d].[SID]
) AS [t] ON [e1].[SID] = [t].[FK_EnqPropResult_SID]
) AS [t0] ON [a0].[SID] = [t0].[FK_AnswerGrp_SID]
) AS [t1] ON [a].[SID] = [t1].[FK_AnswerGrpBatch_SID]
WHERE [e0].[FK_RecordState_SID] = 0
) AS [t2] ON [e].[SID] = [t2].[FK_Enquiry_SID]
WHERE [e].[SID] = 1790797
The problem is that it will use Left Joins for every Include and ThenInclude except the last one. But this will also retrieve information that I do not need. So I tried to make a LINQ query.
Option 2 LINQ
The LINQ query always uses INNER JOINS, so I exactly get what I want from the database. The only problem is that I have no clue how to map this to an Enquiry class where the INNER JOINS are a list within the Enquiry class. Just like with option 1.
var query = from enquiry in _context.Set<Enquiry>()
join enqPartJob in _context.Set<EnqPartJob>()
on enquiry.Id equals enqPartJob.EnquiryId
join answerGrpBatch in _context.Set<AnswerGrpBatch>()
on enqPartJob.Id equals answerGrpBatch.EnqPartJobId
join answerGrp in _context.Set<AnswerGrp>()
on answerGrpBatch.Id equals answerGrp.AnswerGrpBatchId
join enqPropResult in _context.Set<EnqPropResult>()
on answerGrp.Id equals enqPropResult.AnswerGrpId
join enqPropResultDev in _context.Set<EnqPropResultDev>()
on enqPropResult.Id equals enqPropResultDev.EnqPropResultId
join deviation in _context.Set<Deviation>()
on enqPropResultDev.DeviationId equals deviation.Id
where enquiry.Id == id
select new { enquiry, enqPropResultDev };
And yis I get that there is a reason EFCore uses LEFT JOINS but this is not my datbase so I have to adapt to it.

Thx to Ivan Stoev who commented this suggestion.
Starting with Deviation and working my way up to Enquiry did the trick for me.
The query now looks like this (as you can see it contains INNER JOINS).
SELECT [t].[SID], [t].[aDescription], [t].[aName], [t0].[SID], [t0].[FK_Deviation_SID], [t0].[FK_EnqPropResult_SID], [t0].[SID0], [t0].[FK_AnswerGrp_SID], [t0].[FK_EnqPropertyForm_SID], [t0].[FK_PossResult_SID], [t0].[FK_RecordState_SID], [t0].[value], [t0].[SID1], [t0].[FK_AnswerGrpBatch_SID], [t0].[FK_RecordState_SID0], [t0].[SID2], [t0].[FK_EnqPartJob_SID], [t0].[FK_RecordState_SID1], [t0].[SID3], [t0].[FK_Enquiry_SID], [t0].[aName], [t0].[FK_RecordState_SID2], [t0].[remark], [t0].[validFrom], [t0].[validTo], [t0].[SID4], [t0].[FK_CRMPartner_SID], [t0].[FK_Certificate_SID], [t0].[createDate], [t0].[displayDate], [t0].[extID], [t0].[finishDate], [t0].[fixedGenId], [t0].[aName0]
FROM (
SELECT TOP(1) [d].[SID], [d].[aDescription], [d].[aName]
FROM [Deviation] AS [d]
WHERE [d].[SID] = 5038
) AS [t]
LEFT JOIN (
SELECT [e].[SID], [e].[FK_Deviation_SID], [e].[FK_EnqPropResult_SID], [e0].[SID] AS [SID0], [e0].[FK_AnswerGrp_SID], [e0].[FK_EnqPropertyForm_SID], [e0].[FK_PossResult_SID], [e0].[FK_RecordState_SID], [e0].[value], [a].[SID] AS [SID1], [a].[FK_AnswerGrpBatch_SID], [a].[FK_RecordState_SID] AS [FK_RecordState_SID0], [a0].[SID] AS [SID2], [a0].[FK_EnqPartJob_SID], [a0].[FK_RecordState_SID] AS [FK_RecordState_SID1], [e1].[SID] AS [SID3], [e1].[FK_Enquiry_SID], [e1].[aName], [e1].[FK_RecordState_SID] AS [FK_RecordState_SID2], [e1].[remark], [e1].[validFrom], [e1].[validTo], [e2].[SID] AS [SID4], [e2].[FK_CRMPartner_SID], [e2].[FK_Certificate_SID], [e2].[createDate], [e2].[displayDate], [e2].[extID], [e2].[finishDate], [e2].[fixedGenId], [e2].[aName] AS [aName0]
FROM [EnqPropResultDev] AS [e]
INNER JOIN [EnqPropResult] AS [e0] ON [e].[FK_EnqPropResult_SID] = [e0].[SID]
INNER JOIN [AnswerGrp] AS [a] ON [e0].[FK_AnswerGrp_SID] = [a].[SID]
INNER JOIN [AnswerGrpBatch] AS [a0] ON [a].[FK_AnswerGrpBatch_SID] = [a0].[SID]
INNER JOIN [EnqPartJob] AS [e1] ON [a0].[FK_EnqPartJob_SID] = [e1].[SID]
INNER JOIN [Enquiry] AS [e2] ON [e1].[FK_Enquiry_SID] = [e2].[SID]
WHERE [e0].[FK_RecordState_SID] = 0
) AS [t0] ON [t].[SID] = [t0].[FK_Deviation_SID]
ORDER BY [t].[SID], [t0].[SID], [t0].[SID0], [t0].[SID1], [t0].[SID2], [t0].[SID3], [t0].[SID4]

How to improve LINQ statement to use INNER JOIN in resulting SQL statement?

Assuming the following code that applies filtering logic to a passed on collection.
private IQueryable<Customer> ApplyCustomerFilter(CustomerFilter filter, IQueryable<Customer> customers)
{
...
if (filter.HasProductInBackOrder == true)
{
customers = customers.Where(c => c.Orders.Any(o => o.Products.Any(p => p.Status == ProductStatus.BackOrder)))
}
....
return customers;
}
Results in this SQL statement:
SELECT [Extent1].[CustomerId] AS [CustomerId],
[Extent1].[Status] AS [Status]
FROM [Customers] AS [Extent1]
WHERE
(
EXISTS
(
SELECT 1 AS [C1]
FROM
(
SELECT [Extent3].[OrderId] AS [OrderId]
FROM [Orders] AS [Extent3]
WHERE [Extent1].[CustomerId] = [Extent3].[CustomerId]
) AS [Project1]
WHERE EXISTS
(
SELECT 1 AS [C1]
FROM [Products] AS [Extent4]
WHERE ([Project1].[OrderId] = [Extent4].[OrderId])
AND ([Extent4].[Status] = #p__linq__6)
)
)
)
However, I would like to optimize this by forcing to use INNER JOINS so that the result will be similar to this:
SELECT [Extent1].[CustomerId] AS [CustomerId],
[Extent1].[Status] AS [Status]
FROM [Customers] AS [Extent1]
INNER JOIN [Orders] AS [Extent2] ON [Extent1].[CustomerId] = [Extent2].[CustomerId]
INNER JOIN [Products] AS [Extent3] ON [Extent2].[OrderId] = [Extent3].[OrderId]
WHERE [Extent3].[Status] = #p__linq__6
I've tried multiple approaches, but I was unable to accomplish the desired result. Any suggestions on how to force the correct joins and avoiding subselects?

Linq query has a bug?

Just look at the linq query and the SQL profiler resulting query. Linq repeats the subquery which it shouldn't. Is this a bug or am I doing something wrong? By the way, I know how to write this linq to give me a satisfying result, but I want to know why is this one badly interpreted?
return (from k in ctx.UzajamnaKasa
join p in ctx.UzajamnaKasaPozajmice
on k.UzajamnaKasaId equals p.UzajamnaKasaId
where k.ClanId == clanId
select p.Iznos + p.Donos -
(from ot in ctx.UzajamnaKasaPozajmiceOtplate
where p.PozajmicaId == ot.PozajmicaId
group ot.Iznos by ot.PozajmicaId into g
select g.Sum()).FirstOrDefault()
).Sum();
select [GroupBy3].[A1] as [C1]
from
(
select sum([Project2].[A1]) as [A1]
from
(
select ([Project2].[Iznos] + [Project2].[Donos])
- (case
when ([Project2].[C1] is null) then
cast(0 as decimal(18))
else
(
select top (1)
[GroupBy2].[A1] as [C1]
from
(
select [Extent4].[PozajmicaId] as [K1]
, sum([Extent4].[Iznos]) as [A1]
from [dbo].[UzajamnaKasaPozajmiceOtplate] as [Extent4]
where [Project2].[PozajmicaId] = [Extent4].[PozajmicaId]
group by [Extent4].[PozajmicaId]
) as [GroupBy2]
)
end
) as [A1]
from
(
select [Extent2].[PozajmicaId] as [PozajmicaId]
, [Extent2].[Iznos] as [Iznos]
, [Extent2].[Donos] as [Donos]
, (
select top (1)
[GroupBy1].[A1] as [C1]
from
(
select [Extent3].[PozajmicaId] as [K1]
, sum([Extent3].[Iznos]) as [A1]
from [dbo].[UzajamnaKasaPozajmiceOtplate] as [Extent3]
where [Extent2].[PozajmicaId] = [Extent3].[PozajmicaId]
group by [Extent3].[PozajmicaId]
) as [GroupBy1]
) as [C1]
from [dbo].[UzajamnaKasa] as [Extent1]
inner join [dbo].[UzajamnaKasaPozajmice] as [Extent2]
on [Extent1].[UzajamnaKasaId] = [Extent2].[UzajamnaKasaId]
where [Extent1].[ClanId] = 1303
) as [Project2]
) as [Project2]
) as [GroupBy3];
Why is he duplicating this select? It's bad for performace.:
select top (1)
[GroupBy2].[A1] as [C1]
from
(
select [Extent4].[PozajmicaId] as [K1]
, sum([Extent4].[Iznos]) as [A1]
from [dbo].[UzajamnaKasaPozajmiceOtplate] as [Extent4]
where [Project2].[PozajmicaId] = [Extent4].[PozajmicaId]
group by [Extent4].[PozajmicaId]
) as [GroupBy2]

(from ot in ctx.UzajamnaKasaPozajmiceOtplate
where p.PozajmicaId == ot.PozajmicaId
group ot.Iznos by ot.PozajmicaId into g
select g.Sum()).FirstOrDefault()
Due to the FirstOrDefault(), this sub-query will be executed for every p.PozajmicaId founded.

LINQ to SQL LEFT JOIN, GROUP BY, COUNT, MAX and subquery

I have 3 tables: Users, Roles, Bids.
Two of them is created by ASP.NET Identity, Bids is my custom table (it is secondary to Users table)
I'm trying to get all users that aren't in specific role (all non admin users) and show number of bids (they can be null - I have left join), show highest bid per user and show if that bid is max.
SQL query was quire easy:
SELECT
U.Id
,U.UserName
,U.Email
,ISNULL(MAX(B.PriceInPLN), 0)
,COUNT(B.BidID)
,IIF(MAX(B.PriceInPLN)=(SELECT TOP 1 PriceInPLN FROM Bids B2 (NOLOCK) ORDER BY PriceInPLN DESC),1,0 ) AS Max
FROM
AspNetUsers U ( NOLOCK )
JOIN AspNetUserRoles UR ( NOLOCK ) ON U.Id = UR.UserId
LEFT JOIN Bids B ( NOLOCK ) ON B.ApplicationUserId = U.Id
WHERE
UR.RoleId != '9508f9d2-12fb-4175-89a7-3275cb7616ae'
GROUP BY
U.Id
,U.UserName
,U.Email
But I have trouble creating correct LINQ query. Here is what I have so far:
var users =(
from u in db.Users
where !u.Roles.Select(r => r.RoleId).Contains(adminRoleId)//everyone except admins
join b in db.Bids on u equals b.ApplicationUser into ub
from subset in ub.DefaultIfEmpty()//LEFT JOIN
group subset by new { u.Id, u.UserName, u.Email, u.EmailConfirmed, u.Online } into grouped
select new UserReturnModel
{
Id = grouped.Key.Id,
Name = grouped.Key.UserName,
Email = grouped.Key.Email,
EmailConfirmed = grouped.Key.EmailConfirmed,
Online = grouped.Key.Online,
BidsCount = grouped.Count(c => c.ApplicationUserId == grouped.Key.Id),
PriceInPLN = grouped.Max(c => c.PriceInPLN),
IsMax = (grouped.Max(c=>c.PriceInPLN) == db.Bids.Max(b=>b.PriceInPLN))
}).ToList();
I can't get Max column to work correctly - it always says true.
Here are column definitions to easily show what I want to get:
Id = user id,
Name = user name,
Email = user email,
BidsCount = number of bids created by that user,
PriceInPLN = max price for that user,
IsMax = is this user offer highest comparing to other offers, if there are no offers this should be false
I'm using that in ASP.NET, when debugging I've noticed that SQL query created by LINQ is large:
SELECT
1 AS [C1],
[Project6].[Id] AS [Id],
[Project6].[UserName] AS [UserName],
[Project6].[Email] AS [Email],
[Project6].[EmailConfirmed] AS [EmailConfirmed],
[Project6].[Online] AS [Online],
[Project6].[C4] AS [C2],
[Project6].[C1] AS [C3],
CASE WHEN (([Project6].[C2] = [GroupBy3].[A1]) OR (([Project6].[C2] IS NULL) AND ([GroupBy3].[A1] IS NULL))) THEN cast(1 as bit) WHEN ( NOT (([Project6].[C3] = [GroupBy4].[A1]) AND ((CASE WHEN ([Project6].[C3] IS NULL) THEN cast(1 as bit) ELSE cast(0 as bit) END) = (CASE WHEN ([GroupBy4].[A1] IS NULL) THEN cast(1 as bit) ELSE cast(0 as bit) END)))) THEN cast(0 as bit) END AS [C4]
FROM (SELECT
[Project3].[C1] AS [C1],
[Project3].[C2] AS [C2],
[Project3].[C3] AS [C3],
[Project3].[Id] AS [Id],
[Project3].[Online] AS [Online],
[Project3].[Email] AS [Email],
[Project3].[EmailConfirmed] AS [EmailConfirmed],
[Project3].[UserName] AS [UserName],
(SELECT
COUNT(1) AS [A1]
FROM ( SELECT
[Extent4].[Id] AS [Id],
[Extent4].[Online] AS [Online],
[Extent4].[Email] AS [Email],
[Extent4].[EmailConfirmed] AS [EmailConfirmed],
[Extent4].[UserName] AS [UserName],
[Extent5].[ApplicationUserId] AS [ApplicationUserId]
FROM [dbo].[AspNetUsers] AS [Extent4]
INNER JOIN [dbo].[Bids] AS [Extent5] ON [Extent5].[ApplicationUserId] = [Extent4].[Id]
WHERE ( NOT EXISTS (SELECT
1 AS [C1]
FROM [dbo].[AspNetUserRoles] AS [Extent6]
WHERE ([Extent4].[Id] = [Extent6].[UserId]) AND ([Extent6].[RoleId] = #p__linq__0)
)) AND ([Project3].[Id] = [Extent4].[Id]) AND ([Project3].[UserName] = [Extent4].[UserName]) AND (([Project3].[Email] = [Extent4].[Email]) OR (([Project3].[Email] IS NULL) AND ([Extent4].[Email] IS NULL))) AND ([Project3].[EmailConfirmed] = [Extent4].[EmailConfirmed]) AND ([Project3].[Online] = [Extent4].[Online]) AND ([Extent5].[ApplicationUserId] = [Project3].[Id])
) AS [Project5]) AS [C4]
FROM ( SELECT
[GroupBy1].[A1] AS [C1],
[GroupBy1].[A2] AS [C2],
[GroupBy1].[A3] AS [C3],
[GroupBy1].[K1] AS [Id],
[GroupBy1].[K2] AS [Online],
[GroupBy1].[K3] AS [Email],
[GroupBy1].[K4] AS [EmailConfirmed],
[GroupBy1].[K5] AS [UserName]
FROM ( SELECT
[Project2].[Id] AS [K1],
[Project2].[Online] AS [K2],
[Project2].[Email] AS [K3],
[Project2].[EmailConfirmed] AS [K4],
[Project2].[UserName] AS [K5],
MAX([Project2].[PriceInPLN]) AS [A1],
MAX([Project2].[PriceInPLN]) AS [A2],
MAX([Project2].[PriceInPLN]) AS [A3]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Online] AS [Online],
[Extent1].[Email] AS [Email],
[Extent1].[EmailConfirmed] AS [EmailConfirmed],
[Extent1].[UserName] AS [UserName],
[Extent2].[PriceInPLN] AS [PriceInPLN],
[Extent2].[ApplicationUserId] AS [ApplicationUserId]
FROM [dbo].[AspNetUsers] AS [Extent1]
LEFT OUTER JOIN [dbo].[Bids] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ApplicationUserId]
WHERE NOT EXISTS (SELECT
1 AS [C1]
FROM [dbo].[AspNetUserRoles] AS [Extent3]
WHERE ([Extent1].[Id] = [Extent3].[UserId]) AND ([Extent3].[RoleId] = #p__linq__0)
)
) AS [Project2]
GROUP BY [Project2].[Id], [Project2].[Online], [Project2].[Email], [Project2].[EmailConfirmed], [Project2].[UserName]
) AS [GroupBy1]
) AS [Project3] ) AS [Project6]
CROSS JOIN (SELECT
MAX([Extent7].[PriceInPLN]) AS [A1]
FROM [dbo].[Bids] AS [Extent7] ) AS [GroupBy3]
CROSS JOIN (SELECT
MAX([Extent8].[PriceInPLN]) AS [A1]
FROM [dbo].[Bids] AS [Extent8] ) AS [GroupBy4]
Can this be simplified?
Here is model of my my database generated by Entity Framework Power Tools:

I'm not sure why you modified your LINQ query so much from the original SQL query, but translating the SQL query closer to literally using the rules from my SQL to LINQ Recipe I get
var MaxPriceInPLN = db.Bids.Max(b => b.PriceInPLN);
var ans = from U in db.Users
where !U.Roles.Any(r => r.RoleId == adminRoleId)
join b in db.Bids on U.Id equals b.ApplicationUserId into bj
from b in bj.DefaultIfEmpty()
group new { U, b } by new { U.Id, U.UserName, U.Email } into Ubg
let maxBidPriceInPLN = Ubg.Max(Ub => Ub.b.PriceInPLN)
select new {
Ubg.Key.Id,
Ubg.Key.UserName,
Ubg.Key.Email,
PriceInPLN = maxBidPriceInPLN ?? 0,
BidsCount = Ubg.Count(Ub => Ub.b != null),
IsMax = maxBidPriceInPLN == MaxPriceInPLN
};
You can simplify the LINQ query by using the EF navigation properties to hide the join from your query:
var MaxPriceInPLN = db.Bids.Max(b => b.PriceInPLN);
var ans = from U in db.Users
where !U.Roles.Any(r => r.RoleId == adminRoleId)
let maxBidPriceInPLN = U.Bids.Max(b => b.PriceInPLN)
select new {
U.Id,
U.UserName,
U.Email,
PriceInPLN = maxBidPriceInPLN ?? 0,
BidsCount = U.Bids.Count(),
IsMax = maxBidPriceInPLN == MaxPriceInPLN
};

Limit results from multiple individual tables in a single LINQ-to-Entities query. Resultant T-SQL is wrong

I need to query multiple tables with one query, and I need to limit the results from each table individually.
An example ...
I have a ContentItem, Retailer, and Product table.
ContentItem has a Type (int) field that corresponds to an enum of content types like "Retailer" and "Product." I am filtering ContentItem using this field for each sub-subquery.
ContentItem has an Id (pkey) field.
Retailer and Product have an Id (pkey) field. Id is also an FK to ContentItem.Id.
I can select from all three tables with a LEFT JOIN query. From there, I can then limit the total number of rows returned, let's say 6 rows total.
What I want to do is limit the number of rows returned from Retailer and Product individually. This way, I will have 12 rows (max) total: 6 from Retailer, and 6 from Product.
I can already accomplish this with SQL, but I am having a difficult time getting LINQ-to-Entities to "do the right thing."
Here's my SQL
SELECT * From
(
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 0 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Retailers)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 1 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Brands)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 2 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Products)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 3 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Certifications)
UNION ALL
(SELECT * FROM (SELECT * FROM [dbo].[ContentItem] WHERE Type = 4 ORDER BY ContentItem.mtime OFFSET 0 ROWS FETCH NEXT 6 ROWS ONLY) Claims)
) as ContentItem
LEFT JOIN [dbo].[Retailer] ON (Retailer.Id = ContentItem.Id)
LEFT JOIN [dbo].[Brand] ON (Brand.Id = ContentItem.Id)
LEFT JOIN [dbo].[Product] ON (Product.Id = ContentItem.Id)
LEFT JOIN [dbo].[Certification] ON (Certification.Id = ContentItem.Id)
LEFT JOIN [dbo].[Claim] ON (Claim.Id = ContentItem.Id);
Here's one of my many iterations of LINQ queries (which is not returning the desired result).
var queryRetailers = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Retailer)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryBrands = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Brand)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryProducts = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Product)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryCertifications = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Certification)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var queryClaims = contentItemModel
.Where(contentItem => contentItem.Type == ContentTypeEnum.Claim)
.OrderByDescending(o => o.mtime).Skip(skip).Take(take).Select(o => new { Id = o.Id });
var query = from contentItem in
queryRetailers
.Concat(queryBrands)
.Concat(queryProducts)
.Concat(queryCertifications)
.Concat(queryClaims)
join item in context.Retailer on contentItem.Id equals item.Id into retailerGroup
from retailer in retailerGroup.DefaultIfEmpty(null)
join item in context.Brand on contentItem.Id equals item.Id into brandGroup
from brand in brandGroup.DefaultIfEmpty(null)
join item in context.Product on contentItem.Id equals item.Id into productGroup
from product in productGroup.DefaultIfEmpty(null)
join item in context.Certification on contentItem.Id equals item.Id into certificationGroup
from certification in certificationGroup.DefaultIfEmpty(null)
join item in context.Claim on contentItem.Id equals item.Id into claimGroup
from claim in claimGroup.DefaultIfEmpty(null)
select new
{
contentItem,
retailer,
brand,
product,
certification,
claim
};
var results = query.ToList();
This query returns SQL that essentially "nests" my UNION ALL statements, and the server returns all rows from the database.
SELECT
[Distinct4].[C1] AS [C1],
[Distinct4].[C2] AS [C2],
[Extent6].[Id] AS [Id],
[Extent6].[RowVersion] AS [RowVersion],
[Extent6].[ctime] AS [ctime],
[Extent6].[mtime] AS [mtime],
[Extent7].[Id] AS [Id1],
[Extent7].[Recommended] AS [Recommended],
[Extent7].[RowVersion] AS [RowVersion1],
[Extent7].[ctime] AS [ctime1],
[Extent7].[mtime] AS [mtime1],
[Extent8].[Id] AS [Id2],
[Extent8].[OverrideGrade] AS [OverrideGrade],
[Extent8].[PlantBased] AS [PlantBased],
[Extent8].[Recommended] AS [Recommended1],
[Extent8].[RowVersion] AS [RowVersion2],
[Extent8].[ctime] AS [ctime2],
[Extent8].[mtime] AS [mtime2],
[Extent8].[Brand_Id] AS [Brand_Id],
[Extent8].[Grade_Name] AS [Grade_Name],
[Extent8].[Grade_Value] AS [Grade_Value],
[Extent9].[Id] AS [Id3],
[Extent9].[RowVersion] AS [RowVersion3],
[Extent9].[ctime] AS [ctime3],
[Extent9].[mtime] AS [mtime3],
[Extent9].[Grade_Name] AS [Grade_Name1],
[Extent9].[Grade_Value] AS [Grade_Value1],
[Extent10].[Id] AS [Id4],
[Extent10].[RowVersion] AS [RowVersion4],
[Extent10].[ctime] AS [ctime4],
[Extent10].[mtime] AS [mtime4],
[Extent10].[Grade_Name] AS [Grade_Name2],
[Extent10].[Grade_Value] AS [Grade_Value2]
FROM (SELECT DISTINCT
[UnionAll4].[C1] AS [C1],
[UnionAll4].[C2] AS [C2]
FROM (SELECT
[Distinct3].[C1] AS [C1],
[Distinct3].[C2] AS [C2]
FROM ( SELECT DISTINCT
[UnionAll3].[C1] AS [C1],
[UnionAll3].[C2] AS [C2]
FROM (SELECT
[Distinct2].[C1] AS [C1],
[Distinct2].[C2] AS [C2]
FROM ( SELECT DISTINCT
[UnionAll2].[C1] AS [C1],
[UnionAll2].[C2] AS [C2]
FROM (SELECT
[Distinct1].[C1] AS [C1],
[Distinct1].[C2] AS [C2]
FROM ( SELECT DISTINCT
[UnionAll1].[C1] AS [C1],
[UnionAll1].[Id] AS [C2]
FROM (SELECT TOP (1000)
[Project1].[C1] AS [C1],
[Project1].[Id] AS [Id]
FROM ( SELECT [Project1].[Id] AS [Id], [Project1].[mtime] AS [mtime], [Project1].[C1] AS [C1], row_number() OVER (ORDER BY [Project1].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent1]
WHERE 0 = CAST( [Extent1].[Type] AS int)
) AS [Project1]
) AS [Project1]
WHERE [Project1].[row_number] > 0
ORDER BY [Project1].[mtime] DESC
UNION ALL
SELECT TOP (1000)
[Project3].[C1] AS [C1],
[Project3].[Id] AS [Id]
FROM ( SELECT [Project3].[Id] AS [Id], [Project3].[mtime] AS [mtime], [Project3].[C1] AS [C1], row_number() OVER (ORDER BY [Project3].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent2].[Id] AS [Id],
[Extent2].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent2]
WHERE 1 = CAST( [Extent2].[Type] AS int)
) AS [Project3]
) AS [Project3]
WHERE [Project3].[row_number] > 0
ORDER BY [Project3].[mtime] DESC) AS [UnionAll1]
) AS [Distinct1]
UNION ALL
SELECT TOP (1000)
[Project7].[C1] AS [C1],
[Project7].[Id] AS [Id]
FROM ( SELECT [Project7].[Id] AS [Id], [Project7].[mtime] AS [mtime], [Project7].[C1] AS [C1], row_number() OVER (ORDER BY [Project7].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent3].[Id] AS [Id],
[Extent3].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent3]
WHERE 2 = CAST( [Extent3].[Type] AS int)
) AS [Project7]
) AS [Project7]
WHERE [Project7].[row_number] > 0
ORDER BY [Project7].[mtime] DESC) AS [UnionAll2]
) AS [Distinct2]
UNION ALL
SELECT TOP (1000)
[Project11].[C1] AS [C1],
[Project11].[Id] AS [Id]
FROM ( SELECT [Project11].[Id] AS [Id], [Project11].[mtime] AS [mtime], [Project11].[C1] AS [C1], row_number() OVER (ORDER BY [Project11].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent4].[Id] AS [Id],
[Extent4].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent4]
WHERE 3 = CAST( [Extent4].[Type] AS int)
) AS [Project11]
) AS [Project11]
WHERE [Project11].[row_number] > 0
ORDER BY [Project11].[mtime] DESC) AS [UnionAll3]
) AS [Distinct3]
UNION ALL
SELECT TOP (1000)
[Project15].[C1] AS [C1],
[Project15].[Id] AS [Id]
FROM ( SELECT [Project15].[Id] AS [Id], [Project15].[mtime] AS [mtime], [Project15].[C1] AS [C1], row_number() OVER (ORDER BY [Project15].[mtime] DESC) AS [row_number]
FROM ( SELECT
[Extent5].[Id] AS [Id],
[Extent5].[mtime] AS [mtime],
1 AS [C1]
FROM [dbo].[ContentItem] AS [Extent5]
WHERE 4 = CAST( [Extent5].[Type] AS int)
) AS [Project15]
) AS [Project15]
WHERE [Project15].[row_number] > 0
ORDER BY [Project15].[mtime] DESC) AS [UnionAll4] ) AS [Distinct4]
LEFT OUTER JOIN [dbo].[Retailer] AS [Extent6] ON [Distinct4].[C2] = [Extent6].[Id]
LEFT OUTER JOIN [dbo].[Brand] AS [Extent7] ON [Distinct4].[C2] = [Extent7].[Id]
LEFT OUTER JOIN [dbo].[Product] AS [Extent8] ON [Distinct4].[C2] = [Extent8].[Id]
LEFT OUTER JOIN [dbo].[Certification] AS [Extent9] ON [Distinct4].[C2] = [Extent9].[Id]
LEFT OUTER JOIN [dbo].[Claim] AS [Extent10] ON [Distinct4].[C2] = [Extent10].[Id]
So my overall questions are:
1) Is there a simpler SQL query I can execute to get the same results? I know that T-SQL doesn't support offsets per table in a subquery, hence the subquery wrapping.
2) If there isn't, what am I doing wrong in my LINQ query? Is this even possible with LINQ?
I wanted to add the SQL from #radar here all nice and formatted. It at least appears to be an elegant solution to avoid the sub-subqueries, and still accomplishes the offset/fetch.
SELECT *
FROM (SELECT
[ContentItem].*,
row_number() OVER ( PARTITION BY Type ORDER BY ContentItem.mtime ) as rn
FROM [dbo].[ContentItem]
LEFT JOIN [dbo].[Retailer] ON (Retailer.Id = ContentItem.Id)
LEFT JOIN [dbo].[Brand] ON (Brand.Id = ContentItem.Id)
LEFT JOIN [dbo].[Product] ON (Product.Id = ContentItem.Id)
LEFT JOIN [dbo].[Certification] ON (Certification.Id = ContentItem.Id)
LEFT JOIN [dbo].[Claim] ON (Claim.Id = ContentItem.Id)
) as x
WHERE x.rn >= a AND x.rn <= b;
a is the lower threshold (offset) and b is the upper threshold (fetch-ish). The only catch is that b now equals fetch + a instead of just fetch. The first set of results would be WHERE x.rn >= 0 AND x.rn <= 6, the second set WHERE x.rn >= 6 AND x.rn <= 12, third WHERE x.rn >= 12 AND x.rn <= 18, and so on.

As you are looking simpler SQL, you can use row_number analytic function, which would be faster
You need to try and see as there are many left joins and also proper index need to exists in these tables.
select *
from (
select *, row_number() over ( partition by Type order by ContentItem.mtime ) as rn
from [dbo].[ContentItem]
LEFT JOIN [dbo].[Retailer] ON (Retailer.Id = ContentItem.Id)
LEFT JOIN [dbo].[Brand] ON (Brand.Id = ContentItem.Id)
LEFT JOIN [dbo].[Product] ON (Product.Id = ContentItem.Id)
LEFT JOIN [dbo].[Certification] ON (Certification.Id = ContentItem.Id)
LEFT JOIN [dbo].[Claim] ON (Claim.Id = ContentItem.Id);
)
where rn <= 6

Well, it appears that I'm an idiot. That TOP(1000) call should have tipped me off. I assumed that my take variable was set to 6 but it was, in fact, set to 1000. Turns out my giant LINQ query works as expected, but the nested UNION ALL statements threw me off.
Still, I'm going to investigate #radar's answer further. It's hard to argue with better performance.