I've been tasked with converting some crusty old SQL to Linq2SQL and I know this shouldnt be the first choice however it needs to be done.
Problem
I'm stuck attempting to force specific joins to replicate the desired output and the following is a simplified version of the SQL with one parameter variant shown followed by a simplified version of my attempt in Linq2SQL. Several alternative fields may be queried in Table1 hence the use q and q2 in the converted code though the actual parameters are not relevant to the issue.
Aim
The query needs to retrieve relevant rows from Table1, the latest related row from Table2 and additional data from Table2's parent table Table3, preferably in a single pass.
There may be 0-n matches in Table1, Table1 has a 1:n relationship with Table2 with 0-n rows, there is a 1:n relationship between Table3 and Table2.
Regardless of how I structure the expressions, the linq generates an INNER JOIN onto Table2 excluding rows in Table1, how can I structure the linq query to achieve the desired result?
SELECT [...]
FROM Table1 t1
LEFT JOIN (
SELECT MAX(id) AS id, parent_id
FROM Table2
GROUP BY parent_id
) x2 ON t1.id = x2.parent_id
LEFT JOIN Table2 t2 ON x2.id = t2.id
LEFT JOIN Table3 t3 ON t2.table3_id = t3.id
WHERE t1.id = row_id
var q = dc.Table1.AsQueryable();
q = from r in q where r.id == row_id select r;
var q2 = from r in (q)
join x2 in (from r in dc.Table2.DefaultIfEmpty() group r by r.parent_id into maxt2 let max_id = maxt2.Max(f => f.id) select new { maxt2.Key, max_id }) on r.id equals wx.Key
join t2 in dc.Table2.DefaultIfEmpty() on x2.max_id equals t2.id
join t3 in dc.Table3.DefaultIfEmpty() on t2.table3_id equals t3.id
select
{
[...]
};
Here is my translation using my recipe rules:
var Q1 = from t2 in dc.Table2
group t2 by t2.parent_id into t2g
select new { parent_id = t2g.Key, id = t2g.Max(t2 => t2.id) };
var Q2 = from t1 in Table1
where t1.id == row_id
join q1 in Q1 on t1.id equals q1.parent_id into q1j
from q1 in q1j.DefaultIfEmpty()
join t2 in dc.Table2 on q1.id equals t2.id into t2j
from t2 in t2j.DefaultIfEmpty()
join t3 in dc.Table3 on t2.table3_id equals t3.id into t3j
from t3 in t3j.DefaultIfEmpty()
select new { t1, t2, t3 };
I need to join 5 tables in a query. 3 of these tables must have relations, but two of them are optionally connected to an entry.
Because of this, I am trying to do a LEFT JOIN to Table4 and Table5 like this:
var cDesc = (cDesc == null ? "" : cDesc);
var cStreet = (cStreet == null ? "" : cStreet);
var q = await (from t1 in MyContext.Table1
join t2 in MyContext.Table2
on t1.ID equals t2.ObjectID
join t3 in MyContext.Table3
on t2.TeamID equals t3.TeamID
join t4 in MyContext.Table4
on t1.ID equals t4.ObjectID
into join3
from j3 in join3.DefaultIfEmpty()
join t5 in MyContext.Table5
on j3.StorageID equals t5.StorageID
where t2.ObjectType.Equals(16)
&& t3.UserID.Equals(userID)
&& t1.Description.Contains(cDesc)
&& l.Address.Contains(cStreet)
orderby t1.ID descending
select new Table1ListModel
{
ID = t1.ID,
Description = t1.Description,
Address = t5.Address
}
)
.Take(takeThis)
.ToListAsync();
But this query only works for rows that has a connection to Table4, so I'm doing something wrong obviously.
Am I doing the join correctly? Or is the problem that I want to run a where on address that comes from the fifth table?
Basically once you left join one table into a query any additional tables that you want to join to that one should almost always also be done with left joins. In your case you're saying you want keep rows in Table1 that don't have a match in Table4, but then you say you only want matches between Table4 and Table5 which basically will remove all the Table1 results that didn't have a match in Table4. Basically you want something like this
from j3 in join3.DefaultIfEmpty()
join temp5 in MyContext.Table5
on j3.StorageID equals temp5.StorageID into join4
from t5 in join4.DefaultIfEmpty()
This looks like a source of your problem:
join t4 in MyContext.Table4
on t1.ID equals t4.ObjectID
into join3
This means that you are inner joining Table4 to Table1
How can I make a Linq To SQL join on multiple tables where 1 table should produce a Cartesian product.
To shed some more light, here is a sample of the SQL query.
SELECT Table1.MyField, Setup.SomeField
FROM Table1 INNER JOIN Table2 ON Table1.SomeField = Table2.SomeField, Setup
My Linq to SQL are like:
var q = from t1 in db.Table1
join t2 in db.Table2 on t1.SomeField equals t2.SomeField
join setup in db.Setup
select new {t1.MyField, setup.SomeField};
I'm getting an error on the last join that Type inference failed in the call to 'Join'.
Use SelectMany rather than a Join to perform a Cartesian Product.
In query syntax that would be:
var query = from t1 in db.Table1
from t2 in db.Table2
select new {t1, t2};
This will also do:
var q = from t1 in db.Table1
join t2 in db.Table2 on t1.SomeField equals t2.SomeField
from setup in db.Setup
select new {t1.MyField, setup.SomeField};
I have 3 objects that I need to link together
Parent: TblClients
This will have multiple children of Type TblBusinessLeads , the key between the two is ClientID
Type Lead will have multiple children of type TblFeeBreakouts , the key between the two is LeadID
I have written the following LINQ to get the databack, but it is not coming back (out of memory exception)
from t0 in TblClients
join t1 in TblBusinessLeads on t0.ClientID equals t1.ClientID into t1_join
from t1 in t1_join.DefaultIfEmpty()
join t3 in TblFeeBreakouts on t1.LeadID equals t3.LeadID into t3_join
from t3 in t3_join.DefaultIfEmpty()
orderby
t0.ClientID,
t1.LeadID
select new {
client_data = t0,
business_lead_data = t1_join,
fee_breakout_data = t3_join
}
I am not sure of you can even do this, but the idea seems pretty common. Any help would be greatly appreciated. Thanks
EDIT:
Wow lot of comments. Here goes my answers
I am trying to run the query in LinqPad, thats where the Out of Memory is Occuring
If I look at the SQL generated, it gives me
SELECT [t0].[ClientID], [t0].[ClientName], [t0].[ClientDesc], [t0].[EditedBy], [t0].[EditedDate], [t0].[CreatedBy], [t0].[CreatedDate], [t3].[LeadID], [t3].[InitiativeName], [t3].[Description], [t3].[NewBusNeeds], [t3].[CreativeNeeds], [t3].[IdeationNeeds], [t3].[Comments], [t3].[LossReasons], [t3].[OriginDate], [t3].[DateReceivedAssignment], [t3].[DueDate], [t3].[TimelineNotes], [t3].[PendingCode], [t3].[EstStartDate], [t3].[EstEndDate], [t3].[ExeStartDate], [t3].[ExeEndDate], [t3].[Probable80Total], [t3].[Possible50Total], [t3].[Emerging25Total], [t3].[NoBudget0Total], [t3].[TotalBudget], [t3].[FinancialNotes], [t3].[DollarsRecordFor], [t3].[BizDevContactUserID], [t3].[BizDevContact2UserID], [t3].[SVPContactUserID], [t3].[ClientMgmtContactUserID], [t3].[CMAdditionalContactUserID], [t3].[AdditionalContactUserID], [t3].[CreatorUserID], [t3].[OfficeID], [t3].[ClientID] AS [ClientID2], [t3].[LeadTypeID], [t3].[ActionNeeded], [t3].[ActionDate], [t3].[NewBusDeliveryDate], [t3].[NewBusDesc], [t3].[CreativeDeliveryDate], [t3].[CreativeDesc], [t3].[IdeationDeliveryDate], [t3].[IdeationDesc], [t3].[AltMediaDeliveryDate], [t3].[AltMediaDesc], [t3].[MobileOpsDeliveryDate], [t3].[MobileOpsDesc], [t3].[EventsDeliveryDate], [t3].[EventsDesc], [t3].[Routing], [t3].[RoutingDate], [t3].[Deleted], [t3].[LeadSourceID], [t3].[NatureofLeadID], [t3].[NatureofLeadNotes], [t3].[EditedBy] AS [EditedBy2], [t3].[EditedDate] AS [EditedDate2], [t3].[CreatedBy] AS [CreatedBy2], [t3].[CreatedDate] AS [CreatedDate2], [t3].[ClientContactName], [t3].[ClientContactTitle], [t3].[ReportingYear], (
SELECT COUNT(*)
FROM [tblBusinessLead] AS [t4]
WHERE [t0].[ClientID] = [t4].[ClientID]
) AS [value], [t1].[LeadID] AS [LeadID2]
FROM [tblClient] AS [t0]
LEFT OUTER JOIN [tblBusinessLead] AS [t1] ON [t0].[ClientID] = [t1].[ClientID]
LEFT OUTER JOIN [tblFeeBreakout] AS [t2] ON [t1].[LeadID] = [t2].[LeadID]
LEFT OUTER JOIN [tblBusinessLead] AS [t3] ON [t0].[ClientID] = [t3].[ClientID]
ORDER BY [t0].[ClientID], [t1].[LeadID], [t2].[LeadID], [t2].[FeeTypeID], [t3]. [LeadID]
This returns like 1.2 million rows
There is no mapping in the model becuase the DB has no relationships (they are inferred, no foreign keys or anything like that)
The reason I am using t1_join and t3_join is because if I use t1 or t3, I get the single entity, not the IEnumerable of the object, hence I cant loop over it.
If you have more questions, please ask.
First of all, what possible use could a client have for 1.2 million rows? There is no real good use case for this, so your first step should be figuring out how to filter your results appropriately.
Second, I believe the reason your query is returning OutOfMemoryException is because LinQPad is doing a ToList() or something similar so that I can show the results of the query in the bottom pane. The ToList() is loading all 1.2 million rows into memory. If you ran the query in a regular .Net app, the following will return an IQueryable<> which will not load into memory.
var query =
from t0 in TblClients
join t1 in TblBusinessLeads on t0.ClientID equals t1.ClientID into t1_join
from t1 in t1_join.DefaultIfEmpty()
join t3 in TblFeeBreakouts on t1.LeadID equals t3.LeadID into t3_join
from t3 in t3_join.DefaultIfEmpty()
orderby
t0.ClientID,
t1.LeadID
select new {
client_data = t0,
business_lead_data = t1_join,
fee_breakout_data = t3_join
};
As stated above, it is probably a good idea to setup asociations on these tables, which I did... The LINQ for the result after the association was simple
var clientList = (from a in tblClients
select a).ToList();
From there it was just accessing the properties.
I'm trying to do the following query in Linq
SELECT *
FROM Table1 T1
INNER JOIN Table2 T2
ON T1.ID = T2.AnotherID
LEFT OUTER JOIN Table3 T3
on T1.ID = T3.AnotherID
It works as expected in proper SQL syntax, but i'm having a hard time translating it to the corresponding Linq to SQL syntax.
How do i combine left join with an inner join?
Regards,
var results = from t1 in Table1
from t2 in Table2
where t1.ID = t2.AnotherID
join t3 in Table3 on t1.ID equals t3.AnotherID into joined
from j in joined.DefaultIfEmpty()
select new {t1, t2, t3 = j}