I'm working on creating a dynamic linq expression with an optional where clause containing "exists".
Almost everything about this can be dynamic. The user can choose to filter only root level nodes or filter both the root and all descendant nodes. I've mocked up a working example of one of the paths this query takes that is giving me issues.
Example of T-SQL:
SELECT 'Child exists', ge.id AS [run_id], aseq.id, aseq.name
FROM group_execution ge
JOIN automation_sequences aseq ON (aseq.id=ge.automation_sequence_id)
WHERE EXISTS
(SELECT ge2.patriarch_id FROM group_execution ge2
JOIN automation_sequences aseq2 ON (aseq2.id=ge2.automation_sequence_id)
WHERE aseq2.name LIKE '%test%'
AND ge2.patriarch_id = ge.patriarch_id
UNION ALL
SELECT ase.patriarch_id FROM automation_sequence_executions ase
JOIN automation_sequences aseq2 ON (aseq2.id=ase.automation_sequence_id)
WHERE aseq2.name LIKE '%test%'
AND ase.patriarch_id = ge.patriarch_id)
The problem I'm having is LINQ is giving me an execution plan that looks like this:
SELECT 'Child exists', ge.id AS [run_id], aseq.id, aseq.name
FROM group_execution ge
JOIN automation_sequences aseq ON (aseq.id=ge.automation_sequence_id)
WHERE EXISTS
(SELECT 1
FROM
(SELECT ase.patriarch_id FROM automation_sequence_executions ase
JOIN automation_sequences aseq2 ON (aseq2.id=ase.automation_sequence_id)
WHERE aseq2.name LIKE '%test%'
UNION ALL
SELECT ge2.patriarch_id FROM group_execution ge2
JOIN automation_sequences aseq2 ON (aseq2.id=ge2.automation_sequence_id)
WHERE aseq2.name LIKE '%test%'
) AS sub
WHERE sub.patriarch_id = ge.patriarch_id)
Notice how it's putting the "where" clause outside the union. This plan is killing the performance of my query. The first query runs in less than a second but the second takes up to 30 seconds.
I'm having a ton of trouble figuring out how to write the LINQ expression to mimic the first query I have above.
Below is my code so far:
//creates first query from group_execution table and does joins
IQueryable<Contract_SeqExecution> result = this.queryGroups(context);
...
//query for children:
IQueryable<ATSM_DAL.App_Data.automation_sequence_executions> tcChildrenRuns = null;
IQueryable<ATSM_DAL.App_Data.group_execution> groupChildrenRuns = null;
if (queryTestCase)
tcChildrenRuns = (from ase in context.automation_sequence_executions
where ase.parent_group_exec_id != null
select ase);
if (queryGroup)
groupChildrenRuns = (from ge in context.group_execution
where ge.parent_group_exec_id != null
select ge);
//Perform Union All on child results
decendantResults = this.UnionAllTCGroups(tcChildrenRuns, groupChildrenRuns);
//handles dynamic filter, example: aseq2.name LIKE '%test%'
decendantResults = decendantResults.Where(filterExpression);
...
//Perform the EXISTS statement in where clause. This is the problem code
Expression<Func<Contract_SeqExecution, bool>> childFilter = r => decendantResults.AsExpandable().Any(c => c.patriarchId == r.patriarchId);
childFilter = childFilter.Or(filterExpression);
result = result.AsExpandable().Where(childFilter);
This is my Query in SQL :
Select distinct * from tr.Table1
Left Outer join tr.Table2 on tr.Table1.ID = tr.Table2.ID
Left Outer join tr.Table3 on tr.Table2.AId= tr.Table3.ID
where tr.Table1.Deleted =1 and tr.Table1.Ready=1 and tr.Table1.Show=0
The query is working in SQL and gives the expected results.The thing here is that I want the equivalent of this using LINQ. I have tried some variations in LINQ queries such as :
var query = from p in _ctx.Table1
join s in _ctx.Table2 on p.Id equals s.Id into bag1
from to in bag1.DefaultIfEmpty()
join tx in _ctx.Table3 on to.AId equals tx.Id into bag2
from ts in bag2.DefaultIfEmpty()
select new
{
ContactNo = to.Table1.ContactNo
};
But it always doesn't return all the field values. Some are returned as NULL. Also tried referring to some other link as well but they all focus on joining with the parent table whereas I have to join one of the joined tables with the other one. So here I am, struggling with this.
This is the output that I'm getting as of now. Some values are null. The field has values but due to some joining issue, they are returned as NULL.
Guidance here is appreciated. Thank you.
Your query looks fine to me, the reason why you must be getting the Nulls is because when we use DefaultIfEmpty, it returns null for non-matching rows, thus you need to handle that while fetching the actual results. Try doing something like this:-
var query = from p in _ctx.Table1
join s in _ctx.Table2 on p.Id equals s.Id into bag1
from to in bag1.DefaultIfEmpty()
join tx in _ctx.Table3 on to.AId equals tx.Id into bag2
from ts in bag2.DefaultIfEmpty()
select new
{
ContactNo = to == null ? String.Empty : to.Table1.ContactNo
};
Assuming, ContactNo to be of type String, I have used String.Empty you can use any default value.
I have a SQL query that I am trying to convert to LINQ:
SELECT * FROM TABLE1
WHERE LICENSE_RTK NOT IN(
SELECT KEY_VALUE FROM TABLE2
WHERE REFERENCE_RTK = 'FOO')
So I wrote one query for inner query and then one query for the outer one and used Except:
var insideQuery = (from pkcr in this.Repository.Context.TABLE2 where pkcr.Reference_RTK == "FOO" select pkcr.Key_Value);
var outerQuery = (from pl in this.Repository.Context.TABLE1 select pl).Except(insideQuery);
But this is wrong. Cannot even compile it. What is the correct way of writing this?
You cannot compile second query, because Except should be used on Queryables of same type. But you are trying to apply it on Queryable<TABLE1> and Queryable<TypeOfTABLE2Key_Value>. Also I think you should use Contains here:
var keys = from pkcr in this.Repository.Context.TABLE2
where pkcr.Reference_RTK == "FOO"
select pkcr.Key_Value;
var query = from pl in this.Repository.Context.TABLE1
where !keys.Contains(pl.License_RTK)
select pl;
NOTE: Generated query will be NOT EXISTS instead of NOT IN, but that's what you want
SELECT * FROM FROM [dbo].[TABLE1] AS [Extent1]
WHERE NOT EXISTS
(SELECT 1 AS [C1]
FROM [dbo].[TABLE2] AS [Extent2]
WHERE ([Extent2].[Reference_RTK] == #p0) AND
([Extent2].[Key_Value] = [Extent1].[License_RTK]))
Hi I am new to NHibernate and having trouble doing a join on an aggregated set of self. I think showing the SQL I am trying to achieve might help:
The Data Looks Like this:
Message TABLE
ID DIRECTION BATCHID SEQUENCE ISLASTINSEQUENCE
1 Outbound 1 1 0
2 Outbound 1 2 0
3 Outbound 1 3 1
The query looks like this:
--Get all msgs with batchId where full sequence is ready
SELECT *
FROM [Message] M
JOIN (
--Group Msg on batchId and Count Sequence + cross ref using having
SELECT M.BatchId
FROM [Message] M
JOIN (
--Get BatchId of last msg in sequence
SELECT BatchId, Sequence as LastInSequence
FROM [Message]
WHERE Direction = 'Outbound'
AND IsEndOfSequence = 1
) M1 ON M.BatchId = M1.BatchId
WHERE Direction = 'Outbound'
GROUP BY M.BatchId, LastInSequence
HAVING count(1) = M1.LastInSequence
) B ON M.BatchId = B.BatchId
Basically I want to include batches where I have the full sequence
Here is may HNibernate Linq attempt:
var lastInSeqenceMsgs =
from b in Query<Message>(x => x.Direction == MessageDirection.Outbound
&& x.IsEndOfSequence)
select new {b.BatchId, LastInSequence = b.Sequence};
var fullSequenceBatchIds =
from outboundMessage in Query<Message>(x => x.Direction ==
MessageDirection.Outbound)
join lastInSequence in (lastInSeqenceMsgs)
on outboundMessage.BatchId equals lastInSequence.BatchId
group lastInSequence by new {lastInSequence.BatchId, lastInSequence.LastInSequence}
into g
where g.Count() == g.Key.LastInSequence
select g.Key.BatchId;
var allMsgsFromWithCompleteSequences =
from fullSequenceMessage in Query<Message>(x => x.Direction ==
MessageDirection.Outbound)
join test in (fullSequenceBatchIds) on
fullSequenceMessage.BatchId equals test.BatchId
select test;
To which it bombs on the second query (I evaluated the queries - not shown) with the following exception:
Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'Remotion.Linq.Clauses.Expressions.QuerySourceReferenceExpression'.
Of which I refined back to the join on self
var fullSequenceBatchIds =
from outboundMessage in Query<Message>(x => x.Direction ==
MessageDirection.Outbound)
join lastInSequence in (lastInSeqenceMsgs)
on outboundMessage.BatchId equals lastInSequence.BatchId
select outboundMessage;
To which I get the exception
"Specified method is not supported."
I am getting brick imprints on my forehead, so some help here would be greatly appreciated.
In general I can just say that the nhibernate Linq provider does NOT support everything what Linq offers.
In addition, nhibernate does NOT support sub queries. It simply doesn't work and you have to work around it somehow.
I have the following in linq-to-entities
clientprojects = (from p in this.SAPMappingEntities.SAP_Master_Projects
join c in this.SAPMappingEntities.SAP_Master_ProjectPartners on c.project_no equals p.project_no
where c.partner_name.Contains(clientstring)
orderby p.start descending
select new ClientProjects { client = c.partner_name, location = c.city +", "+c.region, project_no = c.project_no, start_dt = p.start, end_dt = p.finish }).Take(50).ToList();
I would like change this query so that for each SAP_Master_Project only get the SAP_Master_ProjectPartners record which has the latest update_dt. How can I do this?
EDIT
There's a project table with a project number and project details including project start and end dates. There's a project partners table with the project partner number, name, project number, update date and other details.
SAP_MASTER_PROJECT
project_no
start
finish
SAP_MASTER_PROJECTPARTNERS
partner_no
project_no
partner_name
city
region
update_dt
When the user enters "ABC" into a text box, the info I want to return is the project number, project start date, project end date plus project partner name, city, and state from the last project partner record for the last 50 projects (based on start date) where the project partner name contains or is like "ABC".
I'm sure there's more than one way to do this, but his SQL gives me the results that I need:
SELECT TOP 50 p.project_no, p.start, p.finish, c.partner_name, c.city, c.region
FROM
(select pp.project_no, pp.partner_name, pp.city, pp.region
from SAP_Master_ProjectPartners pp
where pp.partner_name LIKE #clientstring AND pp.update_dt = (select max(pp1.update_dt)
from SAP_Master_ProjectPartners pp1
where pp1.project_no = pp.project_no)) c
join SAP_Master_Projects p
on (p.project_no = c.project_no)
ORDER BY p.start DESC
EDIT #2
That sql actually returns a few items which have the same update_dt, so I modified the sql to below. Still struggling to convert to linq.
SELECT TOP 50 p.project_no, p.start, p.finish, c.partner_name, c.city, c.region, c.update_dt, c.row_id
FROM SAP_Master_Projects p
join
(select pp.project_no, pp.partner_name, pp.city, pp.region, pp.update_dt, pp.row_id
from SAP_Master_ProjectPartners pp
where pp.partner_name LIKE #clientstring AND pp.row_id = (select TOP 1 row_id
from SAP_Master_ProjectPartners pp1
where pp1.project_no = pp.project_no order by update_dt DESC)) c
on (p.project_no = c.project_no) where p.active_flag = 1
ORDER BY p.start DESC
This query would probably be simpler if you defined an entity relationship between SAP_Master_Projects and SAP_Master_ProjectPartners so the join could be implicit instead of explicit.
Edit Since you can't do that, something like this might work (using let and doing a logical join within a where clause):
var clientProjects =
(
from p in entities.SAP_Master_Projects
let c = entities.SAP_Master_ProjectPartners
.Where(cl => cl.partner_name.Contains(clientstring)
&& cl.project_no == p.project_no
)
.OrderBy(cl => cl.update_dt) // Todo: Might need to be descending?
.FirstOrDefault()
where c != null
orderby p.start descending
select new ClientProjects
{
client = c.partner_name,
location = c.city + ", " + c.region,
project_no = c.project_no,
start_dt = p.start,
end_dt = p.finish
}
)
.Take(50)
.ToList()
;
It sounds like you're trying to come up with the following query:
SELECT *
FROM MasterProjects p
INNER JOIN (SELECT project_no,
partner_name
FROM ProjectPartners o
WHERE o.update_dt = (SELECT MAX(update_dt)
FROM ProjectPartners i
WHERE i.project_no = o.project_no)) c
ON p.project_no = c.project_no
AND p.partner_name = c.partner_name
I'm not entirely sure how to translate this in to LINQ but here is my best attempt:
var clientprojects =
from p in MasterProjects
join c in ProjectPartners on p.project_no == c.project_no
where c.partner_name == (from o in ProjectPartners
where o.project_no == c.project_no
and o.update_dt == (from i in ProjectParters
where o.project_no = i.project_no
select i.update_dt).Max()
select o.partner_name).First();
The above LINQ may not even compile, but hopefully it'll send you in the right direction.
I don't speak your language, sorry. But, for instance, in MySql you might add sort by update_dt DESC LIMIT 1 can you do that or somethign similar?