Develop Reference

Error converting SQL with join to LINQ

I have a SQL query that I'm trying to convert to LINQ and am having trouble understanding the obscure error messages when the query is enumerated.
The SQL query (which works as intended), is:
select a.TestGuid, MIN(a.StartTime) as StartTime, COUNT(b.TestCaseId) as NumTests, COUNT(DINSTINCT a.Id) as NumScenarios
from LoadTestSummary as a
join LoadTestTestSummaryData as b
on a.LoadTestRunid = b.LoadTestRunId
where
a.TargetStack = env and
a.TestGuid IS NOT NULL AND
a.StartTime IS NOT NULL AND
a.LoadTestRunId IS NOT NULL
group by a.TestGuid
Converting to LINQ, I get the following:
var q = from a in _context.LoadTestSummary
where
a.TargetStack == env &&
a.TestGuid != null &&
a.StartTime != null &&
a.LoadTestRunId != null
join b in _context.LoadTestTestSummaryData on new
{
LoadTestRunId = Convert.ToInt32(a.LoadTestRunId)
} equals new
{
LoadTestRunId = b.LoadTestRunId
}
group new { a, b } by new
{
a.TestGuid
}
into g
select new
{
DateCreated = g.Min(p => p.a.StartTime),
NumScenarios = g.Count(),
TestGuid = g.Key.TestGuid
NumTests = // ???
};
Two problems I have:
1) When the query is enumerated I get a run-time error that I'm having trouble deciphering. The query works fine in Linqpad, but gives me a run-time error in my program. I am not sure what would cause this. Just staring at this makes my head hurt:
ArgumentException: Expression of type 'System.Func``2[Microsoft.Data.Entity.Query.EntityQueryModelVisitor+TransparentIdentifier``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType7``1[System.String]]' cannot be used for parameter of type 'System.Func``2[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType7``1[System.String]]' of method 'System.Collections.Generic.IEnumerable``1[System.Linq.IGrouping``2[<>f__AnonymousType7``1[System.String],<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData]]] _GroupBy[<>f__AnonymousType5``2,<>f__AnonymousType7``1,<>f__AnonymousType5``2](System.Collections.Generic.IEnumerable``1[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData]], System.Func``2[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType7``1[System.String]], System.Func``2[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData]])'
2) I am not quite sure how to get the COUNT(DISTINCT a.Id) into the NumTests field. It looks like this isn't supported in LINQ but it looks like other people have asked this question to so I may be able to figure it out once #1 is resolved.
Any thoughts on what's wrong here? I am not even sure exactly what the error is telling me.
All help is appreciated!

Looking just at the SQL query and your LINQ code, I came up with something like this:
from a in LoadTestSummary
join b in LoadTestTestSummaryData
on a.LoadTestRunId equals b.LoadTestRunId
where
a.TargetStack == env &&
a.TestGuid != null &&
a.StartTime != null &&
a.LoadTestRunId != null
group new { a, b } by a.TestGuid into g
select new
{
TestGuid = g.Key,
DateCreated = g.Min(el => el.a.StartTime),
NumTests = g.Select(el => el.b.TestCaseId).Count(),
NumScenarios = g.Select(el => el.a.Id).Distinct().Count()
};
Note, that you don't need to convert LoadTestRunId to int, you may just use standard string comparision.
That horrendous error is most likely caused by grouping and comparing using anonimous objects, thou I prefer not to read that error too much as it's an eldritch abomination not ment to be seen nor comprehend by mere mortals, it seems.

Left outer join IEnumerable<T> to DataTable - parameter cannot be null error

I have this:
var initialRelease = from c in campaignAvailability
where c.reportStatus == "Initial Release"
select c;
var results = from server in initialRelease
join local in table.AsEnumerable()
on server.campaignId equals local.Field<long>("campaignId") into ls
from local in ls.DefaultIfEmpty()
where DateTime.Compare(server.reportDate, local.Field<DateTime>("reportDate")) > 0 || local == null
select server;
//add it to list of campaigns to process
results.Select(m => new { m.campaignId, m.reportDate }).Distinct()
.Select(n => new CampaignReportDate() {
campaignId = n.campaignId,
reportDate = n.reportDate
}).ToList().ForEach(c => campaignsToProcess.Add(c));
Which I want to look like this in SQL:
SELECT a
FROM ienumerable AS a
LEFT OUTER JOIN table AS b
ON
a.id = b.id
WHERE b.some_date > a.some_date
OR b IS NULL
I understand that the where linq clause can't compare a null date. But, as I understand, the || local == null should have taken care of that. What am I missing?

Figured it out. local was coming in as null and I still needed a way to compare it to server.reportDate. I used a ternary operator. And because I was assigning DateTime.MinValue as the default date, I was able to remove || local == null.
where DateTime.Compare(server.reportDate, (local != null) ? (DateTime)local["reportDate"] : DateTime.MinValue) > 0

When doing left joins in linq to objects, you should prefer providing a default value when empty. Then your query doesn't have to make much changes.
var results =
from server in initialRelease
join local in table.AsEnumerable()
on server.campaignId equals local.Field<long>("campaignId") into ls
from local in ls.DefaultIfEmpty(table.NewRow())
where DateTime.Compare(server.reportDate,
local.Field<DateTime?>("reportDate") ?? Datetime.MaxValue) > 0
select server;

Object reference not set to an instance of an object on querying from different lists [duplicate]

This question already has answers here:
What is a NullReferenceException, and how do I fix it?
(27 answers)
Closed 8 years ago.
I have the linq below. Since I am querying from 2 datacontexts, I've brokendown the tables into var list. But then, I have the error "Object reference not set to an instance of an object". This happens because edr is null.
var meetingsQuery = (from s in this.ModelContext.Meetings select s).ToList();
var deliverablesQuery = (from s in this.ModelContext.Deliverables select s).ToList();
var deliverableDatesQuery = (from s in this.ModelContext.DeliverableDates select s).ToList();
var refDateTypesQuery = (from s in this.ModelContext.RefDateTypes select s).ToList();
var refDeliverablesQuery = (from s in this.ModelContext.RefDeliverables select s).ToList();
var updatesQuery = (from s in this.ArenaUpdateBASEModelContext.Updates select s).ToList();
var updateQCsQuery = (from s in this.ArenaUpdateBASEModelContext.UpdateQCs select s).ToList();
var submissionUpdates = (from e in meetingsQuery
from edr in deliverablesQuery.Where(dr => dr.MeetingId == e.MeetingId && !dr.DeletedFlag).DefaultIfEmpty()
from ed in deliverableDatesQuery.Where(d => d.DeliverableId == edr.DeliverableId && !d.DeletedFlag && d.RefDateTypeId == 1).DefaultIfEmpty()
from ed2 in deliverableDatesQuery.Where(d2 => d2.DeliverableId == edr.DeliverableId && !d2.DeletedFlag && d2.RefDateTypeId == 2).DefaultIfEmpty()
join ret in refDateTypesQuery on ed.RefDateTypeId equals ret.RefDateTypeId
join rdt in refDeliverablesQuery on edr.RefDeliverableId equals rdt.RefDeliverableId
join upd in updatesQuery on edr.RefDsgnSubmissionTypeId equals upd.UpdateId
join uqc in updateQCsQuery on upd.UpdateId equals uqc.UpdateId
where
!e.DeletedFlag && !ret.DeletedFlag && !rdt.DeletedFlag && !upd.DeletedFlag && !uqc.DeletedFlag && e.ProjectId == arenaPiD// && rdt. .ObjectIdLink == "Update_UpdateId"
&& uqc.RefQCId == 6 // Distributed
&& uqc.RefQCStatusId == 2 // Complete
orderby e.ScheduledDT descending
select new
{
e.MeetingId,
e.ScheduledDT,
edr.DeliverableId,
edr.RefDeliverableId,
rdt.DeliverableAbbrv,
UpdateId = edr.RefDsgnSubmissionTypeId != null ? edr.RefDsgnSubmissionTypeId : 0,
RefRecommendationId = upd.RefRecommendationId != null ? upd.RefRecommendationId : 0,
uqc.RefQCId,
uqc.RefQCStatusId,
DeadlineDate = ed != null ? ed.DeliverableDateValue.ToString() : "",
ActualDate = ed2 != null ? ed2.DeliverableDateValue.ToString() : ""
}).ToList();

There's a big difference to how DefaultIfEmpty works for Linq To Objects vs Linq To SQL, which is tied into the difference between how nulls are handled between the two.
In SQL an empty record from a LEFT OUTER JOIN is populated with null values. Since your SQL never refer to the record itself this is not a problem. When you write edr.DeliverableId for an unmatched edr, the result is null.
Linq to SQL is different. When you try to reference any field or property of an unmatched edr the resultant error is exactly what you have seen. Every reference to edr after the DefaultIfEmpty call needs to be checked first to see if edr is valid.
Beyond the immediate error however...
You've mixed a couple of join forms - inner and outer - in that query, and the result is convoluted and unexpected. The problems you're taking on with DefaulIfEmpty for an outer join are negated with a subsequent inner join that depends on the outers. Which means that you are going through all the pain and suffering without any of the pay-off.
This block of joins:
join ret in refDateTypesQuery on ed.RefDateTypeId equals ret.RefDateTypeId
join rdt in refDeliverablesQuery on edr.RefDeliverableId equals rdt.RefDeliverableId
join upd in updatesQuery on edr.RefDsgnSubmissionTypeId equals upd.UpdateId
join uqc in updateQCsQuery on upd.UpdateId equals uqc.UpdateId
Every one of those depends ultimately on the outer join for edr, resulting in an output that - even if you put the time in to get the outer join side effects figured out - will negate the effects of DefaultIfEmpty.
You need to re-think your logic.
I suggest breaking this thing down into a series of intermediate queries. Build the query up one step at a time, joining the results in the final stage.
For instance, you have ed as an outer join which is then subjected to an inner join with rdt and further filtered by the properties of rdt in your where clause. Scrape all that out and put it in an intermediate, then join against it later. Do the same with edt: create an intermediate that join the parts. Flatten out the results for use in the final query.
Incidentally, you don't necessarily need to bring all of that data into memory. Even if your data is on different physical servers you can often still get Linq to SQL to talk to them. When you're targeting SQL Server for example you can specify a 3- or 4-part name for the Table attribute to access data in other databases on the same server or databases on linked servers. Might be useful.

LINQ-To-SQL - slow query

I'm just wondering if anyone can offer any advice on how to improve my query.
Basically, it'll be merging 2 rows into 1. The only thing the rows will differ by is a 'Type' char column ('S' or 'C') and the Value. What I want to do is select one row, with the 'S' value and the 'C' value, and calculate the difference (S-C).
My query works, but it's pretty slow - it takes around 8 seconds to get the results, which is not ideal for my application. I wish I could change the database structure but I can't sadly!
Here is my query:
var sales = (from cm in dc.ConsignmentMarginBreakdowns
join sl in dc.SageAccounts on new { LegacyID = cm.Customer, Customer = true } equals new { LegacyID = sl.LegacyID, Customer = sl.Customer }
join ss in dc.SageAccounts on sl.ParentAccount equals ss.ID
join vt in dc.VehicleTypes on cm.ConsignmentTripBreakdown.VehicleType.Trim() equals vt.ID.ToString() into vtg
where cm.ConsignmentTripBreakdown.DeliveryDate >= dates.FromDate && cm.ConsignmentTripBreakdown.DeliveryDate <= dates.ToDate
where (customer == null || ss.SageID == customer)
where cm.BreakdownType == 'S'
orderby cm.Depot, cm.TripNumber
select new
{
NTConsignment = cm.NTConsignment,
Trip = cm.ConsignmentTripBreakdown,
LegacyID = cm.LegacyID,
Costs = dc.ConsignmentMarginBreakdowns.Where(a => a.BreakdownType == 'C' && a.NTConsignment == cm.NTConsignment && a.LegacyID == cm.LegacyID && a.TripDate == cm.TripDate && a.Depot == cm.Depot && a.TripNumber == cm.TripNumber).Single().Value,
Sales = cm.Value ?? 0.00m,
Customer = cm.Customer,
SageID = ss.SageID,
CustomerName = ss.ShortName,
FullCustomerName = ss.Name,
Vehicle = cm.ConsignmentTripBreakdown.Vehicle ?? "None",
VehicleType = vtg.FirstOrDefault().VehicleTypeDescription ?? "Subcontractor"
});

A good place to start when optimizing Linq to SQL queries is the SQL Server Profiler. There you can find what SQL code is being generated by Linq to SQL. From there, you can toy around with the linq query to see if you can get it to write a better query. If that doesn't work, you can always write a stored procedure by hand, and then call it from Linq to SQL.

There really isn't enough information supplied to make an informed opinion. For example, how many rows in each of the tables? What does the generated T-SQL look like?
One thing I would suggest first is to take the outputted T-SQL, generate a query plan and look for table or index scans.

C# LINQ: How to stack LINQ queries correctly

I have a form that allows the user to perform a myriad of searches. The table(s) that need to be joined differ depending on the search criteria entered. (My example below is very simplistic because both tables use the same sub-tables to join on, but the actual problem is not as simple.)
I've been using a technique I call LINQ stacking, like this:
IQueryable<LogENT> results = Context.AssignedLogsENT.Where(l => l.AgencyId);
if(txtFirstName.Text != null)
results = from r in results
join a in Context.LogAssignmentsENT on r.DisplayLogId equals a.LogId
join p in Context.PersonsENT on a.ObjectId equals p.DisplayPersonId
&& !a.Deleted &&
p.FirstName.StartsWith(Object.FirstName)
select r;
if(txtLastName.Text != null)
results = from r in results
join a in Context.LogAssignmentsENT on r.DisplayLogId equals a.LogId
join p in Context.PersonsENT on a.ObjectId equals p.DisplayPersonId
&& !a.Deleted &&
p.LastName.StartsWith(Object.LastName)
select r;
So you see if a certain text field is set, I add to the query as necessary. This actually works fine, except that when I use SQL Profiler to view the generated query, it is INNER JOINing the tables each time I add a new criterion.
i.e. the LogAssignments table is included 3, 4, 5 times. Is there a way I can prevent it from JOINing the same table more than once?
Or, is there a better way I can do this? I've looked at Predicate Builder however it doesn't seem to permit joining tables, which is a requirement in my case.
Thanks!

IQueryable<LogENT> results = Context.AssignedLogsENT.Where(l => l.AgencyId);
results = from r in results
join a in Context.LogAssignmentsENT on r.DisplayLogId equals a.LogId
join p in Context.PersonsENT on a.ObjectId equals p.DisplayPersonId
&& !a.Deleted
select r;
if(txtFirstName.Text != null)
results = from r in results
p.FirstName.StartsWith(Object.LastName)
select r;
if(txtLastName.Text != null)
results = from r in results
p.LastName.StartsWith(Object.LastName)
select r;

If you use just one query, you could modify it something like this:
results = from r in results
join a in Context.LogAssignmentsENT on r.DisplayLogId equals a.LogId
join p in Context.PersonsENT on a.ObjectId equals p.DisplayPersonId
&& !a.Deleted &&
(txtFirstName.Text != null || p.FirstName.StartsWith(Object.FirstName)) &&
(txtLastName.Text != null || p.LastName.StartsWith(Object.LastName))
select r;

You can build your base result and then dynamically add the where clauses.