Develop Reference

LINQ to SQL join generates SQL which joins on IS NULL

I am not good at Linq expression, today I am running into one weird issue as the below of inner join statement,
var orders = (from q in dao.CurrentDBContext.New_OrderForm
join d in dao.CurrentDBContext.New_OrderGoodsDetail on q.billNum equals d.billNum
select new
{
q.billNum,
q.orderSource,
q.sourceOddNum
d.PPT
}
While I traced the linq statement, I am confused of that Entity Framework will convert the linq statement to the below sql statment
SELECT
[Extent1].[billNum] AS [billNum],
[Extent1].[orderSource] AS [orderSource],
[Extent1].[sourceOddNum] AS [sourceOddNum],
[Extent2].[PPT] AS [PPT]
FROM [dbo].[New_OrderForm] AS [Extent1]
INNER JOIN [dbo].[New_OrderGoodsDetail] AS [Extent2]
ON ([Extent1].[billNum] = [Extent2].[billNum]) OR
(([Extent1].[billNum] IS NULL) AND ([Extent2].[billNum] IS NULL))
Do you know why the below SQL segment did automatically append?
OR (([Extent1].[billNum] IS NULL) AND ([Extent2].[billNum] IS NULL)"
I don't expect that the above automatically append, since it did slow down SQL performance. Any suggestions?

Here is what you can do in case you cannot change the billNum columns to be non nullable.
First, set the option mentioned by #Giorgi
class CurrentDBContext
{
public CurrentDBContext()
{
Configuration.UseDatabaseNullSemantics = true;
// ...
}
}
Then change the LINQ query to not use join, but simple where like this
var orders = (from q in dao.CurrentDBContext.New_OrderForm
from d in dao.CurrentDBContext.New_OrderGoodsDetail
where q.billNum == d.billNum
select ...
The result will be the exact SQL query as the one you've shown (with JOIN!) without the OR part.

It seems that Linq translates q.billNum equals d.billNum is such a way that it also includes a valid match in case both q.billNum and d.billNum are NULL (in SQL NULL is never equal to NULL, hence the OR in your query).
Making both fields non-nullable would be the best solution, provided both fields can never be NULL.
If this is not the case, you could also try to add a where clause in your Linq statement to specifiy that both q.billNum and d.billNum cannot be NULL. With any luck, Linq will recognize that nullable values are not possible.
Note: If you are working with Oracle you should check for empty strings as well as NULL (empty string is equivalent to NULL). Empty strings should be fine as a valid value in SQL Server.
As the above did not help, you could try to write the query yourself. If I'm not mistaking it would be something along the following lines (assuming var is an List<Order> in your example code - the results of your query should match the class you are using):
StringBuilder query = new StringBuilder();
query.AppendLine("SELECT [Extent1].[billNum] AS [billNum],");
query.AppendLine(" [Extent1].[orderSource] AS [orderSource],");
query.AppendLine(" [Extent1].[sourceOddNum] AS [sourceOddNum],");
query.AppendLine(" [Extent2].[PPT] AS [PPT]");
query.AppendLine("FROM [dbo].[New_OrderForm] AS [Extent1]");
query.AppendLine("INNER JOIN [dbo].[New_OrderGoodsDetail] AS [Extent2] ON [Extent1].[billNum] = [Extent2].[billNum]");
List<Order> orders = DbContext.Database.SqlQuery<Order>(query.ToString()).ToList();
I have used similar workarounds to get around performance issues in the past.

If you are using EF6 try setting
context.Configuration.UseDatabaseNullSemantics = true;
and it will not generate NULL checks for those columns.
According to documentation
For example (operand1 == operand2) will be translated as: (operand1 = operand2) if UseDatabaseNullSemantics is true, respectively (((operand1 = operand2) AND (NOT (operand1 IS NULL OR operand2 IS NULL))) OR ((operand1 IS NULL) AND (operand2 IS NULL))) if UseDatabaseNullSemantics is false.

Following on from #Giorgi's answer, the UseDatabaseNullSemantics flag will not work with the equals keyword - only the == operand. Thus in order to get round this and ensure the join on billNum is not part of the OR clause this approach should work (in conjunction with the UseDatabaseNullSemantics flag):
var orders = (from q in dao.CurrentDBContext.New_OrderForm
from d in dao.CurrentDBContext.New_OrderGoodsDetail
where q.billNum == d.billNum
select new
{
q.billNum,
q.orderSource,
q.sourceOddNum
d.PPT
}
This will generate the JOIN without the OR.

getting error in joining dictionary with table

I want to have join query from a table with a dictionary based on a common field, my query is:
var query = from c in db.Exp
join d in lstUniprotDic on c.UniID equals d.Key
select new
{
c.UniID,
IdentityPercent=d.Value.ToString(),
c.PrId,
c.SpotNo
}
but i got the following error
Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains() operator.

That pretty much says it all. You can't use the dictionary in your LINQ to SQL query except when using Contains.
Solution:
(from c in db.Exp where lstUniprotDic.Keys.Contains(c.UniID) select c).AsEnumerable()
join d in lstUniprotDic on c.UniID equals d.Key
select new
{
c.UniID,
IdentityPercent=d.Value.ToString(),
c.PrId,
c.SpotNo
}
I am not sure if the usage of lstUniprotDic.Keys in the LINQ to SQL query is actually working.
If not, try using this code instead:
var ids = lstUniprotDic.Keys.ToArray();
(from c in db.Exp where ids.Contains(c.UniID) select c).AsEnumerable()
join d in lstUniprotDic on c.UniID equals d.Key
select new
{
c.UniID,
IdentityPercent=d.Value.ToString(),
c.PrId,
c.SpotNo
}

comparison operator not supported for type int[] - linq to sql

Here is the problematic line:
var originalSummaryCandidates =
(from a in masterDB.tbl_thirty_second_summaries_multi_variant_associations
join d in masterDB.tbl_thirty_second_summaries_multi_variants on a.ThirtySecSummaryId equals d.ThirtySecondSummaryId_this
where d.DrugId == drugId &&
variantGenotypeIds.Contains(new int[] {a.VariantId, a.GenotypeId})
select d.ThirtySecondSummaryId_this)
.Distinct()
.ToList();
variantGeotpeIds is of type List<int[]>. Both a.VariantId and a.GenotypeId are of type int.
I cannot figure out why it why it will not do the comparison. Is this a deferred execution issue? It doesn't seem like it should be...
Thanks in advance.

List<T>.Contains only takes a single parameter of type T. In your case, T is Int32 but you're passing in a Int32[].
If you want to check that both values are in the list, you have to break the calls apart:
where d.DrugId == drugId &&
variantGenotypeIds.Contains(a.VariantId) &&
variantGenotypeIds.Contains(a.GenotypeId)
EDIT
If variantGenotypeIds is actually a List<Int32[]>, then there's another issue. LINQ to SQL will try to convert your query into its SQL equivalent. In this case, there's no way to translate your query into SQL so LINQ to SQL will throw an Exception.
If you really need to query this way, you'll have to read the records into memory first and then query using LINQ to Objects (which may or may not be a big deal depending on how many rows you are reading):
var query =
from a in masterDB.tbl_thirty_second_summaries_multi_variant_associations
join d in masterDB.tbl_thirty_second_summaries_multi_variants
on a.ThirtySecSummaryId equals d.ThirtySecondSummaryId_this
where d.DrugId == drugId
select new { a, d }
var originalSummaryCandidates =
(from q in query.AsEnumerable()
where variantGenotypeIds.Contains(new [] { q.a.VariantId, q.a.GenotypeId})
select d.ThirtySecondSummaryId_this)
.Distinct()
.ToList();

Array comparison uses reference equality by default. It's possible that linq-to-sql just tries to translate that into SQL that compares the values, but you'd have to look at the generated SQL to be sure. Another option would be to use Any instead:
where d.DrugId == drugId &&
variantGenotypeIds.Any(v => v[0] == a.VariantId && v[1] == a.GenotypeId)
but I'm not sure if Linq-to-Sql will be able to translate that to the correct SQL either. Another option would be to project the List` to a > and then do a string comparison:
variantGenotypeStrings = variantGenotypeIds.Select(v => string.Format("{0}|{1}", v[0],v[1]);
var originalSummaryCandidates =
(from a in masterDB.tbl_thirty_second_summaries_multi_variant_associations
join d in masterDB.tbl_thirty_second_summaries_multi_variants on a.ThirtySecSummaryId equals d.ThirtySecondSummaryId_this
where d.DrugId == drugId &&
variantGenotypeStrings.Contains(string.Format("{0}|{1}", a.VariantId, a.GenotypeId))
select d.ThirtySecondSummaryId_this)
.Distinct()
.ToList();

Object reference not set to an instance of an object on querying from different lists [duplicate]

This question already has answers here:
What is a NullReferenceException, and how do I fix it?
(27 answers)
Closed 8 years ago.
I have the linq below. Since I am querying from 2 datacontexts, I've brokendown the tables into var list. But then, I have the error "Object reference not set to an instance of an object". This happens because edr is null.
var meetingsQuery = (from s in this.ModelContext.Meetings select s).ToList();
var deliverablesQuery = (from s in this.ModelContext.Deliverables select s).ToList();
var deliverableDatesQuery = (from s in this.ModelContext.DeliverableDates select s).ToList();
var refDateTypesQuery = (from s in this.ModelContext.RefDateTypes select s).ToList();
var refDeliverablesQuery = (from s in this.ModelContext.RefDeliverables select s).ToList();
var updatesQuery = (from s in this.ArenaUpdateBASEModelContext.Updates select s).ToList();
var updateQCsQuery = (from s in this.ArenaUpdateBASEModelContext.UpdateQCs select s).ToList();
var submissionUpdates = (from e in meetingsQuery
from edr in deliverablesQuery.Where(dr => dr.MeetingId == e.MeetingId && !dr.DeletedFlag).DefaultIfEmpty()
from ed in deliverableDatesQuery.Where(d => d.DeliverableId == edr.DeliverableId && !d.DeletedFlag && d.RefDateTypeId == 1).DefaultIfEmpty()
from ed2 in deliverableDatesQuery.Where(d2 => d2.DeliverableId == edr.DeliverableId && !d2.DeletedFlag && d2.RefDateTypeId == 2).DefaultIfEmpty()
join ret in refDateTypesQuery on ed.RefDateTypeId equals ret.RefDateTypeId
join rdt in refDeliverablesQuery on edr.RefDeliverableId equals rdt.RefDeliverableId
join upd in updatesQuery on edr.RefDsgnSubmissionTypeId equals upd.UpdateId
join uqc in updateQCsQuery on upd.UpdateId equals uqc.UpdateId
where
!e.DeletedFlag && !ret.DeletedFlag && !rdt.DeletedFlag && !upd.DeletedFlag && !uqc.DeletedFlag && e.ProjectId == arenaPiD// && rdt. .ObjectIdLink == "Update_UpdateId"
&& uqc.RefQCId == 6 // Distributed
&& uqc.RefQCStatusId == 2 // Complete
orderby e.ScheduledDT descending
select new
{
e.MeetingId,
e.ScheduledDT,
edr.DeliverableId,
edr.RefDeliverableId,
rdt.DeliverableAbbrv,
UpdateId = edr.RefDsgnSubmissionTypeId != null ? edr.RefDsgnSubmissionTypeId : 0,
RefRecommendationId = upd.RefRecommendationId != null ? upd.RefRecommendationId : 0,
uqc.RefQCId,
uqc.RefQCStatusId,
DeadlineDate = ed != null ? ed.DeliverableDateValue.ToString() : "",
ActualDate = ed2 != null ? ed2.DeliverableDateValue.ToString() : ""
}).ToList();

There's a big difference to how DefaultIfEmpty works for Linq To Objects vs Linq To SQL, which is tied into the difference between how nulls are handled between the two.
In SQL an empty record from a LEFT OUTER JOIN is populated with null values. Since your SQL never refer to the record itself this is not a problem. When you write edr.DeliverableId for an unmatched edr, the result is null.
Linq to SQL is different. When you try to reference any field or property of an unmatched edr the resultant error is exactly what you have seen. Every reference to edr after the DefaultIfEmpty call needs to be checked first to see if edr is valid.
Beyond the immediate error however...
You've mixed a couple of join forms - inner and outer - in that query, and the result is convoluted and unexpected. The problems you're taking on with DefaulIfEmpty for an outer join are negated with a subsequent inner join that depends on the outers. Which means that you are going through all the pain and suffering without any of the pay-off.
This block of joins:
join ret in refDateTypesQuery on ed.RefDateTypeId equals ret.RefDateTypeId
join rdt in refDeliverablesQuery on edr.RefDeliverableId equals rdt.RefDeliverableId
join upd in updatesQuery on edr.RefDsgnSubmissionTypeId equals upd.UpdateId
join uqc in updateQCsQuery on upd.UpdateId equals uqc.UpdateId
Every one of those depends ultimately on the outer join for edr, resulting in an output that - even if you put the time in to get the outer join side effects figured out - will negate the effects of DefaultIfEmpty.
You need to re-think your logic.
I suggest breaking this thing down into a series of intermediate queries. Build the query up one step at a time, joining the results in the final stage.
For instance, you have ed as an outer join which is then subjected to an inner join with rdt and further filtered by the properties of rdt in your where clause. Scrape all that out and put it in an intermediate, then join against it later. Do the same with edt: create an intermediate that join the parts. Flatten out the results for use in the final query.
Incidentally, you don't necessarily need to bring all of that data into memory. Even if your data is on different physical servers you can often still get Linq to SQL to talk to them. When you're targeting SQL Server for example you can specify a 3- or 4-part name for the Table attribute to access data in other databases on the same server or databases on linked servers. Might be useful.

Linq subquery - getting InvalidOperationException as soon as I'm trying to iterate through the results set

The code will compile and even run, but as soon as I'm doing anything with the result, I will get InvalidOperationException or NotSupportedException
var movies = from m in data.Movies
where m.Rating > -1 && m.GenresLinks.Contains
(
(from g in data.GenresLinks
where g.GenreID == queryGenre select g).FirstOrDefault()
)
orderby m.InsertedIn descending
select m;
return movies.ToArray();//Exception here

Maybe I am missing something here, but you are returning movies with a rating > -1 and a genre of queryGenre, why don't you just to this?
var movies = from m in data.Movies
where m.Rating > -1 &&
m.GenreLinks.Any(gr => gr.GenreID == queryGenre)
orderby m.InsteredIn desc
select m;
Really need more info to be of help here though

From where the error is thrown, you can determine that its cause is evaluating the query with ToArray().
LINQ to SQL supports nested queries, so my best guess is that you are doing a Contains(null) and that it doesn't translate to SQL, so an error gets thrown when the IQueryProvider tries to evaluate the expression.