I'm starting with Linq to entities and maybe someone could shed some light.
I have two tables - Vizite (parent table) and AngajatiVizite (child table). I'm using Database first, so I have created the relationship between them using Vizite.Id and AngajatiVizite.IdVizita.
I need to get the rows from Vizite and one more bit field which must be 0 if the DataStart or DataEnd fields are null or count of child records from AngajatiVizite is zero. That's it, if the Vizite has zero subordinate records or any of those Data#### fields is null, the calculated field is 0.
So far so good, the linq I'm using works properly. The syntax I have used is this one:
var list = ctx.Vizite
.OrderBy(p => p.DataEnd != null && p.DataStart != null && p.AngajatiVizite.Count > 0)
.ThenBy(p => p.Data)
.Select(p => new
{
p.Id,
p.Numar,
p.Data,
p.DataStart,
p.DataEnd,
Programat = p.DataEnd != null && p.DataStart != null && p.AngajatiVizite.Count > 0
})
.ToList();
The sql command generated by Linq is extremely complex and I don't understand why it has to be that complex and what's the difference.
What I'm getting from linq is this:
SELECT
[Project6].[Numar] AS [Numar],
[Project6].[Id] AS [Id],
[Project6].[Data] AS [Data],
[Project6].[DataStart] AS [DataStart],
[Project6].[DataEnd] AS [DataEnd],
[Project6].[C2] AS [C1]
FROM ( SELECT
[Project5].[C1] AS [C1],
[Project5].[Id] AS [Id],
[Project5].[Numar] AS [Numar],
[Project5].[Data] AS [Data],
[Project5].[DataStart] AS [DataStart],
[Project5].[DataEnd] AS [DataEnd],
CASE WHEN ([Project5].[C2] > 0) THEN cast(1 as bit) WHEN ( NOT ([Project5].[C3] > 0)) THEN cast(0 as bit) END AS [C2]
FROM ( SELECT
[Project4].[C1] AS [C1],
[Project4].[Id] AS [Id],
[Project4].[Numar] AS [Numar],
[Project4].[Data] AS [Data],
[Project4].[DataStart] AS [DataStart],
[Project4].[DataEnd] AS [DataEnd],
[Project4].[C2] AS [C2],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[AngajatiVizite] AS [Extent5]
WHERE [Project4].[Id] = [Extent5].[IdVizita]) AS [C3]
FROM ( SELECT
[Project3].[C1] AS [C1],
[Project3].[Id] AS [Id],
[Project3].[Numar] AS [Numar],
[Project3].[Data] AS [Data],
[Project3].[DataStart] AS [DataStart],
[Project3].[DataEnd] AS [DataEnd],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[AngajatiVizite] AS [Extent4]
WHERE [Project3].[Id] = [Extent4].[IdVizita]) AS [C2]
FROM ( SELECT
CASE WHEN ([Project2].[C1] > 0) THEN cast(1 as bit) WHEN ( NOT ([Project2].[C2] > 0)) THEN cast(0 as bit) END AS [C1],
[Project2].[Id] AS [Id],
[Project2].[Numar] AS [Numar],
[Project2].[Data] AS [Data],
[Project2].[DataStart] AS [DataStart],
[Project2].[DataEnd] AS [DataEnd]
FROM ( SELECT
[Project1].[Id] AS [Id],
[Project1].[Numar] AS [Numar],
[Project1].[Data] AS [Data],
[Project1].[DataStart] AS [DataStart],
[Project1].[DataEnd] AS [DataEnd],
[Project1].[C1] AS [C1],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[AngajatiVizite] AS [Extent3]
WHERE [Project1].[Id] = [Extent3].[IdVizita]) AS [C2]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Numar] AS [Numar],
[Extent1].[Data] AS [Data],
[Extent1].[DataStart] AS [DataStart],
[Extent1].[DataEnd] AS [DataEnd],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[AngajatiVizite] AS [Extent2]
WHERE [Extent1].[Id] = [Extent2].[IdVizita]) AS [C1]
FROM [dbo].[Vizite] AS [Extent1]
) AS [Project1]
) AS [Project2]
) AS [Project3]
) AS [Project4]
) AS [Project5]
) AS [Project6]
when all I needed was actually this:
Select
Vizite.Id
, Vizite.Numar
, Vizite.Data
, Vizite.DataStart
, Vizite.DataEnd
, Case
When DataStart != Null And DataEnd != Null And (Select Count(Id) From AngajatiVizite Where Vizite.Id = AngajatiVizite.IdVizita) > 0 Then 1
Else 0
End As Programat
From Vizite
Order By Programat, Data
Can anyone please explain to me why the generated SQL is that complex that's even almost impossible to figure it out by simply reading the sql syntax?
Thank you
Entity Framework doesn't build handsome queries, that's a fact; and it's a nuisance sometimes, because it can be really hard to trace SQL logging back to LINQ statements.
It would be a problem though, if the query plan optimizer wouldn't know how to handle them. Fortunately, when Sql Server is concerned, the EF team has managed to make SQL better optimizable in each release since EF5. So generally you shouldn't worry about it too much and only start looking into it when performance is worse than can reasonably be expected.
There are some rules of the thumb though. One of them is to calculate computed values only once. This is where the let keyword comes in handy:
var list = (from p in ctx.Vizite
let Programat = p.DataEnd != null && p.DataStart != null
&& p.AngajatiVizite.Count > 0
order by Programat, p.Data
select new
{
p.Id,
p.Numar,
p.Data,
p.DataStart,
p.DataEnd,
Programat
}).ToList();
This works well in LINQ query syntax. In fluent (method) syntax you can do the exact same thing, but that requires two subsequent Select statements.
What happens to the complexity of your SQL statement if you do the following?
var list = ctx.Vizite
.Select(p => new
{
p.Id,
p.Numar,
p.Data,
p.DataStart,
p.DataEnd,
Programat =
p.DataEnd != null && p.DataStart != null && p.AngajatiVizite.Count > 0
}
.OrderBy(p => p.Programat)
.ThenBy(p => p.Data)
.ToList();
My hope is that by not repeating p.DataEnd != null && p.DataStart != null && p.AngajatiVizite.Count > 0 and moving the OrderBy and ThenBy after the select, you'll get a simpler query.
Edit
To potentially simplify the SQL even further, you could opt to do some of the work after obtaining the raw data from the database:
var list = ctx.Vizite
.Select(p => new
{
p.Id,
p.Numar,
p.Data,
p.DataStart,
p.DataEnd,
AngajatiViziteCount = p.AngajatiVizite.Count
}
.AsEnumerable() // do the rest of the work using LINQ to objects
.OrderBy(p => p.DataEnd != null && p.DataStart != null && p.AngajatiViziteCount > 0)
.ThenBy(p => p.Data)
.ToList();
Related
I'm wondering why the generated SQL is checking for nulls on a column that is not nullable (column Value which is a not-null float),
var query = _context.Events
.Select(e => new
{
eventId = e.EventId,
data = e.Data.Take(10).Select(x => new
{
name = x.Name,
value = Math.Round(x.Value,1),
})
});
Generates the following SQL code:
SELECT
[Project2].[EventId] AS [EventId],
[Project2].[DeviceId] AS [DeviceId],
[Project2].[TimeEnd] AS [TimeEnd],
[Project2].[C2] AS [C1],
[Project2].[EventId1] AS [EventId1],
[Project2].[Name] AS [Name],
[Project2].[C1] AS [C2]
FROM ( SELECT
[Limit1].[EventId] AS [EventId],
[Limit1].[TimeEnd] AS [TimeEnd],
[Limit1].[DeviceId] AS [DeviceId],
[Extent2].[Name] AS [Name],
[Extent2].[EventId] AS [EventId1],
CASE WHEN ([Extent2].[Value] IS NULL) THEN CAST(NULL AS float) ELSE ROUND([Extent2].[Value], 1) END AS [C1],
CASE WHEN ([Extent2].[Value] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C2]
FROM (SELECT [Project1].[EventId] AS [EventId], [Project1].[TimeEnd] AS [TimeEnd], [Project1].[DeviceId] AS [DeviceId]
FROM ( SELECT
[Extent1].[EventId] AS [EventId],
[Extent1].[TimeEnd] AS [TimeEnd],
[Extent1].[DeviceId] AS [DeviceId]
FROM [dbo].[Events] AS [Extent1]
WHERE ([Extent1].[DeviceId] = 1)
) AS [Project1]
ORDER BY [Project1].[TimeEnd] DESC ) AS [Limit1]
LEFT OUTER JOIN [dbo].[Octaves] AS [Extent2] ON [Limit1].[EventId] = [Extent2].[EventId]
) AS [Project2]
ORDER BY [Project2].[TimeEnd] DESC, [Project2].[EventId] ASC, [Project2].[C2] ASC
If I remove the two CASE WHEN ([Extent2].[Value] IS NULL) and just leave ROUND([Limit2].[Value], 1) AS [C1] and also remove the ORDER BY Project2].[C2] ASC in SQL Server Management the query speeds up.
There is a LEFT OUTER JOIN, so the values (in the query) can definitely be NULL (even if the underlying table definition is not nullable).
See http://docs.oracle.com/javadb/10.8.3.0/ref/rrefsqlj18922.html (for Oracle - but the same basic pattern applies for all the main SQL vendors).
In this case, it is a bit pointless since ROUND(NULL, 1) would return NULL anyway (well, it would on SQL Server). But there isn't much you can do about that, given you can't control the SQL it generates.
I have a list of StudentGrades which contains TermNumber, I want to group them by TermNumber. However, when I look at the results, the ones with null TermNumbers are not returned/grouped.
IQueryable<StudentGradeDm> query = GetListQuery().Where(m => m.StudentId == studentId);
IQueryable<StudentGradeDto> groupedQuery = query.GroupBy(m => m.TermNumber)
.Select(m => new StudentGradeDto
{
TermNumber = m.Key,
StudentGrades = m.ToList()
});
return groupedQuery;
As you can see from the lovely screenshot i've taken here .. there are 3 groups, however, the first group is null because their TermNumber is null. But theoretically, who cares if its null? The StudentGrades with null TermNumber should still be a group.
I understand that one fix to this would be to call
query.ToList().GroupBy ......
But this is not an option for me, as the streamlined application will take a query such as the one above, and feed it into a generic pagination and sort function, in which only a subset of records will be fetched from the database to improve performance.
Any expert inputs would be greatly appreciated!
Update: Here is the generated SQL
{SELECT
[Project2].[C2] AS [C1],
[Project2].[C1] AS [C2],
[Project2].[C3] AS [C3],
[Project2].[Id] AS [Id],
[Project2].[StudentId] AS [StudentId],
[Project2].[Year] AS [Year],
[Project2].[TermTypeId] AS [TermTypeId],
[Project2].[TermNumber] AS [TermNumber],
[Project2].[ClassId] AS [ClassId],
[Project2].[GradeId] AS [GradeId],
[Project2].[FileId] AS [FileId],
[Project2].[CreatedById] AS [CreatedById],
[Project2].[CreatedDate] AS [CreatedDate],
[Project2].[ModifiedById] AS [ModifiedById],
[Project2].[ModifiedDate] AS [ModifiedDate],
[Project2].[Deleted] AS [Deleted]
FROM ( SELECT
[Distinct1].[C1] AS [C1],
1 AS [C2],
[Extent2].[Id] AS [Id],
[Extent2].[StudentId] AS [StudentId],
[Extent2].[Year] AS [Year],
[Extent2].[TermTypeId] AS [TermTypeId],
[Extent2].[TermNumber] AS [TermNumber],
[Extent2].[ClassId] AS [ClassId],
[Extent2].[GradeId] AS [GradeId],
[Extent2].[FileId] AS [FileId],
[Extent2].[CreatedById] AS [CreatedById],
[Extent2].[CreatedDate] AS [CreatedDate],
[Extent2].[ModifiedById] AS [ModifiedById],
[Extent2].[ModifiedDate] AS [ModifiedDate],
[Extent2].[Deleted] AS [Deleted],
CASE WHEN ([Extent2].[Id] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C3]
FROM (SELECT DISTINCT
[Extent1].[TermNumber] AS [C1]
FROM [dbo].[StudentGrade] AS [Extent1]
WHERE ([Extent1].[Deleted] <> 1) AND ([Extent1].[StudentId] = #p__linq__0) ) AS [Distinct1]
LEFT OUTER JOIN [dbo].[StudentGrade] AS [Extent2] ON ([Extent2].[Deleted] <> 1) AND ([Extent2].[StudentId] = #p__linq__0) AND ([Distinct1].[C1] = [Extent2].[TermNumber])
) AS [Project2]
ORDER BY [Project2].[C1] ASC, [Project2].[C3] ASC}
I'm trying to write a simple SQL query in LinQ, and no matter how hard I try, I always get a complex query.
Here is the SQL I am trying to achieve (this is not what I'm getting):
SELECT
ClearingAccounts.ID,
SUM(CASE WHEN Payments.StatusID = 1 THEN Payments.TotalAmount ELSE 0 END) AS Sum1,
SUM(CASE WHEN DirectDebits.StatusID = 2 THEN DirectDebits.TotalAmount ELSE 0 END) AS Sum2,
SUM(CASE WHEN Payments.StatusID = 2 THEN Payments.TotalAmount ELSE 0 END) AS Sum3,
SUM(CASE WHEN DirectDebits.StatusID = 1 THEN DirectDebits.TotalAmount ELSE 0 END) AS Sum4
FROM ClearingAccounts
LEFT JOIN Payments ON Payments.ClearingAccountID = ClearingAccounts.ID
LEFT JOIN DirectDebits ON DirectDebits.ClearingAccountID = ClearingAccounts.ID
GROUP BY ClearingAccounts.ID
Here is the code:
from clearingAccount in clearingAccounts
let payments = clearingAccount.Payments
let directDebits = clearingAccount.DirectDebits
select new
{
ID = clearingAccount.ID,
Sum1 = payments.Sum(p => p.StatusID == 1 ? p.TotalAmount : 0),
Sum2 = directDebits.Sum(p => p.StatusID == 2 ? p.TotalAmount : 0),
Sum3 = payments.Sum(p => p.StatusID == 2 ? p.TotalAmount : 0),
Sum4 = directDebits.Sum(p => p.StatusID == 1 ? p.TotalAmount : 0),
}
The generated query gets the data from the respective table for each sum, so four times. I'm not sure if it's even possible to optimize this?
EDIT Here the is generated query:
SELECT
[Project5].[ID] AS [ID],
[Project5].[C1] AS [C1],
[Project5].[C2] AS [C2],
[Project5].[C3] AS [C3],
[Project5].[C4] AS [C4]
FROM ( SELECT
[Project4].[ID] AS [ID],
[Project4].[C1] AS [C1],
[Project4].[C2] AS [C2],
[Project4].[C3] AS [C3],
(SELECT
SUM([Filter5].[A1]) AS [A1]
FROM ( SELECT
CASE WHEN (1 = [Extent5].[StatusID]) THEN [Extent5].[TotalAmount] ELSE cast(0 as decimal(18)) END AS [A1]
FROM [dbo].[DirectDebits] AS [Extent5]
WHERE [Project4].[ID] = [Extent5].[ClearingAccountID]
) AS [Filter5]) AS [C4]
FROM ( SELECT
[Project3].[ID] AS [ID],
[Project3].[C1] AS [C1],
[Project3].[C2] AS [C2],
(SELECT
SUM([Filter4].[A1]) AS [A1]
FROM ( SELECT
CASE WHEN (2 = [Extent4].[StatusID]) THEN [Extent4].[TotalAmount] ELSE cast(0 as decimal(18)) END AS [A1]
FROM [dbo].[Payments] AS [Extent4]
WHERE [Project3].[ID] = [Extent4].[ClearingAccountID]
) AS [Filter4]) AS [C3]
FROM ( SELECT
[Project2].[ID] AS [ID],
[Project2].[C1] AS [C1],
(SELECT
SUM([Filter3].[A1]) AS [A1]
FROM ( SELECT
CASE WHEN (2 = [Extent3].[StatusID]) THEN [Extent3].[TotalAmount] ELSE cast(0 as decimal(18)) END AS [A1]
FROM [dbo].[DirectDebits] AS [Extent3]
WHERE [Project2].[ID] = [Extent3].[ClearingAccountID]
) AS [Filter3]) AS [C2]
FROM ( SELECT
[Project1].[ID] AS [ID],
(SELECT
SUM([Filter2].[A1]) AS [A1]
FROM ( SELECT
CASE WHEN (1 = [Extent2].[StatusID]) THEN [Extent2].[TotalAmount] ELSE cast(0 as decimal(18)) END AS [A1]
FROM [dbo].[Payments] AS [Extent2]
WHERE [Project1].[ID] = [Extent2].[ClearingAccountID]
) AS [Filter2]) AS [C1]
FROM ( SELECT
[Extent1].[ID] AS [ID]
FROM [dbo].[ClearingAccounts] AS [Extent1]
WHERE ([Extent1].[CustomerID] = 3) AND ([Extent1].[Deleted] <> 1)
) AS [Project1]
) AS [Project2]
) AS [Project3]
) AS [Project4]
) AS [Project5]
Edit
Note that as per #usr's comment, that your original Sql Query is broken. By LEFT OUTER joining on two independent tables, and then grouping on the common join key, as soon as one of the DirectDebits or Payments tables returns more than one row, you will erroneously duplicate the TotalAmount value in the 'other' SUMmed colums (and vice versa). e.g. If a given ClearingAccount has 3 DirectDebits and 4 Payments, you will get a total of 12 rows (whereas you should be summing 3 and 4 rows independently for the two tables). A better Sql Query would be:
WITH ctePayments AS
(
SELECT
ClearingAccounts.ID,
-- Note the ELSE 0 projection isn't required as nulls are eliminated from aggregates
SUM(CASE WHEN Payments.StatusID = 1 THEN Payments.TotalAmount END) AS Sum1,
SUM(CASE WHEN Payments.StatusID = 2 THEN Payments.TotalAmount END) AS Sum3
FROM ClearingAccounts
INNER JOIN Payments ON Payments.ClearingAccountID = ClearingAccounts.ID
GROUP BY ClearingAccounts.ID
),
cteDirectDebits AS
(
SELECT
ClearingAccounts.ID,
SUM(CASE WHEN DirectDebits.StatusID = 2 THEN DirectDebits.TotalAmount END) AS Sum2,
SUM(CASE WHEN DirectDebits.StatusID = 1 THEN DirectDebits.TotalAmount END) AS Sum4
FROM ClearingAccounts
INNER JOIN DirectDebits ON DirectDebits.ClearingAccountID = ClearingAccounts.ID
GROUP BY ClearingAccounts.ID
)
SELECT ca.ID, COALESCE(p.Sum1, 0) AS Sum1, COALESCE(d.Sum2, 0) AS Sum2,
COALESCE(p.Sum3, 0) AS Sum3, COALESCE(d.Sum4, 0) AS Sum4
FROM
ClearingAccounts ca
LEFT OUTER JOIN ctePayments p
ON ca.ID = p.ID
LEFT OUTER JOIN cteDirectDebits d
ON ca.ID = d.ID;
-- GROUP BY not required, since we have already guaranteed at most one row
-- per joined table in the CTE's, assuming ClearingAccounts.ID is unique;
You'll want to fix and test this with test cases before you even contemplate conversion to LINQ.
Old Answer(s)
The Sql construct:
SELECT SUM(CASE WHEN ... THEN 1 ELSE 0 END) AS Something
when applied in a SELECT list, is a common hack 'alternative' to pivot data from the 'greater' select into columns which meet the projection criteria (and hence the zero if not matched) . It isn't really a sum at all, its a 'matched' count.
With regards to optimizing the Sql generated, another alternative would be to materialize the data after joining and grouping (and of course, if there is a predicate WHERE clause, apply that in Sql too via IQueryable), and then do the conditional summation in memory:
var result2 = Db.ClearingAccounts
.Include(c => c.Payments)
.Include(c => c.DirectDebits)
.GroupBy(c => c.Id)
.ToList() // or any other means to force materialization here.
.ToDictionary(
grp => grp.Key,
grp => new
{
PaymentsByStatus = grp.SelectMany(x => x.Payments)
.GroupBy(p => p.StatusId),
DirectDebitByStatus = grp.SelectMany(x => x.Payments)
.GroupBy(p => p.StatusId),
})
.Select(ca => new
{
ID = ca.Key,
Sum1 = ca.Value.PaymentsByStatus.Where(pbs => pbs.Key == 1)
.Select(pbs => pbs.Select(x => x.TotalAmount).Sum()),
Sum2 = ca.Value.DirectDebitByStatus.Where(pbs => pbs.Key == 2)
.Select(ddbs => ddbs.Select(x => x.TotalAmount).Sum()),
Sum3 = ca.Value.PaymentsByStatus.Where(pbs => pbs.Key == 2)
.Select(pbs => pbs.Select(x => x.TotalAmount).Sum()),
Sum4 = ca.Value.DirectDebitByStatus.Where(pbs => pbs.Key == 1)
.Select(ddbs => ddbs.Select(x => x.TotalAmount).Sum())
});
However, personally, I would leave this pivot projection directly in Sql, and then use something like SqlQuery to then deserialize the result back from Sql
directly into the final Entity type.
1)
Add AsNoTracking in EF to avoid tracking changes.
Check that you have indexes on the columns you are using for the JOINs. Especially the column that you are using to group by. Profile the query and optimize it. EF has also overhead over a stored procedure.
or
2) If you cannot find a way to make it as fast as you need, create a stored procedure and call it from EF. Even the same query will be faster.
In my database there is a table Question with Content and Answer. I implemented a search engine which uses this:
private Expression<Func<QuestionModel, bool>> CreatePredicate(string query, bool negate = false)
{
query = query.ToUpper();
if (IsATag(ref query))
{
return question => negate == question.Tags.Any(tag => tag.NamesCollection
.Any(translation =>
translation.Language == currentLanguage &&
translation.Translation.ToUpper().Equals(query)));
}
else if(IsSpecial(ref query))
{
return question => true;
}
else
{
return question => negate == question.Content.ToUpper().Contains(query) ||
question.Answer.ToUpper().Contains(query) ||
question.Specialization.NamesCollection.Any(trans =>
trans.Language == currentLanguage &&
trans.Translation.ToUpper().Contains(query));
}
}
Actually it supports multiple queries but this is nicely handled by PrebicateBuilder
I created some performance tests and it doesn't run very fast for 10K records (up to 30 seconds if I use 5 queries. Is there a smart way to enhance the process? Honestly I've never seen a website running so slow, so there must be something.
This is the sql generated by my query:
SELECT TOP (100)
[top].[Id] AS [Id],
[top].[Content] AS [Content],
[top].[Answer] AS [Answer],
[top].[Assessment] AS [Assessment],
[top].[State] AS [State],
[top].[AuthorId] AS [AuthorId],
[top].[AttachmentFileName] AS [AttachmentFileName],
[top].[SubmissionDate] AS [SubmissionDate],
[top].[Specialization_Id] AS [Specialization_Id],
[top].[Doctor_Id] AS [Doctor_Id]
FROM ( SELECT [Project16].[Id] AS [Id], [Project16].[Content] AS [Content], [Project16].[Answer] AS [Answer], [Project16].[Assessment] AS [Assessment], [Project16].[State] AS [State], [Project16].[AuthorId] AS [AuthorId], [Project16].[AttachmentFileName] AS [AttachmentFileName], [Project16].[SubmissionDate] AS [SubmissionDate], [Project16].[Specialization_Id] AS [Specialization_Id], [Project16].[Doctor_Id] AS [Doctor_Id]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Content] AS [Content],
[Extent1].[Answer] AS [Answer],
[Extent1].[Assessment] AS [Assessment],
[Extent1].[State] AS [State],
[Extent1].[AuthorId] AS [AuthorId],
[Extent1].[AttachmentFileName] AS [AttachmentFileName],
[Extent1].[SubmissionDate] AS [SubmissionDate],
[Extent1].[Specialization_Id] AS [Specialization_Id],
[Extent1].[Doctor_Id] AS [Doctor_Id]
FROM [QuestionModels] AS [Extent1]
WHERE ((#p__linq__0 = (CASE WHEN ((CHARINDEX(#p__linq__1, UPPER([Extent1].[Content]))) > 0) THEN cast(1 as bit) WHEN ( NOT ((CHARINDEX(#p__linq__1, UPPER([Extent1].[Content]))) > 0)) THEN cast(0 as bit) END)) OR ((CHARINDEX(#p__linq__2, UPPER([Extent1].[Answer]))) > 0) OR ( EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent2]
WHERE ([Extent2].[SpecializationModel_Id] IS NOT NULL) AND ([Extent1].[Specialization_Id] = [Extent2].[SpecializationModel_Id]) AND ([Extent2].[Language] = #p__linq__3) AND ((CHARINDEX(#p__linq__4, UPPER([Extent2].[Translation]))) > 0)
)) OR (#p__linq__5 = (CASE WHEN ((CHARINDEX(#p__linq__6, UPPER([Extent1].[Content]))) > 0) THEN cast(1 as bit) WHEN ( NOT ((CHARINDEX(#p__linq__6, UPPER([Extent1].[Content]))) > 0)) THEN cast(0 as bit) END)) OR ((CHARINDEX(#p__linq__7, UPPER([Extent1].[Answer]))) > 0) OR ( EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent3]
WHERE ([Extent3].[SpecializationModel_Id] IS NOT NULL) AND ([Extent1].[Specialization_Id] = [Extent3].[SpecializationModel_Id]) AND ([Extent3].[Language] = #p__linq__8) AND ((CHARINDEX(#p__linq__9, UPPER([Extent3].[Translation]))) > 0)
)) OR (#p__linq__10 = (CASE WHEN ((CHARINDEX(#p__linq__11, UPPER([Extent1].[Content]))) > 0) THEN cast(1 as bit) WHEN ( NOT ((CHARINDEX(#p__linq__11, UPPER([Extent1].[Content]))) > 0)) THEN cast(0 as bit) END)) OR ((CHARINDEX(#p__linq__12, UPPER([Extent1].[Answer]))) > 0) OR ( EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent4]
WHERE ([Extent4].[SpecializationModel_Id] IS NOT NULL) AND ([Extent1].[Specialization_Id] = [Extent4].[SpecializationModel_Id]) AND ([Extent4].[Language] = #p__linq__13) AND ((CHARINDEX(#p__linq__14, UPPER([Extent4].[Translation]))) > 0)
)) OR (#p__linq__15 = (CASE WHEN ( EXISTS (SELECT
1 AS [C1]
FROM ( SELECT
[Extent5].[TagModel_Id] AS [TagModel_Id]
FROM [TagModelQuestionModels] AS [Extent5]
WHERE [Extent1].[Id] = [Extent5].[QuestionModel_Id]
) AS [Project4]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent6]
WHERE ([Project4].[TagModel_Id] = [Extent6].[TagModel_Id]) AND ([Extent6].[Language] = #p__linq__16) AND ((UPPER([Extent6].[Translation])) = #p__linq__17)
)
)) THEN cast(1 as bit) WHEN ( NOT EXISTS (SELECT
1 AS [C1]
FROM ( SELECT
[Extent7].[TagModel_Id] AS [TagModel_Id]
FROM [TagModelQuestionModels] AS [Extent7]
WHERE [Extent1].[Id] = [Extent7].[QuestionModel_Id]
) AS [Project7]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent8]
WHERE ([Project7].[TagModel_Id] = [Extent8].[TagModel_Id]) AND ([Extent8].[Language] = #p__linq__16) AND ((UPPER([Extent8].[Translation])) = #p__linq__17)
)
)) THEN cast(0 as bit) END))) AND (#p__linq__18 = (CASE WHEN ( EXISTS (SELECT
1 AS [C1]
FROM ( SELECT
[Extent9].[TagModel_Id] AS [TagModel_Id]
FROM [TagModelQuestionModels] AS [Extent9]
WHERE [Extent1].[Id] = [Extent9].[QuestionModel_Id]
) AS [Project10]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent10]
WHERE ([Project10].[TagModel_Id] = [Extent10].[TagModel_Id]) AND ([Extent10].[Language] = #p__linq__19) AND ((UPPER([Extent10].[Translation])) = #p__linq__20)
)
)) THEN cast(1 as bit) WHEN ( NOT EXISTS (SELECT
1 AS [C1]
FROM ( SELECT
[Extent11].[TagModel_Id] AS [TagModel_Id]
FROM [TagModelQuestionModels] AS [Extent11]
WHERE [Extent1].[Id] = [Extent11].[QuestionModel_Id]
) AS [Project13]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [TranslationModels] AS [Extent12]
WHERE ([Project13].[TagModel_Id] = [Extent12].[TagModel_Id]) AND ([Extent12].[Language] = #p__linq__19) AND ((UPPER([Extent12].[Translation])) = #p__linq__20)
)
)) THEN cast(0 as bit) END))
) AS [Project16]
ORDER BY [Project16].[SubmissionDate] ASC
OFFSET 1000 ROWS
) AS [top]
These db relations don't seem to affect performance very much. If I use a query which doesn't involve them the speed is pretty much the same.
Table full scans and twice-nested per-row subqueries will be slow. A faster approach is to arrange your database such that a query need involve only a single index seek or index range scan.
Many people who want to build a search feature use a search index of some kind. Some relational databases support a limited form of full-text search, and this may suffice for your case.
Many people prefer to use a specialized search index. Lucene is a popular engine, with Solr and Elasticsearch exposing Lucene over the network.
I have the folling LINQ query:
var queryEvents = (from p in db.cl_contact_event
where p.time_of_contact >= beginDate && p.time_of_contact < endDate
group p by p.contact_list_name into g
select new PenRawModel
{
listName = g.Key,
download = g.Max(a => a.total_number_of_records),
dials = g.Where(a => a.ov_dial_start_time != null).Count(), //This fails.
//dials = g.Sum(a => a.ov_dial_start_time != null ? 1 : 0), //This works.
agentConnects = g.Sum(a => a.ov_trunk_released_time != null ? 1 : 0),
abandons = g.Sum(a => a.response_status == "DAC" || a.response_status == "DAD" ? 1 : 0),
rightPartyContacts = g.Sum(a => a.response_status == "PTP" || a.response_status == "RPC" ? 1 : 0),
promiseToPays = g.Sum(a => a.response_status == "PTP" ? 1 : 0),
talkTime = g.Sum(a => EntityFunctions.DiffSeconds(a.ov_call_connected_time, a.ov_trunk_released_time)) ?? 0,
wrapTime = g.Sum(a => EntityFunctions.DiffSeconds(a.ov_trunk_released_time, a.record_released_time)) ?? 0
}
When run it gives me the error:
"The column prefix 'Project1' does not match with a table name or
alias name used in the query. Either the table is not specified in the
FROM clause or it has a correlation name which must be used instead."
The reason for the failure is:
dials = g.Where(a => a.ov_dial_start_time != null).Count(),
If I replace that line of code with the commented out one below it then the query works just fine. I'd prefer to use the .Where/.Count though because it's easier for someone else to decipher and understand the intent of the code.
Can anyone help give me a clue as to why this fails and how I might possibly fix it?
Edit- Here is the SQL that is sent to the Sybase database from the failed query:
SELECT
1 AS [C1],
[Project2].[contact_list_name] AS [contact_list_name],
[Project2].[C1] AS [C2],
[Project2].[C10] AS [C3],
[Project2].[C2] AS [C4],
[Project2].[C3] AS [C5],
[Project2].[C4] AS [C6],
[Project2].[C5] AS [C7],
CASE WHEN ([Project2].[C6] IS NULL) THEN 0 ELSE [Project2].[C7] END AS [C8],
CASE WHEN ([Project2].[C8] IS NULL) THEN 0 ELSE [Project2].[C9] END AS [C9]
FROM ( SELECT
[Project1].[C1] AS [C1],
[Project1].[C2] AS [C2],
[Project1].[C3] AS [C3],
[Project1].[C4] AS [C4],
[Project1].[C5] AS [C5],
[Project1].[C6] AS [C6],
[Project1].[C7] AS [C7],
[Project1].[C8] AS [C8],
[Project1].[C9] AS [C9],
[Project1].[contact_list_name] AS [contact_list_name],
(SELECT
Count([Filter2].[A1]) AS [A1]
FROM ( SELECT
1 AS [A1]
FROM [mel].[cl_contact_event] AS [Extent2]
WHERE ((([Extent2].[time_of_contact] >= #p__linq__0) AND ([Extent2].[time_of_contact] < #p__linq__1)) AND (([Project1].[contact_list_name] = [Extent2].[contact_list_name]) OR (([Project1].[contact_list_name] IS NULL) AND ([Extent2].[contact_list_name] IS NULL)))) AND ([Extent2].[ov_dial_start_time] IS NOT NULL)
) AS [Filter2]) AS [C10]
FROM ( SELECT
[GroupBy1].[A1] AS [C1],
[GroupBy1].[A2] AS [C2],
[GroupBy1].[A3] AS [C3],
[GroupBy1].[A4] AS [C4],
[GroupBy1].[A5] AS [C5],
[GroupBy1].[A6] AS [C6],
[GroupBy1].[A7] AS [C7],
[GroupBy1].[A8] AS [C8],
[GroupBy1].[A9] AS [C9],
[GroupBy1].[K1] AS [contact_list_name]
FROM ( SELECT
[Filter1].[K1] AS [K1],
Max([Filter1].[A1]) AS [A1],
Sum([Filter1].[A2]) AS [A2],
Sum([Filter1].[A3]) AS [A3],
Sum([Filter1].[A4]) AS [A4],
Sum([Filter1].[A5]) AS [A5],
Sum([Filter1].[A6]) AS [A6],
Sum([Filter1].[A7]) AS [A7],
Sum([Filter1].[A8]) AS [A8],
Sum([Filter1].[A9]) AS [A9]
FROM ( SELECT
[Extent1].[contact_list_name] AS [K1],
[Extent1].[total_number_of_records] AS [A1],
CASE WHEN ([Extent1].[ov_trunk_released_time] IS NOT NULL) THEN 1 ELSE 0 END AS [A2],
CASE WHEN ((N'DAC' = [Extent1].[response_status]) OR (N'DAD' = [Extent1].[response_status])) THEN 1 ELSE 0 END AS [A3],
CASE WHEN ((N'PTP' = [Extent1].[response_status]) OR (N'RPC' = [Extent1].[response_status])) THEN 1 ELSE 0 END AS [A4],
CASE WHEN (N'PTP' = [Extent1].[response_status]) THEN 1 ELSE 0 END AS [A5],
DATEDIFF (second, [Extent1].[ov_call_connected_time], [Extent1].[ov_trunk_released_time]) AS [A6],
DATEDIFF (second, [Extent1].[ov_call_connected_time], [Extent1].[ov_trunk_released_time]) AS [A7],
DATEDIFF (second, [Extent1].[ov_trunk_released_time], [Extent1].[record_released_time]) AS [A8],
DATEDIFF (second, [Extent1].[ov_trunk_released_time], [Extent1].[record_released_time]) AS [A9]
FROM [mel].[cl_contact_event] AS [Extent1]
WHERE ([Extent1].[time_of_contact] >= #p__linq__0) AND ([Extent1].[time_of_contact] < #p__linq__1)
) AS [Filter1]
GROUP BY [K1]
) AS [GroupBy1]
) AS [Project1]
) AS [Project2]
FULLY EDITED ANSWER
The problem was finally in the translation from LINQ to SQL of the Sybex ASE provider:
Entity framework 4.1 (shouldn't influence the bug)
Sybex ASE driver for SDK 15.7 SED #02 (responsible for the failure)
The queries should be well rendered using any LINQ sintax. They could produce different queries, but they all should work. However this provider fails to do it right with some of the possible sintax.
Is good to know that:
dials = g.Where(a => a.ov_dial_start_time != null).Count(), // fails.
dials = g.Count(a => a.ov_dial_start_time != null), // also fails.
dials = g.Sum(a => a.ov_dial_start_time != null ? 1 : 0), // works.
According to OP, adding AsEnumerable() before the Where() filter also makes it work.
If you find some failure with this database and provider it's a good idea to check the query that will be sent to the server to find the offending part and try alternative sintax until the problem is solved. You can check the query that will be sent to the server using toString(). Then you can check the sintax, or run it directly in the server and see what is failing.