Develop Reference

How to optimize a slow running LINQ query that runs quickly in SQL Server

I am trying to write a LINQ query that returns rows (items) based on certain conditions on one of the sub-collection of the row (item). The query I wrote works but performs very poorly in LINQ. However if I run the generated query in SQL Server, it almost instantly returns the desired rows.
My assumption is that the query is slow because of the OrderByDescending() that is performed multiple times. Is this correct and how can I improve the performance of this query?
Edit: The goal of this query is to select all Foo objects that have a Bar object
where the price property of the last Baz object is within a lower and upper bound value.
var query = dbContext.Foos.AsQueryable();
query = query.Where(e => e.Bars.Any(p => p.Bazs.OrderByDescending(s => s.BazId).First().Price >= filter.LowerBoundPrice));
query = query.Where(e => e.Bars.Any(p => p.Bazs.OrderByDescending(s => s.BazId).First().Price <= filter.UpperBoundPrice));
Edit2: This is the generated SQL query.
-- Region Parameters
DECLARE #p0 Decimal(6,2) = 2000
DECLARE #p1 Decimal(6,2) = 1000
-- EndRegion
SELECT [t0].[FooId]
FROM [Foo] AS [t0]
WHERE (EXISTS(
SELECT NULL AS [EMPTY]
FROM [Bar] AS [t1]
WHERE (((
SELECT [t3].[Price]
FROM (
SELECT TOP (1) [t2].[Price]
FROM [Baz] AS [t2]
WHERE [t2].[BarId] = [t1].[BarId]
ORDER BY [t2].[BazId] DESC
) AS [t3]
)) <= #p0) AND ([t1].[FooId] = [t0].[FooId])
)) AND (EXISTS(
SELECT NULL AS [EMPTY]
FROM [Bar] AS [t4]
WHERE (((
SELECT [t6].[Price]
FROM (
SELECT TOP (1) [t5].[Price]
FROM [Baz] AS [t5]
WHERE [t5].[BarId] = [t4].[BarId]
ORDER BY [t5].[BazId] DESC
) AS [t6]
)) >= #p1) AND ([t4].[FooId] = [t0].[FooId])
))

LinqPad query does not run in Visual Studio code

I came across this question on Stack Overflow about different results
LINQ - Different results with LINQ to SQL vs LINQPad
I am sort of having a similar problem, but it is with stored procs.
I have some Linq
var op = (from o in Entities.Opportunities
where (o.OpportunityId == opportunityId)
select new
{
IsActiveStatus = o.IsActiveStatus,
opportunityReasonId = o.OpportunityLostReasonId,
comments = o.Comment,
competitorId = o.LostToCompetitorId,
sourceId = o.DataSourceId,
GetCrossSellOptionForOpportunity = Entities.GetCrossSellOptionForOpportunity(o.OpportunityId),
projects = Entities.GetOpportunityProjectPaul(o.OpportunityId),
items = Entities.GetOpportunityItemPaul(o.OpportunityId),
lastUpdated = o.UpdatedDate
}).FirstOrDefault();
This works in LinqPad, but fails to run within a C# VS2015 environment.
The error relates to the lines in the select statement
GetCrossSellOptionForOpportunity = Entities.GetCrossSellOptionForOpportunity(o.OpportunityId),
projects = Entities.GetOpportunityProjectPaul(o.OpportunityId),
items = Entities.GetOpportunityItemPaul(o.OpportunityId),
These are either Stored procedures or functions.
As an example:
GetCrossSellOptionForOpportunity returns an object of type
GetOpportunityProjectPaulReturnModel. the context.cs file has the correct
settings, and GetOpportunityProjectPaulReturnModel is generated as a class
against the context.tt4 file. The app compiles with no errors, but it still
crashes
Our database is mapped into c# using the EntityFramework Reverse POCO Generator addin, located here: https://marketplace.visualstudio.com/items?itemName=SimonHughes.EntityFrameworkReversePOCOGenerator
The physical error returned is:
LINQ to Entities does not recognize the method
'System.Collections.Generic.List1[GetOpportunityProjectPaulReturnModel]
GetOpportunityProjectPaul(System.Nullable1[System.Int64])' method,
and this method cannot be translated into a store expression.
Update:
Please find the image of a screenshot showing that it works it LinqPad.
Update 2
Generated SQL....
DECLARE #p0 Int = 1
DECLARE #p1 Int = 7
DECLARE #p2 BigInt = 1798609
-- EndRegion
SELECT [t2].[IsActiveStatus], [t0].[OpportunityLostReasonID] AS [OpportunityReasonId], [t0].[Comment] AS [Comments], [t0].[LostToCompetitorID] AS [CompetitorId], [t0].[DataSourceID] AS [SourceId], [t3].[OpportunityCrossSellOptionID], [t3].[OpportunityID], [t3].[CrossSellOptionID], [t3].[DistanceRank], [t3].[BusinessLineID], [t3].[UpdatedDate], (
SELECT COUNT(*)
FROM [dbo].[GetCrossSellOptionForOpportunity]([t0].[OpportunityID]) AS [t4]
) AS [value], [t0].[UpdatedDate] AS [LastUpdated]
FROM [Opportunity] AS [t0]
INNER JOIN [OpportunityAssignmentUserPage] AS [t1] ON [t0].[OpportunityID] = [t1].[OpportunityID]
INNER JOIN [OpportunityStatus] AS [t2] ON [t2].[OpportunityStatusID] = [t0].[OpportunityStatusID]
OUTER APPLY [dbo].[GetCrossSellOptionForOpportunity]([t0].[OpportunityID]) AS [t3]
WHERE (([t0].[OpportunityStatusID] = #p0) OR ([t0].[OpportunityStatusID] = #p1)) AND ([t0].[OpportunityID] = #p2)
ORDER BY [t0].[OpportunityID], [t2].[OpportunityStatusID]

SQL Server - Reuse previous query - similar logic to Linq

I am trying to write to achieve the logic of the following query:
select ......
from (select ..... from ... complex and long query #1) R1
(select ..... from ... complex and long query #2) R2
Where
NOT ( #DateStart > R2.IstEnde OR #DateEnd <= R2.IstStart)
OR
(
(select count(*)
from R2 // <---- BUG IS HERE
Where R2.IsOutsideTaskTimeFrame = 1 AND R2.IsManuallyFixed = 1
) > 0
)
order BY R2.PersonName, R1.YearOfWeek, R1.Week
Which obvious do not work because I am trying to "Reuse" the R2 in the "count()" where condition.
I am writing this based on the ".NET Linq" logic where we can "reuse" the previous query.
Can I even write this logic of "reusing" the previous query and not repeating it again?

Maybe you are looking for CTE - Common Table Expression:
;WITH R1 AS
(SELECT ...)
,R2 AS
(SELECT ...)
SELECT * FROM R1
... use R1 and R2 like any other table here (it's called "derived table")
If you need the result within independant queries, you might fill a declared table variable or a temp table.

You can use CTE to define a base query which can be used multiple times in the same scope of the code:
WITH Sales_CTE (returnParam1, returnParam2, ...)
AS
-- Define the CTE query.
(
select ..... from ... complex and long query #2
)
Or you can create a user defined function which gets params and returns the result, then call it multiple times:
CREATE FUNCTION dbo.ufnGetInventoryStock(param1 int, ...)
RETURNS int
AS
RETURN (
select ..... from ... complex and long query #2
)

Entity Framework v6.1 query compilation performance

I am confused how EF LINQ queries are compiled and executed. When I run a piece of program in LINQPad couple of times, I get varied performance results (each time the same query takes different amount of time). Please find below my test execution environment.
tools used: EF v6.1 & LINQPad v5.08.
Ref DB : ContosoUniversity DB downloaded from MSDN.
For queries, I am using Persons, Courses & Departments tables from the above DB; see below.
Now, I have below data:
Query goal: get the second person and associated departments.
Query:
var test = (
from p in Persons
join d in Departments on p.ID equals d.InstructorID
select new {
person = p,
dept = d
}
);
var result = (from pd in test
group pd by pd.person.ID into grp
orderby grp.Key
select new {
ID = grp.Key,
FirstName = grp.First().person.FirstName,
Deps = grp.Where(x => x.dept != null).Select(x => x.dept).Distinct().ToList()
}).Skip(1).Take(1).ToList();
foreach(var r in result)
{
Console.WriteLine("person is..." + r.FirstName);
Console.WriteLine(r.FirstName + "' deps are...");
foreach(var d in r.Deps){
Console.WriteLine(d.Name);
}
}
When I run this I get the result and LINQPad shows time taken value from 3.515 sec to 0.004 sec (depending how much gap I take between different runs).
If I take the generated SQL query and execute it, that query always runs between 0.015 sec to 0.001sec.
Generated query:
-- Region Parameters
DECLARE #p0 Int = 1
DECLARE #p1 Int = 1
-- EndRegion
SELECT [t7].[ID], [t7].[value] AS [FirstName]
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY [t6].[ID]) AS [ROW_NUMBER], [t6].[ID], [t6].[value]
FROM (
SELECT [t2].[ID], (
SELECT [t5].[FirstName]
FROM (
SELECT TOP (1) [t3].[FirstName]
FROM [Person] AS [t3]
INNER JOIN [Department] AS [t4] ON ([t3].[ID]) = [t4]. [InstructorID]
WHERE [t2].[ID] = [t3].[ID]
) AS [t5]
) AS [value]
FROM (
SELECT [t0].[ID]
FROM [Person] AS [t0]
INNER JOIN [Department] AS [t1] ON ([t0].[ID]) = [t1].[InstructorID]
GROUP BY [t0].[ID]
) AS [t2]
) AS [t6]
) AS [t7]
WHERE [t7].[ROW_NUMBER] BETWEEN #p0 + 1 AND #p0 + #p1
ORDER BY [t7].[ROW_NUMBER]
GO
-- Region Parameters
DECLARE #x1 Int = 2
-- EndRegion
SELECT DISTINCT [t1].[DepartmentID], [t1].[Name], [t1].[Budget], [t1]. [StartDate], [t1].[InstructorID], [t1].[RowVersion]
FROM [Person] AS [t0]
INNER JOIN [Department] AS [t1] ON ([t0].[ID]) = [t1].[InstructorID]
WHERE #x1 = [t0].[ID]
My questions:
1) Are those LINQ statements correct? Or can they be optimized?
2) Is the time difference for LINQ query execution normal?
Another different question:
I have modified the first query to execute immediately (called ToList before the second query). This time generated SQL is very simple as shown below (it doesn't look like there is a SQL query for the first LINQ statement with ToList() included):
SELECT [t0].[ID], [t0].[LastName], [t0].[FirstName], [t0].[HireDate], [t0]. [EnrollmentDate], [t0].[Discriminator], [t1].[DepartmentID], [t1].[Name], [t1]. [Budget], [t1].[StartDate], [t1].[InstructorID], [t1].[RowVersion]
FROM [Person] AS [t0]
INNER JOIN [Department] AS [t1] ON ([t0].[ID]) = [t1].[InstructorID]
Running this modified query also took varied amount of time but the difference is not as big as the first query set run.
In my application, there going to be lot of rows and I prefer first query set to second one but I am confused.
Please guide.
(Note: I have a little SQL Server knowledge so, I am using LINQPad to fine tune queries based on the performance)
Thanks

EF Pre Compile query and return of a scalar value

I use asp.net 4 c# and ef4.
I have this code, it should compile a query and return a single scalar value (I use anonymous type).
My code does not have apparently errors, but because is the first time I write a compiled query I would like to know if is well written or could be improved for a performance boost.
var query = CompiledQuery.Compile((CmsConnectionStringEntityDataModel ctx)
=> from o in ctx.CmsOptions
where o.OptionId == 7
select new
{
Value = o.Value
});
uxHtmlHead.Text = query(context).FirstOrDefault().Value;// I print the result in a Label
SQL Profile Output:
SELECT TOP (1)
[Extent1].[OptionId] AS [OptionId],
[Extent1].[Value] AS [Value]
FROM [dbo].[CmsOptions] AS [Extent1]
WHERE 7 = [Extent1].[OptionId]
Many Thanks
Result after Wouter advice (please guys have a double check again):
static readonly Func<CmsConnectionStringEntityDataModel, int, string> compiledQueryHtmlHead =
CompiledQuery.Compile<CmsConnectionStringEntityDataModel, int, string>(
(ctx, id) => ctx.CmsOptions.FirstOrDefault(o => o.OptionId == id).Value);
using (var context = new CmsConnectionStringEntityDataModel())
{
int id = 7;
uxHtmlHead.Text = compiledQueryHtmlHead.Invoke(context, id);
}
Resulting SQL (I do not understand why with a LEFT JOIN)
exec sp_executesql N'SELECT
[Project1].[Value] AS [Value]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT
[Extent1].[Value] AS [Value]
FROM [dbo].[CmsOptions] AS [Extent1]
WHERE [Extent1].[OptionId] = #p__linq__0 ) AS [Project1] ON 1 = 1',N'#p__linq__0 int',#p__linq__0=7

There are 2 things you can improve on.
First, precompiling a query is definitely a good idea but if you have a look at your code you will see that it precompiles the query each and every time instead of only once.
You need to move the precompiled query to a static variable that is initialized only once.
Another thing you need to be careful of is that when precompiling a query you shouldn't modify the query anymore before executing it.
You are building a precompiled query that will select all rows and then you say 'firstordefault' which changes the precompiled query to a SELECT TOP (1) and you lose the benefit of precompiling. You need to move the FirstOrDefault part inside your precompiled query and return only one result.
Have a look at this documentation. If you look at the examples you can see how they use a static field to hold the compiled query and how they specify the return value.