I'm trying to execute this query on my c# application, linked with SQL Server.
This is the query:
SELECT DISTINCT TOP(1) pil1.nome, pil1.cognome, pil1.scuderia, pil2.nome, pil2.cognome, pil2.scuderia, p1.punteggio AS punteggio1, p2.punteggio AS punteggio2, (p1.punteggio+p2.punteggio) AS punteggio_tot
FROM pilota pil1, pilota pil2, punti p1, punti p2
WHERE EXISTS
(SELECT *
FROM risultato r1, risultato r2
WHERE r1.pilota < r2.pilota
AND r1.posizione = p1.posizione
AND r2.posizione = p2.posizione
AND r1.data_gran_premio = #date
AND r2.data_gran_premio = #date
AND r1.pilota = pil1.codice
AND r2.pilota = pil2.codice)
AND pil1.scuderia = pil2.scuderia
AND pil1.codice <> pil2.codice
GROUP BY pil1.nome, pil2.nome, pil1.cognome, pil1.scuderia, pil2.cognome, pil2.scuderia, p1.punteggio, p2.punteggio
ORDER BY punteggio_tot ASC
The query is working properly cause I already checked it on workbrench and the output was correct, number and name of columns as well (to check it on workbrench i just changed TOP(1) in LIMIT 1).
But when I execute it on my application the name of the columns are different and the output is incomplete.
WORKBRECH OUTPUT: (correct)
SQL Server/ VISUAL STUDIO OUTPUT:
I am doing a join on two SQL functions using Entity Framework as my ORM. When the query gets executed I get this error message:
The query attempted to call 'Outer Apply' over a nested query,
but 'OuterApply' did not have the appropriate keys
This is my query:
var ingredientAllergenData = (from ings in db.fnListIngredientsFromItem(productId, (short)itemType, productId)
join ingAllergens in db.fnListAllergensFromItems(productId.ToString(CultureInfo.InvariantCulture), (short)itemType, currentLang)
on ings.id equals ingAllergens.ingredientId into ingAllergensData
from allergens in ingAllergensData.DefaultIfEmpty()
where ings.table == "tblIng" || ings.table == ""
select new {ings, allergens}).ToList();
I wrote the same query in LINQPad and I got back results, so I'm not sure what the issue is:
var ingredientAllergenData = (from ings in fnListIngredientsFromItem(1232, 0, 1232)
join ingAllergens in fnListAllergensFromItems("1232", 0, 1)
on ings.Id equals ingAllergens.IngredientId into ingAllergensData
from allergens in ingAllergensData.DefaultIfEmpty()
where ings.Table == "tblIng" || ings.Table == ""
select new {ings, allergens}).ToList();
The response from linqpad:
EDIT
This is the generated SQL query in LINQPad:
-- Region Parameters
DECLARE #p0 Int = 1232
DECLARE #p1 Int = 0
DECLARE #p2 Int = 1232
DECLARE #p3 VarChar(1000) = '1232'
DECLARE #p4 SmallInt = 0
DECLARE #p5 Int = 1
DECLARE #p6 VarChar(1000) = 'tblIng'
DECLARE #p7 VarChar(1000) = ''
-- EndRegion
SELECT [t0].[prodId] AS [ProdId], [t0].[id] AS [Id], [t0].[parent] AS [Parent], [t0].[name] AS [Name], [t0].[ing_gtin] AS [Ing_gtin], [t0].[ing_artsup] AS [Ing_artsup], [t0].[table] AS [Table], [t0].[quantity] AS [Quantity], [t2].[test], [t2].[prodId] AS [ProdId2], [t2].[ingredientId] AS [IngredientId], [t2].[allergenId] AS [AllergenId], [t2].[allergenName] AS [AllergenName], [t2].[level_of_containment] AS [Level_of_containment]
FROM [dbo].[fnListIngredientsFromItem](#p0, #p1, #p2) AS [t0]
LEFT OUTER JOIN (
SELECT 1 AS [test], [t1].[prodId], [t1].[ingredientId], [t1].[allergenId], [t1].[allergenName], [t1].[level_of_containment]
FROM [dbo].[fnListAllergensFromItems](#p3, #p4, #p5) AS [t1]
) AS [t2] ON [t0].[id] = ([t2].[ingredientId])
WHERE ([t0].[table] = #p6) OR ([t0].[table] = #p7)
I also tried hardcoding the same numbers into C# and got the same error again.
The problem is that Entity Framework needs to know what the primary key columns of the TVF results are to do a left join, and the default generated EDMX file does not contain that information. You can add the key value information by mapping the TVF results to an entity (instead of the default of mapping to a complex type).
The reason the same query works in LINQPad is that the default Data Context driver for connecting to a database in LINQPad uses LINQ to SQL (not Entity Framework). But I was able to get the query to run in Entity Framework (eventually).
I set up a local SQL Server database similar table-valued functions:
CREATE FUNCTION fnListIngredientsFromItem(#prodId int, #itemType1 smallint, #parent int)
RETURNS TABLE
AS
RETURN (
select prodId = 1232, id = 1827, parent = 1232, name = 'Ossenhaaspunten', ing_gtin = 3003210089821, ing_artsup=141020, [table] = 'tblIng', quantity = '2 K'
);
go
CREATE FUNCTION fnListAllergensFromItems(#prodIdString varchar(1000), #itemType2 smallint, #lang int)
RETURNS TABLE
AS
RETURN (
select prodId = '1232', ingredientId = 1827, allergenId = 11, allergenName = 'fish', level_of_containment = 2
union all
select prodId = '1232', ingredientId = 1827, allergenId = 16, allergenName = 'tree nuts', level_of_containment = 2
union all
select prodId = '1232', ingredientId = 1827, allergenId = 12, allergenName = 'crustacean and shellfish', level_of_containment = 2
);
go
And I created a test project using Entity Framework 6.1.2 and generated an EDMX file from the database using the Entity Data Model Designer in Visual Studio 2013. With this setup, I was able to get the same error when trying to run that query:
System.NotSupportedException
HResult=-2146233067
Message=The query attempted to call 'OuterApply' over a nested query, but 'OuterApply' did not have the appropriate keys.
Source=EntityFramework
StackTrace:
at System.Data.Entity.Core.Query.PlanCompiler.NestPullup.ApplyOpJoinOp(Op op, Node n)
at System.Data.Entity.Core.Query.PlanCompiler.NestPullup.VisitApplyOp(ApplyBaseOp op, Node n)
at System.Data.Entity.Core.Query.InternalTrees.BasicOpVisitorOfT`1.Visit(OuterApplyOp op, Node n)
...
Running an alternate expression for a left join resulted in a slightly different error:
var ingredientAllergenData = (db.fnListIngredientsFromItem(1323, (short)0, 1)
.GroupJoin(db.fnListAllergensFromItems("1232", 0, 1),
ing => ing.id,
allergen => allergen.ingredientId,
(ing, allergen) => new { ing, allergen }
)
).ToList();
Here is a truncated stacktrace from the new exception:
System.NotSupportedException
HResult=-2146233067
Message=The nested query does not have the appropriate keys.
Source=EntityFramework
StackTrace:
at System.Data.Entity.Core.Query.PlanCompiler.NestPullup.ConvertToSingleStreamNest(Node nestNode, Dictionary`2 varRefReplacementMap, VarList flattenedOutputVarList, SimpleColumnMap[]& parentKeyColumnMaps)
at System.Data.Entity.Core.Query.PlanCompiler.NestPullup.Visit(PhysicalProjectOp op, Node n)
at System.Data.Entity.Core.Query.InternalTrees.PhysicalProjectOp.Accept[TResultType](BasicOpVisitorOfT`1 v, Node n)
...
Entity Framework is open source, so we can actually look at the source code where this exception is thrown. The comments in this snippet explains what the problem is (https://entityframework.codeplex.com/SourceControl/latest#src/EntityFramework/Core/Query/PlanCompiler/NestPullup.cs):
// Make sure that the driving node has keys defined. Otherwise we're in
// trouble; we must be able to infer keys from the driving node.
var drivingNode = nestNode.Child0;
var drivingNodeKeys = Command.PullupKeys(drivingNode);
if (drivingNodeKeys.NoKeys)
{
// ALMINEEV: In this case we used to wrap drivingNode into a projection that would also project Edm.NewGuid() thus giving us a synthetic key.
// This solution did not work however due to a bug in SQL Server that allowed pulling non-deterministic functions above joins and applies, thus
// producing incorrect results. SQL Server bug was filed in "sqlbuvsts01\Sql Server" database as #725272.
// The only known path how we can get a keyless drivingNode is if
// - drivingNode is over a TVF call
// - TVF is declared as Collection(Row) is SSDL (the only form of TVF definitions at the moment)
// - TVF is not mapped to entities
// Note that if TVF is mapped to entities via function import mapping, and the user query is actually the call of the
// function import, we infer keys for the TVF from the c-space entity keys and their mappings.
throw new NotSupportedException(Strings.ADP_KeysRequiredForNesting);
}
That explains the path that leads to that error, so anything we can do to get off that path should fix the problem. Assuming we have to do that left join on the results of a table-valued function, one option (maybe the only option?) is to map the results of the TVF to an entity that has a primary key. Then Entity Framework will know the key values of the TVF results based on the mapping to that entity, and we should avoid these errors related to missing keys.
By default when generating an EDMX file from the database, a TVF is mapped to a complex type. There are instructions for how to change it at https://msdn.microsoft.com/en-us/library/vstudio/ee534438%28v=vs.100%29.aspx.
In my test project, I added an empty table with a schema that matched the output of the TVFs to get the model designer to generate Entities, then I went to the model browser and updated the function imports to return a collection of these entities (instead of the auto-generated complex types). After making these changes, that same LINQ query ran without errors.
var ingredientAllergenData = (from ings in db.fnListIngredientsFromItem(productId, (short)itemType, productId)
join ingAllergens in db.fnListAllergensFromItems(productId.ToString(CultureInfo.InvariantCulture), (short)itemType, currentLang)
on ings.id equals ingAllergens.ingredientId into ingAllergensData
from allergens in ingAllergensData.DefaultIfEmpty()
where ings.table == "tblIng" || ings.table == ""
select new {ings, allergens}).ToList();
Here is the trace SQL that the query gave me:
SELECT
1 AS [C1],
[Extent1].[prodId] AS [prodId],
[Extent1].[id] AS [id],
[Extent1].[parent] AS [parent],
[Extent1].[name] AS [name],
[Extent1].[ing_gtin] AS [ing_gtin],
[Extent1].[ing_artsup] AS [ing_artsup],
[Extent1].[table] AS [table],
[Extent1].[quantity] AS [quantity],
[Extent2].[prodId] AS [prodId1],
[Extent2].[ingredientId] AS [ingredientId],
[Extent2].[allergenId] AS [allergenId],
[Extent2].[allergenName] AS [allergenName],
[Extent2].[level_of_containment] AS [level_of_containment]
FROM [dbo].[fnListIngredientsFromItem](#prodId, #itemType1, #parent) AS [Extent1]
LEFT OUTER JOIN [dbo].[fnListAllergensFromItems](#prodIdString, #itemType2, #lang) AS [Extent2] ON ([Extent1].[id] = [Extent2].[ingredientId]) OR (([Extent1].[id] IS NULL) AND ([Extent2].[ingredientId] IS NULL))
WHERE [Extent1].[table] IN ('tblIng','')
Entity Framework 4 is generating a SQL statement from a fairly innocuous linq query that includes the standard parameter variables, however the numeric parameter suffix is incrementing continually for each successive run of the query. The calling code is not being run in any asynchronous or transactional manner.
This only seems to happening in our production environment and I cannot replicate it in the development environment, wherein each run of the code that calls this query generates parameters #p__linq__0 and #p__linq__1
Sample linq:
var myJob =
(from job in this.context.Jobs
where job.Contract.ContractNo == contractNo && job.JobNo == jobNo
select job).FirstOrDefault();
Sample sql queries being generated:
SELECT ... WHERE ([Extent2].[ContractNo] = #p__linq__1799) AND ([Extent1].[JobNo] = #p__linq__1800) ) AS [Limit1]
SELECT ... WHERE ([Extent2].[ContractNo] = #p__linq__8262) AND ([Extent1].[JobNo] = #p__linq__8263) ) AS [Limit1]
SELECT ... WHERE ([Extent2].[ContractNo] = #p__linq__3590) AND ([Extent1].[JobNo] = #p__linq__3591) ) AS [Limit1]
SELECT ... WHERE ([Extent2].[ContractNo] = #p__linq__788) AND ([Extent1].[JobNo] = #p__linq__789) ) AS [Limit1]
SELECT ... WHERE ([Extent2].[ContractNo] = #p__linq__2250) AND ([Extent1].[JobNo] = #p__linq__2251) ) AS [Limit1]
I can't work out why these numbers are increasing, and the problem with the parameter names changing like this is that it's causing SQL Server to generate a new execution plan for each query.
I have a SQL query that I am trying to convert to LINQ:
SELECT * FROM TABLE1
WHERE LICENSE_RTK NOT IN(
SELECT KEY_VALUE FROM TABLE2
WHERE REFERENCE_RTK = 'FOO')
So I wrote one query for inner query and then one query for the outer one and used Except:
var insideQuery = (from pkcr in this.Repository.Context.TABLE2 where pkcr.Reference_RTK == "FOO" select pkcr.Key_Value);
var outerQuery = (from pl in this.Repository.Context.TABLE1 select pl).Except(insideQuery);
But this is wrong. Cannot even compile it. What is the correct way of writing this?
You cannot compile second query, because Except should be used on Queryables of same type. But you are trying to apply it on Queryable<TABLE1> and Queryable<TypeOfTABLE2Key_Value>. Also I think you should use Contains here:
var keys = from pkcr in this.Repository.Context.TABLE2
where pkcr.Reference_RTK == "FOO"
select pkcr.Key_Value;
var query = from pl in this.Repository.Context.TABLE1
where !keys.Contains(pl.License_RTK)
select pl;
NOTE: Generated query will be NOT EXISTS instead of NOT IN, but that's what you want
SELECT * FROM FROM [dbo].[TABLE1] AS [Extent1]
WHERE NOT EXISTS
(SELECT 1 AS [C1]
FROM [dbo].[TABLE2] AS [Extent2]
WHERE ([Extent2].[Reference_RTK] == #p0) AND
([Extent2].[Key_Value] = [Extent1].[License_RTK]))
I use asp.net 4 c# and ef4.
I have this code, it should compile a query and return a single scalar value (I use anonymous type).
My code does not have apparently errors, but because is the first time I write a compiled query I would like to know if is well written or could be improved for a performance boost.
var query = CompiledQuery.Compile((CmsConnectionStringEntityDataModel ctx)
=> from o in ctx.CmsOptions
where o.OptionId == 7
select new
{
Value = o.Value
});
uxHtmlHead.Text = query(context).FirstOrDefault().Value;// I print the result in a Label
SQL Profile Output:
SELECT TOP (1)
[Extent1].[OptionId] AS [OptionId],
[Extent1].[Value] AS [Value]
FROM [dbo].[CmsOptions] AS [Extent1]
WHERE 7 = [Extent1].[OptionId]
Many Thanks
Result after Wouter advice (please guys have a double check again):
static readonly Func<CmsConnectionStringEntityDataModel, int, string> compiledQueryHtmlHead =
CompiledQuery.Compile<CmsConnectionStringEntityDataModel, int, string>(
(ctx, id) => ctx.CmsOptions.FirstOrDefault(o => o.OptionId == id).Value);
using (var context = new CmsConnectionStringEntityDataModel())
{
int id = 7;
uxHtmlHead.Text = compiledQueryHtmlHead.Invoke(context, id);
}
Resulting SQL (I do not understand why with a LEFT JOIN)
exec sp_executesql N'SELECT
[Project1].[Value] AS [Value]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT
[Extent1].[Value] AS [Value]
FROM [dbo].[CmsOptions] AS [Extent1]
WHERE [Extent1].[OptionId] = #p__linq__0 ) AS [Project1] ON 1 = 1',N'#p__linq__0 int',#p__linq__0=7
There are 2 things you can improve on.
First, precompiling a query is definitely a good idea but if you have a look at your code you will see that it precompiles the query each and every time instead of only once.
You need to move the precompiled query to a static variable that is initialized only once.
Another thing you need to be careful of is that when precompiling a query you shouldn't modify the query anymore before executing it.
You are building a precompiled query that will select all rows and then you say 'firstordefault' which changes the precompiled query to a SELECT TOP (1) and you lose the benefit of precompiling. You need to move the FirstOrDefault part inside your precompiled query and return only one result.
Have a look at this documentation. If you look at the examples you can see how they use a static field to hold the compiled query and how they specify the return value.