Develop Reference

Entity Framework - extremely long execution time for stored procedure

I'm following up on my question yesterday, Entity Framework 6 get complext return value from a stored procedure. My stored procedure now runs under entity framework. However, it times out after 3 minutes, the connection time out.
I run the stored procedure in my SQL Server Management Studio with the line (customer information omitted):
EXEC spGetDupWOs #ProjectName=N'...', #City=N'...', #State=N'LA', #ProposalNum=N'201703080740-001', #County=N'...', #Owner=N'...', #QuoteRecipients=N'...', #ProjectID=-1
It executes in less than a second. When Entity framwork executes it, it takes forever.
Using the SQL Server Profiler, I determined that Entity Framework is sending this line to the SQL server:
exec sp_executesql N'EXEC spGetDupWOs',N'#ProjectName nvarchar(19),#City nvarchar(6),#State nvarchar(2),#ProjectNum nvarchar(12),#County nvarchar(10),#Owner nvarchar(23),#QuoteRecipients nvarchar(23),#ProjectID bigint',#ProjectName=N'...',#City=N'Holden',#State=N'LA',#ProposalNum=N'201703080740-001',#County=N'Livingston',#Owner=N'...',#BID_RECIP=N'...',#ProjectID=-1
When I run this in SSMS, it takes forever to run.
Reading the similar questions it looks like the issue is Parameter Sniffing and a change in execution plan.
Here is my call to execute the stored procedure in my application:
List<DuplicateProposals> duplicateCheckResults =
db.Database.SqlQuery<DuplicateProposals>("spGetDupWOs",
spl.ToArray())
.ToList();
After reading a bunch of articles online, I'm even more confused. How can I change my call to resolve this?

The issue identified is parameter sniffing in SQL Server. There are multiple approaches to handle this, but the most optimal for your scenario depends on your real use case, utilization, etc.
Here are some options.
Recompile the stored procedure with every execution. This may become very heavy CPU utilization, and is typically overkill. I would NOT recommend this option unless you have a very good reason. To implement: Use
WITH RECOMPILE or the OPTION(RECOMPILE) hint at the end of your query.
Optimize for hint. This can be a workaround for the parameter sniffing, but may result in a subpar execution plan for all of your queries. Typically, not an optimal approach. To implement: Use OPTION(OPTIMIZE FOR UNKNOWN)
Copy parameter to a local variable. Was more common in older versions of SQL Server. To implement: Declare a local variable, then copy the value from your input parameter to your local variable. DECLARE #local_var1 char(1) = #InputParam1;
Turn off parameter sniffing at query level. This approach uses the QUERYTRACEON hint. This may be the most optimal approach for this case scenario. I would recommend to explore this option as a primary strategy. To implement: add OPTION(QUERYTRACEON 4136) to the end of your query.
Example:
SELECT * FROM dbo.MyTable T
WHERE T.Col1 = #Param1 and T.Col2 = #Param2
OPTION(QUERYTRACEON 4136)

I ended up having to convert the entire call into a single string that I passed to the SqlQuery function.
string sql = string.Format("exec spGetDupWOs #ProjectName=N'{0}',#City=N'{1}',#State=N'{2}',#ProjectNumber=N'{3}',#County=N'{4}',#Owner=N'{5}',#QuoteRecipients=N'{6}',#ProjectID={7}",
project.ProjectName,
project.City,
project.State,
project.ProjectNumber,
project.County,
project.Owner,
quoteRecipientsList,
"null");
Yes, I had to include the N prefix to the strings to make it work, I'm not sure why but it worked.
Thanks for all of the help everyone. I could not have solved this without your help.

Stored procedure returns int instead of result set

I have a stored procedure that contains dynamic select. Something like this:
ALTER PROCEDURE [dbo].[usp_GetTestRecords]
--#p1 int = 0,
--#p2 int = 0
#groupId nvarchar(10) = 0
AS
BEGIN
SET NOCOUNT ON;
DECLARE #query NVARCHAR(max)
SET #query = 'SELECT * FROM CUSTOMERS WHERE Id = ' + #groupId
/* This actually contains a dynamic pivot select statement */
EXECUTE(#query);
END
In SSMS the stored procedure runs fine and shows result set.
In C# using Entity Framework it shows returning an int instead of IEnumerable?
private void LoadTestRecords()
{
TestRecordsDBEntities dataContext = new TestRecordsDBEntities();
string id = ddlGroupId.SelectedValue;
List<TestRecord> list = dataContext.usp_GetTestRecords(id); //This part doesn't work returns int
GridView1.DataSource = list;
}
Generated function for usp_GetTestRecords
public virtual int usp_GetTestRecords(string groupId)
{
var groupIdParameter = groupId != null ?
new ObjectParameter("groupId", groupId) :
new ObjectParameter("groupId", typeof(string));
return ((IObjectContextAdapter)this).ObjectContext.ExecuteFunction("usp_GetTestRecords", groupIdParameter);
}

I get this when I have a stored procedure that includes an "exec" call into a temporary table, such as:
insert into #codes (Code, ActionCodes, Description)
exec TreatmentCodes_sps 0
It appears that Entity Framework gets confused as to what should be returned by the procedure. The solution I've come across is to add this at the top of the sproc:
SET FMTONLY OFF
After this, all is well.

I got the same problem, and found solution here
Move to your .edmx
At Model Browser Window/Function Imports find your procedure then double click it
Change the return type to you want
Save .edmx and check the return type again.
It should be what you need now.

Entity Framework can't tell what your stored procedure is returning. I've had success creating a table variable that mirrors the data from your SELECT statement. Just insert into the table variable then do a select from that table variable. EF should pick it up.

See Ladislav Mrnka's answer in this Stack Overflow post
https://stackoverflow.com/a/7131344/4318324
I had the same basic problem.
Adding
SET FMTONLY OFF
To a procedure you are trying to import during the import will address this problem.
It's a good practice to remove the line afterwards unless the purpose of the database is solely to provide schema for EF (Entity Framework).
The main reason for caution is that EF uses this setting to prevent data mutations when trying to obtain metadata.
If you refresh your entity model from a database any procedures with this line in them can potentially update the data in that database just by trying to obtain the schema.
I wanted to add a further note on this so it's not needed to fully scan through the other link.
if you want to try to use FMTONLY here are a couple things to keep in mind.
when FMTONLY is on:
1) only the schema is returned (no) rows.
similar to adding a blanket false statement to your where clause (ie "where 1=0")
2) flow control statements are ignored
Example
set fmtonly on
if 1=1
begin
select 1 a
end
else
begin
select 1 a,2 b
end
while 1=1
select 1 c
The above returns NO rows whatsoever and the metadata for each of the three queries
For this reason some people suggest toggling it off in a way that takes advantage of it's non-observance of flow control
if 1=0
begin
set fmtonly off
end
In fact you could use this to introduce logic that tracks this
set fmtonly off
declare #g varchar(30)
set #g = 'fmtonly was set to off'
if 1=0
begin
set fmtonly off
set #g = 'fmtonly was set to on'
end
select #g
Think VERY CAREFULLY before trying to use this feature as it is both deprecated and potentially makes sql extremely hard to follow
the MAIN concepts that need to be understood are the following
1. EF turns FMTONLY on to prevent MUTATING data from executing stored procedures
when it executes them during a model update.
(from which it follows)
2. setting FMTONLY off in any procedure that EF will attempt to do a schema scan
(potentially ANY and EACHONE) introduces the potential to mutate database
data whenever *anyone* attempts to update their database model.

Entity Framework will automatically return a scalar value if your stored procedure doesn't have a primary key in your result set. Thus, you'd have to include a primary key column in your select statement, or create a temp table with a primary key in order for Entity Framework to return a result set for your stored procedure.

I had the same problem, I changed the name of return fields by 'AS' keyword and addressed my problem. One reason for this problem is naming column names with SQL Server reserved keywords.
The example is fallows:
ALTER PROCEDURE [dbo].[usp_GetProducts]
AS
BEGIN
SET NOCOUNT ON;
SELECT
, p.Id
, p.Title
, p.Description AS 'Description'
FROM dbo.Products AS p
END

Best solution I found is to cheat a little.
In the store procedure, comment everything, put a first line with a select [foo]='', [bar]='' etc...
Now update the model, go to the mapped function, select complex type and click on Get Column Information and then Create Complex Type.
Now comment the fake select and un-comment the real store procedure body.

When you generated your model class for your stored procedure, you chose scalar return result by mistake. you should remove your stored procedure from your entity model, then re-add the stored procedure. In the dialog for the stored procedure, you can choose the return type you are expecting. Do not just edit the generated code.. this may work now, but the generated code can be replaced if you make other changes to your model.

I have pondered this a bit and I think I have a better/simpler answer
If you have a complex stored that gives entity framework some difficultly (for current versions of Entity Framework that are using the FMTONLY tag to aquire schema)
consider doing the folowing at the beginning of your stored procedure.
--where [columnlist] matches the schema you want EF to pick up for your stored procedure
if 1=0
begin
select
[columnlist]
from [table list and joins]
where 1=0
end
if you are okay loading your result set into a table variable
you can do the following to help keep your schema in sync
declare #tablevar as table
(
blah int
,moreblah varchar(20)
)
if 1=0
begin
select * from #tablevar
end
...
-- load data into #tablevar
select * from #tablevar

If you need to do this, then you might be better off just making a partial of the dbcontext and creating the C# function yourself that will use SqlQuery to return the data you need. Advantages over some of the other options is:
Don't have to change anything when the model updates
Won't get overwritten if you do it directly in the generated class (someone above mention this as if it's an option :) )
Don't have to add anything to the proc itself that could have side effects now or later on
Example Code:
public partial class myEntities
{
public List<MyClass> usp_GetTestRecords(int _p1, int _p2, string _groupId)
{
// fill out params
SqlParameter p1 = new SqlParameter("#p1", _p1);
...
obj[] parameters = new object[] { p1, p2, groupId };
// call the proc
return this.Database.SqlQuery<MyClass>(#"EXECUTE usp_GetTestRecords #p1, #p2, #groupId", parameters).ToList();
}
}

Just change to
ALTER PROCEDURE [dbo].[usp_GetTestRecords]
--#p1 int = 0,
--#p2 int = 0
#groupId nvarchar(10) = 0
AS
BEGIN
SET NOCOUNT ON;
SELECT * FROM CUSTOMERS WHERE Id = #groupId
END

I know this is an old thread but in case someone has the same problems I'll tell my woes.
As a help to find the issue, run sql profiler when you add your stored proc. Then you can see what entity framework is passing as parameters to generate your resultset. I imagine nearly always it will pass null parameter values. If you are generating sql on the fly by concatenating string values and parameter values and some are null then the sql will break and you wont get a return set.
I haven't needed to generate temp tables or anything just an exec command.
Hope it helps

During import
SET FMTONLY ON
can be used for taking the sp schema.
If you change the sp and want to update the new one, you should delete the old defined function from edmx file (from xml), because although deleting sp from model browser, it is not deleted in edmx. For example;
<FunctionImport Name="GetInvoiceByNumber" ReturnType="Collection(Model.Invoice_Result)">
<Parameter Name="InvoiceNumber" Mode="In" Type="Int32" />
</FunctionImport>
I had the same problem, and when I delete the FuctionImport tag of corresponding sp totally, the model updated right. You can find the tag by searching the function name from visual studio.

You may have luck opening up the model browser, then going to Function Imports, double clicking the stored procedure in question and then manually clicking "Get Column Information" and then clicking "Create New Complex Type". This usually sorts out the problem.

Well I had this issue as well but after hours of online searching none of above methods helped.
Finally I got to know that It will happen if your store procedure is getting some parameters as null and which generate any error in query execution.
Entity Framework will generate method for store procedure by defining the complex entity model. Due to that null value your store procedure will return and int value.
Please check your store procedure either its providing empty result set with null values. It will fix your problem. Hopefully.

I think this is a problem of permissions on the database, I don't know what exactly could be, but, in my job we use Active Directory users to grant applications connect to databases, this accounts are specially created for the applications, each app has its own user account, well, as a developers I have permissions for read, write and other basic things, no alter, and no advanced features, and I have this same problem running Visual Studio with my normal account, then, what I did was to open Visual Studio selecting the option "as a different user" on the context menu, and I put the AD login granted for the application and voila!, now my Stored Procedures are loading with all the fields I was expected, before that, my Stored Procedures was returning as int. I hope this help someone, maybe the VIEW DEFINITION permissions on database account do the trick

If SQL Authentication is in place, verify that the user credential that is being used to connect Entity Framework to the database has the proper rights to read from CUSTOMERS table.
When Entity Framework uses SQL Authentication to map complex objects (i.e stored procedures that SELECTs more than one column), if any of the tables from within such stored procedure don't have set up the Read permission, the mapping will result in returning INT instead of the desired Result set.

What's the best practice in executing a stored procedure with multiple parameters using MVC3?

I am working at implementing a search feature into my MVC3 application. I'm looking to pass two parameters into and execute a stored procedure that will look basically something like this:
create procedure MyProc
(
#FirstParam nvarchar(50),
#SecondParam nvarchar(20)
)
as select * from MyTable where #FirstParam like #SecondParam
MyTable has about 30 fields that will be returned for each object and I need to create a procedure like this for several tables, so I am trying to avoid using a SqlDataReader and converting the returned Sql data to C#.
I would like to use something like this method but I am not sure if this can be done with multiple parameters.
Ideally I would like to use EF4, but I have not found any good information on executing stored procedures while using EF4.
Any insight on the most painless way and/or best practice for executing this task will be greatly appreciated.

My sugestion is use dynamic linq (and here, and here). You can pass valid linq expressions as regular strings:
var column = "Name";
var value = "Marvin";
var query = DbCtx.MyEntity.Where("{0} == #1", columnName, value);
The benefits (IMO) is that you can keep the search logic in the application and, if you need to do this for many tables, you can create a T4 template to generate the bootstrap code for you.

What you are suggesting can indeed be done through parameters, and you should be using an ORM like EF4 for you data access. Like most ORM that have support for stored procedure, you can indeed pass multiple parameters to the stored procedure.
The issue you will find, however, is that you can't have dynamic column names in SQL Server (or any other SQL database that I am aware of) - you can't give a column name in a variable.
You will need to use dynamic SQL to achieve this, either within the stored procedure or otherwise.

Dataset times out while query runs immediately

I am running a query directly, it is trivial in nature:
SELECT * FROM [dbo].[vwUnloadedJobDetailsWithData] WHERE JobId = 36963
When I run this from Management studio the query doesn't even take a second. When I run it from within the table adapter it times out. I have fixed this multiple times, but the fix is ludicrous. If I delete the table adapter from my xsd file and recreate it the query time matches that of management studio for about two days, but I have to redeploy which is asinine.
Any insight into what could be causing this would be greatly appreciated. I've seen another question about this but the solution involving set arithabort on before the query had no effect for me.
Edit: It was asked that I show my code for calling the query. Now this happens when I go into my xsd file and just do preview data as well, but for sake of clarity, here it is:
using (TEAMSConnection connection = new TEAMSConnection())
{
connection.OpenConnection();
_JobDetailsDAO jobDetailDao= new _JobDetailsDAO(connection);
return jobDetailDao.GetUnloadedJobDetailsByJobId(jobId);
}
On disposal of connection the database connection is closed. using this line of code:
if (_DBConnection != null && _DBConnection.State == ConnectionState.Open)
_DBConnection.Close();
Edit2: I ran a trace and here are the set options that are being set
set quoted_identifier on
set arithabort off
set numeric_roundabort off
set ansi_warnings on
set ansi_padding on
set ansi_nulls on
set concat_null_yields_null on
set cursor_close_on_commit off
set implicit_transactions off
set language us_english
set dateformat mdy
set datefirst 7
set transaction isolation level read committed
I went and added that to the query that I generated in management studio and it still ran in less than a second. I even copied the query exactly as in the trace.
exec sp_executesql N'SELECT * FROM [dbo].[vwUnloadedJobDetailsWithData] WHERE JobID = #JobId',N'#JobId int',#JobId=36963
and it is still less than a second return time. I am so very confused.
Thanks,
Josh

the most likely scenarion why this would be happening is the difference in SET options between ssms and ado.net. that difference causes (re)building of execution plans that might not be optimal.

Alright, well I could not find any solution that would continue to allow me to use the dataset, so I went straight to using the SqlDataAdapter in code rather than using the auto generated TableAdapters.
According to the trace it performs the exact same query, but so far it works. It may not in two days, but for now it works it seems.

Just trying to think loudly:
Maybe there is a lock caused by another process/person? Is there anybody who updates the same row at the same time? Is there anybody who opens the table from Management studio or Query Analyzer with Open Table feature and plays with the filters?
Try looking for locks using sp_who2

Some thoughts:
What I'd call parameter sniffing for stored proc. Try the OPTION (RECOMPILE) hint, so your sent SQL looks like this:
exec sp_executesql
N'SELECT *
FROM [dbo].[vwUnloadedJobDetailsWithData]
WHERE JobID = #JobId
OPTION (RECOMPILE)',
N'#JobId int',
#JobId=36963
Explanation: When a query plan is produced and cached, it may be a bad, atypical value. Say JobID is usually very selective, but for that one execution it's not. When you run the query the next plan the cached plan is wrong for the next selective JobId. A plan will be recompiled for various reasons, but the value on recompilation matters.
Otherwise, what is the exact datatype of Jobid? If it's smallint, then the column will be converted to int in the parameterised query. When using a constant it will be smallint. Make sure the type is defined correctly: this matters in SQL code.

Determine whether to generate ExecuteNonQuery or ExecuteReader from schema

I am attempting to create a stored procedure/ADO.NET mapping mechanism where a stored procedure with parameters becomes
object MyStoredProcedure.Execute(out returnValue, param1, param2, ...)
The problem comes when trying to generate the actual data retrieval method. I can easily obtain most of the schema information I need from the Information Schema views, but I can't reliably find what type of return (output param vs. SELECT/SqlDataReader vs. both) should come from the procedure and whether to call ExecuteNonQuery or ExecuteReader.
Worst-case, I can probably parse the procedure's text, but there are all kinds of funky things that could go wrong there.
The reason for the code generation is that the application database contains many hundreds of stored procedures. And we inherited this project, so there is no way to wrap our heads around that many procs that we didn't create.
I actually have two main goals for the ADO.NET generation:
1) Remove all string literals (stored proc names in SqlCommand creation, parameter names in SqlParameter creation, etc.) from the application. This way, when a procedure or database schema changes, we can regenerate the ADO.NET wrappers, and any errors resulting from those changes will be caught at compile time.
2) Remove the need to dig through a proc to determine params, returns types, etc. So basically, the database itself becomes an API, with all of the internal stored procedure details abstracted away.

Yup; that isn't easy. For simple cases you can try running the sp (awooga!) passing nulls for all the parameters, and using SET FMTONLY ON - a bit risky (extended sprocs are stll executed, for example) and not robust since the TSQL could branch on the input. But an option.
The "out" ets should be available via metadata; the "old" way would be syscolumns (there is probably an info-schema alternative for doing it the right way).
Just as an update; if you want the database to describe itself as an API, perhaps consider UDFs for the selects; advantages:
the metadata for the return values is rigid and easy to query
it is composable at the caller
Or; just use an ORM. LINQ-to-SQL will work happily with this type of setup (including composability); Entity Framework will certainly do all that hard work for you for stored procedures. Or any of the others; NHibernate, LLBLGen Pro, etc. They have all tackled this exact problem. It isn't trivial; why re-invent it?

I don't think you should be worried about it.
Wouldn't it be better to provide overloads & let the user decide which method to call?
Let me know, if I have not understood the question correctly.

Assuming that all of your stored procedures are consistent, that is, each returns at most one result-set, with the same column-set irrespective of it's parameter values or the data-state of the database, and all OUTPUT parameters are always written to, ..
And also assuming that this is a development or at latest a build-time tool (and not a run-time tool) and that you have some control over Srored Procedure's content, .. THEN here is what I would try:
Institute a retroactive Stored Procedure standard that requires all sProcs to have a comment of the following form, that must work correctly in the test (or development?) database:
'TEST: EXEC spThisProc [, ...]
Write your tool to extract the list of stored procedures, their parameters, data types and output parameters from the system catalogs (you can use the INFORMATION_SCHEMA tables ROUTINES and PARAMETERS for this).
Also retrieve the script for all sProcs (from the ROUTINE_DEFINITON column of the INFORMATION_SCHEMA.ROUTINES table, for one place), then extract the text of the "'TEST:" command (above) and ExecuteReader that command against the Test database. Then test the Resultset returned to see if it contains any datasets (or is it "Tables"?). If not, then mark this sProc as needing ExecuteNonQuery, otherwise, extract the column definitions of the returned dataset and use than to generate the corresponding ExecuteReader code for your Class definitions.

hope this could help retrieving the type of params
select s.id, s.name, t.name as [type], t.length,s.isoutparam
from syscolumns s inner join systypes t
on s.xtype = t.xtype
where id = (select id from sysobjects where name = 'SP NAME')
And check out this excellent link - get the list of columns returned from the SQL select
this is a sample query
USE tempdb
GO
IF OBJECT_ID('dbo.TestSP') IS NOT NULL
DROP PROCEDURE dbo.TestSP;
GO
IF NOT EXISTS (SELECT * FROM sys.servers WHERE name = 'Loopback')
BEGIN
EXEC sp_addlinkedserver #server = 'Loopback', #srvproduct = '',
#provider = 'SQLOLEDB', #datasrc = ##servername
END
GO
CREATE PROC dbo.TestSP
AS
SELECT 1 as [ID], 'Name' as [Name], 'Boston' as [City]
GO
DECLARE #MyXML XML
SELECT #MyXML = (SELECT * FROM
(SELECT *
FROM OPENQUERY(Loopback,'SET FMTONLY ON;
EXEC tempdb.dbo.TestSP;
SET FMTONLY OFF')) vw FOR XML AUTO, XMLSCHEMA)
SELECT #MyXML
SET FMTONLY--> It will allow you to run a SP, but only get back metadata

Also, you can try looking at SqlServer SMO http://msdn.microsoft.com/en-us/library/ms162169.aspx for querying information on your database without having to write queries against the information schema directly.