Some friends and I are making a system based on C#, they make the Desktop application with CodeFirst and I make the Web application from their CodeFirst, but with DataBaseFirst, add DataBase.
It turns out that when I am going to add the model, I select the tables and start generating them, but it takes at least two hours.
I need your help because it takes a lot of time and I can not find a solution online.
As discussed in this question, it seems to be a bug in reverse engineering using SQL Server.
This issue was fixed by a cumulative update or a service pack, depending on the version you are running on.
The update is available here
I've got Entity Framework 4.1 with .NET 4.5 running on ASP.NET in Windows 2008R2. I'm using EF code-first to connect to SQL Server 2008R2, and executing a fairly complex LINQ query, but resulting in just a Count().
I've reproduced the problem on two different web servers but only one database (production of course). It recently started happening with no application, database structure, or server changes on the web or database side.
My problem is that executing the query under certain circumstances takes a ridiculous amount of time (close to 4 minutes). I can take the actual query, pulled from SQL Profiler, and execute in SSMS in about 1 second. This is consistent and reproducible for me, but if I change the value of one of the parameters (a "Date after 2015-01-22" parameter) to something earlier, like 2015-01-01, or later like 2015-02-01, it works fine in EF. But I put it back to 2015-01-22 and it's slow again. I can repeat this over and over again.
I can then run a similar but unrelated query in EF, then come back to the original, and it runs fine this time - same exact query as before. But if I open a new browser, the cycle starts over again. That part also makes no sense - we're not doing anything to retain the data context in a user session, so I have no clue whatsoever why that comes into play.
But this all tells me that the data itself is fine.
In Profiler, when the query runs properly, it takes about a second or two, and shows about 2,000,000 in reads and about 2,000 in CPU. When it runs slowly, it takes 3.5 minutes, and the values are 300,000,000 and 200,000 - so reads are about 150 times higher and CPU is 100 times higher. Again, for the identical SQL statement.
Any suggestions on what EF might be doing differently that wouldn't show up in the query text? Is there some kind of hidden connection property which might cause a different execution plan in certain circumstances?
EDIT
The query that EF builds is one of the ones where it builds a giant string with the parameter included in the text, not as a SQL parameter:
exec sp_executesql
N'SELECT [GroupBy1].[A1] AS [C1]
FROM (
SELECT COUNT(1) AS [A1]
...
AND ([Extent1].[Added_Time] >= convert(datetime2, ''2015-01-22 00:00:00.0000000'', 121))
...
) AS [GroupBy1]'
EDIT
I'm not adding this as an answer since it doesn't actually address the underlying issue, but this did end up getting resolved by rebuilding indexes and recomputing statistics. That hadn't been done in longer than usual, and it seems to have cleared up whatever caused the issue.
I'll keep reading up on some of the links here in case this happens again, but since it's all working now and unreproduceable, I don't know if I'll ever know for sure exactly what it was doing.
Thanks for all the ideas.
I recently had a very similar scenario, a query would run very fast executing it directly in the database, but had terrible performance using EF (version 5, in my case). It was not a network issue, the difference was from 4ms to 10 minutes.
The problem ended up being a mapping problem. I had a column mapped to NVARCHAR, while it was VARCHAR in the database. Seems inoffensive, but that resulted in an implicit conversion in the database, which totally ruined the performance.
I'm not entirely sure on why this happens, but from the tests I made, this resulted in the database doing an Index Scan instead of an Index Seek, and apparently they are very different performance-wise.
I blogged about this here (disclaimer: it is in Portuguese), but later I found that Jimmy Bogard described this exact problem in a post from 2012, I suggest you check it out.
Since you do have a convert in your query, I would say start from there. Double check all your column mappings and check for differences between your table's column and your entity's property. Avoid having implicit conversions in your query.
If you can, check your execution plan to find any inconsistencies, be aware of the yellow warning triangle that may indicate problems like this one about doing implicit conversion:
I hope this helps you somehow, it was a really difficult problem for us to find out, but made sense in the end.
Just to put this out there since it has not been addressed as a possibility:
Given that you are using Entity Framework (EF), if you are using Lazy Loading of entities, then EF requires Multiple Active Result Sets (MARS) to be enabled via the connection string. While it might seem entirely unrelated, MARS does sometimes produce this exact behavior of something running quickly in SSMS but horribly slow (seconds become several minutes) via EF.
One way to test this is to turn off Lazy Loading and either remove MultipleActiveResultSets=True; (the default is "false") or at least change it to be MultipleActiveResultSets=False;.
As far as I know, there is unfortunately no work-around or fix (currently) for this behavior.
Here is an instance of this issue: Same query with the same query plan takes ~10x longer when executed from ADO.NET vs. SMSS
There is an excellent article about Entity Framework performance consideration here.
I would like to draw your attention to the section on Cold vs. Warm Query Execution:
The very first time any query is made against a given model, the
Entity Framework does a lot of work behind the scenes to load and
validate the model. We frequently refer to this first query as a
"cold" query. Further queries against an already loaded model are
known as "warm" queries, and are much faster.
During LINQ query execution, the step "Metadata loading" has a high impact on performance for Cold query execution. However, once loaded metadata will be cached and future queries will run much faster. The metadata are cached outside of the DbContext and will be re-usable as long as the application pool lives.
In order to improve performance, consider the following actions:
use pre-generated views
use query plan caching
use no tracking queries (only if accessing for read-only)
create a native image of Entity Framework (only relevant if using EF 6 or later)
All those points are well documented in the link provided above. In addition, you can find additional information about creating a native image of Entity Framework here.
I don't have an specific answer as to WHY this is happening, but it certainly looks to be related with how the query is handled more than the query itself. If you say that you don't have any issues running the same generated query from SSMS, then it isn't the problem.
A workaround you can try: A stored procedure. EF can handle them very well, and it is the ideal way to deal with potentially complicated or expensive queries.
Realizing you are using Entity Framework 4.1, I would suggest you upgrade to Entity Framework 6.
There has been a lot of performance improvement and EF 6 is much faster than EF 4.1.
The MSDN article about Entity Framework performance consideration mentioned in my other response has also a comparison between EF 4.1 and EF 6.
There might be a bit of refactoring needed as a result, but the improvement in performance should be worth it (and that would reduce the technical debt at the same time).
I updated my ASP.NET site from Linq-to-SQL to EF 4. I've had a bunch of hiccups along the way but have ironed most out.
Not sure if it has anything to do with EF but recently my application has started going nuts at random times. I'll delete one or multiple records from my SQL Server database, but when I re-display the page that was listing those records they're still there. I check my database and the records are gone yet when my code queries for this data after it's deleted my data model is still getting that deleted data and displaying it on my site. The only way I've found to get around the issue is closing the web browser or stopping my local host when debugging. Obviously not the fix I want. It happens on server code and local host.
This problem is very rare and difficult to reproduce and it only happens some days but not most days. I'd like to say it's completely random but I know there has to be some variable and I can't figure it out. Any suggestions?
I've been working with the Sync Framework from microsoft with c# trying to synchronize about 35 tables from a local database to a database stored in a central server, the main problem is that one of my tables has more or less than 1million records, and even with the filters it takes to long to synchronize, i dont know if there's a way or any other framework that works a bit faster than this.
for the complete sync it takes about 4-6 hours.
any help will be good, thanks in advance.
Sync framework is just like any other database app and you can troubleshoot/optimize it similarly.
You can enable sync framework tracing to see where its spending its time: querying for changes, serialising changes, applying changes, locks/concurrency issues, network latency, etc...
Do you have pre-existing data on the destination DB? Have you tried initializing replicas from backup? Have you enabled batching? etc...
I am not sure if you still looking for a solution, but while using sync select upload / download incremental changes. This way every time you sync only new changes will be transferred. This way the process will consume time on the first sync only, later on only changes will be uploaded / downloaded, this it will be fairly quick.
If it has million records it will take lot of time to sync...Better is to build ur own sync architecture...we faced same issue then we created our own framework with rest based web apis returning Json formats.Only last updated rows will be synced based on last updated time...dnt need to compare whole database...
I saw similar questions around the same topic, but i couldn't solve my issue.
I have asp.net web application with DB2 backend. And we have the entity framework model 3.5
So when i load the page for the very first time, it takes close to 15 seconds for executing the first query. And the query is very simple, selecting a row from one table and the where clauses are indexed. This is the query
protected Detail getProgramDetail(string id1,string id2, string id3)
{
Detail result = (from d in context.Detail
where d.id1.equals(id1) &&
d.id2.equals(id2) &&
d.id3.equals(id3)
select d).FirstorDefault();
return result;
}
I tried updating the statistics too, but it didnt help either.
After reading other performance tuning articles, i made this query as a compiled one, but still its taking close 15 seconds. But the subsequent calls are pretty fast in milliseconds. I think its taking time to establish the connection and run the query.
Is there a way to improve the initial performance?. I tried to generate views. But i am receiving the below error. Not sure how to handle this one.
The specified store provider cannot be found in the configuration, or
is not valid
My connection string is in machine.config.
Thanks in advance.
Srividhya
Is there a way to improve the initial performance?.
What about it is not ASp.NET / Entity Framework but IIS - STARTING THE APPLICATION, including doing the compilation needed ;)?
Make it hot - there is an extension that can call a page when IIS starts, so the applicatin is hot (compiled, loaded in memory).
http://weblogs.asp.net/scottgu/archive/2009/09/15/auto-start-asp-net-applications-vs-2010-and-net-4-0-series.aspx
There are multiple options which you look forward for solving this issue
1) Add the second level caching with the Entity framework .This article is very helpful
http://msdn.microsoft.com/hi-in/magazine/hh394143(en-us).aspx
2) Querying and ORM dont stick together very well ,Can you use a stored procedure and Datareader which will definitely optimize performance.
3)If you can use Optimize the first or default in your code