I have a very simple application using EF. But when it runs a week, the memory usage is awful (only 80MB at first, 700MB after one week).
When I use dotMemory to profile my application. I find the memory of Heap generation 2 is increasing all the time.
I Get a snapshot, finally find the retained bytes of ef dbcontext is the most.
I am so confused. My application is so simple. Code sample：
protected CarbonBrushMonitorEntities _entities = new MYEntities();
public void Add(HistoryData data)
{
_entities.HistoryDatas.Add(data);
_entities.SaveChanges();
}
_entities only initials once at the starting time, then used all the time.
The function Add is frequently called，about 3 times/second
I google a long time, and try some methods such as:
_entities.Configuration.ValidateOnSaveEnabled = false;
_entities.Configuration.AutoDetectChangesEnabled = false;
_entities.Configuration.LazyLoadingEnabled = false;
but these do not work.
If you use entity framework, you should create the context just before you need it and dispose it as soon as possible:
using (var someContext = new SomeContext())
{
// your commands/queries
}
Never keep context in memory or share it across different calls.
What I typically do is register the context with an IoC container:
DependencyFactory.RegisterType(typeof(SomeContext));
and use a context resolver (also registered with IoC of course) like:
using (var someContext = _contextResolver.ResolveContext())
{
// your commands/queries
}
where resolution is done like:
public class ContextResolver : IContextResolver
{
public ISomeContext ResolveContext()
{
return DependencyFactory.Resolve<SomeContext>();
}
}
The EF context is actually your unit of work, which should be disposed of once you don't need it anymore.
The other way is to clear the changetracker of the respective entities of concern or even
of all the entities. This is done by changing the entity state to "Detached". This called after dbContext.SaveChangesAsync()
protected void DisposeDbset<T>() where T : class
{
var Tname = typeof(T).Name;
var changetrackercollection = _unitOfWork.dbContext.ChangeTracker.Entries<T>();
foreach (var item in changetrackercollection.ToList())
{
item.State = EntityState.Detached;
}
GC.Collect();
}
I recently faced a similar situation where I was inserting 3,00,000 rows in batch operation. After inserting the rows, the change tracking info for all the rows remained in the memory with the entity state as Unchanged. Hence after every SaveChangesAsync() call, the changetracker accumulated.
I could not resolve new instance dbcontext for every batch, as it was a more expensive operation.
Just FYI, i had configured dbConetext.ChangeTracker.QueryTrackingBehavior = NoTracking. But this is applicable to while fetching the data.
Hopefully this is helpful. I found my solution with the help of this link http://andreyzavadskiy.com/2016/09/23/entries-in-entity-framework-changetracker-could-degrade-database-write-performance/?unapproved=19301&moderation-hash=4acc61a32ead7232959c2ec1ca268180#comment-19301
Based on Chintan shah answer I made an extension method and an example.
public static class DbContextExtensions
{
/// <summary>
/// Set all entries in ChangeTracker to detached to get them collected by the GC
/// </summary>
/// <param name="context"></param>
public static void DetachAllEntriesInChangeTracker(this DbContext context)
{
try
{
foreach (var entityEntry in context.ChangeTracker.Entries())
{
entityEntry.State = EntityState.Detached;
}
}
catch (Exception e)
{
LogManager.GetLogger(context.GetType().FullName).Error(e, "error when detaching all entries in changeTracker");
}
}
}
public class FooDbContext : DbContext
{
public override void Dispose()
{
this.DetachAllEntriesInChangeTracker();
base.Dispose();
}
}
Related
I have a LogContext Model :
using System.Data.Entity;
namespace Logging.Models
{
public class LogContext : DbContext
{
// You can add custom code to this file. Changes will not be overwritten.
//
// If you want Entity Framework to drop and regenerate your database
// automatically whenever you change your model schema, add the following
// code to the Application_Start method in your Global.asax file.
// Note: this will destroy and re-create your database with every model change.
//
// System.Data.Entity.Database.SetInitializer(new System.Data.Entity.DropCreateDatabaseIfModelChanges<Logging.Models.ProductContext>());
public LogContext() : base("name=LogContext")
{
Database.SetInitializer<LogContext>(null);
}
public DbSet<Log> Logs { get; set; }
}
}
but when I try to reference the Logs in my other LogContext class under App_code I'm getting an error trying to reference the context.Logs.Load();
"cannot be accessed with an instance reference; qualify it with a type name"
How do I reference and render all the rows in my table? What am i doing wrong?
Thanks
using System;
using System.Collections.Generic;
using System.Linq;
using Logging.Controllers;
using Logging.Models;
namespace Logging
{
public class LogContext : IDisposable
{
private static readonly List<Log> Logs = new List<Log>();
static LogContext()
{
using (var context = new LogContext())
{
**context.Logs.Load();**
}
//Logs.Add(new Log() { Id = 1, LoggerName = "TESTSYS1", InnerException = "InnerException", LogText = "LogText", ThreadID = 1, StackTrace = "Stack Trace", eLevel = "INFO" });
//Logs.Add(new Log() { Id = 2, LoggerName = "TESTSYS2", InnerException = "InnerException", LogText = "LogText", ThreadID = 2, StackTrace = "Stack Trace", eLevel = "ERROR" });
//Logs.Add(new Log() { Id = 3, LoggerName = "TESTSYS3", InnerException = "InnerException", LogText = "LogText", ThreadID = 3, StackTrace = "Stack Trace", eLevel = "WARN" });
}
void IDisposable.Dispose()
{
}
public void GetLoggies()
{
using (var context = new LogContext())
{
foreach (var log in context.GetLogs())
{
Logs.Add(log);
}
}
}
public Log GetLog(int id)
{
var log = Logs.Find(p => p.Id == id);
return log;
}
public IEnumerable<Log> GetLogs()
{
return LogContext.Logs;
}
public Log AddLog(Log p)
{
Logs.Add(p);
return p;
}
public void Delete(int id)
{
var product = Logs.FirstOrDefault(p => p.Id == id);
if (product != null)
{
Logs.Remove(product);
}
}
public bool Update(int id, Log log)
{
Log rLog = Logs.FirstOrDefault(p => p.Id == id);
if (rLog != null)
{
rLog = log;
return true;
}
return false;
}
}
}
The problem is frankly very bad design.
Your class here has the same name as your context and also has a member with the same name as a member on your context, i.e. Logs. This is a case study in how intelligent the compiler is, in that the only reason the whole thing doesn't explode, is because it's able to make some sense out of which you want in which place, given context. Still, it might guess wrong, and you will certainly get confused at some point. If you insist on maintaining it this way, you should fully-qualify all uses of your actual context class, i.e. new Namespace.To.LogContext(), so the compiler isn't just guessing.
Using using around a context is a hugely bad idea. A context instance should ideally be request-scoped. Among other things, the context employs change tracking, and when you start passing entities between different context instances, you're going to run headlong into a brick wall. Instead, you should inject your context into this class and save it as a field on the class.
Implementing IDisposable is not something you should do lightly. There's a very particular way it needs to be implemented or you're actually causing more harm than good.
public class Base: IDisposable
{
private bool disposed = false;
//Implement IDisposable.
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (!disposed)
{
if (disposing)
{
// Free other state (managed objects).
}
// Free your own state (unmanaged objects).
// Set large fields to null.
disposed = true;
}
}
// Use C# destructor syntax for finalization code.
~Base()
{
// Simply call Dispose(false).
Dispose (false);
}
}
See: https://msdn.microsoft.com/en-us/library/b1yfkh5e(v=vs.100).aspx
However, if you inject your context, this class will no longer own the context, and therefore wouldn't need to even implement IDisposable. And, for the love of everything good and holy, don't implement IDisposable when you're injecting dependencies. I see far too many developers do this and end up with strange bugs because resources are being disposed incorrectly.
Finally, just throw this class away completely. What you're essentially trying to create here (incorrectly) is a repository, and you don't need that. Entity Framework already implements the repository and unit of work patterns. As you can see from your methods here, all you're doing is basically proxying from your method to a nearly equivalent method on the DbSet. You're buying yourself nothing but just an additional layer that now has to be maintained, more entropy for your application code, and technical debt. For a more detail description of why this is the wrong approach see: https://softwareengineering.stackexchange.com/a/220126/65618
I'm trying to add a singleton pattern to my DbContext with Entity Framework. I've always used the singleton pattern for this, and never experienced this error before. I know that singleton is best practice (Apparently not), but if any of you have the time to spare, could you please explain why singleton is best practice?
Problem
Other than that, I get this error:
The underlying provider failed on open
Let's have a look at my code
DAO.cs
public class DAO
{
private static HourRegistrationEntities hourRegInstance;
public static HourRegistrationEntities HourRegInstance { get { return hourRegInstance = hourRegInstance ?? new HourRegistrationEntities(); } }
}
Service.cs (example method)
/// <summary>
/// Return a list of all denied Hour Registration for the Login with the given stringId
/// </summary>
/// <param name="stringId"></param>
/// <returns>A list of HourRegistrationDTO</returns>
public List<HourRegistrationDTO> GetAllDeniedHoursForLogin(string stringId)
{
var id = Int32.Parse(stringId);
using (var db = DAO.HourRegInstance)
{
var dbHours = db.HourRegistration.Where(x => x.LoginProject.Login.ID == id && x.Denied == true).ToList();
var returnList = new List<HourRegistrationDTO>();
foreach (var item in dbHours)
{
returnList.Add(new HourRegistrationDTO()
{
Id = item.ID,
Hours = item.Hours,
Date = item.Date.ToShortDateString(),
Comment = item.Comment,
CommentDeny = item.CommentDeny,
LoginProject = new LoginProjectDTO()
{
Project = new ProjectDTO()
{
Title = item.LoginProject.Project.Title
}
}
});
}
return returnList;
}
}
As mentioned, I've ALWAYS used the singleton pattern but NEVER had this error before. What's causing this, and why?
UPDATE:
I basically do like this (code below) instead, as that solves the problem. Now I'm more curious about what's causing the error.
Service.cs
using (var db = new HourRegistrationEntities())
The reason it doesn't work is that your using clause is disposing your singleton instance after first use. After that, it becomes useless, disposed but still not null.
The fact that you insist you always used singletons and it always worked doesn't really mean anything. Using singletons for data contexts is considered a terribly bad habit that leads to numerous issues, including memory, concurrency and transaction problems.
My guess is that before you always worked on single threaded desktop apps where a singleton is still risky but doesn't lead to immediate issues. Here, however, in a concurrent world of a web service, it just won't work.
On the other hand, creating a new instance per a wcf call is perfectly valid, different instances do not interfere and you correctly dispose them after used.
Make sure you have this code as your constructor:
public HourRegistrationEntities()
: base("ConnectionStringName")
{
}
And then define a connection string in App.config/web.config named ConnectionStringName
This will most probably solve your problem.
Now about the singleton: you're creating a new instance of HourRegistrationEntities on each call to GetAllDeniedHoursForLogin which is not singleton pattern, it's Unit of Work pattern which is THE best practice for EntityFramework. Keep using it and forget about singleton DbContext, unless you REALLY know what you do. As EF keeps track of entities and their relationship, a long lived singleton DbContext will get slow over time. There will be many other strange problems, like unsaved transactions, side effects and many other hard to debug problems.
May I suggest you to use "lock" functionality to access your singleton object :
public sealed class XModelInstance
{
private static XDBEntities edmx = null;
private static readonly object padlock = new object();
private XModelInstance() { }
public static XDBEntities Edmx
{
get
{
lock (padlock)
{
if (edmx == null)
{
edmx = new XDBEntities();
}
return edmx;
}
}
}
}
This will avoid concurrent access to your context.
And of course do not use "using" clause any more to access your context :)
Call example:
var dbHours = XModelInstance.Edmx.HourRegistration.Where(...);
If I use Microsoft implementation unit of work from this tutorial:
http://www.asp.net/mvc/tutorials/getting-started-with-ef-5-using-mvc-4/implementing-the-repository-and-unit-of-work-patterns-in-an-asp-net-mvc-application
public class UnitOfWork : IDisposable
{
private SchoolContext context = new SchoolContext();
private GenericRepository<Department> departmentRepository;
private GenericRepository<Course> courseRepository;
public GenericRepository<Department> DepartmentRepository
{
get
{
if (this.departmentRepository == null)
{
this.departmentRepository = new GenericRepository<Department>(context);
}
return departmentRepository;
}
}
public GenericRepository<Course> CourseRepository
{
get
{
if (this.courseRepository == null)
{
this.courseRepository = new GenericRepository<Course>(context);
}
return courseRepository;
}
}
public void Save()
{
context.SaveChanges();
}
//......
}
I don't need to use transactions when I must add related items? For example when I must add order and order positions to database I don't need to start transaction because if something will go wrong then method Save() won't execute yes? Am I right?
_unitOfWork.OrdersRepository.Insert(order);
_unitOfWork.OrderPositionsRepository.Insert(orderPosition);
_unitOfWork.Save();
??
SaveChanges itself is transactional. Nothing happens at the database level when you call Insert, which based on the tutorial merely calls Add on the DbSet. Only once SaveChanges is called on the context does the database get hit and everything that happened up to that point is sent in one transaction.
You need transactions if you have multiple save changes in one method ... or chain of method calls using the same context.
Then you can roll back over the multiple save changes when your final update fails.
An example would be multiple repositories wrapping crud for an entity under the unit of work (IE a generic class). You may have many functions inserting and saving in each repository. However at the end you may find an issue which causes you to roll back previous saves.
EG in a service layer that needs to hit many repositories and execute a complex operation.
We create one Linq2Sql DataContext for every request using the following class:
public static class DataContextManager
{
private const string HTTPCONTEXT_KEY = "DataContextManagerKey";
private static CompanyDataContext _staticContext; //when there's no HttpContext (in test/debug situations).
public static CompanyDataContext Context
{
get
{
if (_Context == null)
{
_Context = NewContext();
}
return _Context;
}
}
private static CompanyDataContext _Context
{
get
{
return (CompanyDataContext)(HttpContext.Current != null ? HttpContext.Current.Items[HTTPCONTEXT_KEY] : _staticContext);
}
set
{
if (HttpContext.Current != null)
{
HttpContext.Current.Items[HTTPCONTEXT_KEY] = value;
}
else
{
DataContextManager._staticContext = value;
}
}
}
public static void Dispose()
{
CompanyDataContext context = _Context;
if (context != null)
{
if (Config.Instance.TestMode) context.Log.WriteLine("--- DISPOSING DATACONTEXT ---");
context.Dispose();
_Context = null;
}
}
public static CompanyDataContext NewContext()
{
CompanyDataContext db = new CompanyDataContext();
db.CommandTimeout = Config.SQL_COMMAND_TIMEOUT;
if (Config.Instance.TestMode)
{
db.Log = new ConsoleTextWriter();
db.Log.WriteLine("--- CREATING NEW DATACONTEXT ---");
}
return db;
}
}
And in Global.asax:
protected void Application_EndRequest(Object sender, EventArgs e)
{
DataContextManager.Dispose();
}
The reason why I'm asking is that we're suddenly getting random "SqlException: Server failed to resume the transaction" exceptions once or twice every day with code that used to work perfectly. After the exception we get a lot of other exceptions until we restart the web application. Anyone seen this behaviour before?
We're running ASP .Net 2.0 with SQL server 2005 on IIS 6.
UPDATE:
Just so no one else does the same horrible mistake as we did:
It turned out that some worker threads also used the DataContext but with no HttpContext they of course got the _staticContext (a feature in DataContextManager only to be used when testing). We rewrote the code in the worker threads to make sure one DataContext per thread and disposing it when done. And so far everything has worked for 2 weeks :)
This is a bad pattern. First you should never have a static data context it implements IDisposable so one thread could try to use the context while another is disposing of it plus many other potential problems. One data context per http request is no good either, data contexts are designed to be used for a single transaction then disposed of. You get problems if you retrieve update/insert/delete and retrieve using the same context, the second retrieve does not reflect the changes of the update/insert/delete. Remove the static context and just have the Context property return a new context every time. You could still dispose of the all at the end of a request by sticking them all in a List property and iterating through it.
In ASP.NET MVC 2, using Entity Framework 4, I'm getting this error "An entity object cannot be referenced by multiple instances of IEntityChangeTracker".
A search of SO shows that it is probably because I have different instances of the Entity Framework ObjectContext, when it should be only one ObjectContext instance for each HttpContext.
I have this code (written long before I joined) that appears to do just that - have one ObjectContext for every HttpContext. But I am getting the "IEntityChangeTracker" exception frequently so it is probably not working as intended:
// in ObjectContextManager.cs
public const string ConnectionString = "name=MyAppEntities";
public const string ContainerName = "MyAppEntities";
public static ObjectContext GetObjectContext()
{
ObjectContext objectContext = GetCurrentObjectContext();
if (objectContext == null) // create and store the object context
{
objectContext = new ObjectContext(ConnectionString, ContainerName);
objectContext.ContextOptions.LazyLoadingEnabled = true;
StoreCurrentObjectContext(objectContext);
}
return objectContext;
}
private static void StoreCurrentObjectContext(ObjectContext objectContext)
{
if (HttpContext.Current.Items.Contains("EF.ObjectContext"))
HttpContext.Current.Items["EF.ObjectContext"] = objectContext;
else
HttpContext.Current.Items.Add("EF.ObjectContext", objectContext);
}
private static ObjectContext GetCurrentObjectContext()
{
ObjectContext objectContext = null;
if (HttpContext.Current.Items.Contains("EF.ObjectContext")
objectContext = (ObjectContext)HttpContext.Current.Items["EF.ObjectContext"];
return objectContext;
}
I've examined this code and it looks correct. It does as far as I can tell return one ObjectContext instance for each HttpContext. Is the code wrong?
If the code is not wrong, why else would I get the "An entity object cannot be referenced by multiple instances of IEntityChangeTracker" exception?
EDIT: To show how the ObjectContext is disposed:
// in HttpRequestModule.cs
private void Application_EndRequest(object source, EventArgs e)
{
ServiceLocator.Current.GetInstance<IRepositoryContext>().Terminate();
}
// in RepositoryContext.cs
public void Terminate()
{
ObjectContextManager.RemoveCurrentObjectContext();
}
// in ObjectContextManager.cs
public static void RemoveCurrentObjectContext()
{
ObjectContext objectContext = GetCurrentObjectContext();
if (objectContext != null)
{
HttpContext.Current.Items.Remove("EF.ObjectContext");
objectContext.Dispose();
}
}
My guess is that you've stored an object somewhere in memory (most likely the http cache using in-process mode, but could also be any manual cache such as a shared dictionary), and now you've somehow associated that object with something else, for example:
newOrder.OwnerUser = currentUser; // <== let's say currentUser came from cache
// and newOrder was on your new entity context
Hence, a problem if the cached object still thinks it is attached to a context; not least, you are probably keeping an entire graph alive accidentally.
The code looks OK (as long as you are disposing it at the end of the request), but this would be a good time to add:
private const string EFContextKey = "EF.ObjectContext";
and use that in place of the 5 literals. Avoids a few risks ;p