Develop Reference

EC Core Lazy Database Connection

I'm using EF Core with .NET Core 5 and have a database connection that is dependency injected into my different controllers. Here's how the database context is created via ConfigureServices:
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<MyDbContext>(opt =>
{
opt.UseMySql(Settings.Instance.SQLConnectionString, ServerVersion.AutoDetect(Settings.Instance.SQLConnectionString), b =>
{
b.UseNewtonsoftJson();
})
});
}
This works great, but I've realized this code runs for every API method that gets called, even those that don't need a database context. For example, this controller has an empty constructor and no database context usage, but still calls UseMySql:
[Produces("application/json")]
[Route("client")]
public class SimpleClientController : Controller
{
[HttpPost("GetTime")]
public IActionResult GetTime([FromBody] GetTimeRequest request)
{
// return the current UTC server time
return Json(new GetTimeResponse()
{
Time = DateTime.UtcNow
});
}
}
In general this isn't a big deal, but some issues were highlighted during the recent us-east-2 AWS outage, which showed that methods that do not rely on the database were blocked by the lack of database connection. My redis/dynamodb methods do not suffer the same fate if there is a redis or dynamodb outage, as they are handled via a singleton service and only used lazily by methods that require them.
Is there a way to do something similar with EF Core and the database context? Ideally we only initialize EF/the database if the controller has to use the context.

Looks like my issue was actually with the automatic version detection, which is creating a connection to MySQL every single time to detect the version. I've now cached the version and it seems to have fixed the issue.

Entity Framework query throws 'async error' after many requests

In my project using .NET framework 4.6.1, EF 6.1.4 and IdentityServer3, I set the following DbContext:
public class ValueContext : DbContext
{
public IValueContext(bool lazyLoadingEnabled = false) : base("MyConnectionString")
{
Database.SetInitializer<IValueContext>(null);
Configuration.LazyLoadingEnabled = lazyLoadingEnabled;
}
public DbSet<NetworkUser> NetworkUser { get; set; }
public DbSet<User> User { get; set; }
[...]
And my Entity model User:
[Table("shared.tb_usuarios")]
public class NetworkUser
{
[Column("id")]
[Key()]
public int Id { get; set; }
[Required]
[StringLength(255)]
[Column("email")]
public string Email { get; set; }
[...]
public virtual Office Office { get; set; }
[...]
So far I think its all good.
Then I set this following query in my UserRepository (using DI)
protected readonly ValueContext Db;
public RepositoryBase(ValueContext db)
{
Db = db;
}
public async Task<ImobUser> GetUser(string email)
{
//sometimes I get some error here
return await Db.User.AsNoTracking()
.Include(im => im.Office)
.Include(off => off.Office.Agency)
.Where(u => u.Email == email &&
u.Office.Agency.Active)
.FirstOrDefaultAsync();
}
And everything runs well, until it starts to get many sequential requests, then I begin to get these type of errors, randomly in any function that uses my ValueContext as data source:
System.NotSupportedException: 'A second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe.'
This is my last hope, as I tried a bunch of different things. Some of them work, and some dont, like:
Convert dbContext to use DI: no difference.
Use context lifetime to run the queries: works, but isnt the solution I want.
Remove asyncronous from requests: works, but also I feel is not the correct way to do.
What Im doing wrong?
EDIT 1
This is how I set up DI in Startup.cs:
private void AddAuth()
{
Builder.Map("/identity", app =>
{
var factory = new IdentityServerServiceFactory()
{
//here I implemented the IdentityServer services to work
ClientStore = new Registration<IClientStore>(typeof(ClientStore)),
[...]
};
AddDependencyInjector(factory);
}
[...]
}
private void AddDependencyInjector(IdentityServerServiceFactory factory)
{
//here I inject all the services I need, as my DbContext
factory.Register(new Registration<ValueContext>(typeof(ValueContext)));
[...]
}
And this is how my UserService is working:
public class UserService : IUserService
{
[Service injection goes here]
//this is a identityServer method using my dbContext implementation on UserRepository
public async Task AuthenticateLocalAsync(LocalAuthenticationContext context)
{
SystemType clientId;
Enum.TryParse(context.SignInMessage.ClientId, true, out clientId);
switch (clientId)
{
case 2:
result = await _userService.GetUser(context.UserName);
break;
case 3:
//also using async/await correctly
result = await _userService.Authenticate(context.UserName, context.Password);
break;
default:
result = false;
break;
}
if (result)
context.AuthenticateResult = new AuthenticateResult(context.UserName, context.UserName);
}

Update - After code posted
When using ASP.Net DI and IdentityServer DI together, we have to be careful to make sure that both the IdentityServer and the underlying DbContext are scoped to the OWIN request context, we do that by Injecting the DbContext into the IdentityServer context. this answer has some useful background: https://stackoverflow.com/a/42586456/1690217
I suspect all you need to do is resolve the DbContext, instead of explicitly instantiating it:
private void AddDependencyInjector(IdentityServerServiceFactory factory)
{
//here I inject all the services I need, as my DbContext
factory.Register(new Registration<ValueContext>(resolver => new ValueContext()));
[...]
}
Supporting dicussion, largely irrelevant now...
With EF it is important to make sure that there are no concurrent queries against the same DbContext instance at the same time. Even though you have specified AsNoTracking() for this endpoint there is no indication that this endpoint is actually the culprit. The reason for synchronicity is so that the context can manage the original state, there are many internals that are simply not designed for multiple concurrent queries, including the way the database connection and transactions are managed.
(under the hood the DbContext will pool and re-use connections to the database if they are available, but ADO.Net does this for us, it happens at a lower level and so is NOT an argument for maintaining a singleton DbContext)
As a safety precaution, the context will actively block any attempts to re-query while an existing query is still pending.
EF implements the Unit-Of-Work pattern, you are only expected to maintain the same context for the current operation and should dispose of it when you are done. It can be perfectly acceptable to instantiate a DbContext scoped for a single method, you could instantiate multiple contexts if you so need them.
There is some anecdotal advice floating around the web based on previous versions of EF that suggest there is a heavy initialization sequence when you create the context and so they encourage the singleton use of the EF context. This advice worked in non-async environments like WinForms apps, but it was never good advice for entity framework.
When using EF in a HTTP based service architecture, the correct pattern is to create a new context for each HTTP request and not try to maintain the context or state between requests. You can manually do this in each method if you want to, however DI can help to minimise the plumbing code, just make sure that the HTTP request gets a new instance, and not a shared or recycled one.
Because most client-side programming can create multiple concurrent HTTP requests (this of a web site, how many concurrent requests might go to the same server for a single page load) it is a frivolous exercise to synchronise the incoming requests, or introduce a blocking pattern to ensure that the requests to the DbContext are synchronous or queued.
The overheads to creating a new context instance are expected to be minimal and the DbContext is expected to be used in this way especially for HTTP service implementations, so don't try to fight the EF runtime, work with it.
Repositories and EF
When you are using a repository pattern over the top of EF... (IMO an antipattern itself) it is important that each new instance of the repository gets its own unique instance of the DbContext. Your repo should function the same if you instead created the DbContext instance from scratch inside the Repo init logic. The only reason to pass in the context is to have DI or another common routine to pre-create the DbContext instance for you.
Once the DbContext instance is passed into the Repo, we lose the ability to maintain synchronicity of the queries against it, this is an easy pain point that should be avoided.
No amount of await or using synchronous methods on the DbContext will help you if multiple repos are trying to service requests at the same time against the same DbContext.

Why is DBContext is disposed after putting it in IMemoryCache (.NET Core / EF Core)

I'm trying to put a subset of db-data in an IMemoryCache, but the 2nd time I call the application, I get an error:
ObjectDisposedException: Cannot access a disposed object. A common cause of this error is disposing a context that was resolved from dependency injection and then later trying to use the same context instance elsewhere in your application. This may occur if you are calling Dispose() on the context, or wrapping the context in a using statement. If you are using dependency injection, you should let the dependency injection container take care of disposing context instances.
Object name: 'WebDbContext'.
My Code snippet:
public class ArticleRepository : IArticleRepository
{
private readonly WebDbContext _WebDbContext;
private readonly IMemoryCache _cache;
public ArticleRepository(WebDbContext WebDbContext, IMemoryCache cache)
{
_WebDbContext = WebDbContext;
_cache = cache;
}
public IQueryable<Articles> WebshopArticles
{
get
{
return _cache.GetOrCreate("WebshopArticles", entry =>
{
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1);
return _WebDbContext.Article.Include(s => s.Details);
});
}
}
public IQueryable<Articles> GetArticles(string category)
{
return WebshopArticles.FirstOrDefault(s => s.Category == Category);
}
}
It looks like DBContext is disposed after the first time I put it in cache. How can i handle this?

You're using dependency injection to get an instance of your WebDbContext through your constructor. ASP.NET Core does this by initiating a WebDbContext object for you and injecting it into the constructor call when it creates an instance of your repository class.
But that WebDbContext object is only available for the life of the current HTTP request. Once that HTTP request is complete, ASP.NET Core gets rid of it. That's why you see it disposed.
Update: I see what you're doing. The problem is here:
return _WebDbContext.Article.Include(s => s.Details);
That does not cache the data. That caches the query (IQueryable). The query doesn't get executed until you enumerate that (loop through it). This is refered to as "lazy loading". So your GetArticles actually performs the query again each time it's called.
The first time you use it (in the same HTTP request you cached it), it works. But when you use it the second time, the context is disposed and the query can't be executed.
You need to force it to execute the query right away. An easy way is to call ToList():
return _WebDbContext.Article.Include(s => s.Details).ToList();
You'll need to change the property type to IEnumerable too.

Unable to cast object of type 'System.Data.ProviderBase.DbConnectionClosedConnecting' to type 'System.Data.SqlClient.SqlInternalConnectionTds

I am getting the following error on the first db access after the application starts - "Unable to cast object of type 'System.Data.ProviderBase.DbConnectionClosedConnecting' to type 'System.Data.SqlClient.SqlInternalConnectionTds"
The error only thrown once, at the first method tries to read data from the database, after the application starts.
Re-calling the same method for the 2nd time and further, everything works fine.
Using .net core 1.1 with Entity Framework

I recently had this same exception in an ASP.NET Core 2 app with EF Core. In my case, the root cause was a problem with the scopes my dependency-injected DbContext. I had a controller and a service both using an injected DbContext. The service was a singleton, like this:
public class TestService{
public TestService(FooDbContext db)
{
this.db = db;
}
}
public class FooController{
public FooController(FooDbContext db, TestService testService)
{
this.testService = testService;
this.db = db;
}
}
public class Startup{
public void ConfigureServices(IServiceCollection services){
//...
services.AddDbContext<FooDbContext>(options =>
options
.UseSqlServer(Configuration.GetConnectionString("FooDbContext"))
);
services.AddSingleton<TestService>();
}
}
So the controller would use it's instance, and then if the singleton service also tried to use it's own instance, then it would give the error above about 90% of the time. I'm a little fuzzy on why this would be an issue, or be intermittent, but it became pretty clear in debugging that EF was reusing some underlying resources. I didn't dig into EF code debugging, but I suspect the controller instance was closed, and the service instance reused the connection, expecting it to be open. In reading, others suggested MultipleActiveResultSet=true in the connection string would fix, but this did not resolve the issue in my case. In my case, the fix was to change the service to Transient in Startup.cs, which was acceptable in this case, and possibly better:
services.AddTransient<TestService>();

How to force Entity Framework to always get updated data from the database?

I am using EntityFramework.Extended library to perform batch updates. The only problem is EF does not keep track of the batch updates performed by the library. So when I query the DbContext again it does not return the updated entities.
I found that using AsNoTracking() method while querying disables the tracking and gets fresh data from the database. However, since EF does not keep track of the entities queried with AsNoTracking(), I am not able to perform any update on the queried data.
Is there any way to force EF to get the latest data while tracking changes?

Please try this to refresh a single entity:
Context.Entry<T>(entity).Reload()
Edit:
To get fresh data for a collection of entities is worth trying to dispose the DbContext instance after each request.

I stumbled upon this question while searching for a solution to a problem I was having where the navigation properties were not populating after updating the entity. Whenever I attempted to reload the entity from the database, it would grab the entry from the local store instead which would not populate the navigation properties via lazy loading. Instead of destroying the context and recreating one, I found this allowed me to get fresh data with the proxies working:
_db.Entry(entity).State = EntityState.Detached;
The logic behind it was - my update attached the entity so it would track changes to it. This adds it to the local store. Thereafter, any attempts to retrieve the entity with functional proxies would result in it grabbing the local one instead of going out to the db and returning a fresh, proxy-enabled entity. I tried the reload option above, which does refresh the object from the database, but that doesn't give you the proxied object with lazy-loading. I tried doing a Find(id), Where(t => t.Id = id), First(t => t.Id = id). Finally, I checked the available states that were provided and saw there was a "Detached" state. Eureka! Hope this helps someone.

I declared the entity variable, without assignment, as part of the class. This allowed me to dispose of an instance without losing the variable for reference by other methods. I just came across this so it doesn't have alot of runtime under it's belt, but so far it seems to be working fine.
public partial class frmMyForm
{
private My_Entities entity;
public frmMyForm()
{
InitializeComponent();
}
private void SomeControl_Click(object sender, EventArgs e)
{
db.SaveChanges();
db.Dispose();
entity = new My_Entities();
//more code using entity ...
}

Stumbled onto this problem. My app wasn't returning fresh data from the database.
These seems to be 3 solutions:
Reload on select: first you select the object, then reload. Loading it twice if it's not cached?
Detach after use: if you forget to detach an object after use, it's going to cause bugs in completely separate parts of the application that are going to be extremely hard to track down.
Disposing the DbContext after use. Definitely seems like the way to go.
I was creating my DbContext instance in the Repository class. If the DbContext is declared at the Repository level, then I have no control over how it gets disposed. That's a no-no. If I create a new DbContext on every call, then I cannot call Select, modify data, and then call Update.
Seems like something is fundamentally missing in my Repository pattern.
After some research on fundamental Repository pattern, I found the solution: Unit of Work pattern alongside the Repository pattern.
This is an excellent article on the Unit of Work pattern
Or this article from Microsoft. What I currently have is the Repository further up in the page, and what's missing is the section "Implement a Generic Repository and a Unit of Work Class"
Basically, instead of injecting repositories into your services, you access all repositories via a UnitOfWork that you inject into your service. It will solve many problems.
public class UnitOfWork : IUnitOfWork
{
private readonly ApplicationContext _context;
public UnitOfWork(ApplicationContext context)
{
_context = context;
Developers = new DeveloperRepository(_context);
Projects = new ProjectRepository(_context);
}
public IDeveloperRepository Developers { get; private set; }
public IProjectRepository Projects { get; private set; }
public int Complete()
{
return _context.SaveChanges();
}
public void Dispose()
{
_context.Dispose();
}
}
Remains the question: how to create the IUnitOfWork instance?
If I create it in the class constructor to be injected just like the repository, then it gets created and destroyed exactly the same way and we're back to the same problem. In ASP.NET and MVC, class instances are short-lived so injecting in the constructor may be fine, but in Blazor and desktop apps, class instances are much more long-lived and it's more of a problem.
This article from Microsoft clearly states that Dependency Injection isn't suitable to manage the lifetime of DbContext in Blazor:
In Blazor Server apps, scoped service registrations can be problematic
because the instance is shared across components within the user's
circuit. DbContext isn't thread safe and isn't designed for concurrent
use. The existing lifetimes are inappropriate for these reasons:
Singleton shares state across all users of the app and leads to
inappropriate concurrent use.
Scoped (the default) poses a similar
issue between components for the same user.
Transient results in a new
instance per request; but as components can be long-lived, this
results in a longer-lived context than may be intended.
They suggest using the Factory pattern, which can be implemented like this
/// <summary>
/// Creates instances of UnitOfWork. Repositories and UnitOfWork are not automatically injected through dependency injection,
/// and this class is the only one injected into classes to give access to the rest.
/// </summary>
public class UnitOfWorkFactory : IUnitOfWorkFactory
{
private readonly IDateTimeService _dateService;
private readonly DbContextOptions<PaymentsContext> _options;
public UnitOfWorkFactory(IDateTimeService dateService, DbContextOptions<PaymentsContext> options)
{
_dateService = dateService;
_options = options;
}
/// <summary>
/// Creates a new Unit of Work, which can be viewed as a transaction. It provides access to all data repositories.
/// </summary>
/// <returns>The new Unit of Work.</returns>
public IUnitOfWork Create() => new UnitOfWork(CreateContext(), _dateService);
/// <summary>
/// Creates a new DbContext.
/// </summary>
/// <returns>The new DbContext.</returns>
public PaymentsContext CreateContext() => new(_options);
}
Neither IWorkOfUnit nor any repository will be registered into the IoC container. Only IWorkOfUnitFactory.
And finally... how to share a transaction between various services?
I have a SetStatus method that updates the status field in the database. How is this method supposed to know whether it's a stand-alone operation or part of a larger transaction?
Since class-level dependency injection isn't suitable to manage and share the Work of Unit, then the only option is to pass it as parameters to the methods that need it.
I add an optional IUnitOfWork? workScope = null parameter to every method that needs it, and call Save only if this parameter is null. Here's an implementation.
public virtual async Task<TempOrder?> SetStatusAsync(int orderId, PaymentStatus status, IUnitOfWork? workScope = null)
{
using var unitOfWork = _workFactory.Create();
var work = workScope ?? unitOfWork;
var order = await work.Orders.GetByIdAsync(orderId);
if (order != null)
{
order.Status = status;
work.Orders.Update(order); // DateModified gets set here
if (workScope == null)
{
await work.SaveAsync();
}
}
return order;
}
Another option is to have IUnitOfWorkFactory.Create take the workScope parameter, and when set:
Re-use the existing DbContext
Do not dispose
IUnitOfWork.Save won't submit
My final implementation can be used like this
public virtual async Task<TempOrder?> SetStatusAsync(int orderId, PaymentStatus status, IUnitOfWork? workScope = null)
{
using var unitOfWork = _workFactory.Create(workScope);
var order = await unitOfWork.Orders.GetByIdAsync(orderId);
if (order != null)
{
order.Status = status;
work.Orders.Update(order); // DateModified gets set here
await unitOfWork.SaveAsync(); // Ignored if workScope != null
}
return order;
}
Pheww! That bug was a rabbit hole. It's a pretty long solution but should solve it for good with a solid architecture.

Making the code run on the same context will not yield you updated entities. It will only append new entities created in the database between runs. EF force reload can be done like this:
ObjectQuery _query = Entity.MyEntity;
_query.MergeOption = MergeOption.OverwriteChanges;
var myEntity = _query.Where(x => x.Id > 0).ToList();

For me ...
I access my DbContext like this:
_viewModel.Repo.Context
To force EF to hit the database I do this:
_viewModel.Repo.Context = new NewDispatchContext();
Overwriting the current DbContext with a new instance. Then the next time I use my data services they get the data from the database.

Reloading specific entities was not an option for me because I didn't know the exact entity. I also did not want to create a new DbContext as it is injected by DI. So I resorted to the following trick to "reset" the whole context.
foreach (var entry in db.ChangeTracker.Entries())
{
entry.State = EntityState.Detached;
}