I have a problem. I have ASP .NET Core REST API app and in one mothod I"m trying to write more changes into a DB asynchronously. Everytime the different amount of objects is written into the DB and everytime one of three different errors ocured. Any suggestions what can be wrong?
This is my code:
Startup.cs
public void ConfigureServices(IServiceCollection services)
{
...
services.AddDbContext<MyDbContext>(options => options.UseSqlServer(connection_string), ServiceLifetime.Transient);
services.AddScoped<IHelper, Helper>();
...
}
Helper.cs
private MyDbContext _dbContext;
public Helper(IOptions<HelperSettings> settings, ILogger<Helper> logger, MyDbContext dbContext)
{
...
_dbContext = dbContext;
...
}
public void Save(object entity)
{
...
_dbContext.Add(entity);
}
This is the controller and the method that throws the exceptions.
public class MyController : ControllerBase
{
private readonly Helper _db;
public MyController(IHelper helper)
{
_db = helper;
}
...
[HttpPost]
[Route("something")]
[Produces("application/json")]
public async Task<ActionResult<Guid>> CreateSomethingAsync([FromBody] DataRequest data)
{
...
if (data.Answers != null)
{
List<Task> saveTasks = new List<Task>();
foreach (AnswerData ans in data.Answers)
{
Answer answer = ans.ConvertToAnswer(); //just create new Answer instance and filll it with data from AnswerData
saveTasks.Add(Task.Run(() => _db.Save(answer)));
}
await Task.WhenAll(saveTasks);
await _db.DbContext.SaveChangesAsync();
}
return Ok(...);
}
}
I call the CreateSomethingAsync() in a cycle in another app. It throws one of this three exceptions:
System.IndexOutOfRangeException: 'Index was outside the bounds of the array.'
or
System.InvalidOperationException: 'Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.'
or
System.InvalidOperationException: Cannot start tracking InternalEntityEntry for entity type 'Answer' because another InternalEntityEntry is already tracking the same entity
on the line _dbContext.Add(entity); in my Helper.cs.
I know that the problem is in the paralelism, but I dont know how to solve it. Any ideas?
DbContext is not thread-safe (that's what the exceptions you get are telling you), and calling DbContext.Set<T>.Add() does not take a significant amount of time. By parallelizing the Add() you are not adding multiple entities to the database asynchronously - you're just marking the entities as to-be-added upon calling SaveChanges().
So while I'm sure you have your reasons to parallelize your _db.Save(answer) calls, it probably has no performance improvements whatsoever, so you can remove it entirely, serializing the work.
If the work you do in there does benefit from parallelization, simply move the call to DbContext.Set<T>.Add() out of there - it is not thread safe.
Related
In my asp mvc Framework project, using EF, I have some objects (from classes) whose fields store data coming from my database.
My question is :
How to populate these fields, or manage methods of these objects using a dbcontext variable ?
Sol 1: Is it better to use each time I need a connection with db in my classes with the instruction (using (resource), see below ) ?
Sol 2: Is it betterI to code a singleton class to use one instance of the context ?
Sol 3: Or should I use another way for the links beetween my classes and the database ?
What is the best method considering performances and code quality.
Thanks for your attention .
Solution 1
public class Test
{
private T1 a;
private T2 b;
public Test()
{}
public void CreateFrom (int id)
{
using (var db=new WebApplicationMVCTest.Models.dbCtx())
{
a=db.T1s.Find(id);
b= db.T2s.Find(a.id2);
}
}
Solution 2:
public class DbSingleton
{
private static dbCtx instance;
private int foo;
private DbSingleton ()
{}
public static dbCtx Current
{
get
{
if (instance == null)
{
instance = new dbCtx();
}
return instance;
}
}
public static void Set (dbCtx x)
{
if (instance==null)
{
instance = x;
}
}
}
For a web project, never use a static DbContext. EF DbContexts are not thread safe so handling multiple requests will lead to exceptions.
A DbContext's lifespan should only be as long as it is needed. Outside of the first time setup cost when a DbContext is used for the first time, instantiating DbContexts is fast.
My advice is to start simple:
public ActionResult Create(/* details */)
{
using (var context = new AppDbContext())
{
// do stuff.
}
}
When you progress to a point where you learn about, and want to start implementing dependency injection applications then the DbContext can be injected into your controller / service constructors. Again, from the IoC container managing the DbContext, the lifetime scope of the Context should be set to PerWebRequest or equivalent.
private readonly AppDbContext _context;
public MyController(AppDbContext context)
{
_context = context ?? throw new ArgumentNullException("context");
}
public ActionResult Create(/* details */)
{
// do stuff with _context.
}
The gold standard for enabling unit testing would be injecting a Unit of Work pattern and considering something like the Repository pattern to make your dependencies easier to unit test.
The best advice I can give you starting out with EF and MVC is to avoid the temptation to pass Entities between the controller (server) and the UI. (views) You will come across dozens of examples doing just this, but it is a poor choice for performance, it also hides a LOT of land mines and booby traps for both performance, exceptions, and data security issues. The most important detail is that when the UI calls the controller passing what you expect will be an entity, you are not actually getting an entity, but a de-serialized JSON object cast to an entity. It is not an entity that is tracked by the DbContext handling the request. Instead, get accustomed to passing view models (serializable data containers with the data the view needs or can provide) and IDs + values where the controller will re-load entities to update the data only as needed.
I'm trying to implement SignalR with Angular, I want to send data to all ConnectionsIds.
I have Mapped each Connection Id to SectionId having 1-to-Many Relation as below,
public Class SectionConnectionMapping{
public string SectionId {get; set:}
public List<string> ConnectionId {get; Set;}
}
I want to Fetch SectionRelated data from DB and SendAsync to mapped ConnectionsIds parallelly,
Parallel.For(0, SectionConnectionMapping.Count, index => {
_hubContext.Clients.Clients(SectionConnectionMapping[index].Connections).SendAsync("SendMessage",
new {
somVal = getPatientData(SectionConnectionMapping[index].SectionId),
})
});
But it is throwing an exception as below,
System.InvalidOperationException: 'A second operation started on this context before a previous operation completed. This is usually caused by different threads using the same instance of DbContext, however instance members are not guaranteed to be thread safe. This could also be caused by a nested query being evaluated on the client, if this is the case rewrite the query avoiding nested invocations.'
If I Make for loop sequential, it works fine, but how can I make it parallel.
I have added Context in Startup.cs as,
services.AddDbContext<CustomContext>(options => options.UseSqlServer(connection), ServiceLifetime.Transient);
Also implemented it in DI of that Service as,
public class LiveHostedService : IHostedService, IDisposable
{
private readonly CustomContext _context;
public LiveHostedService(CustomContext context)
{
_context = context;
}
}
EntityFramework DbContext is not a thread-safe. This means it can not execute multiple queries at the same time.
getPatientData(SectionConnectionMapping[index].SectionId) is executing multiple times on same EF DbContext. Prepare data separately and then pass to _hubContext.Clients.Clients.SendAsync, or create a new instance of EF DbContext.
I have a working app utilizing EF Core 3.0 (preview 8 right now). There's one data set that changes very infrequently, but is queried very frequently. The services that actually perform operations on the data are isolated via a very simple interface.
interface IDataProvider
{
ValueTask<IQueryable<DataRow>> GetDataAsync();
}
Until now, there was only one implementation of the data provider built upon an actual SQL database. So if we have a DbContext with DbSet<DataRow> Data:
class LiveDataProvider : IDataProvider
{
private readonly DataDbContext _dbContext;
public LiveDataProvider(DataDbContext dbContext)
{
_dbContext = dbContext;
}
public ValueTask<IQueryable<DataRow>> GetDataAsync() =>
new ValueTask<IQueryable<DataRow>>(_dbContext.Data.AsNoTracking());
}
Now I'm tasked with adding caching to this application. So the idea is to store the entire Data set in-memory, because retrieving it is a fairly expensive query on the DB, but the amount of data is just a few hundred MB. Since updates are infrequent, the vast majority of request would hit the cache. With this idea, the implementation seemed fairly simple:
class InMemoryDataCache : IDataProvider
{
private readonly IPicksDbContext _picksDbContext;
private readonly IMemoryCache _memoryCache;
public async ValueTask<IQueryable<DataRow>> GetDataAsync()
{
var entryKey = GetEntryKey(); // Implementation is irrelevant.
if (!_memoryCache.TryGetValue(entryKey, out IQueryable<DataRow> data))
{
data = (await _dbContext.Data.AsNoTracking().ToListAsync()).AsQueryable();
_memoryCache.Set(entryKey, data);
}
return data;
}
InMemoryDataCache(DataDbContext dbContext, IMemoryCache memoryCache)
{
_dbContext = dbContext;
_memoryCache = memoryCache;
}
}
Sounds good, doesn't work - the service implementations use asynchronous operations on the IQueryable extensively, so ToListAsync(), AnyAsync() etc. Calling any of these on the cached List will results in EF Core throwing an InvalidOperationException with message Only sources that implement IAsyncEnumerable can be used for Entity Framework asynchronous operations.
That error is pretty familiar, as unit testing the services threw the same thing. There I mocked an IAsyncQueryProvider implementation (shout-out to MockQueryable), so here I could the same thing.
Alternatively, I could write my own extension methods for ToListAsync, AnyAsync, etc. with an implementation of
public static Task<List<T>> ToListAsync<T>(this IQueryable<T> queryable)
{
if(queryable is IAsyncEnumerable<T>)
{
return Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync(queryable);
}
return Task.FromResult(queryable.ToList());
}
Both are a lot of work and tests for something so stupidly simple. Is there a more robust solution? Or have I chased myself into a corner by tying my services to the EntityFrameworkQueryableExtensions in the first place and I cannot transparently use an in-memory provider.
I have a Web API architecture in following way:
Controller -> Services -> Repositories
Dependency is injected into using Unity framework via constructor injection pattern.
i have something similar in my project:
public class OrderService : IOrderService
{
private readonly IOrderRepo _orderRepo;
private readonly IProductRepo _prodRepo;
public OrderService(IOrderRepo orderRepo, IProductRepo prodRepo)
{
_orderRepo = orderRepo;
_prodRepo = prodRepo;
}
public void DoSomething()
{
_orderRepo.Update();
_prodRepo.Insert();
}
}
public class OrderRepo : IOrderRepo
{
private readonly CreditPortalContext _context;
public OrderRepo(MyContext context)
{
_context = context;
}
public void Update()
{
//Do something
_context.SaveChanges();
}
}
public class ProductRepo : IProductRepo
{
private readonly CreditPortalContext _context;
public ProductRepo(MyContext context)
{
_context = context;
}
public void Insert()
{
//Do something
_context.SaveChanges();
}
}
Now, OrderService is calling two different repository method to perform some database operation.
What I want is if something fails in _prodRepo.Insert(); method, it should rollback _orderRepo.Update(); call as well.
I am too far in this project so can not perform major architecture change at this point. Also, the context object is getting instantiated only once.(using Unity HierarchicalLifetimeManager)
Is there anyway i can achieve this?
this is not an ideal situation at all and I am afraid you're going to have to be quite creative.
You can't really rollback a statement that has already been committed, but you can do something else else.
So the situation is like this : you have 2 calls, call A and call B.
call A commits successfully. call B fails and you want to rollback A.
My suggestion is to perform the first commit to a different table.
Create a copy of your table in the database and commit your data there. if B succeeds then you issue another command to transfer the data to the main table where it should be. If B fails then you simply remove the data from the second table and your main table remains untouched.
This can be done via a stored procedure maybe, that you can call, give it the id of the record and then it does an update on the main table using the row from the second table. You can easily call a stored procedure from entity framework so that part should not be a problem.
So bottom line, use a temporary storage place and only move the data when your second call succeeds
I cannot seem to find an answer to this question.
So in the frontend when the user loads a page we call an API for each item on that page (10 items). So that equals 10 API calls.
Most of the calls work but there are always a few that fail when trying to query the database resulting in the following error:
InvalidOperationException: A second operation started on this
context before a previous operation completed. Any instance members
are not guaranteed to be thread safe.
Now I understand that Entity Framework is not thread safe but I am unsure how to get around this error.
Everywhere where I am using a DBContext it is always injected in using the built in .net core Ioc container.
Here is the DI setup
services.AddScoped<IOmbiContext, OmbiContext>();
services.AddTransient<ISettingsRepository, SettingsJsonRepository>();
All of my repositories are setup in a Transient scope with the Context as Scoped according to this article: https://learn.microsoft.com/en-us/aspnet/core/data/entity-framework-6
Now I have tried changing the context to Transient and it still happens.
How can I avoid this?
More Information
The API Method:
[HttpGet("movie/info/{theMovieDbId}")]
public async Task<SearchMovieViewModel> GetExtraMovieInfo(int theMovieDbId)
{
return await MovieEngine.LookupImdbInformation(theMovieDbId);
}
Which eventually calls the following where the exception is being thrown:
public async Task<RuleResult> Execute(SearchViewModel obj)
{
var item = await PlexContentRepository.Get(obj.CustomId); <-- Here
if (item != null)
{
obj.Available = true;
obj.PlexUrl = item.Url;
obj.Quality = item.Quality;
}
return Success();
}
PlexContentRepository
public PlexContentRepository(IOmbiContext db)
{
Db = db;
}
private IOmbiContext Db { get; }
public async Task<PlexContent> Get(string providerId)
{
return await Db.PlexContent.FirstOrDefaultAsync(x => x.ProviderId == providerId); <-- Here
}
If you use Entity Framework Core usually you do not need to add your Database Context as an additional service
I recommend to setup your DbContext in the Startup.cs as following:
services.AddEntityFrameworkSqlServer()
.AddDbContext<OmbiContext>();
Followed by a Controller class for your API calls taking the DBContext as constructor parameter.
public class ApiController : Controller
{
protected OmbiContext ctx;
public ApiController(OmbiContext dbctx)
{
ctx = dbctx;
}
public async Task<IActionResult> yourAsyncAction()
{
// access ctx here
}
}