Within an ASP.NET MVC Application I'm recieving the following error message for one of my controller methods that uses my Entity Framework context.
A second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe.
I'm aware that you cannot run queries in parallel, and everything appears to be awaited properly. If I debug the program and step and inspect some of the data returned from EF then it works, probably because this forces the queries to complete.
EDIT If I place a breakpoint at the null check in the controller method and inspect the data of shipmentDetail the exception is NOT thrown.
Here's a snippit of the code:
Controller Method:
[Route("{id:int}/Deliveries")]
public async Task<ActionResult> DeliveryInfo(int id)
{
var shipmentDetail = await db.ShipmentDetails.SingleOrDefaultAsync(s => s.Id == id);
if (shipmentDetail == null)
return HttpNotFound(string.Format("No shipment detail found with id {0}", id));
var model = await DeliveryInfoModel.CreateModel(db, shipmentDetail);
return View("DeliveryInfo", model);
}
CreateModel Method:
public static async Task<DeliveryInfoModel> CreateModel(Context db, ShipmentDetail shipment)
{
DeliveryInfoModel model = new DeliveryInfoModel()
{
ShipmentInfo = shipment
};
//initialize processing dictionary
Dictionary<int, bool> boxesProcessed = new Dictionary<int, bool>();
List<DeliveryBoxStatus> statuses = new List<DeliveryBoxStatus>();
for (int i = 1; i <= shipment.BoxCount; i++ )
{
boxesProcessed.Add(i, false);
}
//work backwards through process
//check for dispositions from this shipment
if(shipment.Dispositions.Count > 0)
{
foreach (var d in shipment.Dispositions)
{
DeliveryBoxStatus status = new DeliveryBoxStatus()
{
BoxNumber = d.BoxNumber,
LastUpdated = d.Date,
Status = d.Type.GetDescription().ToUpper()
};
statuses.Add(status);
boxesProcessed[d.BoxNumber] = true;
}
}
//return if all boxes have been accounted for
if (boxesProcessed.Count(kv => kv.Value) == shipment.BoxCount)
{
model.BoxStatuses = statuses;
return model;
}
//check for deliveries
if(shipment.Job_Detail.Count > 0)
{
foreach (var j in shipment.Job_Detail.SelectMany(d => d.DeliveryInfos))
{
DeliveryBoxStatus status = new DeliveryBoxStatus()
{
BoxNumber = j.BoxNumber,
LastUpdated = j.Job_Detail.To_Client.GetValueOrDefault(),
Status = "DELIVERED"
};
statuses.Add(status);
boxesProcessed[j.BoxNumber] = true;
}
}
//check for items still in processing & where
foreach (int boxNum in boxesProcessed.Where(kv => !kv.Value).Select(kv => kv.Key))
{
//THIS LINE THROWS THE EXCEPTION
var processInfo = await db.Processes.Where(p => p.Jobs__.Equals(shipment.Job.Job__, StringComparison.InvariantCultureIgnoreCase) && p.Shipment == shipment.ShipmentNum && p.Box == boxNum)
.OrderByDescending(p => p.date)
.FirstOrDefaultAsync();
//process returned data
//...
}
model.BoxStatuses = statuses;
return model;
}
I'm not completely sure if it's because of the query made in the controller, or because of the queries made in the loop that aren't completing causing this behavior. Is there something I'm not understanding about when the queries are actually made/returned due to EF's laziness, or how async/await works in this situation? I have a lot of other methods & controllers that make async EF calls and haven't run into this previously.
EDIT
My context is injected into my controller using Ninject as my IoC container. Here's its config inside of NinjectWebCommon's RegisterServices method:
kernel.Bind<Context>().ToSelf().InRequestScope();
Avoid lazy loading when using async with Entity Framework. Instead, either load the data you need first, or use Include()'s to ensure the data you need is loaded with the query.
https://msdn.microsoft.com/en-gb/magazine/dn802603.aspx
Current State of Async Support
... Async
support was added to Entity Framework (in the EntityFramework NuGet
package) in version 6. You do have to be careful to avoid lazy
loading when working asynchronously, though, because lazy loading is
always performed synchronously. ...
(Emphasis mine)
Also:
https://entityframework.codeplex.com/wikipage?title=Task-based%20Asynchronous%20Pattern%20support%20in%20EF.#ThreadSafety
Related
I'm working on a game, where players can get points, and players can earn points that are saved in my database. I have one method that looks like this:
private async Task SendAnswer(Answer answer)
{
clicked = true;
answerText = answer.AnswerText;
team = await gameRepo.GetTeamByNameAndGameSession(Teamname, GameSessionId);
if (answer.isCorrect)
{
team.TeamPoints = team.TeamPoints + answer.Points;
}
team.Answer = answer;
await gameRepo.UpdateTeam(team);
if (hubConnection is not null)
{
await hubConnection.SendAsync("SendTeamAnswer", team, GameSessionId);
}
}
That one works just fine, but then I also have this one in another view:
private async Task ChooseBestAnswer(Team team)
{
var answer = currentQuestion.Answers.FirstOrDefault();
team.TeamPoints = team.TeamPoints + answer.Points;
await gameRepo.UpdateTeam(team);
}
Both of them uses this method
public async Task UpdateTeam(Team teamToUpdate)
{
var oldTeam = await _context.Teams.FirstOrDefaultAsync(t => t.TeamName == teamToUpdate.TeamName && t.GameSessionGuid == teamToUpdate.GameSessionGuid);
if (teamToUpdate is not null)
{
oldTeam = teamToUpdate;
}
_context.SaveChangesAsync();
}
In the first method everything works as it should but at the second "oldteam" suddenly returns that points = 0 although I can see in the database that it is not 0, how is this possible, I use the same method put it fetches a 0 where there isn't any. All the other variables that are returned from the db to "oldteam" are correct it is just the points that suddenly are zero.
Does anyone know what is going on?
A couple of problems with this code:
public async Task UpdateTeam(Team teamToUpdate)
{
var oldTeam = await _context.Teams.FirstOrDefaultAsync(t => t.TeamName == teamToUpdate.TeamName && t.GameSessionGuid == teamToUpdate.GameSessionGuid);
if (teamToUpdate is not null)
{
oldTeam = teamToUpdate;
}
_context.SaveChangesAsync();
}
As mentioned in the comments, the SaveChangesAsync isn't awaited, but more importantly, this code doesn't update the team in the database. You are loading the existing team, but then simply overwriting the in-memory reference. That doesn't copy values across. Instead:
public async Task UpdateTeam(Team teamToUpdate)
{
if (teamToUpdate == null) throw new NullReferenceException(nameof(teamToUpdate));
var existingTeam = await _context.Teams.SingleAsync(t => t.TeamName == teamToUpdate.TeamName && t.GameSessionGuid == teamToUpdate.GameSessionGuid);
existingTeam.TeamPoints = teamToUpdate.TeamPoints;
// copy any additional fields that are allowed to be updated.
await _context.SaveChangesAsync();
}
Key changes to consider here. Assert the passed in state early and handle if it's invalid. If you expect 1 team to be found, use Single rather than First, and if an entry is expected, don't use the OrDefault variations. Those should be used only if you expect that an item might not be found. Once we have the existing data record, copy the values that can change across and call SaveChanges, awaiting the async operation.
This code will throw exceptions if expected state isn't valid, but it will throw meaningful exceptions to be handled at an appropriate level. (Rather than less descriptive exceptions when assumptions aren't met, or failing silently.)
I've never attempted to use WaitAll() or WhenAll() when running async functionality. After looking at many documentations, SO posts, and tutorials, I haven't found enough information for this, so here I am.
I'm trying to figure out the best/proper way(s) to do the following:
Using EF6, get data as List<Entity>.
Iterate through each Entity and call an external API to perform some action.
External API returns data per Entity which I need to store on the same Entity.
Currently I have built (not tested) the following (without the error handling code):
public IEnumerable<Entity> Process() {
bool hasChanged = false;
var data = _db.Entity.Where(x => !x.IsRegistered);
foreach (var entity in data) {
var result = await CallExternalApi(entity.Id, entity.Name);
entity.RegistrationId = result.RegistrationId;
entity.IsRegistered = true;
_db.Entry(entity).State = EntityState.Modified;
hasChanges = true;
}
if (hasChanges) {
uow.Commit();
}
return data;
}
I feel like I may be able to take advantage of some other functionality/feature in async, but if I can I'm not sure how to implement it here.
Any guidance is really appreciated.
Update
The API I'm calling is the Zoom Api to add Registrants. While they do have an route to batch add Registrants, it does not return the RegistrantId and the Join Url I need.
First, figure out if your external API might have a way to get all the items you want in a batch. If it does, use that instead of sending a whole bunch of requests.
If you need to send a separate request for each item, but want to do it concurrently, you could do this:
public async Task<IReadOnlyCollection<Entity>> Process() {
var data = _db.Entity.Where(x => !x.IsRegistered).ToList();
if(!data.Any()) { return data; }
var entityResultTasks = data
.Select(async entity => new { entity, result = await CallExternalApi(entity.Id, entity.Name) })
.ToList();
var entityResults = await Task.WhenAll(entityResultTasks);
foreach (var entityResult in entityResults) {
var entity = entityResult.entity;
var result = entityResult.result;
entity.RegistrationId = result.RegistrationId;
entity.IsRegistered = true;
_db.Entry(entity).State = EntityState.Modified;
}
uow.Commit();
return data;
}
You will want to watch out for possible concurrency limits on the target source. Consider using Chunk to break your work into batches of acceptable sizes, or leveraging a semaphore or something to throttle the number of calls you're making.
I have a an aync method that is looped:
private Task<HttpResponseMessage> GetResponseMessage(Region region, DateTime startDate, DateTime endDate)
{
var longLatString = $"q={region.LongLat.Lat},{region.LongLat.Long}";
var startDateString = $"{startDateQueryParam}={ConvertDateTimeToApixuQueryString(startDate)}";
var endDateString = $"{endDateQueryParam}={ConvertDateTimeToApixuQueryString(endDate)}";
var url = $"http://api?key={Config.Key}&{longLatString}&{startDateString}&{endDateString}";
return Client.GetAsync(url);
}
I then take the response and save it to my ef core database, however in some instances I get this Exception message: The Operaiton was canceled
I really dont understand that. This is a TCP handshake issue?
Edit:
For context I am making many of these calls, passing response to the method that writes to db (which is also so slow Its unbelievable):
private async Task<int> WriteResult(Response apiResponse, Region region)
{
// since context is not thread safe we ensure we have a new one for each insert
// since a .net core app can insert data at the same time from different users different instances of context
// must be thread safe
using (var context = new DalContext(ContextOptions))
{
var batch = new List<HistoricalWeather>();
foreach (var forecast in apiResponse.Forecast.Forecastday)
{
// avoid inserting duplicates
var existingRecord = context.HistoricalWeather
.FirstOrDefault(x => x.RegionId == region.Id &&
IsOnSameDate(x.Date.UtcDateTime, forecast.Date));
if (existingRecord != null)
{
continue;
}
var newHistoricalWeather = new HistoricalWeather
{
RegionId = region.Id,
CelsiusMin = forecast.Day.Mintemp_c,
CelsiusMax = forecast.Day.Maxtemp_c,
CelsiusAverage = forecast.Day.Avgtemp_c,
MaxWindMph = forecast.Day.Maxwind_mph,
PrecipitationMillimeters = forecast.Day.Totalprecip_mm,
AverageHumidity = forecast.Day.Avghumidity,
AverageVisibilityMph = forecast.Day.Avgvis_miles,
UvIndex = forecast.Day.Uv,
Date = new DateTimeOffset(forecast.Date),
Condition = forecast.Day.Condition.Text
};
batch.Add(newHistoricalWeather);
}
context.HistoricalWeather.AddRange(batch);
var inserts = await context.SaveChangesAsync();
return inserts;
}
Edit: I am making 150,000 calls. I know this is questionable since It all goes in memory I guess before even doing a save but this is where I got to in trying to make this run faster... only I guess my actual writing code is blocking :/
var dbInserts = await Task.WhenAll(
getTasks // the list of all api get requests
.Select(async x => {
// parsed can be null if get failed
var parsed = await ParseApixuResponse(x.Item1); // readcontentasync and just return the deserialized json
return new Tuple<ApiResult, Region>(parsed, x.Item2);
})
.Select(async x => {
var finishedGet = await x;
if(finishedGet.Item1 == null)
{
return 0;
}
return await writeResult(finishedGet.Item1, finishedGet.Item2);
})
);
.net core has a DefaultConnectionLimit setting as answered in comments.
this limits outgoing connections to specific domains to ensure all ports are not taken etc.
i did my parallel work incorrectly causing it to go over the limit - which everything i read says should not be 2 on .net core but it was - and that caused connections to close before receiving responses.
I made it greater, did parallel work correctly, lowered it again.
In my ASP.NET Web API application, I get the following error in one of my actions, at the line below:
var dateExists = await _unitOfWork.GetRepository<Booking>().All()
.AnyAsync(b => b.User.UserName == booking.User.UserName && b.Date == model.Date);
Here's the error:
A second operation started on this context before a previous
asynchronous operation completed. Use 'await' to ensure that any
asynchronous operations have completed before calling another method
on this context. Any instance members are not guaranteed to be thread
safe.
But, as you can see, I am using await. I'm also using await on all the other async operations in the action. Then, what might be the problem?
Edit:
Here's my entire action:
[HttpPut]
[Route("Update")]
public async Task<IHttpActionResult> Put(BookingEditBindingModel model)
{
if (!ModelState.IsValid)
{
// Return a bad request response if the model state is invalid...
return BadRequest(ModelState);
}
try
{
var booking = await _unitOfWork.GetRepository<Booking>().FindAsync(model.BookingId);
if (booking == null)
{
return NotFound();
}
// Only Admin is allowed to update other users' data...
if (!User.IsInRole("Admin") && booking.User.UserName != User.Identity.Name)
{
return Unauthorized();
}
// There can only be one entry per date/user.
if (model.Date != booking.Date)
{
var dateExists = await _unitOfWork.GetRepository<Booking>().All()
.AnyAsync(b => b.User.UserName == booking.User.UserName && b.Date == model.Date);
if (dateExists)
{
ModelState.AddModelError("Date", "Data already exists for this date.");
return BadRequest(ModelState);
}
}
booking.Date = model.Date;
// Change other properties...
await _unitOfWork.SaveAsync();
return Ok();
}
catch (Exception ex)
{
return InternalServerError(ex);
}
}
Update:
I disabled lazy loading, but I still had a problem. It looks like, as other people had guessed, the issue was with booking.User, because the FindAsync() in the first query does not include the User. I replaced the first query with the one below, and the issue was resolved.
var booking = await _unitOfWork.GetRepository<Booking>().All()
.Include(b => b.User)
.SingleOrDefaultAsync(b => b.BookingId == model.BookingId);
I am going to go out on a limb and guess that you have lazy loading enabled. This would cause a call like booking.User.UserName to go to the database to retrieve the value and putting that inside of a call like AnyAsync could cause the DbContext to try to open a second connection to retrieve that value after it already opened one for the AnyAsync statement.
The quick fix in this case is to retrieve the user name before the call to AnyAsync.
var bookingUserName = booking.User.UserName;
var dateExists = await _unitOfWork.GetRepository<Booking>().All()
.AnyAsync(b => b.User.UserName == bookingUserName && b.Date == model.Date);
A better fix would be to retrieve this information in advance in the call where you get the booking instance so you do not have to make an additional round trip.
Personally I always disable lazy loading on my DbContext types and use Include statements to explicitly specify what I want to be retrieved with any call. This gives me more control over my execution.
I used the following approach long time (approx 5 years):
Create one big class with initialization of XXXEntities in controller and create each method for each action with DB. Example:
public class DBRepository
{
private MyEntities _dbContext;
public DBRepository()
{
_dbContext = new MyEntities();
}
public NewsItem NewsItem(int ID)
{
var q = from i in _dbContext.News where i.ID == ID select new NewsItem() { ID = i.ID, FullText = i.FullText, Time = i.Time, Topic = i.Topic };
return q.FirstOrDefault();
}
public List<Screenshot> LastPublicScreenshots()
{
var q = from i in _dbContext.Screenshots where i.isPublic == true && i.ScreenshotStatus.Status == ScreenshotStatusKeys.LIVE orderby i.dateTimeServer descending select i;
return q.Take(5).ToList();
}
public void SetPublicScreenshot(string filename, bool val)
{
var screenshot = Get<Screenshot>(p => p.filename == filename);
if (screenshot != null)
{
screenshot.isPublic = val;
_dbContext.SaveChanges();
}
}
public void SomeMethod()
{
SomeEntity1 s1 = new SomeEntity1() { field1="fff", field2="aaa" };
_dbContext.SomeEntity1.Add(s1);
SomeEntity2 s2 = new SomeEntity2() { SE1 = s1 };
_dbContext.SomeEntity1.Add(s2);
_dbContext.SaveChanges();
}
And some external code create DBRepository object and call methods.
It worked fine. But now Async operations came in. So, if I use code like
public async void AddStatSimplePageAsync(string IPAddress, string login, string txt)
{
DateTime dateAdded2MinsAgo = DateTime.Now.AddMinutes(-2);
if ((from i in _dbContext.StatSimplePages where i.page == txt && i.dateAdded > dateAdded2MinsAgo select i).Count() == 0)
{
StatSimplePage item = new StatSimplePage() { IPAddress = IPAddress, login = login, page = txt, dateAdded = DateTime.Now };
_dbContext.StatSimplePages.Add(item);
await _dbContext.SaveChangesAsync();
}
}
can be a situation, when next code will be executed before SaveChanged completed and one more entity will be added to _dbContext, which should not be saved before some actions. For example, some code:
DBRepository _rep = new DBRepository();
_rep.AddStatSimplePageAsync("A", "b", "c");
_rep.SomeMethod();
I worry, that SaveChanged will be called after line
_dbContext.SomeEntity1.Add(s1);
but before
_dbContext.SomeEntity2.Add(s2);
(i.e. these 2 actions is atomic operation)
Am I right? My approach is wrong now? Which approach should be used?
PS. As I understand, will be the following stack:
1. calling AddStatSimplePageAsync
2. start calling await _dbContext.SaveChangesAsync(); inside AddStatSimplePageAsync
3. start calling SomeMethod(), _dbContext.SaveChangesAsync() in AddStatSimplePageAsync is executing in another (child) thread.
4. complete _dbContext.SaveChangesAsync() in child thread. Main thread is executing something in SomeMethod()
Ok this time I (think)'ve got your problem.
At first, it's weird that you have two separate calls to SaveChangesmethod. Usually you should try to have it at the end of all your operations and then dispose it.
Even thought yes, your concerns are right, but some clarifications are needed here.
When encountering an asyncor await do not think about threads, but about tasks, that are two different concepts.
Have a read to this great article. There is an image that will practically explain you everything.
To say that in few words, if you do not await an async method, you can have the risk that your subsequent operation could "harm" the execution of the first one. To solve it, simply await it.