I have a race condition on nHibernate that's creating duplicate entries on my database. Unfortunately, I cannot create an UNIQUE index on database, thus I would like to solve this error using only nHibernate methods. It is a web application that might run on a web farm (hence I guess a system lock should not solve the problem neither). The simplified situation follows:
var current = UnitOfWorkManager.Instance.Current;
current.BeginTransaction(IsolationLevel.Serializable);
try {
var myEntity = MyFactory.MyEntityRepository.GetBy(product, company);
// race condition happens between the previous statement and Save() method.
if (myEntity == null)
{
myEntity = new MyEntity();
myEntity.Product = product;
myEntity.Company = company;
myEntity.Date = date;
myEntity.CurrentUser = currentUser;
myEntity.IsManual = true;
myEntity.Save();
}
else
{
myEntity.IsManual = false;
myEntity.Save();
}
current.CommitTransaction();
}
catch {
current.RollbackTransaction();
throw;
}
I am new to nHibernate so maybe I am missing some basics here. I'd appreciate any feedback. :)
After reading the nHibernate Manual, I think your problem is maybe your second call to save if subProjectToSupplier isn't null. Because the nHibernate manual says "save" does an insert.
Try SaveOrUpdate
You should wrap your Save() in a transaction, ideally you could implement a pattern such as IUnitOfWork or use the SessionFactory, for example:
using (var transaction = session.BeginTransaction()) {
myEntity = new MyEntity();
myEntity.Product = product;
myEntity.Company = company;
myEntity.Date = date;
myEntity.CurrentUser = currentUser;
myEntity.IsManual = true;
myEntity.Save();
transaction.Commit();
}
OK, so you are asking for NHibernate solution here. I am not sure if this can solve in 100% your problem but try this one.
Use ReadCommitted isolation level.
Implement an interceptor for your needs http://nhibernate.info/doc/nh/en/#manipulatingdata-interceptors
Within the interceptor you can do:
Query the DB if there is a record on your unique column. Then you can apply some sort of a merge mechanism
Check the current session if there are other objects like yours pending to be persisted.
================
Other solution is to keep shared cache of between the farm (memcache) and track objects you do not want to be duplicated.
PS: I can talk more about each point. Just tell me if something sounds like a solution for you
Related
I am writing a system that has a concept of idempotent operations: If clients give the system an operation id more than once the system will reject those "duplicated" operations immediately.
I want to implement this by storing UUID Primary-Key values in a table in SQL server such that SQL server will just reject duplicated writes, as expected. My problem comes when EF Core tries to be smart about these values and cache them: EF Core will reject the addition of the entity without ever pinging SQL server because it knows there's already a tracked entity with that same PK. This behavior is ideal in most scenarios but in my specific scenario it will become very memory-intensive real quick. I don't want this behavior.
This is the code that I'm using to manually trigger the specific error I need to give clients of the system:
Action throwIdempotentOpError = () => {
throw new ExecutionError("The operation you are trying to perform was already performed, please try again with a new client mutation id");
};
if (opsRepo.IdempotentOperations.Local.Any(op => op.ClientMutationId == mutationGuid)) {
throwIdempotentOpError();
}
var operation = new IdempotentOperation {
ClientMutationId = mutationGuid,
CreatedAt = DateTime.UtcNow,
UpdatedAt = DateTime.UtcNow,
RawDocument = context.Document.OriginalQuery,
Status = IdempotentOperationStatus.Started
};
try {
opsRepo.IdempotentOperations.Add(operation);
await opsRepo.SaveChangesAsync();
} catch (DbUpdateException ex) {
if (ex.InnerException != null && ex.InnerException.Message.StartsWith("Violation of PRIMARY KEY constraint 'PK_IdempotentOperations'")) {
throwIdempotentOpError();
}
throw;
}
Ideally I would only have to throw the error inside the catch block.
How can I disable that entity tracking behavior on the .Add call?
For context: opsRepo is a DbContext
You can solve your problem with cashing added Ids and save them into for example Redis, check new ids with cached Ids and prevent to insert duplicate items and reject them. But for solving problem with DbContext, you should detach the inserted items like this:
...
try {
opsRepo.IdempotentOperations.Add(operation);
await opsRepo.SaveChangesAsync();
//detached inserted entities
ClearDbContextState();
}
...
public void ClearDbContextState()
{
var entities = opsRepo.ChangeTracker.Entries<IdempotentOperations>().Where(e => e.State == EntityState.Added ||
e.State == EntityState.Modified).ToList();
foreach (var entry in entities)
entry.State = EntityState.Detached;
}
I ended up using IHttpContextAccessor as a way to scope out the injection of my DbContext implementation, side-stepping the issue #Panagiotis Kanavos commented.
I have just read about TransactionScope. It is very good and informative.
First of all, I was wondering if I really need transactions in MVC 4 / EF 6+. The reason for that is we always invoke DbContext.SaveChanges() to save changes. I'm wondering if SaveChanges() is something that simulates transaction close meaning if I invoke SaveChanges() I commit a transactions.
On the other hand, if I need transactions, then how to implement TransactionScope in MVC / EF applications. My scenario is something similar to the steps below:
I save valid record in database
I save a copy of an old and a new record in another table which is sort of archived version of the original table
I save user's activity in another table
I also provided code. As you can see if something goes wrong in the middle I have inconsistent data. I would be grateful for some examples on how to use TransactionScope. I may also need more to save in other tables. I would like to be certain either I save everything or nothing, such that I save everything if transaction is OK or roll back anything that happened up to the problem.
Thanks.
[HttpPost]
public ActionResult Edit(ApplicationViewModel viewmodel)
{
using(MyDbCOntext dbContext = new MyDbContext())
{
if(!MoselState.IsValid)
return View(application);
// Copy old data from database and assign to an object
ApplicationArchive applicationOld = CopyApplicationFromDB(db, viewmodel.ApplicationID);
// Update model
if (TryUpdateModel(applicationNew), null, null, new string[] { "ApplicationID" })
{
try
{
dbContext.Entry(userToUpdate).State = EntityState.Modified;
dbContext.SaveChanges();
// Archive old application
ApplicationArchive applicationNew = CopyApplicationFromDB(db, viewmodel.ApplicationID);
try
{
dbContext.ApplicationsArchive.Add(applicationOld);
dbCOntext.ApplicationsArchive.Add(applicationNew);
dbContext.SaveChanges();
// Register user activity
string username = GetUserNameFromCookie();
UserActivity useractivity = new UserActivity() { UserName = username, activity = "edit", Table = "application" };
try
{
dbContext.UserActivities.Add(useractivity);
dbContext.SaveChanges();
return RedirectView("Index");
}
}
}
catch
{
ModelState.AddModelError("", "Cannot update this application");
}
}
//
return View(application);
}
}
You need to wrap your database operation within a DbContextTransaction. See this link for Entity Framework transaction examples:
https://msdn.microsoft.com/en-us/data/dn456843.aspx
The code which is throwing exception is extremely easy - this is very regular insert and then submit changes statement which looks:
context.tb_dayErrorLog.InsertOnSubmit(data);
context.SubmitChanges();
So really nothing special. This statement is executed about 50 thousands times a day without any problem, but:
about 6 - 10 times a day it finishes with:
The operation cannot be performed during a call to SubmitChanges.
StackTrace: at System.Data.Linq.DataContext.CheckNotInSubmitChanges()
at System.Data.Linq.Table`1.InsertOnSubmit(TEntity entity)
I was trying to find out what that can be but can't find a clue
This behavior is very not deterministic politely saying - how it can finish 50k times correctly and few times not?
DataContext was firstly initialized as a static one, and then reused for all the calls, so I was thinking maybe that's the problem. Then I changed it to be initialized with every call but results are quite similar. Still few exceptions a day.
Any idea?
some additions:
function looks like:
public override bool Log(ErrorLogData logData)
{
try
{
logData.ProcessID = _processID;
//Create new log dataset
var data = new DataRecord
{
application = logData.Application,
date = DateTime.Now,
Other = logData.Other,
process = logData.ProcessName,
processid = logData.ProcessID,
severity = logData.Severity,
username = logData.UserName,
Type = (short)logData.ErrorType
};
var context = new DataContext(ConnectionString);
context.tb_dayErrorLog.InsertOnSubmit(data);
context.SubmitChanges();
}
catch (Exception ex)
{
//log log in eventviewer
LogEvent(logData.ToString(), ex);
return false;
}
return true;
}
so simple record initialization and then insert.
As I wrote in the comment, while making same thing by Ado.Net and SqlCommand this problem is not occuring...
So my curiosity makes me think why?
This sounds like a threading issue where you are calling Log and hence SubmitChanges on one thread when another thread is in the middle of SubmitChanges.
I suspect your DataContext is still a global static variable.
Try changing your Log method to
using (var context = new DataContext(ConnectionString))
{
context.tb_dayErrorLog.InsertOnSubmit(data);
context.SubmitChanges();
}
#SgMoore points to concurrency problem and in my case it really was. If that's the case another approach is to use lock like this:
String lockValue = "";
lock (lockValue)
{
context.tb_dayErrorLog.InsertOnSubmit(data);//UPDATE: concurrency error can occur here too
context.dc.SubmitChanges();
}
This code is giving me optimisticconcurrencyexception was caught. I am simply using a Stub Entity to get an existing record and trying to update a couple values. I am not sure how to resolve the exception. Any help is very much appreciated:
using (MiscEntities ctx = new MiscEntities())
{
var m = ctx.Rates.FirstOrDefault(m => m.UserId == UserIdGuid);
DataAccess.Rate oldDbRate = new DataAccess.Rate { RatingId = m.RatingId };
ctx.AttachTo("Rates", dbRate);
dbRate.Rating = Rating;
dbRate.DateLastModified = DateTime.Now;
ctx.SaveChanges();
}
EF, by default, uses an optimistic concurrency model, meaning that locks are not held on data in the source between when the data is queried and when it is updated. So it doesn't check for any conflicts before saving changes to the database. With any conflicts an OptimisticConcurrencyException is raised (for more information check out How to: Manage Data Concurrency in the Object Context).
It's good practice (when you make updates in a high concurrency scenario) to call Refresh quite often. In this case try using a RefreshMode of ClientWins to to refresh the values in the client store before sending them to the database, like this:
using (MiscEntities ctx = new MiscEntities())
{
try
{
var m = ctx.Rates.FirstOrDefault(m => m.UserId == UserIdGuid);
DataAccess.Rate oldDbRate = new DataAccess.Rate { RatingId = m.RatingId };
ctx.AttachTo("Rates", dbRate);
dbRate.Rating = Rating;
dbRate.DateLastModified = DateTime.Now;
ctx.SaveChanges();
}
catch (OptimisticConcurrencyException)
{
ctx.Refresh(RefreshMode.ClientWins, dbRate);
ctx.SaveChanges();
}
}
EDIT: After more reading, and re-reading that error message it makes sense, you cannot attach an object to an ObjectContext is that object has already that has already been cached by the ObjectStateManager.
The solution is real simple, attach your objects before doing any operations/query in your ObjectContext. This allows you to prevent any double-tracking requests. If the ObjectContext needs your Entity later, it will retrieve the instance you attached before and you're good to go. Take a look at this code and see if it helps (Sorry don't have Visual Studio 2010 opened right now)
using (MiscEntities ctx = new MiscEntities())
{
try
{
ctx.AttachTo("Rates", dbRates);
var m = ctx.Rates.FirstOrDefault(m => m.UserId == UserIdGuid);
DataAccess.Rate oldDbRate = new DataAccess.Rate { RatingId = m.RatingId };
dbRate.Rating = Rating;
dbRate.DateLastModified = DateTime.Now;
ctx.SaveChanges();
}
catch (OptimisticConcurrencyException)
{
ctx.Refresh(RefreshMode.ClientWins, dbRate);
ctx.SaveChanges();
}
}
In the following code doesn't work as
public void Foo()
{
CompanyDataContext db = new CompanyDataContext();
Client client = (select c from db.Clients ....).Single();
Bar(client);
}
public void Bar(Client client)
{
CompanyDataContext db = new CompanyDataContext();
db.Client.Attach(client);
client.SomeValue = "foo";
db.SubmitChanges();
}
This doens't work, I get error msg. "An attempt has been made to Attach or Add an entity that is not new, perhaps having been loaded from another DataContext. This is not supported."
How do you work with DataContexts throughout an application so you don't need to pass around a reference?
What
They really mean it with 'This is not supported.'. Attaching to an object fetched from another data context is not implemented.
There are a number of workarounds to the problem, the recommended way is by serializing objects, however this is not easy nor a clean approach.
The most simple approach I found is to use a readonly DataContext for fetching objects like this:
MyDataContext dataContext = new MyDataContext()
{
DeferredLoadingEnabled = false,
ObjectTrackingEnabled = false
};
The objects obtained from this context can be attached to another context but only applies to some scenarios.
The PLINQO framework generates detach for all entities making it easy to detach and reattach objects without receiving that error.
public void Foo()
{
CompanyDataContext db = new CompanyDataContext();
Client client = (select c from db.Clients ....).Single();
// makes it possible to call detach here
client.Detach();
Bar(client);
}
public void Bar(Client client)
{
CompanyDataContext db = new CompanyDataContext();
db.Client.Attach(client);
client.SomeValue = "foo";
db.SubmitChanges();
}
Here is the article that describing how the detach was implemented.
http://www.codeproject.com/KB/linq/linq-to-sql-detach.aspx
Yep. That's how it works.
You have tagged this asp.net so I guess it's a web app. Maybe you want one datacontext per request?
http://blogs.vertigo.com/personal/keithc/Blog/archive/2007/06/28/linq-to-sql-and-the-quote-request-scoped-datacontext-quote-pattern.aspx
(P.S. It's a lot harder in WinForms!)
I've created data access classes that encapsulate all the communication with Linq2Sql.
These classes have their own datacontext that they use on their objects.
public class ClientDataLogic
{
private DataContext _db = new DataContext();
public Client GetClient(int id)
{
return _db.Clients.SingleOrDefault(c => c.Id == id);
}
public void SaveClient(Client c)
{
if (ChangeSetOnlyIncludesClient(c))
_db.SubmitChanges();
}
}
Ofcourse you will need to keep this object instantiated as long as you need the objects.
Checking if only the rigth object has been changed is altso somewhat bothersom, you could make methods like
void ChangeClientValue(int clientId, int value);
but that can become a lot of code.
Attaching and detaching is a somewhat missing feature from Linq2Sql, if you need to use that a lot, you sould probably use Linq2Entities.
I took a look at this and found that it appears to work fine as long as the original DataContext has been disposed.
Try wrapping the DataContext with using() and make sure your changes occur after you've attached to the second DataContext? It worked for me..
public static void CreateEntity()
{
User user = null;
using (DataClassesDataContext dc = new DataClassesDataContext())
{
user = (from u in dc.Users
select u).FirstOrDefault();
}
UpdateObject(user);
}
public static void UpdateObject(User user)
{
using (DataClassesDataContext dc = new DataClassesDataContext())
{
dc.Users.Attach(user);
user.LastName = "Test B";
dc.SubmitChanges();
}
}
You need to handle object versioning.
An entity can only be attached as modified without original state if it declares a version member or does not have an update check policy.
So, if there's no timestamp member or other 'versioning' mechanism provided there's no way for LINQ to determine whether that data has changed - hence the error you are seeing.
I resolved this issue by adding a timestamp column to my tables but there are other ways around it. Rick Strahl has written some decent articles about exactly this issue.
Also, see this and this for a bit more info.