I want to use System.Transactions and update multiple rows. My database is connected using Entity Framework.
Below is the code I tried but it throws an error :
public void Update(List<PortfolioCompanyLinkModel> record)
{
var transaction = _context.Database.BeginTransaction();
try
{
foreach (var item in record)
{
var portfolioCompanyLink = _context.PortfolioCompanyLink.FirstOrDefault(p => p.Id == item.Id);
portfolioCompanyLink.ModifiedBy = _loggedInUser;
portfolioCompanyLink.ModifiedOn = DateTime.UtcNow;
portfolioCompanyLink.URL = item.URL;
_context.SaveChanges();
//_context.PortfolioCompanyLink.Update(portfolioCompanyLink);
}
transaction.Commit();
}
catch(Exception ex)
{
transaction.Rollback();
}
}
Error:
The configured execution strategy 'SqlServerRetryingExecutionStrategy' does not support user initiated transactions. Use the execution strategy returned by 'DbContext.Database.CreateExecutionStrategy()' to execute all the operations in the transaction as a retriable unit.
Can someone help me on how to proceed with this?
You problem is the SqlServerRetryingExecutionStrategy as described in Microsoft documentation
When not using a retrying execution strategy you can wrap multiple operations in a single transaction. For example, the following code wraps two SaveChanges calls in a single transaction. If any part of either operation fails then none of the changes are applied.
MS docs on resiliency
System.InvalidOperationException: The configured execution strategy 'SqlServerRetryingExecutionStrategy' does not support user initiated transactions. Use the execution strategy returned by 'DbContext.Database.CreateExecutionStrategy()' to execute all the operations in the transaction as a retriable unit.
Solution: Manually Call Execution Strategy
var executionStrategy = _context.db.CreateExecutionStrategy();
executionStrategy.Execute(
() =>
{
// execute your logic here
using(var transaction = _context.Database.BeginTransaction())
{
try
{
foreach (var item in record)
{
var portfolioCompanyLink = _context.PortfolioCompanyLink.FirstOrDefault(p => p.Id == item.Id);
portfolioCompanyLink.ModifiedBy = _loggedInUser;
portfolioCompanyLink.ModifiedOn = DateTime.UtcNow;
portfolioCompanyLink.URL = item.URL;
_context.SaveChanges();
//_context.PortfolioCompanyLink.Update(portfolioCompanyLink);
}
transaction.Commit();
}
catch(Exception ex) {
transaction.Rollback();
}
}
});
You can set the strategy globally too, but that depends on what you are trying to achieve.
Related
I have an application which runs multiple threads to insert data into a SQL Server 2017 database table using EF Core 5.
The C# code for inserting the domain model entities using EF Core 5 is as follows:
using (var ctx = this.dbContextFactory.CreateDbContext())
{
//ctx.Database.AutoTransactionsEnabled = false;
foreach (var rootEntity in request.RootEntities)
{
ctx.ChangeTracker.TrackGraph(rootEntity, node =>
{
if ((request.EntityTypes != null && request.EntityTypes.Contains(node.Entry.Entity.GetType()))
|| rootEntity == node.Entry.Entity)
{
if (node.Entry.IsKeySet)
node.Entry.State = Microsoft.EntityFrameworkCore.EntityState.Modified;
else
node.Entry.State = Microsoft.EntityFrameworkCore.EntityState.Added;
}
});
}
await ctx.SaveChangesAsync(cancellationToken);
}
Each thread is responsible for instantiating its own DbContext instance hence the use of dbContextFactory.
Some example SQL generated for the INSERT (MERGE) is as follows:
SET NOCOUNT ON;
DECLARE #inserted0 TABLE ([OrderId] bigint, [_Position] [int]);
MERGE [dbo].[Orders] USING (
VALUES (#p0, 0),
(#p1, 1),
(#p2, 2),
...
(#43, 41)) AS i ([SomeColumn], _Position) ON 1=0
WHEN NOT MATCHED THEN
INSERT ([SomeColumn])
VALUES (i.[SomeColumn])
OUTPUT INSERTED.[OrderId], i._Position
INTO #inserted0;
SELECT [t].[OrderId] FROM [dbo].[Orders] t
INNER JOIN #inserted0 i ON ([t].[OrderId] = [i].[OrderId])
ORDER BY [i].[_Position];
As these threads frequently run at the same time I get the following SQL exception:
Transaction (Process ID 99) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
EF Core implicitly sets the isolation level to READ COMMITTED.
Using SQL Profiler the transaction deadlock was caused by the following:
My concerns:
Frustratingly, the SQL generated by EF Core includes two statements: a MERGE, and then a SELECT. I do not understand the purpose of the SELECT given the identities of the primary key are available from the #inserted0 table variable. Given this answer, the MERGE statement in isolation would be sufficient enough to make this atomic.
I believe it is this SELECT which is causing the transaction deadlock.
I tried to resolve the problem by using READ COMMITTED SNAPSHOT to avoid the conflict with the primary key lookup, however I still got the same error even though this isolation level should avoid locks and use row versioning instead.
My attempt at solving the problem:
The only way I could find to solve this problem was to explicitly prevent a transaction being started by EF Core, hence the following code:
ctx.Database.AutoTransactionsEnabled = false;
I have tested this numerous times and haven't received a transaction deadlock. Given the logic is merely inserting new records I believe this can be done.
Does anyone have any advice to fixing this problem?
Thanks for your time.
We had the same issues with INSERT (MERGE) statements on multiple threads. We didn't want to enable the EnableRetryOnFailure() option for all transactions, so we wrote the following DbContent extension method.
public static async Task<TResult> SaveWithRetryAsync<TResult>(this DbContext context,
Func<Task<TResult>> bulkInsertOperation,
Func<TResult, Task<bool>> verifyBulkOperationSucceeded,
IsolationLevel isolationLevel = IsolationLevel.Unspecified,
int retryLimit = 6,
int maxRetryDelayInSeconds = 30)
{
var existingTransaction = context.Database.CurrentTransaction?.GetDbTransaction();
if (existingTransaction != null)
throw new InvalidOperationException($"Cannot run {nameof(SaveWithRetryAsync)} inside a transaction");
if (context.ChangeTracker.HasChanges())
{
throw new InvalidOperationException(
"DbContext should be saved before running this action to revert only the changes of this action in case of a concurrency conflict.");
}
const int sqlErrorNrOnDuplicatePrimaryKey = 2627;
const int sqlErrorNrOnSnapshotIsolation = 3960;
const int sqlErrorDeadlock = 1205;
int[] sqlErrorsToRetry = { sqlErrorNrOnDuplicatePrimaryKey, sqlErrorNrOnSnapshotIsolation, sqlErrorDeadlock };
var retryState = new SaveWithRetryState<TResult>(bulkInsertOperation);
// Use EF Cores connection resiliency feature for retrying (see https://learn.microsoft.com/en-us/ef/core/miscellaneous/connection-resiliency)
// Usually the IExecutionStrategy is configured DbContextOptionsBuilder.UseSqlServer(..., options.EnableRetryOnFailure()).
// In ASP.NET, the DbContext is configured in Startup.cs and we don't want this retry behaviour everywhere for each db operation.
var executionStrategyDependencies = context.Database.GetService<ExecutionStrategyDependencies>();
var retryStrategy = new CustomSqlServerRetryingExecutionStrategy(executionStrategyDependencies, retryLimit, TimeSpan.FromSeconds(maxRetryDelayInSeconds), sqlErrorsToRetry);
try
{
var result = await retryStrategy.ExecuteInTransactionAsync(
retryState,
async (state, cancelToken) =>
{
try
{
var r = await state.Action();
await context.SaveChangesAsync(false, cancelToken);
if (state.FirstException != null)
{
Log.Logger.Warning(
$"Action passed to {nameof(SaveWithRetryAsync)} failed {state.NumberOfRetries} times " +
$"(retry limit={retryLimit}, ThreadId={Thread.CurrentThread.ManagedThreadId}).\nFirst exception was: {state.FirstException}");
}
state.Result = r;
return r;
}
catch (Exception ex)
{
context.RevertChanges();
state.NumberOfRetries++;
state.FirstException ??= ex;
state.LastException = ex;
throw;
}
},
(state, cancelToken) => verifyBulkOperationSucceeded(retryState.Result),
context.GetSupportedIsolationLevel(isolationLevel));
context.ChangeTracker.AcceptAllChanges();
return result;
}
catch (Exception ex)
{
throw new InvalidOperationException(
$"DB Transaction in {nameof(SaveWithRetryAsync)} failed. " +
$"Tried {retryState.NumberOfRetries} times (retry limit={retryLimit}, ThreadId={Thread.CurrentThread.ManagedThreadId}).\n" +
$"First exception was: {retryState.FirstException}.\nLast exception was: {retryState.LastException}",
ex);
}
}
With the following CustomSqlServerRetryingExecutionStrategy
public class CustomSqlServerRetryingExecutionStrategy : SqlServerRetryingExecutionStrategy
{
public CustomSqlServerRetryingExecutionStrategy(ExecutionStrategyDependencies executionStrategyDependencies, int retryLimit, TimeSpan fromSeconds, int[] sqlErrorsToRetry)
: base(executionStrategyDependencies, retryLimit, fromSeconds, sqlErrorsToRetry)
{
}
protected override bool ShouldRetryOn(Exception exception)
{
//SqlServerRetryingExecutionStrategy does not check the base exception, maybe a bug in EF core ?!
return base.ShouldRetryOn(exception) || base.ShouldRetryOn(exception.GetBaseException());
}
}
Helper class to save the current (retry) state:
private class SaveWithRetryState<T>
{
public SaveWithRetryState(Func<Task<T>> action)
{
Action = action;
}
public Exception FirstException { get; set; }
public Exception LastException { get; set; }
public int NumberOfRetries { get; set; }
public Func<Task<T>> Action { get; }
public T Result { get; set; }
}
Now, the extension method can be used as follow. The code will try to add the bulk multiple times (5).
await _context.SaveWithRetryAsync(
// method to insert the bulk
async () =>
{
var listOfAddedItems = new List<string>();
foreach (var item in bulkImport)
{
listOfAddedItems.Add(item.Guid);
await context.Import.AddAsync(item);
}
return listOfAddedItems;
},
// method to check if the bulk insert was successful
listOfAddedItems =>
{
if (listOfAddedItems == null)
return Task.FromResult(true);
return _context.Import.AsNoTracking().AnyAsync(x => x.Guid == listOfAddedItems.First());
},
IsolationLevel.Snapshot,
5, // max retry attempts
100); // max retry time
For background information why this can happen, have a look at this discussion: https://github.com/dotnet/efcore/issues/21899
I'm on a team using an EF, Code-first approach with ODP.Net (Oracle). We need to attempt to write updates to multiple rows in a table, and store any exceptions in a collection to be bubbled up to a handler (so writing doesn't halt because one record can't be written). However, this code throws an exception saying
System.InvalidOperationException: The operation cannot be completed because the DbContext has been disposed.
I'm not sure why. The same behavior occurs if the method is changed to be a synchronous method and uses .Find().
InvModel _model;
public InvoiceRepository(InvModel model)
{
_model = model;
}
public void SetStatusesToSent(IEnumerable<Invoice> Invoices)
{
var exceptions = new List<Exception>();
foreach (var id in invoices)
{
try
{
var iDL = await _model.INVOICES.FindAsync(id);/*THROWS A DBCONTEXT EXCEPTION HERE*/
iDL.STATUS = Statuses.Sent; // get value from Statuses and assign
_model.SaveChanges(); //save changes to the model
}
catch (Exception ex)
{
exceptions.Add(ex);
continue; //not necessary but makes the intent more legible
}
}
}
Additional detail update: _model is injected by DI.
Remember that LINQ executes lazily - that is when you actually use the information.
The problem might be, that Your DbContext has gone out of scope...
Use .ToList() or .ToArray() to force execution at that time.
I have a process where I retrieve records from a database periodically, and run 3 operations on each. For each record, the 3 operations must either all succeed, or none at all. In case of a failure on one of the operations, I want the operations that have been already processed for the previous records to be
committed, so that next time the process runs, it picks up on the record for which one of the 3 transactions failed previously.
I thought of wrapping the 3 operations in a transaction per record, and loop for each record, but I want to ensure that using a database transaction in this scenario is efficient. The following is what have in mind. Is it correct?
public async Task OrderCollectionProcessorWorker()
{
using (var context = new DbContext())
{
try
{
IList<Order> ordersToCollect =
await context.Orders.Where(
x => x.OrderStatusId == OrderStatusCodes.DeliveredId)
.ToListAsync(_cancellationTokenSource.Token);
await ProcessCollectionsAsync(context, ordersToCollect);
}
catch (Exception ex)
{
Log.Error("Exception in OrderCollectionProcessorWorker", ex);
}
}
}
/// <summary>
/// For each order to collect, perform 3 operations
/// </summary>
/// <param name="context">db context</param>
/// <param name="ordersToCollect">List of Orders for collection</param>
private async Task ProcessCollectionsAsync(DbContext context, IList<Order> ordersToCollect)
{
if (ordersToCollect.Count == 0) return;
Log.Debug($"ProcessCollections: processing {ordersToCollect.Count} orders");
foreach (var order in ordersToCollect)
{
// group the 3 operations in one transaction for each order
// so that if one operation fails, the operations performend on the previous orders
// are committed
using (var transaction = context.Database.BeginTransaction())
{
try
{
// *************************
// run the 3 operations here
// operations consist of updating the order itself, and other database updates
Operation1(order);
Operation2(order);
Operation3(order);
// *************************
await context.SaveChangesAsync();
transaction.Commit();
}
catch (Exception ex)
{
transaction?.Rollback();
Log.Error("General exception when executing ProcessCollectionsAsync on Order " + order.Id, ex);
throw new Exception("ProcessCollections failed on Order " + order.Id, ex);
}
}
}
}
It seems like a correct way of doing it, apart perhaps from fact that in catch you should rethrow the exception or do something else to stop progressing on the loop (if I understood correctly your requirments). It is even not necessary to use
var transaction = context.Database.BeginTransaction()
because
await context.SaveChangesAsync();
creates its own transaction. Every change you made is stored in the context and when you call SaveChanges there is transaction made and all the changes are written as 1 batch. If something fails all the changes are rollbacked. Another call to SaveChanges will make another transaction on new changes. Please bear in mind however that in case transaction fails you should no longer use the same context but create a new one. To summarize I would write your method as follows:
private async Task ProcessCollectionsAsync(DbContext context, IList<Order> ordersToCollect)
{
if (ordersToCollect.Count == 0) return;
Log.Debug($"ProcessCollections: processing {ordersToCollect.Count} orders");
foreach (var order in ordersToCollect)
{
// group the 3 operations in one transaction for each order
// so that if one operation fails, the operations performend on the previous orders
// are committed
try
{
// *************************
// run the 3 operations here
// operations consist of updating the order itself, and other database updates
Operation1(order);
Operation2(order);
Operation3(order);
// *************************
await context.SaveChangesAsync();
}
catch (Exception ex)
{
Log.Error("General exception when executing ProcessCollectionsAsync on Order " + order.Id, ex);
throw;
}
}
I have below method
public void UpdateQuantity()
{
Sql ss = new Sql();
M3 m3 = new M3();
TransactionOptions ff = new TransactionOptions();
ff.IsolationLevel = IsolationLevel.ReadUncommitted;
using (TransactionScope dd = new TransactionScope(TransactionScopeOption.Required, ff))
{
try
{
ss.AddRegion("ALFKI", "SES1"); //step 1
m3.UpdateAnotherSystem(); //step2
dd.Complete();
}
catch (Exception)
{
}
}
}
public void AddRegion(string customerName, string Deception)
{
using (NorthWind context = new NorthWind())
{
Region rr = new Region();
rr.RegionID = 5;
rr.RegionDescription = "Ssaman";
context.Regions.Add(rr);
try
{
context.SaveChanges();
}
catch (Exception)
{
throw;
}
}
}
In that first im going to update Sql server data base .After that im going to perform another update on other system.If step2 fails(may be network failure) then i need to reverse step 1.There for i put two method calls inside the transactionscope. I'm use entity framework to work with sql.Entity framework always set the transaction isolation level as read committed(according to the sql profiler).
but my problem is after context.SaveChanges() called my target table is locked till transaction completes(dd.Complete()).
Are there are any way to change entity framework transaction isolation level?(My entity framework version is 5).
SQL Server does not release locks that were taken due to writes until the end of the transaction. This is so that writes can be rolled back. You cannot do anything about this.
End your transaction or live with the fact that the rows written are still in use. Normally, this is not a problem. You should probably have a single context, connection and transaction for most work that happens in an HTTP request or WCF request. Transactions do not block on themselves.
using (var context = new BloggingContext())
{
using (var dbContextTransaction = context.Database.BeginTransaction())
{
try
{
context.Database.ExecuteSqlCommand(
#"UPDATE Blogs SET Rating = 5" +
" WHERE Name LIKE '%Entity Framework%'"
);
var query = context.Posts.Where(p => p.Blog.Rating >= 5);
foreach (var post in query)
{
post.Title += "[Cool Blog]";
}
context.SaveChanges();
dbContextTransaction.Commit();
}
catch (Exception)
{
dbContextTransaction.Rollback();
}
}
}
string[] usersToAdd = new string[] { "asd", "asdert", "gasdff6" };
using (Entities context = new Entities())
{
foreach (string user in usersToAdd)
{
context.AddToUsers(new User { Name = user });
}
try
{
context.SaveChanges(); //Exception thrown: user 'gasdff6' already exist.
}
catch (Exception e)
{
//Roll back all changes including the two previous users.
}
Or maybe this is done automatically, meaning that if error occurs, committing changes are canceled for all the changes.
is it?
OK
I created a sample a application like the example from the the question and afterwords I checked in the DB and no users were added.
Conclusion: ObjectContext.SaveChange it's automatically a transaction.
Note: I believe transactions will be needed if executing sprocs etc.
I believe (but I am no long time expert in EF) that until the call to context.SaveChanges goes through, the transaction is not started. I'd expect an Exception from that call would automatically rollback any transaction it started.
Alternatives (in case you want to be in control of the transaction) [from J.Lerman's "Programming Entity Framework" O'Reilly, pg. 618]
using (var transaction = new System.Transactions.TransactionScope())
{
try
{
context.SaveChanges();
transaction.Complete();
context.AcceptAllChanges();
}
catch(OptimisticConcurrencyException e)
{
//Handle the exception
context.SaveChanges();
}
}
or
bool saved = false;
using (var transaction = new System.Transactions.TransactionScope())
{
try
{
context.SaveChanges();
saved = true;
}
catch(OptimisticConcurrencyException e)
{
//Handle the exception
context.SaveChanges();
}
finally
{
if(saved)
{
transaction.Complete();
context.AcceptAllChanges();
}
}
}