I have N process to run over SQL Server 2008. If any of the processes fails I need to rollback all the others.
I was thinking to use the TPL creating a parent task and N child task. All of this enclosed with a transactionScope (IsolationLevel.ReadCommitted) but in my example below, the child2 throws an error(customers2 is not a valid table) and the child1 doesn't rolled back.
Am I assuming something wrong here? is there other way to manage this scenario?
Here is my test code:
edit
I modified the code as below using the DependClone on the current transaction. I think is working.
try
{
using (TransactionScope mainTransaction = TransactionUtils.CreateTransactionScope())
{
var parentTransactionClone1 = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var parentTransactionClone2 = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var parentTask = Task.Factory.StartNew(() =>
{
var childTask1 = Task.Factory.StartNew(() =>
{
using (TransactionScope childScope1 = new TransactionScope(parentTransactionClone1))
{
SqlConnection cnn = new SqlConnection("Server=.\\sqlexpress;Database=northwind;Trusted_Connection=True;");
cnn.Open();
SqlCommand cmd = new SqlCommand("update customers set city ='valXXX' where customerID= 'ALFKI'", cnn);
cmd.ExecuteNonQuery();
cnn.Close();
childScope1.Complete();
}
parentTransactionClone1.Complete();
}, TaskCreationOptions.AttachedToParent);
var childTask2 = Task.Factory.StartNew(() =>
{
using (TransactionScope childScope2 = new TransactionScope(parentTransactionClone2))
{
SqlConnection cnn = new SqlConnection("Server=.\\sqlexpress;Database=northwind;Trusted_Connection=True;");
cnn.Open();
SqlCommand cmd = new SqlCommand("update customers2 set city ='valyyy' where customerID= 'ANATR'", cnn);
cmd.ExecuteNonQuery();
cnn.Close();
childScope2.Complete();
}
parentTransactionClone2.Complete();
}, TaskCreationOptions.AttachedToParent);
});
parentTask.Wait();
mainTransaction.Complete();
}
}
catch (Exception ex)
{
// manage ex
}
public static TransactionScope CreateTransactionScope()
{
var transactionOptions = new TransactionOptions();
transactionOptions.IsolationLevel = IsolationLevel.ReadCommitted;
transactionOptions.Timeout = TransactionManager.MaximumTimeout;
return new TransactionScope(TransactionScopeOption.Required, transactionOptions);
}
The TransactionScope class sets the ambient transaction for the current thread (see also Transaction.Current only.
You should at least assume that each task runs in a separate thread (although that is not a necessity with the TPL).
Review the "important" box in the remarks section of the relevant article - if you want to share a transaction between threads, you need to use the DependentTransaction class.
Personally, I am sure that the whole facility to share a transaction amongst multiple threads works technically, however, I have always found it easier to write up a design that uses a seperate transaction per thread.
Task Parallel Library cannot figure out on its own the details of the task, and it won't be rolling back automatically, the closest you can do is from the parent task you define another task child1-rollback that gets executed only if child1 fails, and you can define this very nicely by specifying a TaskContinuationOption set to OnlyOnFailure so the task will execute only if child1 fails, same can be said about child2.
Related
I have an odd situation with TransactionScope and async/synchronous SQL calls that I'm having difficulty understanding. I hope that someone with a deeper understanding of the ins and outs of these kinds of operations can shed some light on the issue.
The situation:
I have a NUnit testfixture which creates a TransactionScope during [SetUp] and Disposes it at [TearDown] to let each test run on the same data. I have a series of tests which kick off an asynchronous operation on the database and then execute a synchronous operation on the database. The first such test completes successfully. The second such test fails with "There is already an open DataReader associated with this Command which must be closed first.".
If I comment out the TransactionScope entirely, all the tests pass.
I tried various different TransactionScope options, and Complete / Dispose, but the same issue occurs.
I am using the Resharper test runner on an NUnit test, .NET 4.5.1.
I realize the "correct" answer may be "make everything async await". That's not an option for me, unfortunately.
I don't want to enable MARS, as this issue only occurs in tests.
I don't want to use GetAwaiter().GetResult() due to the potential deadlocks.
What it looks like to me is that once a TransactionScope.Dispose/Complete is called, the automatic SQLConnection pooling loses track of which connections have open DataReaders. It hands out the same SqlConnection to two simultaneously running operations, and the second dies.
My primary question is "what is causing this behavior (specifically)?"
My secondary question is "is there anything that can be done to safely resolve the issue?"
The replicating code below prints out the client connection Ids. On my machine, the ClientConnectionId for the ASYNC and SYNC calls in the Second test case are always the same.
Replicating Code:
[TestFixture]
public class DataReaderTests
{
private TransactionScope _scope;
private string _connString = #"my connection string";
[SetUp]
public void Setup()
{
var options = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = TimeSpan.FromMinutes(1)
};
_scope = new TransactionScope(TransactionScopeOption.RequiresNew, options, TransactionScopeAsyncFlowOption.Enabled);
}
[Test]
[TestCase("First")]
[TestCase("Second")]
public void Test(string name)
{
DoAsyncThing().ConfigureAwait(false);
using (var conn = new SqlConnection(_connString))
{
try
{
conn.Open();
Console.WriteLine("SYNC: " + conn.ClientConnectionId);
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT 1";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
int id = reader.GetInt32(0);
}
}
}
}
catch (TransactionAbortedException tax)
{
Console.WriteLine("ERROR: " + ((SqlException)tax.InnerException.InnerException).ClientConnectionId);
throw;
}
}
}
private async Task DoAsyncThing()
{
using (var connection = new SqlConnection(_connString))
{
await connection.OpenAsync();
Console.WriteLine("ASYNC: " + connection.ClientConnectionId);
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = "WAITFOR DELAY '00:02';";
await cmd.ExecuteNonQueryAsync();
Console.WriteLine("ASYNC COMPLETE");
}
}
}
[TearDown]
public void Teardown()
{
_scope.Dispose();
}
}`
Check out this answer
I think the gist is that you cannot have two active sql commands executing over the same connection at the same time without a special connection string property. When you are operating under the transaction scope, you should find that both SqlConnection objects have the same client ID. However, if you remove the transaction scope they are different, which I believe implies that they are operating on separate connections.
Adding "MultipleActiveResultSets=true" to the connection string fixed the issue for me. Another alternative is to replace
DoAsyncThing().ConfigureAwait(false);
with
DoAsyncThing().ConfigureAwait(false).GetAwaiter().GetResult();
which will terminate the first command before starting the second command.
I'm having an issue getting my transaction scope to rollback while using async/await. Everything works as it should without the transaction scope, but whenever I intentionally cause an exception (duplicate primary key on the insert for 2nd iteration), no rollback (for the update) or any sort of transaction related error occurs.
I should also note that unless "OLE DB Services=-4" is in the connection string, I receive the error:
"The ITransactionLocal interface is not supported by the 'Microsoft.ACE.OLEDB.12.0' provider. Local transactions are unavailable with the current provider."
The code in the button event handler below is just an example for testing the transaction scope. The main goal is to be able to update multiple tables in a loop that's contained in a transaction asynchronously, so I can avoid UI deadlocks and perform rollbacks for any exceptions that may occur during the loop. Any alternatives or suggestions to my problem are appreciated, thanks :)
private async void button1_Click(object sender, EventArgs e)
{
try
{
int customerCount = 150; // First 150 rows of customer table
TransactionScope transaction = null;
using (OleDbConnection dbConn = new OleDbConnection(Provider = Microsoft.ACE.OLEDB.12.0; OLE DB Services=-4; Data Source = " + filePath))
{
dbConn.Open();
using (transaction = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
for (int i = 0; i < customerCount; i++)
{
// Update field indicating customer made an invoice
var taskName = sql.executeAsync("UPDATE Customer SET lastInvoiceDate = #date WHERE customerID = #custID", dbConn,
new OleDbParameter("#date", DateTime.Today),
new OleDbParameter("#custID", i));
// Insert new invoice - Breaks here
var taskInsert = sql.executeAsync("INSERT INTO Invoice VALUES (1, 'thisisatestinvoice', '$100.50')", dbConn);
await Task.WhenAll(taskName, taskInsert);
}
}
// All updates executed properly
transaction.Complete();
}
}
catch (AggregateException exception)
{
foreach (Exception ex in exception.InnerExceptions)
{
MessageBox.Show(ex.Message);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
public async Task executeAsync(string dbQuery, OleDbConnection dbConn, params OleDbParameter[] parameters)
{
var dbComm = new OleDbCommand(dbQuery, dbConn);
if (parameters != null)
dbComm.Parameters.AddRange(parameters);
await dbComm.ExecuteNonQueryAsync().ConfigureAwait(false);
}
I wasn't able to get the transaction scope to work, and I'm not entirely sure what the issue is, I think it's due to me being on ACE.OLEDB.12.0, but I found another alternative with OleDbTransaction that will rollback if any failures occur.
private async void button1_Click(object sender, EventArgs e)
{
try
{
using (OleDbConnection dbConn = new OleDbConnection(SQLWrapper.CONNECT_STRING))
{
dbConn.Open();
OleDbTransaction dbTrans = dbConn.BeginTransaction();
var taskName = sql.executeAsync("UPDATE Config SET Busname = #name", dbConn, dbTrans,
new OleDbParameter("#name", "name"));
var taskInsert = sql.executeAsync("INSERT INTO Callout VALUES (16, 'ryanistesting')", dbConn, dbTrans);
await Task.WhenAll(taskName, taskInsert);
dbTrans.Commit();
}
}
}
public async Task executeAsync(string dbQuery, OleDbConnection dbConn, OleDbTransaction dbTrans, params OleDbParameter[] parameters)
{
using (var dbComm = new OleDbCommand(dbQuery, dbConn))
{
if (parameters != null)
dbComm.Parameters.AddRange(parameters);
if (dbTrans != null)
dbComm.Transaction = dbTrans;
await dbComm.ExecuteNonQueryAsync().ConfigureAwait(false);
}
}
Maybe you know the answer by now as is' been a while. Same issue happened to me. Thing is that when using ConfigureAwait(false) you are telling the runtime to pick any free thread from the thread pool instead of waiting for the previous used when calling the async method. A different thread is used with a different context, creating another connection under the same transaction scope which was meant for only one connection. Then the transaction gets promoted to a distributed transaction. That was my experience, I had to abandon the idea of using async as it best and wait to for the previos thread to finish by not using configureAwait(false) or Task.WhenAll. Hope that help!
I read a lot of articles on Transaction but I want to build my own example of nested Transactions to see how it really works in c#. I already have a good idea about them in SQl but C# is giving me a tough time. SO I came here for an example that can explain how Nested Transactions work.
I tried the following code to check if the inner transaction will be committed or not. Since the TransactionScope property is RequiresNew the inner transaction should execute but I introduced a deliberate Unique key violation in outer Transaction and my inner transaction didn't execute. Why? Is my concept messed up?
Database _Cataloguedatabase = DatabaseFactory.CreateDatabase();
public void TransferAmountSuppress(Account a)
{
var option = new TransactionOptions();
option.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
try
{
using (TransactionScope outerScope = new TransactionScope(TransactionScopeOption.RequiresNew, option))
{
using (DbCommand cmd = _Cataloguedatabase.GetStoredProcCommand(SpUpdateCredit))
{
_Cataloguedatabase.AddInParameter(cmd, CreditAmountParameter, DbType.String, a.Amount);
_Cataloguedatabase.AddInParameter(cmd, CodeParameter, DbType.String, a.Code);
_Cataloguedatabase.ExecuteNonQuery(cmd);
}
using (TransactionScope innerScope = new TransactionScope(TransactionScopeOption.RequiresNew, option))
{
using (DbCommand cmd = _Cataloguedatabase.GetStoredProcCommand(SpUpdateDebit))
{
_Cataloguedatabase.AddInParameter(cmd, DebitAmountParameter, DbType.String, a.Amount);
_Cataloguedatabase.ExecuteNonQuery(cmd);
}
innerScope.Complete();
}
outerScope.Complete();
}
}
catch (Exception ex)
{
throw new FaultException(new FaultReason(new FaultReasonText(ex.Message)));
}
}
I have multiple methods inside a Parallel.Invoke() that need to run inside of a transaction. These methods all invoke instances of SqlBulkCopy The use-case is "all-or-none", so if one method fails nothing gets committed. I am getting a TransactionAbortedException ({"Transaction Timeout"}) when I call the Complete() method on the parent transaction.
This is the parent transaction:
using (var ts = new TransactionScope())
{
var saveClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var saveErrorsClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var saveADClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var saveEnrollmentsClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
Parallel.Invoke(_options, () =>
{
Save(data, saveClone);
},
() =>
{
SaveErrors(saveErrorsClone);
},
() =>
{
SaveEnrollments(data, saveEnrollmentsClone);
});
ts.Complete();
}//***** GET THE EXCEPTION HERE *****
Here's a dependent transaction that makes use of SqlBulkCopy (they're all the same structure). I'm passing-in the parent and assigning it to the child's TransactionScope
private void Save(IDictionary<string, string> data, Transaction transaction)
{
var dTs = (DependentTransaction)transaction;
if (transaction.TransactionInformation.Status != TransactionStatus.Aborted)
{
using (var ts = new TransactionScope(dTs))
{
_walmartData.Save(data);
Debug.WriteLine("Completed Processing XML - {0}", _stopWatch.Elapsed);
ts.Complete();
}
}
else
{
Debug.WriteLine("Save Not Executed - Transaction Aborted - {0}", _stopWatch.Elapsed);
dTs.Complete();
}
dTs.Complete();
}
EDIT (added my SqlBulkCopy method...notice null for the transaction param)
private void SqlBulkCopy(DataTable dt, SqlBulkCopyColumnMappingCollection mappings)
{
try
{
using (var sbc = new SqlBulkCopy(_conn, SqlBulkCopyOptions.TableLock, null))
{
sbc.BatchSize = 100;
sbc.BulkCopyTimeout = 0;
sbc.DestinationTableName = dt.TableName;
foreach (SqlBulkCopyColumnMapping mapping in mappings)
{
sbc.ColumnMappings.Add(mapping);
}
sbc.WriteToServer(dt);
}
}
catch (Exception)
{
throw;
}
}
Besides fixing the error, I'm open to alternatives. Thanks.
You're creating a form of deadlock with your choice of DependentCloneOption.BlockCommitUntilComplete.
Parallel.Invoke blocks the calling thread until all of its processing is complete. The jobs trying to be completed by Parallel.Invoke are all blocking while waiting for the parent transaction to complete (due to the DependentCloneOption). So the 2 are waiting on each other... deadlock. The parent transaction eventually times out and releases the dependent transactions from blocking, which unblocks your calling thread.
Can you use DependentCloneOption.RollbackIfNotComplete ?
http://msdn.microsoft.com/en-us/library/system.transactions.transactionscope.complete.aspx says that TransactionScope.Complete only commits the transaction it contains if it was the one that created it. Since you are creating the scope from an existing transaction I believe you will need to commit the transaction before calling complete on the scope.
From MSDN:
The actual work of commit between the resources manager happens at the
End Using statement if the TransactionScope object created the
transaction. If it did not create the transaction, the commit occurs
whenever Commit is called by the owner of the CommittableTransaction
object. At that point the Transaction Manager calls the resource
managers and informs them to either commit or rollback, based on
whether this method was called on the TransactionScope object
.
After a lot of pain, research, and lack of a valid answer, I've got to believe that it's not possible with the stack that I described in my question. The pain-point, I believe, is between TransactionScope and SqlBulkCopy. I put this answer here for the benefit of future viewers. If someone can prove that it can be done, I'll gladly remove this as the answer.
I believe that how you create your _conn-instance matters a lot, if you create it and open it within your TransactionScope-instance any SqlBulkCopy-related issues should be solved.
Have a look at Can I use SqlBulkCopy inside Transaction and Is it possible to use System.Transactions.TransactionScope with SqlBulkCopy? and see if it helps you.
void MyMainMethod()
{
using (var ts = new TransactionScope())
{
Parallell.InvokeOrWhatNotOrWhatEver(() => DoStuff());
}
}
void DoStuff()
{
using (var sqlCon = new SqlConnection(conStr))
{
sqlCon.Open(); // ensure to open it before SqlBulkCopy can open it in another transactionscope.
using (var bulk = new SqlBulkCopy(sqlCon))
{
// Do you stuff
bulk.WriteToServer...
}
ts.Complete(); // finish the transaction, ie commit
}
}
In short:
Create transaction scope
Create sql-connection and open it under the transaction scope
Create and use SqlBulkCopy-instance with above created conncection
Call transaction.Complete()
Dispose of everything :-)
can I do something like this:
using (var scope = new TransactionScope())
{
using (var conn = new SqlConnection(Cs))
{
using (var cmd = conn.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
...
scope.complete();
}
}
}
is going to be the same thing as using the SqlTransaction with catch(){rollback;}
TransactionScope creates an implicit transaction, therefore any action within the transaction scope that supports transactional processing, will be handled by their according Transaction Manager.
In the case of database processing, it will use SqlTransaction. So yes, your given code will do the same as doing it by hand using the SqlTransaction.
You can call transaction.Current to get a handler to the current transaction manager to ensure this.
Note that one TransactionScope object can have óne implicit transaction type, so you will need to nest TransactionScope objects to ensure transactional processing over multiple managers. F.e.:
using(var ts = new TransactionScope)
{
using(var db = new TransactionScope)
{
//do db processing
}
using(var msmq = new TransactionScope)
{
//do msmq processing
}
ts.Complete();
}