Develop Reference

Parallel.Invoke(), TransactionScope() and SqlBulkCopy

I have multiple methods inside a Parallel.Invoke() that need to run inside of a transaction. These methods all invoke instances of SqlBulkCopy The use-case is "all-or-none", so if one method fails nothing gets committed. I am getting a TransactionAbortedException ({"Transaction Timeout"}) when I call the Complete() method on the parent transaction.
This is the parent transaction:
using (var ts = new TransactionScope())
{
var saveClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var saveErrorsClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var saveADClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var saveEnrollmentsClone = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
Parallel.Invoke(_options, () =>
{
Save(data, saveClone);
},
() =>
{
SaveErrors(saveErrorsClone);
},
() =>
{
SaveEnrollments(data, saveEnrollmentsClone);
});
ts.Complete();
}//***** GET THE EXCEPTION HERE *****
Here's a dependent transaction that makes use of SqlBulkCopy (they're all the same structure). I'm passing-in the parent and assigning it to the child's TransactionScope
private void Save(IDictionary<string, string> data, Transaction transaction)
{
var dTs = (DependentTransaction)transaction;
if (transaction.TransactionInformation.Status != TransactionStatus.Aborted)
{
using (var ts = new TransactionScope(dTs))
{
_walmartData.Save(data);
Debug.WriteLine("Completed Processing XML - {0}", _stopWatch.Elapsed);
ts.Complete();
}
}
else
{
Debug.WriteLine("Save Not Executed - Transaction Aborted - {0}", _stopWatch.Elapsed);
dTs.Complete();
}
dTs.Complete();
}
EDIT (added my SqlBulkCopy method...notice null for the transaction param)
private void SqlBulkCopy(DataTable dt, SqlBulkCopyColumnMappingCollection mappings)
{
try
{
using (var sbc = new SqlBulkCopy(_conn, SqlBulkCopyOptions.TableLock, null))
{
sbc.BatchSize = 100;
sbc.BulkCopyTimeout = 0;
sbc.DestinationTableName = dt.TableName;
foreach (SqlBulkCopyColumnMapping mapping in mappings)
{
sbc.ColumnMappings.Add(mapping);
}
sbc.WriteToServer(dt);
}
}
catch (Exception)
{
throw;
}
}
Besides fixing the error, I'm open to alternatives. Thanks.

You're creating a form of deadlock with your choice of DependentCloneOption.BlockCommitUntilComplete.
Parallel.Invoke blocks the calling thread until all of its processing is complete. The jobs trying to be completed by Parallel.Invoke are all blocking while waiting for the parent transaction to complete (due to the DependentCloneOption). So the 2 are waiting on each other... deadlock. The parent transaction eventually times out and releases the dependent transactions from blocking, which unblocks your calling thread.
Can you use DependentCloneOption.RollbackIfNotComplete ?

http://msdn.microsoft.com/en-us/library/system.transactions.transactionscope.complete.aspx says that TransactionScope.Complete only commits the transaction it contains if it was the one that created it. Since you are creating the scope from an existing transaction I believe you will need to commit the transaction before calling complete on the scope.
From MSDN:
The actual work of commit between the resources manager happens at the
End Using statement if the TransactionScope object created the
transaction. If it did not create the transaction, the commit occurs
whenever Commit is called by the owner of the CommittableTransaction
object. At that point the Transaction Manager calls the resource
managers and informs them to either commit or rollback, based on
whether this method was called on the TransactionScope object
.

After a lot of pain, research, and lack of a valid answer, I've got to believe that it's not possible with the stack that I described in my question. The pain-point, I believe, is between TransactionScope and SqlBulkCopy. I put this answer here for the benefit of future viewers. If someone can prove that it can be done, I'll gladly remove this as the answer.

I believe that how you create your _conn-instance matters a lot, if you create it and open it within your TransactionScope-instance any SqlBulkCopy-related issues should be solved.
Have a look at Can I use SqlBulkCopy inside Transaction and Is it possible to use System.Transactions.TransactionScope with SqlBulkCopy? and see if it helps you.
void MyMainMethod()
{
using (var ts = new TransactionScope())
{
Parallell.InvokeOrWhatNotOrWhatEver(() => DoStuff());
}
}
void DoStuff()
{
using (var sqlCon = new SqlConnection(conStr))
{
sqlCon.Open(); // ensure to open it before SqlBulkCopy can open it in another transactionscope.
using (var bulk = new SqlBulkCopy(sqlCon))
{
// Do you stuff
bulk.WriteToServer...
}
ts.Complete(); // finish the transaction, ie commit
}
}
In short:
Create transaction scope
Create sql-connection and open it under the transaction scope
Create and use SqlBulkCopy-instance with above created conncection
Call transaction.Complete()
Dispose of everything :-)

TransactionScope has aborted transaction before disposal

When using TransactionScope it apperars that if internally executed code rolled back the transaction than the parent transaction will rollback as well. Which is good for me. But when disposing that scope it throws an exception meaning that transaction was rolled back already and aborted.
So what is the right way to handle that and properly dispose the scope?
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew))
{
using (var conn = GetConnection())
{
string query =
#"some query that may contain transaction itself
or some SP whith transaction included"
using (var command = new SqlCommand(query, conn))
command.ExecuteNonQuery();
}
}
scope.Complete();
} // Exception here

scope.Dispose() may throw TransactionAborted exception even if scope.Complete() has been called. For example some stored procedures a smart enough to handle exceptions and abort transaction inside T-SQL script using T-SQL TRY/CATCH construct w/o throwing exception to the caller.
So I would consider the safest approach I would suggest is as follows:
try
{
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew))
{
try
{
using (var conn = GetConnection())
{
string query =
#"some query that may contain transaction itself
or some SP whith transaction included"
using (var command = new SqlCommand(query, conn))
command.ExecuteNonQuery();
}
}
catch (SqlException ex)
{
// log SQL Exception, if any
throw; // re-throw exception
}
scope.Complete();
}
}
catch (TransactionAbortedException ex)
{
// we can get here even if scope.Complete() was called.
// log TransactionAborted exception if necessary
}
And don't worry about disposing TransactionScope. scope.Dispose performs whatever necessary to clean it up before throwing TransactionAborted exception.

I do not use stored procs or SQL-specific try/catch, so my situation was slightly different but I got the exact same transaction aborted exception mentioned in the post. I found that if I have a SELECT somewhere inside the primary TransactionScope, that will cause the Transaction to commit, although I am not sure why. I had a case where in order to create an object in the database it first checked to make sure the object didn't already exist with a SELECT, and then the abort exception occurred when Dispose was called. I looked at the Inner Exception and it said the Transaction was trying to commit without a begin. I finally tried wrapping my SELECT in a Suppressed TransactionScope, and then it worked. So:
using(TransactionScope tx = new TransactionScope())
{
//UPDATE command for logging that I want rolled back if CREATE fails
using(TransactionScope tx2 = new TransactionScope(TransactionScopeOption.Suppress))
{
// SELECT command
}
//If not exists logic
//CREATE command
} //Used to error here, but not with the SELECT Suppressed
I hope this helps anyone else who might have gotten this exception without using stored procs.

If an exception throws from your inner query the scope.Complete() line 'll not executed.
Please refer the link below.. And I've made some changes to your query also. I hope it should work for you.
Transaction Scope
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew))
{
using (var conn = GetConnection())
{
string query =
#"some query that may contain transaction itself
or some SP whith transaction included"
using (var command = new SqlCommand(query, conn))
command.ExecuteNonQuery();
}
scope.Complete();
}

Is Rollback automatic in a "Using" scope with C# SQL Server calls?

When you create 'using' blocks for your SQL Connection, Transaction, and Command, it is well known that the connection, transaction, or command that the using block is associated with is disposed on its own properly after you leave the using block.
If an exception occurs in one of these blocks though, for instance in the command block - Would the transaction be rolled back on its own, or do developers need to do a try catch inside of the command 'using' block, and add a rollback transaction statement in the catch for this try?

The transaction is rolled back automatically as long as you haven't successfully called Commit. So your using blocks can look something like this, and the transaction will be rolled back if an exception is thrown before the Commit.
using (IDbConnection connection = ...)
{
connection.Open();
using (IDbTransaction transaction = connection.BeginTransaction())
{
using (IDbCommand command = ...)
{
command.Connection = connection;
command.Transaction = transaction;
...
}
...
transaction.Commit();
}
}

It's not guaranteed to get disposed. The Dispose(bool) method of the SqlTransaction will in fact roll it back conditionally:
// System.Data.SqlClient.SqlTransaction
protected override void Dispose(bool disposing)
{
if (disposing)
{
SNIHandle target = null;
RuntimeHelpers.PrepareConstrainedRegions();
try
{
target = SqlInternalConnection.GetBestEffortCleanupTarget(this._connection);
if (!this.IsZombied && !this.IsYukonPartialZombie)
{
this._internalTransaction.Dispose();
}
}
catch (OutOfMemoryException e)
{
this._connection.Abort(e);
throw;
}
catch (StackOverflowException e2)
{
this._connection.Abort(e2);
throw;
}
catch (ThreadAbortException e3)
{
this._connection.Abort(e3);
SqlInternalConnection.BestEffortCleanup(target);
throw;
}
}
base.Dispose(disposing);
}
and if you notice, it would only happen if this._internalTransaction.Dispose(); got called. The problem here is that if GetBestEffortCleanupTarget throws an exception it won't get cleaned up.
In your case, as long as an exception isn't thrown as already stated, you will fall into the category of being Zombied and so it will then actually issue a Rollback call in the _internalTransaction.Dispose() call.
Finally, if this is called with false it will most certainly not get disposed.
Now, unless I'm really missing something here I'm a bit appalled at how fragile this code is.
An interesting note is that I think the MSDN documentation is actually wrong because it states, for the Rollback() method:
The transaction can only be rolled back from a pending state (after BeginTransaction has been called, but before Commit is called). The transaction is rolled back in the event it is disposed before Commit or Rollback is called.

How to handle sql transaction in this scenario?

I am a C# programmer. I want to clear this complex concept.
If there are 2 databases: A and B. Suppose I want to insert records in both but first in A and then in B. Say if while inserting in db B an exception occurs. The situation is that if B crashes, transaction with db A should also be rolled back. What do I have to do?
I know I can use SqlTransaction object with SqlConnectionString class. Can I have some code for this?

Already asked here : Implementing transactions over multiple databases.
Best answer from keithwarren7 :
use the TransactionScope class like this
using(TransactionScope ts = new TransactionScope())
{
//all db code here
// if error occurs jump out of the using block and it will dispose and rollback
ts.Complete();
}
The class will automatically convert to a distributed transaction if necessary.
.
Edit : adding explanations to original answer
You've got a good example in the MSDN : http://msdn.microsoft.com/fr-fr/library/system.transactions.transactionscope%28v=vs.80%29.aspx.
This example shows you how to use 2 Database Connections in one TransactionScope.
// Create the TransactionScope to execute the commands, guaranteeing
// that both commands can commit or roll back as a single unit of work.
using (TransactionScope scope = new TransactionScope())
{
using (SqlConnection connection1 = new SqlConnection(connectString1))
{
try
{
// Opening the connection automatically enlists it in the
// TransactionScope as a lightweight transaction.
connection1.Open();
// Create the SqlCommand object and execute the first command.
SqlCommand command1 = new SqlCommand(commandText1, connection1);
returnValue = command1.ExecuteNonQuery();
writer.WriteLine("Rows to be affected by command1: {0}", returnValue);
// If you get here, this means that command1 succeeded. By nesting
// the using block for connection2 inside that of connection1, you
// conserve server and network resources as connection2 is opened
// only when there is a chance that the transaction can commit.
using (SqlConnection connection2 = new SqlConnection(connectString2))
try
{
// The transaction is escalated to a full distributed
// transaction when connection2 is opened.
connection2.Open();
// Execute the second command in the second database.
returnValue = 0;
SqlCommand command2 = new SqlCommand(commandText2, connection2);
returnValue = command2.ExecuteNonQuery();
writer.WriteLine("Rows to be affected by command2: {0}", returnValue);
}
catch (Exception ex)
{
// Display information that command2 failed.
writer.WriteLine("returnValue for command2: {0}", returnValue);
writer.WriteLine("Exception Message2: {0}", ex.Message);
}
}
catch (Exception ex)
{
// Display information that command1 failed.
writer.WriteLine("returnValue for command1: {0}", returnValue);
writer.WriteLine("Exception Message1: {0}", ex.Message);
}
}
// The Complete method commits the transaction. If an exception has been thrown,
// Complete is not called and the transaction is rolled back.
scope.Complete();
}

If you like to have only one connection and like to manage the things then you may using the Linked Server and call the SP from server A which can call the SP from Server B
:)

TransactionScope not rolling back transaction

Here is the current architecture of my transaction scope source code. The third insert throws an .NET exception (Not a SQL Exception) and it is not rolling back the two previous insert statements. What I am doing wrong?
EDIT: I removed the try/catch from insert2 and insert3. I also removed the exception handling utility from the insert1 try/catch and put "throw ex". It still does not rollback the transaction.
EDIT 2: I added the try/catch back on the Insert3 method and just put a "throw" in the catch statement. It still does not rollback the transaction.
UPDATE:Based on the feedback I received, the "SqlHelper" class is using the SqlConnection object to establish a connection to the database, then creates a SqlCommand object, set the CommandType property to "StoredProcedure" and calls the ExecuteNonQuery method of the SqlCommand.
I also did not add Transaction Binding=Explicit Unbind to the current connection string. I will add that during my next test.
public void InsertStuff()
{
try
{
using(TransactionScope ts = new TransactionScope())
{
//perform insert 1
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /* create parameters for first insert */ };
sh.Insert("MyInsert1", sp);
}
//perform insert 2
this.Insert2();
//perform insert 3 - breaks here!!!!!
this.Insert3();
ts.Complete();
}
}
catch(Exception ex)
{
throw ex;
}
}
public void Insert2()
{
//perform insert 2
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /* create parameters for second insert */ };
sh.Insert("MyInsert2", sp);
}
}
public void Insert3()
{
//perform insert 3
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /*create parameters for third insert */ };
sh.Insert("MyInsert3", sp);
}
}

I have also run into a similar issue. My problem occurred because the SqlConnection I used in my SqlCommands was already open before the TransactionScope was created, so it never got enlisted in the TransactionScope as a transaction.
Is it possible that the SqlHelper class is reusing an instance of SqlConnection that is open before you enter your TransactionScope block?

It looks like you are catching the exception in Insert3() so your code continues after the call. If you want it to rollback you'll need to let the exception bubble up to the try/catch block in the main routine so that the ts.Complete() statement never gets called.

An implicit rollback will only occur if the using is exited without calling ts.complete. Because you are handling the exception in Insert3() the exception never causes an the using statement to exit.
Either rethrow the exception or notify the caller that a rollback is needed (make change the signature of Insert3() to bool Insert3()?)

(based on the edited version that doesn't swallow exceptions)
How long do the operations take? If any of them are very long running, it is possible that the Transaction Binding bug feature has bitten you - i.e. the connection has become detached. Try adding Transaction Binding=Explicit Unbind to the connection string.

I dont see your helper class, but transaction scope rollsback if you don't call complete statement even if you get error from .NET code. I copied one example for you. You may be doing something wrong in debugging. This example has error in .net code and similar catch block as yours.
private static readonly string _connectionString = ConnectionString.GetDbConnection();
private const string inserttStr = #"INSERT INTO dbo.testTable (col1) VALUES(#test);";
/// <summary>
/// Execute command on DBMS.
/// </summary>
/// <param name="command">Command to execute.</param>
private void ExecuteNonQuery(IDbCommand command)
{
if (command == null)
throw new ArgumentNullException("Parameter 'command' can't be null!");
using (IDbConnection connection = new SqlConnection(_connectionString))
{
command.Connection = connection;
connection.Open();
command.ExecuteNonQuery();
}
}
public void FirstMethod()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello1"));
ExecuteNonQuery(command);
}
public void SecondMethod()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello2"));
ExecuteNonQuery(command);
}
public void ThirdMethodCauseNetException()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello3"));
ExecuteNonQuery(command);
int a = 0;
int b = 1/a;
}
public void MainWrap()
{
TransactionOptions tso = new TransactionOptions();
tso.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
//TransactionScopeOption.Required, tso
try
{
using (TransactionScope sc = new TransactionScope())
{
FirstMethod();
SecondMethod();
ThirdMethodCauseNetException();
sc.Complete();
}
}
catch (Exception ex)
{
logger.ErrorException("eee ",ex);
}
}
If you want to debug your transactions, you can use this script to see locks and waiting status etc.
SELECT
request_session_id AS spid,
CASE transaction_isolation_level
WHEN 0 THEN 'Unspecified'
WHEN 1 THEN 'ReadUncomitted'
WHEN 2 THEN 'Readcomitted'
WHEN 3 THEN 'Repeatable'
WHEN 4 THEN 'Serializable'
WHEN 5 THEN 'Snapshot' END AS TRANSACTION_ISOLATION_LEVEL ,
resource_type AS restype,
resource_database_id AS dbid,
DB_NAME(resource_database_id) as DBNAME,
resource_description AS res,
resource_associated_entity_id AS resid,
CASE
when resource_type = 'OBJECT' then OBJECT_NAME( resource_associated_entity_id)
ELSE 'N/A'
END as ObjectName,
request_mode AS mode,
request_status AS status
FROM sys.dm_tran_locks l
left join sys.dm_exec_sessions s on l.request_session_id = s.session_id
where resource_database_id = 24
order by spid, restype, dbname;
You will see one SPID for two method calls before calling exception method.
Default isolation level is serializable.You can read more about locks and transactions here

I ran into a similar issue when I had a call to a WCF service operation in TransactionScope.
I noticed transaction flow was not allowed due to the 'TransactionFlow' attribute in the service interface. Therefore, the WCF service operation was not using the transaction used by the outer transaction scope. Changing it to allow transaction flow as shown below fixed my problem.
[TransactionFlow(TransactionFlowOption.NotAllowed)]
to
[TransactionFlow(TransactionFlowOption.Allowed)]