Develop Reference

TransactionScope breaking SqlConnection pooling?

I have an odd situation with TransactionScope and async/synchronous SQL calls that I'm having difficulty understanding. I hope that someone with a deeper understanding of the ins and outs of these kinds of operations can shed some light on the issue.
The situation:
I have a NUnit testfixture which creates a TransactionScope during [SetUp] and Disposes it at [TearDown] to let each test run on the same data. I have a series of tests which kick off an asynchronous operation on the database and then execute a synchronous operation on the database. The first such test completes successfully. The second such test fails with "There is already an open DataReader associated with this Command which must be closed first.".
If I comment out the TransactionScope entirely, all the tests pass.
I tried various different TransactionScope options, and Complete / Dispose, but the same issue occurs.
I am using the Resharper test runner on an NUnit test, .NET 4.5.1.
I realize the "correct" answer may be "make everything async await". That's not an option for me, unfortunately.
I don't want to enable MARS, as this issue only occurs in tests.
I don't want to use GetAwaiter().GetResult() due to the potential deadlocks.
What it looks like to me is that once a TransactionScope.Dispose/Complete is called, the automatic SQLConnection pooling loses track of which connections have open DataReaders. It hands out the same SqlConnection to two simultaneously running operations, and the second dies.
My primary question is "what is causing this behavior (specifically)?"
My secondary question is "is there anything that can be done to safely resolve the issue?"
The replicating code below prints out the client connection Ids. On my machine, the ClientConnectionId for the ASYNC and SYNC calls in the Second test case are always the same.
Replicating Code:
[TestFixture]
public class DataReaderTests
{
private TransactionScope _scope;
private string _connString = #"my connection string";
[SetUp]
public void Setup()
{
var options = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = TimeSpan.FromMinutes(1)
};
_scope = new TransactionScope(TransactionScopeOption.RequiresNew, options, TransactionScopeAsyncFlowOption.Enabled);
}
[Test]
[TestCase("First")]
[TestCase("Second")]
public void Test(string name)
{
DoAsyncThing().ConfigureAwait(false);
using (var conn = new SqlConnection(_connString))
{
try
{
conn.Open();
Console.WriteLine("SYNC: " + conn.ClientConnectionId);
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT 1";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
int id = reader.GetInt32(0);
}
}
}
}
catch (TransactionAbortedException tax)
{
Console.WriteLine("ERROR: " + ((SqlException)tax.InnerException.InnerException).ClientConnectionId);
throw;
}
}
}
private async Task DoAsyncThing()
{
using (var connection = new SqlConnection(_connString))
{
await connection.OpenAsync();
Console.WriteLine("ASYNC: " + connection.ClientConnectionId);
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = "WAITFOR DELAY '00:02';";
await cmd.ExecuteNonQueryAsync();
Console.WriteLine("ASYNC COMPLETE");
}
}
}
[TearDown]
public void Teardown()
{
_scope.Dispose();
}
}`

Check out this answer
I think the gist is that you cannot have two active sql commands executing over the same connection at the same time without a special connection string property. When you are operating under the transaction scope, you should find that both SqlConnection objects have the same client ID. However, if you remove the transaction scope they are different, which I believe implies that they are operating on separate connections.
Adding "MultipleActiveResultSets=true" to the connection string fixed the issue for me. Another alternative is to replace
DoAsyncThing().ConfigureAwait(false);
with
DoAsyncThing().ConfigureAwait(false).GetAwaiter().GetResult();
which will terminate the first command before starting the second command.

Proper way of using BeginTransaction with Dapper.IDbConnection

Which is the proper way of using BeginTransaction() with IDbConnection in Dapper ?
I have created a method in which i have to use BeginTransaction(). Here is the code.
using (IDbConnection cn = DBConnection)
{
var oTransaction = cn.BeginTransaction();
try
{
// SAVE BASIC CONSULT DETAIL
var oPara = new DynamicParameters();
oPara.Add("#PatientID", iPatientID, dbType: DbType.Int32);
..........blah......blah............
}
catch (Exception ex)
{
oTransaction.Rollback();
return new SaveResponse { Success = false, ResponseString = ex.Message };
}
}
When i executed above method - i got an exception -
Invalid operation. The connection is closed.
This is because you can't begin a transaction before the connection is opened. So when i add this line: cn.Open();, the error gets resolved. But i have read somewhere that manually opening the connection is bad practice!! Dapper opens a connection only when it needs to.
In Entity framework you can handle a transaction using a TransactionScope.
So my question is what is a good practice to handle transaction without adding the line cn.Open()... in Dapper ? I guess there should be some proper way for this.

Manually opening a connection is not "bad practice"; dapper works with open or closed connections as a convenience, nothing more. A common gotcha is people having connections that are left open, unused, for too long without ever releasing them to the pool - however, this isn't a problem in most cases, and you can certainly do:
using(var cn = CreateConnection()) {
cn.Open();
using(var tran = cn.BeginTransaction()) {
try {
// multiple operations involving cn and tran here
tran.Commit();
} catch {
tran.Rollback();
throw;
}
}
}
Note that dapper has an optional parameter to pass in the transaction, for example:
cn.Execute(sql, args, transaction: tran);
I am actually tempted to make extension methods on IDbTransaction that work similarly, since a transaction always exposes .Connection; this would allow:
tran.Execute(sql, args);
But this does not exist today.
TransactionScope is another option, but has different semantics: this could involve the LTM or DTC, depending on ... well, luck, mainly. It is also tempting to create a wrapper around IDbTransaction that doesn't need the try/catch - more like how TransactionScope works; something like (this also does not exist):
using(var cn = CreateConnection())
using(var tran = cn.SimpleTransaction())
{
tran.Execute(...);
tran.Execute(...);
tran.Complete();
}

You should not call
cn.Close();
because the using block will try to close too.
For the transaction part, yes you can use TransactionScope as well, since it is not an Entity Framework related technique.
Have a look at this SO answer: https://stackoverflow.com/a/6874617/566608
It explain how to enlist your connection in the transaction scope.
The important aspect is: connection are automatically enlisted in the transaction IIF you open the connection inside the scope.

Take a look at Tim Schreiber solution which is simple yet powerful and implemented using repository pattern and has Dapper Transactions in mind.
The Commit() in the code below shows it.
public class UnitOfWork : IUnitOfWork
{
private IDbConnection _connection;
private IDbTransaction _transaction;
private IBreedRepository _breedRepository;
private ICatRepository _catRepository;
private bool _disposed;
public UnitOfWork(string connectionString)
{
_connection = new SqlConnection(connectionString);
_connection.Open();
_transaction = _connection.BeginTransaction();
}
public IBreedRepository BreedRepository
{
get { return _breedRepository ?? (_breedRepository = new BreedRepository(_transaction)); }
}
public ICatRepository CatRepository
{
get { return _catRepository ?? (_catRepository = new CatRepository(_transaction)); }
}
public void Commit()
{
try
{
_transaction.Commit();
}
catch
{
_transaction.Rollback();
throw;
}
finally
{
_transaction.Dispose();
_transaction = _connection.BeginTransaction();
resetRepositories();
}
}
private void resetRepositories()
{
_breedRepository = null;
_catRepository = null;
}
public void Dispose()
{
dispose(true);
GC.SuppressFinalize(this);
}
private void dispose(bool disposing)
{
if (!_disposed)
{
if(disposing)
{
if (_transaction != null)
{
_transaction.Dispose();
_transaction = null;
}
if(_connection != null)
{
_connection.Dispose();
_connection = null;
}
}
_disposed = true;
}
}
~UnitOfWork()
{
dispose(false);
}
}

There are two intended ways to use transactions with Dapper.
Pass your IDbTranasction to your normal Dapper call.
Before:
var affectedRows = connection.Execute(sql, new {CustomerName = "Mark"});
After:
var affectedRows = connection.Execute(sql, new {CustomerName = "Mark"}, transaction=tx);
Use the new .Execute extension method that Dapper adds to IDbTransaction itself:
tx.Execute(sql, new {CustomerName = "Mark"});
Note: the variable tx comes from IDbTransaction tx = connection.BeginTransaction();
This is how you're supposed to use transactions with Dapper; neither of them are TranasctionScope.
Bonus Reading
https://stackoverflow.com/a/67474832/12597

Application Crash on SQL Connection.Close method

I have a windows services which is performing lot of Data processing. At some point, my service is crashed on closing a SQLConnection. When I comment the Close connection method call, Service is working consistently without crashing.
What could be the problem ? Below is a code snippet
private void DeleteTempTable()
{
_logger.Info("DeleteTempTable");
try
{
foreach (KeyValuePair<string, string> item in _stables)
{
string dropSql = string.Empty;
dropSql = string.Format("DROP TABLE [{0}];", item.Value);
SqlConnection oConnDropTables = new SqlConnection(connectionString);
oConnDropTables.Open();
if (!string.IsNullOrEmpty(dropSql))
{
using (SqlCommand cmd = new SqlCommand(dropSql, oConnDropTables))
{
cmd.ExecuteNonQuery();
}
}
if (oConnDropTables != null && oConnDropTables.State == ConnectionState.Open)
oConnDropTables.Close();
oConnDropTables = null;
}
}
catch (Exception ex)
{
_logger.Error("Error " + ex.Message);
throw ex;
}
}
When I comment the Close connection, service is working without crashing. Also it is not caught in the catch block. Also Connection is not Null and connectionstate is open only..
What I have tried:
1) Put "using" construct for connection - Didn't help
2) catching SQLException to check if anything I get- Didn't help

Removing that Close() should not make any problems go away, and frankly I don't believe it has. Since you don't yet understand the problem, it is premature to assume that a random code change has fixed it. Specifically:
with the Close(), it is returning the connection to the pool each time; when you call Open(), it will get back the same connection from the pool (cleansed, except for a few minor things)
without the Close(), the previous connection will be left to be garbage collected, which can cause either the connection-pool to become saturated, or the database-server's connection count to saturate; basically - bad things
What I suspect is happening is that you had some random error, that you now aren't seeing, by random. For example, network connectivity, or the unpredictable ordering of a Dictionary<,> (which means you don't know what order the tables are being DROPped, which is very important if there are foreign keys between them).
The only major problem with the current code is that it isn't using using. There are some redundant lines, though. This would be better:
foreach (var item in _stables)
{
var dropSql = string.Format("DROP TABLE [{0}];", item.Value);
using(var oConnDropTables = new SqlConnection(connectionString))
using (var cmd = new SqlCommand(dropSql, oConnDropTables))
{
oConnDropTables.Open();
cmd.ExecuteNonQuery();
}
}
or (preferable):
using(var oConnDropTables = new SqlConnection(connectionString))
{
oConnDropTables.Open();
foreach (var item in _stables)
{
var dropSql = string.Format("DROP TABLE [{0}];", item.Value);
using (var cmd = new SqlCommand(dropSql, oConnDropTables))
{
cmd.ExecuteNonQuery();
}
}
}

The issue is the creation of a new connection object each time the loop runs. When you close a SQL Connection, it is not actually closed but its returned to the app pool ready to be re-used. There is a limited number of connections you can open in SQL at once.
Try moving the SQLConnection object out of the loop and just execute commands in the loop and close the connection after the loop finishes.
private void DeleteTempTable()
{
_logger.Info("DeleteTempTable");
try
{
using(SqlConnection oConnDropTables = new SqlConnection(connectionString))
{
oConnDropTables.Open();
foreach (KeyValuePair<string, string> item in _stables)
{
string dropSql = string.Empty;
dropSql = string.Format("DROP TABLE [{0}];", item.Value);
if (!string.IsNullOrEmpty(dropSql))
{
using (SqlCommand cmd = new SqlCommand(dropSql, oConnDropTables))
{
cmd.ExecuteNonQuery();
}
}
}
}
}
catch (Exception ex)
{
_logger.Error("Error " + ex.Message);
throw ex;
}
}

Task launched from a Threading.Timer

*Edit: Please see my answer below for the solution.
Is there any danger in the following? I'm trying to track down what I think might be a race condition. I figured I'd start with this and go from there.
private BlockingCollection<MyTaskType>_MainQ = new BlockingCollection<MyTaskType>();
private void Start()
{
_CheckTask = new Timer(new TimerCallback(CheckTasks), null, 10, 5000);
}
private void CheckTasks(object state)
{
_CheckTask.Change(Timeout.Infinite, Timeout.Infinite);
GetTask();
_CheckTask.Change(5000,5000);
}
private void GetTask()
{
//get task from database to object
Task.Factory.StartNew( delegate {
AddToWorkQueue(); //this adds to _MainQ which is a BlockingCollection
});
}
private void AddToWorkQueue()
{
//do some stuff to get stuff to move
_MainQ.Add(dataobject);
}
edit: I am also using a static class to handle writing to the database. Each call should have it's own unique data called from many threads, so it is not sharing data. Do you think this could be a source of contention?
Code below:
public static void ExecuteNonQuery(string connectionString, string sql, CommandType commandType, List<FastSqlParam> paramCollection = null, int timeout = 60)
{
//Console.WriteLine("{0} [Thread {1}] called ExecuteNonQuery", DateTime.Now.ToString("HH:mm:ss:ffffff"), System.Threading.Thread.CurrentThread.ManagedThreadId);
using (SqlConnection connection = new SqlConnection(connectionString))
using (SqlCommand command = new SqlCommand(sql, connection))
{
try
{
if (paramCollection != null)
{
foreach (FastSqlParam fsqlParam in paramCollection)
{
try
{
SqlParameter param = new SqlParameter();
param.Direction = fsqlParam.ParamDirection;
param.Value = fsqlParam.ParamValue;
param.ParameterName = fsqlParam.ParamName;
param.SqlDbType = fsqlParam.ParamType;
command.Parameters.Add(param);
}
catch (ArgumentNullException anx)
{
throw new Exception("Parameter value was null", anx);
}
catch (InvalidCastException icx)
{
throw new Exception("Could not cast parameter value", icx);
}
}
}
connection.Open();
command.CommandType = commandType;
command.CommandTimeout = timeout;
command.ExecuteNonQuery();
if (paramCollection != null)
{
foreach (FastSqlParam fsqlParam in paramCollection)
{
if (fsqlParam.ParamDirection == ParameterDirection.InputOutput || fsqlParam.ParamDirection == ParameterDirection.Output)
try
{
fsqlParam.ParamValue = command.Parameters[fsqlParam.ParamName].Value;
}
catch (ArgumentNullException anx)
{
throw new Exception("Output parameter value was null", anx);
}
catch (InvalidCastException icx)
{
throw new Exception("Could not cast parameter value", icx);
}
}
}
}
catch (SqlException ex)
{
throw ex;
}
catch (ArgumentException ex)
{
throw ex;
}
}
}
per request:
FastSql.ExecuteNonQuery(connectionString, "someProc", System.Data.CommandType.StoredProcedure, new List<FastSqlParam>() { new FastSqlParam(SqlDbType.Int, "#SomeParam", variable)});
Also, I wanted to note that this code seems to fail at random running it from VS2010 [Debug or Release]. When I do a release build, run setup on a dev server that will be hosting it, the application has failed to crash and has been running smoothly.
per request:
Current architecture of threads:
Thread A reading 1 record from a database scheduling table
Thread A, if a row is returned, launches a Task to login to resource to see if there are files to transfer. The task is referencing an object that contains data from the DataTable that was creating using a static call. Basically as below.
If there are files found, Task adds to _MainQ the files to move
//Called from Thread A
void ProcessTask()
{
var parameters = new List<FastSqlParam>() { new FastSqlParam(SqlDbType.Int, "#SomeParam", variable) };
using (DataTable someTable = FastSql.ExecuteDataTable(connectionString, "someProc", CommandType.StoredProcedure, parameters))
{
SomeTask task = new Task();
//assign task some data from dt.Rows[0]
if (task != null)
{
Task.Factory.StartNew(delegate { AddFilesToQueue(task); });
}
}
}
void AddFilesToQueue(Task task)
{
//connect to remote system and build collection of files to WorkItem
//e.g, WorkItem will have a collection of collections to transfer. We control this throttling mechanism to allow more threads to split up the work
_MainQ.Add(WorkItem);
}
Do you think there could be a problem returning a value from FastSql.ExecuteDataTable since it is a static class and then using it with a using block?

I'd personally be wary of introducing extra threads in quite so many places - "Here be Dragons" is a useful rule when it comes to working with threads! I can't see any problems with what you have, but if it were simpler it'd be easier to be more certain. I'm assuming you want the call to "AddToWorkQueue" to be done in a different thread (to test the race condition) so I've left that in.
Does this do what you need it to? (eye compiled so may be wrong)
while(true) {
Task.Factory.StartNew( delegate { AddToWorkQueue(); });
Thread.Sleep(5000);
}
random aside - prefer "throw;" to "throw ex;" - the former preserves the original call stack, the latter will only give you the line number of the "throw ex;" call. Even better, omit the try/catch in this case as all you do is re-throw the exceptions, so you may as well save yourself the overhead of the try.

It turns out the problem was a very, very strange one.
I converted the original solution from a .NET 3.5 solution to a .NET 4.0 solution. The locking up problem went away when I re-created the entire solution in a brand new .NET 4.0 solution. No other changes were introduced, so I am very confident the problem was the upgrade to 4.0.

TransactionScope not rolling back transaction

Here is the current architecture of my transaction scope source code. The third insert throws an .NET exception (Not a SQL Exception) and it is not rolling back the two previous insert statements. What I am doing wrong?
EDIT: I removed the try/catch from insert2 and insert3. I also removed the exception handling utility from the insert1 try/catch and put "throw ex". It still does not rollback the transaction.
EDIT 2: I added the try/catch back on the Insert3 method and just put a "throw" in the catch statement. It still does not rollback the transaction.
UPDATE:Based on the feedback I received, the "SqlHelper" class is using the SqlConnection object to establish a connection to the database, then creates a SqlCommand object, set the CommandType property to "StoredProcedure" and calls the ExecuteNonQuery method of the SqlCommand.
I also did not add Transaction Binding=Explicit Unbind to the current connection string. I will add that during my next test.
public void InsertStuff()
{
try
{
using(TransactionScope ts = new TransactionScope())
{
//perform insert 1
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /* create parameters for first insert */ };
sh.Insert("MyInsert1", sp);
}
//perform insert 2
this.Insert2();
//perform insert 3 - breaks here!!!!!
this.Insert3();
ts.Complete();
}
}
catch(Exception ex)
{
throw ex;
}
}
public void Insert2()
{
//perform insert 2
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /* create parameters for second insert */ };
sh.Insert("MyInsert2", sp);
}
}
public void Insert3()
{
//perform insert 3
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /*create parameters for third insert */ };
sh.Insert("MyInsert3", sp);
}
}

I have also run into a similar issue. My problem occurred because the SqlConnection I used in my SqlCommands was already open before the TransactionScope was created, so it never got enlisted in the TransactionScope as a transaction.
Is it possible that the SqlHelper class is reusing an instance of SqlConnection that is open before you enter your TransactionScope block?

It looks like you are catching the exception in Insert3() so your code continues after the call. If you want it to rollback you'll need to let the exception bubble up to the try/catch block in the main routine so that the ts.Complete() statement never gets called.

An implicit rollback will only occur if the using is exited without calling ts.complete. Because you are handling the exception in Insert3() the exception never causes an the using statement to exit.
Either rethrow the exception or notify the caller that a rollback is needed (make change the signature of Insert3() to bool Insert3()?)

(based on the edited version that doesn't swallow exceptions)
How long do the operations take? If any of them are very long running, it is possible that the Transaction Binding bug feature has bitten you - i.e. the connection has become detached. Try adding Transaction Binding=Explicit Unbind to the connection string.

I dont see your helper class, but transaction scope rollsback if you don't call complete statement even if you get error from .NET code. I copied one example for you. You may be doing something wrong in debugging. This example has error in .net code and similar catch block as yours.
private static readonly string _connectionString = ConnectionString.GetDbConnection();
private const string inserttStr = #"INSERT INTO dbo.testTable (col1) VALUES(#test);";
/// <summary>
/// Execute command on DBMS.
/// </summary>
/// <param name="command">Command to execute.</param>
private void ExecuteNonQuery(IDbCommand command)
{
if (command == null)
throw new ArgumentNullException("Parameter 'command' can't be null!");
using (IDbConnection connection = new SqlConnection(_connectionString))
{
command.Connection = connection;
connection.Open();
command.ExecuteNonQuery();
}
}
public void FirstMethod()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello1"));
ExecuteNonQuery(command);
}
public void SecondMethod()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello2"));
ExecuteNonQuery(command);
}
public void ThirdMethodCauseNetException()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello3"));
ExecuteNonQuery(command);
int a = 0;
int b = 1/a;
}
public void MainWrap()
{
TransactionOptions tso = new TransactionOptions();
tso.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
//TransactionScopeOption.Required, tso
try
{
using (TransactionScope sc = new TransactionScope())
{
FirstMethod();
SecondMethod();
ThirdMethodCauseNetException();
sc.Complete();
}
}
catch (Exception ex)
{
logger.ErrorException("eee ",ex);
}
}
If you want to debug your transactions, you can use this script to see locks and waiting status etc.
SELECT
request_session_id AS spid,
CASE transaction_isolation_level
WHEN 0 THEN 'Unspecified'
WHEN 1 THEN 'ReadUncomitted'
WHEN 2 THEN 'Readcomitted'
WHEN 3 THEN 'Repeatable'
WHEN 4 THEN 'Serializable'
WHEN 5 THEN 'Snapshot' END AS TRANSACTION_ISOLATION_LEVEL ,
resource_type AS restype,
resource_database_id AS dbid,
DB_NAME(resource_database_id) as DBNAME,
resource_description AS res,
resource_associated_entity_id AS resid,
CASE
when resource_type = 'OBJECT' then OBJECT_NAME( resource_associated_entity_id)
ELSE 'N/A'
END as ObjectName,
request_mode AS mode,
request_status AS status
FROM sys.dm_tran_locks l
left join sys.dm_exec_sessions s on l.request_session_id = s.session_id
where resource_database_id = 24
order by spid, restype, dbname;
You will see one SPID for two method calls before calling exception method.
Default isolation level is serializable.You can read more about locks and transactions here

I ran into a similar issue when I had a call to a WCF service operation in TransactionScope.
I noticed transaction flow was not allowed due to the 'TransactionFlow' attribute in the service interface. Therefore, the WCF service operation was not using the transaction used by the outer transaction scope. Changing it to allow transaction flow as shown below fixed my problem.
[TransactionFlow(TransactionFlowOption.NotAllowed)]
to
[TransactionFlow(TransactionFlowOption.Allowed)]