Develop Reference

C# SqlClient | Connection Pooling | Error When Opening Second Connection

I'm trying to figure out how to setup my database access correctly while using the SqlClient hitting a Microsoft SQL Server. For the most part it is working, but there's a particular scenario that is giving me trouble. Namely: attempting to simultaneously use two connections in the same thread; one with an open data reader and the other performing a delete operation.
The following code demonstrates my conundrum:
public class Database {
...
internal SqlConnection CreateConnection() => new SqlConnection(connectionString);
...
}
public IEnumerable<Model> GetModel() {
var cmd = new SqlCommand() { ... };
using(var conn = db.CreateConnection()) {
conn.Open();
cmd.Connection = conn;
using(var reader = cmd.ExecuteReader()) {
while(reader.Read()) {
var m = new Model();
// deserialization logic
yield return m;
}
}
}
}
public void Delete(int id) {
var cmd = new SqlCommand() { ... }
using(var conn = db.CreateConnection()) {
conn.Open(); // throwing the error here
cmd.Connection = conn;
cmd.ExecuteNonQuery();
}
}
Application Code:
using(var scope = new TransactionScope()) {
var models = GetModels();
foreach(var m in models) {
Delete(m.Id); // throws an exception
}
scope.Complete();
}
For whatever reason, an exception is thrown by the above code while trying to execute the Delete operation:
quote
System.Transactions.TransactionAbortedException: The transaction has aborted. ---> System.Transactions.TransactionPromotionException: Failure while attempting to promote transaction. ---> System.Data.SqlClient.SqlException: There is already an open DataReader associated with this Command which must be closed first. ---> System.ComponentModel.Win32Exception: The wait operation timed out
quote
Now, I have confirmed that if I either set MultipleActiveResultSets=true or Pooling=false on the ConnectionString, that then the above application code will work without error. However, it doesn't seem like I should need to set either of those. If I open two connections simultaneously, should they not be separate connections? Why then am I getting an error from the Delete connection saying that there's an open DataReader?
Please help <3

By far the easiest fix here is to simply load all the models outside the transaction before you go deleting any. Eg
var models = GetModels().ToList();
using(var scope = new TransactionScope()) {
foreach(var m in models) {
Delete(m.Id); // throws an exception
}
scope.Complete();
}
Even fetching the models inside the transaction shold work
using(var scope = new TransactionScope()) {
var models = GetModels().ToList();
foreach(var m in models) {
Delete(m.Id); // throws an exception
}
scope.Complete();
}
so long as you don't leave the connection open during the iteration. If you allow the connection in GetModels() to close, it will be returned to the connection pool, and be available for use for subsequent methods that are enlisted in the same transaction.
In the current code the connection in GetModels() is kept open during the foreach loop and Delete(id) has to open a second connection and try to create a distributed transaction, which is failing.
Without MultipleActiveResultsets, the GetModels connection can't be promoted to a distributed transaction in the middle of returning query results. Setting pooling=false will not make this error go away.
Here's a simplified repro to play with:
using Microsoft.Data.SqlClient;
using System.Collections.Generic;
using System.Transactions;
namespace SqlClientTest
{
class Program
{
static void Main(string[] args)
{
Setup();
var topt = new TransactionOptions();
topt.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
using (new TransactionScope(TransactionScopeOption.Required, topt ))
{
foreach (var id in GetIds())
{
Delete(id);
}
}
}
static string constr = #"server=.;database=tempdb;Integrated Security=true;TrustServerCertificate=true;";
public static void Setup()
{
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "drop table if exists ids; select object_id id into ids from sys.objects";
cmd.ExecuteNonQuery();
}
}
public static IEnumerable<int> GetIds()
{
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "select object_id id from sys.objects";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
yield return reader.GetInt32(0);
}
}
}
}
public static void Delete(int id)
{
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "insert into ids(id) values (#id)";
cmd.Parameters.Add(new SqlParameter("#id", id));
cmd.ExecuteNonQuery();
}
}
}
}
And here's what Profiler shows when run:

The main reason here as far as I understand is your yielding iteration.
So the DB connection has not yet called disposed as it's still being used in your iteration (foreach). If for example, you called .ToList() at that point it should return all the entries and then dispose of the connection.
See here for a better explanation on how yield works in an iteration: https://stackoverflow.com/a/58449809/3329836

The transaction operation cannot be performed because there are pending requests working

background
I have some code which opens a sql connection, begins a transaction and performs some operations on the DB. This code creates an object from the DB (dequeue), gets some values and saves it back. The whole operation needs to take place in a transaction. All the code works perfectly without the transaction.
using (var connection = new SqlConnection(connectionString))
{
connection.Open();
var transaction = connection.BeginTransaction();
try
{
var myObject = foo.Dequeue(connection, transaction);
var url = myObj.GetFilePathUri(connection, transaction);
//some other code that sets object values
myObj.SaveMessage(connection, transaction);
transaction.Commit(); //error here
}
catch(Exception ex)
{
transaction.Rollback();
//logging
}
finally
{
//cleanup code
}
}
dequeue method code
public foo Dequeue(SqlConnection connection, SqlTransaction transaction)
{
using (var command = new SqlCommand(DEQUEUE_SPROC, connection) {CommandType = CommandType.StoredProcedure, Transaction = transaction})
{
var reader = command.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
ID = (Guid) reader["ID"];
Name = reader["Name"].ToString();
return this;
}
return null;
}
}
Get Path Code
public string GetFilePathUri(SqlConnection connection, SqlTransaction transaction)
{
using (var command = new SqlCommand(FILEPATH_SPROC, connection) {CommandType = CommandType.StoredProcedure, Transaction = transaction})
{
var reader = command.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
return reader["Path"].ToString();
}
return "";
}
}
Save Code
public void SaveMessage(SqlConnection connection, SqlTransaction transaction)
{
using (var command = new SqlCommand(SAVE_SPROC, connection) {CommandType = CommandType.StoredProcedure, Transaction = transaction})
{
command.Parameters.Add("#ID", SqlDbType.UniqueIdentifier).Value = ID;
command.Parameters.Add("#Name", SqlDbType.VarChar).Value = Name;
//other object params here
command.ExecuteNonQuery();
}
}
The problem
When transaction.Commit() is called, I get the following error:
The transaction operation cannot be performed because there are pending requests working on this transaction.
What am I doing wrong?
EDIT: Quick edit to say I have read the other questions about this problem on SO, but couldn't find any related to ADO.net

I have had this issue before and the problem was the reader needed to be closed.
Try this:
public foo Dequeue(SqlConnection connection, SqlTransaction transaction)
{
using (var command = new SqlCommand(DEQUEUE_SPROC, connection) {CommandType = CommandType.StoredProcedure, Transaction = transaction})
{
var reader = command.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
ID = (Guid) reader["ID"];
Name = reader["Name"].ToString();
reader.Close();//Closing the reader
return this;
}
return null;
}
}
public string GetFilePathUri(SqlConnection connection, SqlTransaction transaction)
{
string filePathUri = "";
using (var command = new SqlCommand(FILEPATH_SPROC, connection) {CommandType = CommandType.StoredProcedure, Transaction = transaction})
{
var reader = command.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
filePathUri = reader["Path"].ToString();
}
reader.Close();//Closing the reader
}
return filePathUri;
}

I had this problem when I forgot to use await on an async method that was doing the DB call - the connection was being disposed while the transaction was running because the program was not waiting for the query to complete before trying to dispose everything.

We ran into this problem and while the accepted answer probably would've worked, it wouldn't have been the most correct thing to do -- it did, however, give us a clue as to what we were doing wrong.
In our case, there was more data to read. Here's a simplified version of the problem (using Dapper):
public Foo GetFooById(int fooId, IDbTransaction transaction = null) {
var sql = #"
SELECT * FROM dbo.Foo WHERE FooID = #FooID;
SELECT * FROM dbo.FooBar WHERE FooID = #FooID;
SELECT * FROM dbo.FooBaz WHERE FooID = #FooID;
";
using var data = await _connection.QueryMultipleAsync(sql, new { fooId }, transaction);
var foo = data.ReadSingle<Foo>();
foo.Bars = data.Read<Bar>();
// Oops, didn't read FooBaz.
return foo;
}
Even though data gets disposed, it leaves a "pending request" on the transaction, which then blows up later when transaction.Commit() is called.
The solution is to either read FooBaz records out of data or get rid of the unread SELECT.

For who will come here in the future, I had this issue with a combination of IQueryable and AutoMapper.
I passed an IQueryable for mapping to Mapper.Map<>(myData) by thinking that since it maps data from a class to another, it also materializes it.
Apparently, I was wrong and the issue mentioned in the question was thrown when trying to transaction.Commit().
Solution
As easy as forcing the materialization with a .ToList() in Mapper.Map<>(myData.ToList()).
Just wasted an hour finding that, I hope this will save some time for someone else.

Would this code waste resources?

I am using gridview to select multiple records and then play loop over it to individually send each record ID to database and update it but I don't find it very good way to implement because it opens and close connection each time so what is the good way ?
foreach (GridViewRow r in grdViewLastHearingDates.Rows)
{
int CaseHearingID = Convert.ToInt32(r.Cells[0].Text);
CheckBox chkBox = r.FindControl("chkBoxIsConveyed") as CheckBox;
TextBox txtboxConvenienceRemarks = r.FindControl("txtBoxConvenienceRemarks") as TextBox;
string ConvenienceRemarks = txtboxConvenienceRemarks.Text;
MngCaseHearings.UpdateCasesIsConveyed(CaseHearingID, ConvenienceRemarks, chkBox.Checked);
}
MngCaseHearings.UpdateCasesIsConveyed sends and executes this code every time for updating each ID. Please review and give suggestions
public Boolean UpdateCasesIsConveyed(int CaseHearingID, string ConvenienceRemarks, bool IsConveyed)
{
try
{
SqlCommand SqlCom = new SqlCommand("UpdateCasesIsConveyed", DatabaseConnection.OpenConnection());
SqlCom.CommandType = CommandType.StoredProcedure;
SqlCom.Parameters.AddWithValue("#pk_CaseHearings_ID ", CaseHearingID);
SqlCom.Parameters.AddWithValue("#IsConveyed", IsConveyed);
SqlCom.Parameters.AddWithValue("#ConvenienceRemarks", ConvenienceRemarks);
SqlParameter SqlParamReturnStatus = new SqlParameter("#ReturnStatus", SqlDbType.Bit);
SqlCom.Parameters.Add(SqlParamReturnStatus);
SqlParamReturnStatus.Direction = ParameterDirection.Output;
SqlParameter SqlParamReturnStatusMessage = new SqlParameter("#ReturnStatusMessage", SqlDbType.VarChar, -1);
SqlCom.Parameters.Add(SqlParamReturnStatusMessage);
SqlParamReturnStatusMessage.Direction = ParameterDirection.Output;
SqlCom.ExecuteNonQuery();
string ReturnStatusMessage = Convert.ToString(SqlParamReturnStatusMessage);
Boolean ReturnStatus = Convert.ToBoolean(SqlParamReturnStatus.Value);
return ReturnStatus;
}
catch (Exception)
{
throw;
}
finally
{
DatabaseConnection.CloseConnection();
}
this would waste the resources so if any good work around ?

Since the SqlConnection and SqlCommand objects both implement IDisposable you should probably dispose of them when you're done. The simplest way would be via the using statement:
using (var conn = DatabaseConnection.OpenConnection())
using (var SqlCom = new SqlCommand("UpdateCasesIsConveyed", conn))
{
// setup and execute the SP, can return from in here
}
This will ensure that the resources used by the objects are properly closed as soon as you are done. While this isn't strictly necessary - the Dispose method will be called when the garbage collector destroys the objects - it will ensure that you aren't holding open the database objects any longer than necessary. Depending on how often this is called you can end up with resource shortages on the SQL server, excessive handle usage, etc.
In general anything that implements IDisposable should be disposed as soon as practical.
(Sorry, missed the loop part of the question)
This type of create/destroy cycle is fine for single operations, but becomes wasteful when used to update a lot of records. I would put the loop in the middle of the code rather than repeatedly calling this code from outside.
I would create a record class or struct that holds the SP parameters and pass an IEnumerable of that record class to your update method. This way you can do the setup once, process all of the updates, then tear down the database objects after all of the changes have been made. Throw a transaction in too so you can undo it all if one of the records fails.
Something like:
public struct UpdateCaseConveyanceRec
{
public int CaseHearingID;
public string ConvenienceRemarks;
public bool IsConveyed;
}
public bool UpdateCasesIsConveyed(IEnumerable<UpdateCaseConveyanceRec> uopdates)
{
using (SqlConnection conn = DatabaseConnection.OpenConnection())
using (SqlCommand cmd = new SqlCommand("UpdateCasesIsConveyed", conn))
using (SqlTransaction trans = conn.BeginTransaction("UpdateCasesIsConveyed"))
{
cmd.CommandType = CommandType.StoredProcedure;
var pID = cmd.Parameters.Add("#pk_CaseHearings_ID", SqlDbType.Int);
var pConveyed = cmd.Parameters.Add("#IsConveyed", SqlDbType.Bit);
var pRemarks = cmd.Parameters.Add("#ConvenienceRemarks", SqlDbType.VarChar, -1);
var retStatus = cmd.Parameters.Add("#ReturnStatus", SqlDbType.Bit);
retStatus.Direction = ParameterDirection.Output;
var retStatusMsg = cmd.Parameters.Add("#ReturnStatusMessage", SqlDbType.VarChar, -1);
retStatusMsg.Direction = ParameterDirection.Output;
try
{
foreach (var row in updates)
{
pID.Value = row.CaseHearingID;
pConveyed.Value = row.IsConveyed;
pRemarks.Value = row.ConvenienceRemarks;
cmd.ExecuteNonQuery();
if (!Convert.ToBoolean(retStatus))
{
trans.Rollback();
return false;
}
}
trans.Commit();
}
catch ()
{
trans.Rollback();
throw;
}
return true;
}
}
You can then feed that with a LINQ to Objects query:
var source =
from r in grdViewLastHearingDates.Rows.OfType<GridViewRow>()
select new UpdateCaseConveyanceRec
{
CaseHearingID = Convert.ToInt32(r.Cells[0].Text),
ConvenienceRemarks = (r.FindControl("txtBoxConvenienceRemarks") as TextBox).Text;
IsConveyed = (r.FindControl("chkBoxIsConveyed") as CheckBox).Checked
};
bool updated = UpdateCasesIsConveyed(source);

Is this correct usage of TransactionSope?

I have decided to try using a TransactionScope, rather than the SqlTransaction class.
The following is my code, wrapped in a TransactionScope:
using (var transaction = new System.Transactions.TransactionScope())
{
using (MySqlCommand cmd = new MySqlCommand(sql, connection))
{
if (listParameters != null && listParameters.Count > 0)
{
foreach (string currentKey in listParameters.Keys)
{
cmd.Parameters.Add(new MySqlParameter(currentKey, GetDictionaryValue(listParameters, currentKey)));
}
}
using (MySqlDataReader reader = cmd.ExecuteReader())
{
dtResults.Load(reader);
}
}
transaction.Complete();
}
The code works, however I am not binding the MySqlCommand cmd object with a transaction at any point. Is this a problem?

No, this is not the correct use.
The correct use is to create a connection after creating TransactionScope. Then the connection will detect the ambient TransactionScope and enlist itself.
using (var transaction = new System.Transactions.TransactionScope())
{
using (var connection = new MySqlConnection())
{
...
}
}
If you create the connection before the scope, that connection will be out of that scope, even if you create the command after creating the scope.
Also note that TransactionScope defaults to Serializable level of isolation. This is the most secure level, but also the least concurrent one. You often want to explicitly set a more common isolation level:
using (var transaction = new TransactionScope(
TransactionScopeOption.Required,
new TransactionOptions() { IsolationLevel = IsolationLevel.ReadCommitted }))
{
}

Difference between Transaction Scope and manually defined transactions?

Hi just reading about using transaction scrope, previously I am used to making transactions inside a single DB class like
try
{
con.Open();
tran = con.BeginTransaction();
OleDbCommand myCommand1 = new OleDbCommand(query1, con);
OleDbCommand myCommand2 = new OleDbCommand(query2, con);
myCommand .Transaction = tran;
// Save Master
myCommand1.ExecuteNonQuery();
// Save Childred
myCommand2.ExecuteNonQuery();
// Commit transaction
tran.Commit();
}
catch (OleDbException ex)
{
tran.Rollback();
lblError.Text = "An error occured " + ex.ToString();
}
finally
{
if (con != null)
{
con.Close();
}
}
But now I come to know that I can execute transaction inside the Business Logic layer simply by using a transaction scope object and using separate DB classes like
public static int Save(Employee myEmployee)
{
using (TransactionScope myTransactionScope = new TransactionScope())
{
int RecordId = EmpDB.Save(myEmployee);
foreach (Address myAddress in myEmployee.Addresses)
{
myAddress.EmployeeId = EmployeeId;
AddressDB.Save(myAddress);
}
foreach (PhoneNumber myPhoneNumber in myEmployee.PhoneNumbers)
{
myPhoneNumber.EmployeeId = EmployeeId;
PhoneNumberDB.Save(myPhoneNumber);
}
myTransactionScope.Complete();
return EmployeeId;
}
}
Which one is the recommended coding practice and why ? Is using a Transaction Scope safe ? Is it the latest way to do things ? I am confused about both methods.
Thanks in advance.

One of the nice things about Transaction scope is that you don't need the try/catch block. You just have to call Complete on the scope in order to commit the transaction, and it rollsback automatically if an exception does occur.
You can also use other components that are able to participate in transactions, not just the DB connection. This is because the components (including the connection) look for a Transaction on the current thread. It is this transaction that is created by the call to
using (TransactionScope myTransactionScope = new TransactionScope())