As demonstrated by previous Stack Overflow questions (TransactionScope and Connection Pooling and How does SqlConnection manage IsolationLevel?), the transaction isolation level leaks across pooled connections with SQL Server and ADO.NET (also System.Transactions and EF, because they build on top of ADO.NET).
This means, that the following dangerous sequence of events can happen in any application:
A request happens which requires an explicit transaction to ensure data consistency
Any other request comes in which does not use an explicit transaction because it is only doing uncritical reads. This request will now execute as serializable, potentially causing dangerous blocking and deadlocks
The question: What is the best way to prevent this scenario? Is it really required to use explicit transactions everywhere now?
Here is a self-contained repro. You will see that the third query will have inherited the Serializable level from the second query.
class Program
{
static void Main(string[] args)
{
RunTest(null);
RunTest(IsolationLevel.Serializable);
RunTest(null);
Console.ReadKey();
}
static void RunTest(IsolationLevel? isolationLevel)
{
using (var tran = isolationLevel == null ? null : new TransactionScope(0, new TransactionOptions() { IsolationLevel = isolationLevel.Value }))
using (var conn = new SqlConnection("Data Source=(local); Integrated Security=true; Initial Catalog=master;"))
{
conn.Open();
var cmd = new SqlCommand(#"
select
case transaction_isolation_level
WHEN 0 THEN 'Unspecified'
WHEN 1 THEN 'ReadUncommitted'
WHEN 2 THEN 'ReadCommitted'
WHEN 3 THEN 'RepeatableRead'
WHEN 4 THEN 'Serializable'
WHEN 5 THEN 'Snapshot'
end as lvl, ##SPID
from sys.dm_exec_sessions
where session_id = ##SPID", conn);
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
Console.WriteLine("Isolation Level = " + reader.GetValue(0) + ", SPID = " + reader.GetValue(1));
}
}
if (tran != null) tran.Complete();
}
}
}
Output:
Isolation Level = ReadCommitted, SPID = 51
Isolation Level = Serializable, SPID = 51
Isolation Level = Serializable, SPID = 51 //leaked!
The connection pool calls sp_resetconnection before recycling a connection. Resetting the transaction isolation level is not in the list of things that sp_resetconnection does. That would explain why "serializable" leaks across pooled connections.
I guess you could start each query by making sure it's at the right isolation level:
if not exists (
select *
from sys.dm_exec_sessions
where session_id = ##SPID
and transaction_isolation_level = 2
)
set transaction isolation level read committed
Another option: connections with a different connection string do not share a connection pool. So if you use another connection string for the "serializable" queries, they won't share a pool with the "read committed" queries. An easy way to alter the connection string is to use a different login. You could also add a random option like Persist Security Info=False;.
Finally, you could make sure every "serializable" query resets the isolation level before it returns. If a "serializable" query fails to complete, you could clear the connection pool to force the tainted connection out of the pool:
SqlConnection.ClearPool(yourSqlConnection);
This is potentially expensive, but failing queries are rare, so you should not have to call ClearPool() often.
In SQL Server 2014 this seem to have been fixed. If using TDS protocol 7.3 or higher.
Running on SQL Server version 12.0.2000.8 the output is:
ReadCommitted
Serializable
ReadCommitted
Unfortunately this change is not mentioned in any documentation such as:
Behavior Changes to Database Engine Features in SQL Server 2014
Breaking Changes to Database Engine Features in SQL Server 2014
But the change has been documented on a Microsoft Forum.
Update 2017-03-08
Unfortunately this was later "unfixed" in SQL Server 2014 CU6 and SQL Server 2014 SP1 CU1 since it introduced a bug:
FIX: The transaction isolation level is reset incorrectly when the SQL Server connection is released in SQL Server 2014
"Assume that you use the TransactionScope class in SQL Server client-side source code, and you do not explicitly open the SQL Server connection in a transaction. When the SQL Server connection is released, the transaction isolation level is reset incorrectly."
Workaround
It appears that, since passing through a parameter makes the driver use sp_executesql, this forces a new scope, similar to a stored procedure. The scope is rolled back after the end of the batch.
Therefore, to avoid the leak, pass through a dummy parameter, as show below.
using (var conn = new SqlConnection(connString))
using (var comm = new SqlCommand(#"
SELECT transaction_isolation_level FROM sys.dm_exec_sessions where session_id = ##SPID
", conn))
{
conn.Open();
Console.WriteLine(comm.ExecuteScalar());
}
using (var conn = new SqlConnection(connString))
using (var comm = new SqlCommand(#"
SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
SELECT transaction_isolation_level FROM sys.dm_exec_sessions where session_id = ##SPID
", conn))
{
comm.Parameters.Add("#dummy", SqlDbType.Int).Value = 0; // see with and without
conn.Open();
Console.WriteLine(comm.ExecuteScalar());
}
using (var conn = new SqlConnection(connString))
using (var comm = new SqlCommand(#"
SELECT transaction_isolation_level FROM sys.dm_exec_sessions where session_id = ##SPID
", conn))
{
conn.Open();
Console.WriteLine(comm.ExecuteScalar());
}
For those using EF in .NET, you can fix this for your whole application by setting a different appname per isolation level (as also stated by #Andomar):
//prevent isolationlevel leaks
//https://stackoverflow.com/questions/9851415/sql-server-isolation-level-leaks-across-pooled-connections
public static DataContext CreateContext()
{
string isolationlevel = Transaction.Current?.IsolationLevel.ToString();
string connectionString = ConfigurationManager.ConnectionStrings["yourconnection"].ConnectionString;
connectionString = Regex.Replace(connectionString, "APP=([^;]+)", "App=$1-" + isolationlevel, RegexOptions.IgnoreCase);
return new DataContext(connectionString);
}
Strange this is still an issue 8 years later ...
I just asked a question on this topic and added a piece of C# code, which can help around this problem (meaning: change isolation level only for one transaction).
Change isolation level in individual ADO.NET transactions only
It is basically a class to be wrapped in an 'using' block, which queries the original isolation level before and restores it later.
It does, however, require two additional round trips to the DB to check and restore the default isolation level, and I am not absolutely sure that it will never leak the altered isolation level, although I see very little danger of that.
Related
I have this code:
var to = new TransactionOptions();
to.IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted;
using (var ts = new TransactionScope(TransactionScopeOption.Required, to))
{
someQuery.ToList();
ts.Complete();
}
No matter what, the SQL Server Profiler shows that "someQuery" (and any other query on this transaction) run with an isolation level of "Read Committed".
The only way I can force to run as ReadUncommitted if, before someQuery.ToList();, I execute this line:
myContext.Database.ExecuteSqlCommand("SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;");
Why does SQL Server not respect the .Net isolation level? What can I do about this?
We have a C# application using ADO.NET and an SQL-Server with SNAPSHOT Transaction Isolation Level. This is 'as is' and unfortunately cannot be modified.
Now we need insert stuff on a linked server.
We execute the following code (reduced to illustrate the problem):
// Create a snapshot Transaction and close the connection
using (var con = new SqlConnection(myConStr))
{
con.BeginTransaction(TransactionLevel.Snapshot);
}
// do something with a linked server
using (var con = new SqlConnection(myConStr))
{
using (var cmd = con.CreateCommand()
{
cmd.CommandText = "insert into LinkedServer.SomeDb..Table ...";
cmd.ExecuteNonQuery();
}
}
We get can an exception when trying to insert something into the linked server
'Remote access is not supported for transaction isolation level "SNAPSHOT"'
I wonder why it is not possible: We open the connection, make sure it is disposed (clearing all transactions, I guess) and use a second connection for the linked server call.
Executing the stuff in SSMS using plain SQL seems to work.
What are we missing? Is there a proper way to do it?
Thanks for any hints in the right direction.
The secret to understand the problem is the 'connection pooling' which is done by ADO.NET in the Background. The real connection is actually set to SNAPSHOT.
In the second part of the sample code, that connection simply gets reused thus still being in 'snapshot mode'.
The solution is to explicitly set the transaction isolation Level to something else right after opening the Connection.
using (var con = new SqlConnection(myConStr))
{
using (var cmd = con.CreateCommand()
{
cmd.CommandText = "set transaction isolation Level read committed";
cmd.ExecuteNonQuery();
}
using (var cmd = con.CreateCommand()
{
cmd.CommandText = "insert into LinkedServer.SomeDb..Table ...";
cmd.ExecuteNonQuery();
}
}
My current application has all database operations in a giant Using statement with a connection to ensure that the transaction is committed or rolled back in full, currently if i have common methods they pass the current open OleDbConnection so that it can be used.
I would like to use TransactionScope in place of the outer using section. Please see my test code below:
private void Test() {
string _connectionString = "Provider=OraOLEDB.Oracle.1;Password=XXXXXXXX;Persist Security Info=True;User ID=XXXXXXXX;Data Source=XXXXXXX;min pool size=1;incr pool size=5;decr pool size=2;connection timeout=60;";
using (TransactionScope _ts = new TransactionScope(TransactionScopeOption.Required))
{
using (OleDbConnection _cn = new OleDbConnection(_connectionString))
{
_cn.Open(); // Errors Here!
using (OleDbCommand _cmd = new OleDbCommand())
{
_cmd.Connection = _cn;
_cmd.CommandText = "insert into testtable (TEST) values ('FIRST')";
_cmd.CommandType = CommandType.Text;
_cmd.ExecuteNonQuery();
}
}
using (OleDbConnection _cn = new OleDbConnection(_connectionString))
{
_cn.Open();
using (OleDbCommand _cmd = new OleDbCommand())
{
_cmd.Connection = _cn;
_cmd.CommandText = "insert into testtable (TEST) values ('SECOND')";
_cmd.CommandType = CommandType.Text;
_cmd.ExecuteNonQuery();
}
}
}
}
The error i receive is "Unable to enlist in the transaction." I have read that Oracle doesn't like using a TransactionScope (Problems with TransactionScope and Oracle) , but it seems to fit with what i need to achieve. I have found very little information about how to bridge single transactions across connection pooled connections.
EDIT - 11th Feb
I switched from OleDB to ODP.Net and managed to get an official Oracle ORA error out...
ORA-02048: attempt to begin distributed transaction without logging on
Sadly from what i can find i think its an Oracle bug? I have found forum posts which suggest that version 10.2.0.2 has this bug, but i am on 10.2.0.4?
Hoping somebody can help! Thanks
To use with TransactionScope the connection string must contain "enlist=dynamic"
https://docs.oracle.com/database/121/ODPNT/InstallConfig.htm#r6c1-t14
Specifies whether the application enlists in distributed transactions explicitly after an OracleConnection.Open method invocation through EnlistTransaction() or EnlistDistributedTransaction(). To configure ODP.NET to enable dynamic enlistment programmatically, the connection string must contain "enlist=dynamic".
connection string for example:
enlist=dynamic;User Id=USER;Password=pass;Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=TestDB)));
So, my answer was a couple of things:
Firstly I needed to install the OracleMTS service on my client. Secondly i changed from OleDB (via Oracle.ManagedDataAccess v12) to Oracle.DataAccess v11, and it works!
I discovered that v12 of the ODP.Net client and 10.2.0.4 have a bug where the distributed transaction fails, but version 11 works. Still puzzled why it doesn't work with OleDB but i have resolved it now. Hope this can help somebody else in my position!
I'm trying to write some unit tests for my code that connect to SQL Server for persistence, but I would like to be able to run my unit tests by just pointing it at a SQL Server instance, and let the tests create their own database to run tests in, so after each test it can just drop the database and then on setup before the next test recreate it, so I know there is no legacy data or structures left over from a previous test effecting the next test.
In brief: no, you cannot do that. You might be able to leave out the database from the connection string, but in that case, that connection will be made to the configured default database of the login that's connecting to SQL Server (and that default database must exist at the time the connection is made)
If you want to have this scenario, you need to
first connect to your instance and database master and create your new testdb (or whatever it's called)
disconnect
in your tests, connect to the instance and the testdb database
Better yet: use a mocking framework of some sort so you don't even need an actual database in your testing scenario!
I use the following class to facilitate the OP's scenario:
public class MsSqlDatabaseCreator
{
public void Create(string connectionstring)
{
if (DatabaseExists(connectionstring))
{
DropDatabase(connectionstring);
}
CreateDatabase(connectionstring);
}
private static void CreateDatabase(string connectionString)
{
var sqlConnectionStringBuilder = new SqlConnectionStringBuilder(connectionString);
var databaseName = sqlConnectionStringBuilder.InitialCatalog;
sqlConnectionStringBuilder.InitialCatalog = "master";
using (var sqlConnection = new SqlConnection(sqlConnectionStringBuilder.ConnectionString))
{
sqlConnection.Open();
using (var sqlCommand = sqlConnection.CreateCommand())
{
sqlCommand.CommandText = $"CREATE DATABASE {databaseName}";
sqlCommand.ExecuteNonQuery();
}
}
}
private static bool DatabaseExists(string connectionString)
{
var sqlConnectionStringBuilder = new SqlConnectionStringBuilder(connectionString);
var databaseName = sqlConnectionStringBuilder.InitialCatalog;
sqlConnectionStringBuilder.InitialCatalog = "master";
using (var sqlConnection = new SqlConnection(sqlConnectionStringBuilder.ConnectionString))
{
sqlConnection.Open();
using (var command = sqlConnection.CreateCommand())
{
command.CommandText = $"SELECT db_id('{databaseName}')";
return command.ExecuteScalar() != DBNull.Value;
}
}
}
private static void DropDatabase(string connectionString)
{
var sqlConnectionStringBuilder = new SqlConnectionStringBuilder(connectionString);
var databaseName = sqlConnectionStringBuilder.InitialCatalog;
sqlConnectionStringBuilder.InitialCatalog = "master";
using (var sqlConnection = new SqlConnection(sqlConnectionStringBuilder.ConnectionString))
{
sqlConnection.Open();
using (var sqlCommand = sqlConnection.CreateCommand())
{
sqlCommand.CommandText = $#"
ALTER DATABASE {databaseName} SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
DROP DATABASE [{databaseName}]
";
sqlCommand.ExecuteNonQuery();
}
}
}
}
The important part is the switching of the database name (initial catalog) to master. This way you can have just one connectionstring.
What you want to accomplish is possible using a mocking framework, in which case you don't even have to "connect to a database", you simply mock the return values that the database should return in order for you to test your "db handler" implementation.
There are several to choose from when it comes to C#, I can recommend Rhino Mocks and Moq to name two. Here's a question detailing a bit more; https://stackoverflow.com/questions/37359/what-c-sharp-mocking-framework-to-use
Why not have the same named database dedicated for tests? and drop-create it every time. This way you won't need to mess about with connection strings - it is always the same.
And yet, there is a better solution: within all your tests, start transaction, do your test, where your data is messed up. Once you verified (or failed) the test, unroll the transaction. This way you don't need to drop-create your tests for every test, because the data is never changed.
But you'll need to make sure schema in test-database is always up to date. So you'll need to drop-create test database whenever your schema is changed.
I've blogged about database tests and how we deal with Entity Framework migrations. This might not be completely applicable to your situation, but might help with ideas.
Regarding using mocks in your tests - yes this is absolutely valid suggestion and should be followed most of the time. Unless you are trying to test the database layer. In that case no mocks will save you, and you just have to go to DB. Many times over I have tried to mock DbContext in EF, but never managed to simulate realistic DB behavior. So going to DB was easier for me, rather than simulating DB-mock.
I'd use SQL Server Management Objects for the task. It's Server and Database APIs doesn't necessarily need a connection string but I think you might still need to specify a database. You can use master for that. (Check jeroenh's answer about creating object using SMO API as well)
By the way, if you are using .Net 4.0.2 and up you can use LocalDB as well, which is even better.
Edit: Note that actually LocalDB is an SQL Server 2012 feature however you still need .Net Framework > 4.0.2 to be able to use it.
My web application is in asp.net 2.0,c#2.0 and sql server 208 how can i find number of open connections on my sql server 2008 database.and is there any way to clear connection pool.because my site is hosted on shared hosting and they have provided limited connections. In my codding i have closed all the connection after use, but still i am getting warning for suspending database.
Can any one tell me how to find number open connections on database and and how to clear connection pool.
i used using statements for connections and closed all connections after used in finally block. so though there is error it closes oped connections.
Thanks in advance.
This shows the number of connections per each DB:
SELECT
DB_NAME(dbid) as DBName,
COUNT(dbid) as NoOfConnections,
loginame as LoginName
FROM
sys.sysprocesses
WHERE
dbid > 0
GROUP BY
dbid, loginame
And this gives total connections:
SELECT
COUNT(dbid) as TotalConnections
FROM
sys.sysprocesses
WHERE
dbid > 0
From c#, you can follow :
http://www.c-sharpcorner.com/UploadFile/dsdaf/ConnPooling07262006093645AM/ConnPooling.aspx
Another good reference can be found at :
http://www.wduffy.co.uk/blog/monitoring-database-connections/
Call the static method ReleaseObjectPool on the the OleDbConnection - see http://msdn.microsoft.com/en-us/library/system.data.oledb.oledbconnection.releaseobjectpool.aspx
Sql query to get the current active connection
SELECT DB_NAME(dbid) as 'DbNAme', COUNT(dbid) as 'Connections' from master.dbo.sysprocesses with (nolock) WHERE dbid > 0 GROUP BY dbid
you can define dbid if you want connection specific to database
You might want to read up on connection pooling: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
A separate connection pool is created for each distinct connect string. Further, if you are connecting via integraged security and your web site is using Basic or Windows authentication (rather than anonymous), a separate connection pool will be created for each user of the web site.
To clear connection pools, the SqlConnection object provides the methods ClearPool() and ClearAllPool()`. However, an individual connection won't be closed and removed from the pool until it is closed or disposed.
All the various objects involved in the execution of the sql query that implement IDisposable should be wrapped in a using statement to guaranteed proper disposal. Something along these lines:
IEnumerable<BusinessObject> list = new List<BusinessObject>() ;
using ( SqlConnection connection = new SqlConnection( credentials ) )
using ( SqlCommand command = connection.CreateCommand() )
using ( SqlDataAdapter adapter = new SqlDataAdapter( command ) )
using ( DataSet results = new DataSet() )
{
command.CommandType = CommandType.StoredProcedure ;
command.CommandText = #"someStoredProcedure" ;
try
{
connection.Open() ;
adapter.Fill( results ) ;
connection.Close() ;
list = TransformResults( results ) ;
}
catch
{
command.Cancel() ;
throw
}
}
return list ;
You can examine what SPIDs are open in Sql Server either by executing the stored procedure sp_who (must have the appropriate admin permissions in the SQL Server). You can also use perfmon.