Develop Reference

Lock on Static List or access by Key

Please give expert opinion, refer to below static sorted list based on key value pair.
Method1 for close connection uses approach of accessing sorted list using key.
Method2 for close connection uses lock statement on the Sorted List and access it by index.
Please guide which approach is better as thousands of users simultaneously creating thousands of connections on web application. Note, accessing by index without locking can raise Index out of bound exception.
internal class ConnA
{
static internal SortedList slCons = new SortedList();
internal static bool CreateCon(string ConnID)
{
string constring = "sqlconnectionstring_containing_DataSource_UserInfo_InitialCatalog";
SqlConnection objSqlCon = new SqlConnection(constring);
objSqlCon.Open();
bool connSuccess = (objSqlCon.State == ConnectionState.Open) ? true : false;
if (connSuccess && slCons.ContainsKey(ConnID) == false)
{
slCons.Add(ConnID, objSqlCon);
}
return connSuccess;
}
//Method1
internal static void CloseConnection(string ConnID)
{
if (slCons.ContainsKey(ConnID))
{
SqlConnection objSqlCon = slCons[ConnID] as SqlConnection;
objSqlCon.Close();
objSqlCon.Dispose();
objSqlCon.ResetStatistics();
slCons.Remove(ConnID);
}
}
//Method2
internal static void CloseConnection(string ConnID)
{
lock (slCons)
{
int nIndex = slCons.IndexOfKey(ConnID);
if (nIndex != -1)
{
SqlConnection objSqlCon = (SqlConnection)slCons.GetByIndex(nIndex);
objSqlCon.Close();
objSqlCon.Dispose();
objSqlCon.ResetStatistics();
slCons.RemoveAt(nIndex);
}
}
}
internal class UserA
{
public string ConnectionID { get { return HttpContext.Current.Session.SessionID; } }
private ConnA objConnA = new objConnA();
public void ConnectDB()
{
objConnA.CreateCon(ConnectionID));
}
public void DisConnectDB()
{
objConnA.CloseConnection(ConnectionID));
}
}

Access to the SortedList isn't thread safe.
In CreateCon, two threads could access this simultaneously:
if (connSuccess && slCons.ContainsKey(ConnID) == false)
Both threads could determine that the key isn't present, and then both threads try to add it, so that one of them fails.
In method 2:
When this is called - slCons.RemoveAt(nIndex); - the lock guarantees that another call to the same method won't remove another connection, which is good. But nothing guarantees that another thread won't call CreateCon and insert a new connection string, changing the indexes so that nIndex now refers to a different item in the collection. You would end up closing, disposing, and deleting the wrong connection string, likely one that another thread was still using.
It looks like you're attempting an orchestration which ensures that a single connection string will be used across multiple operations. But there's no need to introduce that complication. Whatever class or method needs a connection, there's no need for it to collaborate with this collection and these methods. You can just let each of them open a connection when it needs it and dispose the connection when it's done.
That's expensive, but that's why the framework implements connection pooling. From the perspective of your code connections are being created, opened, closed, and disposed.
But behind the scenes, the "closed" connection isn't really closed, at least not right away. It's actually kept open. If, in a brief period, you "open" another connection with the same connection string, you're actually getting the same connection again, which is still open. That's how the number of connections opened and closed is reduced without us having to manually manage it.
That in turn prevents us from having to do what it looks like you're doing. This might be different if we were opening a transaction on a connection, and then we had to ensure that multiple operations were performed on the same connection. But even then it would likely be clearer and simpler to pass around the connection, not an ID.

Managing and Closing Dynamically created SQL connections in .net

I have a c# windows form application that connects to databases dynamically where each user may connect to different databases.
The current implementation is as follows:
Connection Repository that contains a dynamically populated list of connections (per user).
When a user initiates a request that requires a database connection the respective connection is looked up from the connection repository ,opened , and then used in the user request .
Code Sample from the connection repository
public class RepoItem
{
public string databasename;
public SqlConnection sqlcnn;
}
public class ConnectionRepository
{
private List<RepoItem> connectionrepositroylist;
public SqlConnection getConnection(String dbname)
{
SqlConnection cnn = (from n in connectionrepositroylist
where n.databasename == dbname
select n.sqlcnn).Single;
cnn.Open();
return cnn;
}
}
sorry for any code errors i just improvised a small version of the implementation for demonstration purpose.
I'am not closing connections after a command execution because it may be used by another command simultaneously.
The questions are:
Should i be worried about closing the connections ?
Does connection close automatically if it is idle for a specific period ?
I have a method in mind to implement a timer in the created Connection Repository and check for idle connections through the Executing ConnectionState Enumeration and close them manually.
Any suggestions are welcome .
When i want a specific connection i call the getConnection function in the ConnectionRepository class and pass the database name as a parameter.
PS: I didn't post the complete implemented code because it is quite big and includes the preferences that affect the populating of the connection list.

I would suggest not to return the SQLConnection to the calling method at all.
Instead, create a method that will accept an Action<SqlConnection>, create the connection inside a using block, and execute the action inside that block
This way you know that the connection will always be correctly closed and disposed, while giving the using code the freedom to do whatever it needs:
public class RepoItem
{
public string databasename;
public SqlConnection sqlcnn;
}
public class DatabaseConnector
{
private List<RepoItem> connectionrepositroylist;
private SqlConnection GetConnection(String dbname)
{
return (from n in connectionrepositroylist
where n.databasename == dbname
select n.sqlcnn).SingleOrDefault();
}
public void Execute(String dbname, Action<SqlConnection> action)
{
using (var cnn = GetConnection(dbname))
{
if (cnn != null) // in case dbname is not in the list...
{
cnn.Open();
action(cnn);
}
}
}
}
Then, to execute an sql statement you can do something like this:
public void ExecuteReaderExample(string dbName, string sql)
{
Execute("dbName",
connection =>
{
using (var cmd = new SqlCommand(sql, connection))
{
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
// do stuff with data form the database
}
}
}
});
}
Of course, you can also wrap the SqlCommand in a method like this.
I've been working with this approach for quite some time now, and as far as I can tell it's working well. In fact, It's working so well I've published a project on git hub based on this approach.
It saves you a lot of the plumbing when dealing with ado.net, by wrapping the connection, command, reader and adapter much the same way.
Feel free to download it and adapt to your needs.
P.S.
To answer your questions directly:
Should i be worried about closing the connections ?
Yes, you should.
Does connection close automatically if it is idle for a specific period ?
No, it doesn't.
However, implementing a method like I suggested will handle closing and disposing the connection object for you, so you don't need to worry about it.
Update
As Yahfoufi wrote in his comment, this design has a flaw, since multiple commands are using the same instance of SqlConnection, you are risking closing the connection while other commands are running.
However, fixing this design flaw is very easy - instead of holding SqlConnection in RepoItem you can simply hold the connection string:
public class RepoItem
{
public string DatabaseName {get; set;}
public string ConnectionString {get; set;}
}
Then you change the GetConnection method like this:
private SqlConnection GetConnection(String dbname)
{
return new SqlConnection(from n in connectionrepositroylist
where n.databasename == dbname
select n.sqlcnn).SingleOrDefault());
}
Now each Execute method is working on it's own individual instance of SqlConnection so you don't need to worry about closing in the middle of some other command executing.
However, While we are on the subject of refactoring, I would suggest removing the RepoItem class all together and instead of using a List<RepoItem> to hold the connection strings simply use a Dictionary<string, string>, where the database name is the key and the connection string is the value. This way you can only have one connection string per database name, and your GetConnection method is simplified to this:
private Dictionary<string, string> connectionrepositroylist;
private string GetConnectionString(String dbname)
{
return connectionrepositroylist.ContainsKey(dbname) ? connectionrepositroylist[dbname] : "";
}
So, the complete DatabaseConnector class will look like this:
public class DatabaseConnector
{
private Dictionary<string, string> connectionrepositroylist;
private string GetConnectionString(String dbname)
{
return connectionrepositroylist.ContainsKey(dbname) ? connectionrepositroylist[dbname] : "";
}
public void Execute(String dbname, Action<SqlConnection> action)
{
var connectionString = GetConnectionString(dbname);
if(!string.IsNullOrEmpty(connectionString))
{
using (var cnn = new SqlConnection(connectionString))
{
cnn.Open();
action(cnn);
}
}
}
// Of course, You will need a way to populate your dictionary -
// I suggest having a couple of methods like this to add, update and remove items.
public bool AddOrUpdateDataBaseName(string dbname, string connectionString)
{
if(connectionrepositroylist.ContainsKey(dbname))
{
connectionrepositroylist[dbname] = connectionString;
}
else
{
connectionrepositroylist.Add(dbname, connectionString);
}
}
}

The good news is that ADO.Net manages your connection pools dynamically, so there's minimal overhead in you dynamically opening and closing connections in code. There's a good document here if you want to look through the detail.
To answer the specific questions you've raised:
Should i be worried about closing the connections ?
Yes, but not for the reasons you may think. Microsoft encourage you to close your connections, so as to return them to the pool for (re)use elsewhere in your code. Closing the connection doesn't actually close it - it merely returns the underlying connection to the pool. Failure to close your connections properly can lead to delays in them being returned to the pool, thus adversely affecting your applications performance as more connections need to be added to the pool to cope with demand.
Does connection close automatically if it is idle for a specific
period ?
A connection is only returned to the pool when it's Dispose or Finalise methods get called. If you create a connection and drop it into a static container then it will not be returned to the pool at all. As such, your ConnectionRepository may actually be harming performance.
I have a method in mind to implement a timer in the created Connection
Repository and check for idle connections
This is unnecessary - close your connections to allow them to return to the pool. This way they will be available for other threads to use
Personally, I'd suggest that you modify your RepoItem class to store connection strings, rather than connection objects, and let ADO.Net's pooling do all the heavy lifting.
public static class ConnectionRepository
{
private static readonly Dictionary<string, string> Connections = new Dictionary<string, string>(StringComparer.CurrentCultureIgnoreCase);
public static bool Contains(string key)
{
return Connections.ContainsKey(key);
}
public static void Add(string key, string connectionString)
{
Connections.Add(key, connectionString);
}
public static SqlConnection Get(string key)
{
var con = new SqlConnection(Connections[key]);
con.Open();
return con;
}
}
With this in place, you can query the database as follows:
public static void foo()
{
using (var con = ConnectionRepository.Get("MyConnection"))
using (var cmd = new SqlCommand("SELECT * FROM MyTable", con))
{
var dr = cmd.ExecuteReader();
//...
}
}
Once the query has executed and the connection is no longer required, the using() block calls its Dispose() method and releases the underlying connection back to the pool for re-use.

As #tinudu says, the SqlConnection class reuses existing connections automatically - you don't have to implement that yourself. See SQL Server Connection Pooling.
If you create the SqlConnection object in a using statement, C# will close the connection automatically as required.
Wrapping the whole thing (create connection, open, run query, close connection) in a function is the best idea. You can put the function in a repository base-class, so it is available to all your repositories.
You would need several functions for the different types of SQL query (select, update, stored proc) but you only need to write one of each - they will get reused.

if you worried about so many conditions let say parallel execution as
well so consider reserving connections for app and close all at once
when app is closing.
public class RepoItem
{
public string databasename;
public SqlConnection sqlcnn;
}
public class ConnectionRepository
{
private List<RepoItem> connectionrepositroylist;
public SqlConnection getConnection(String dbname)
{
SqlConnection cnn = (from n in connectionrepositroylist
where n.databasename == dbname
select n.sqlcnn).Single;
if (cnn!= null && cnn.State == cnn.Closed) // Impelement other checks as well
{
cnn.Open();
}
return cnn;
}
}
Implement CloseConnections and call while application closing i.e
Application.ApplicationExit event
public void CloseConnections()
{
foreach (var connection in connectionrepositroylist)
{
try
{
if (connection.State == System.Data.ConnectionState.Open) // check other conditions
{
connection.Close();
}
}
catch (Exception)
{
//logging or special handling
}
}
}
Points to be note
If some query is still executing and user tries to shut down or close the
app can consider following implementations
Wont allow the application shutdown . Callback delegate will help in this
case to ensure that query is returned.
Force stop and close the connection

It is a better Practice to close the sqlconnection manually, Since it can release the connection which can be used for other processes. Also note that you should open the connection as much as late you can and close it early as possible.

In-memory SQLite and database disappearing

I am using Nancy to create an api to a database, and am wanting to test / develop against an in-memory database.
I am using an instance of my database class in a custom bootstrapper class with the connection string Data Source=:memory:, which in turn is creating the necessary tables - I have stepped through this and I'm confident this is occuring.
I am then obtaining a new connection using the same connection string to load/save data, but even a simple select is coming up with the sql error that the table doesn't exist.
Is there a fault in this logic of creating and using a new connection with the same connection string?

OK, so straight from the docs:
The database ceases to exist as soon as the database connection is closed. "
However, this can be worked around with multiple connections by using cache=shared in your connection string.
However, this isn't a solution to the problem because as soon as the last connection closes, the database ceases to exist.

You need to keep the SQLiteConnection open and ensure that the DbContext does not own the connection. This way, when the DbContext is disposed by the container, the connection doesn't get closed with it. Note: This only works with EF6. You can still pass the flag in EF5, but the context will still close the connection when the context gets disposed.
I created a TestBootstrapper which inherited the working Bootstrapper from the web project.
As part of the ConfigureRequestContainer method, I used a method factory on the DbContext registration which created a new DbContext each time, but used the existing connection. If you don't do this, the DbContext will get disposed after your first request and the second request will fail.
public class TestBootstrapper : Bootstrapper
{
private const string ConnectionString = "data source=:memory:;cache=shared;";
private static SQLiteConnection _connection;
private static TestInitializer _initializer = new TestInitializer();
protected override void ConfigureRequestContainer(TinyIoCContainer, NancyContext context)
{
container.Register<Context>((_,__) =>
{
if (_connection == null)
_connection = new SQLiteConnection(ConnectionString);
// The false flag tells the context it does not own the connection, i.e. it cannot close it. (EF6 behaviour only)
var dbContext = new Context(_connection, _initializer, false);
if (_connection.State == ConnectionState.Closed)
_connection.Open();
// I build the DB and seed it here
_initializer.InitializeDatabase(context);
return dbContext;
});
// Additional type registrations
}
// Call this method on a [TearDown]
public static void Cleanup()
{
if (_connection != null && _connection.State == ConnectionState.Open)
_connection.Close();
_connection = null;
}
}

Recommended practice for stopping transactions escalating to distributed when using transactionscope

Using the TransactionScope object to set up an implicit transaction that doesn't need to be passed across function calls is great! However, if a connection is opened whilst another is already open, the transaction coordinator silently escalates the transaction to be distributed (needing MSDTC service to be running and taking up much more resources and time).
So, this is fine:
using (var ts = new TransactionScope())
{
using (var c = DatabaseManager.GetOpenConnection())
{
// Do Work
}
using (var c = DatabaseManager.GetOpenConnection())
{
// Do more work in same transaction using different connection
}
ts.Complete();
}
But this escalates the transaction:
using (var ts = new TransactionScope())
{
using (var c = DatabaseManager.GetOpenConnection())
{
// Do Work
using (var nestedConnection = DatabaseManager.GetOpenConnection())
{
// Do more work in same transaction using different nested connection - escalated transaction to distributed
}
}
ts.Complete();
}
Is there a recommended practise to avoid escalating transactions in this way, whilst still using nested connections?
The best I can come up with at the moment is having a ThreadStatic connection and reusing that if Transaction.Current is set, like so:
public static class DatabaseManager
{
private const string _connectionString = "data source=.\\sql2008; initial catalog=test; integrated security=true";
[ThreadStatic]
private static SqlConnection _transactionConnection;
[ThreadStatic] private static int _connectionNesting;
private static SqlConnection GetTransactionConnection()
{
if (_transactionConnection == null)
{
Transaction.Current.TransactionCompleted += ((s, e) =>
{
_connectionNesting = 0;
if (_transactionConnection != null)
{
_transactionConnection.Dispose();
_transactionConnection = null;
}
});
_transactionConnection = new SqlConnection(_connectionString);
_transactionConnection.Disposed += ((s, e) =>
{
if (Transaction.Current != null)
{
_connectionNesting--;
if (_connectionNesting > 0)
{
// Since connection is nested and same as parent, need to keep it open as parent is not expecting it to be closed!
_transactionConnection.ConnectionString = _connectionString;
_transactionConnection.Open();
}
else
{
// Can forget transaction connection and spin up a new one next time one's asked for inside this transaction
_transactionConnection = null;
}
}
});
}
return _transactionConnection;
}
public static SqlConnection GetOpenConnection()
{
SqlConnection connection;
if (Transaction.Current != null)
{
connection = GetTransactionConnection();
_connectionNesting++;
}
else
{
connection = new SqlConnection(_connectionString);
}
if (connection.State != ConnectionState.Open)
{
connection.Open();
}
return connection;
}
}
Edit: So, if the answer is to reuse the same connection when it's nested inside a transactionscope, as the code above does, I wonder about the implications of disposing of this connection mid-transaction.
So far as I can see (using Reflector to examine code), the connection's settings (connection string etc.) are reset and the connection is closed. So (in theory), re-setting the connection string and opening the connection on subsequent calls should "reuse" the connection and prevent escalation (and my initial testing agrees with this).
It does seem a little hacky though... and I'm sure there must be a best-practise somewhere that states that one should not continue to use an object after it's been disposed of!
However, since I cannot subclass the sealed SqlConnection, and want to maintain my transaction-agnostic connection-pool-friendly methods, I struggle (but would be delighted) to see a better way.
Also, realised that I could force non-nested connections by throwing exception if application code attempts to open nested connection (which in most cases is unnecessary, in our codebase)
public static class DatabaseManager
{
private const string _connectionString = "data source=.\\sql2008; initial catalog=test; integrated security=true; enlist=true;Application Name='jimmy'";
[ThreadStatic]
private static bool _transactionHooked;
[ThreadStatic]
private static bool _openConnection;
public static SqlConnection GetOpenConnection()
{
var connection = new SqlConnection(_connectionString);
if (Transaction.Current != null)
{
if (_openConnection)
{
throw new ApplicationException("Nested connections in transaction not allowed");
}
_openConnection = true;
connection.Disposed += ((s, e) => _openConnection = false);
if (!_transactionHooked)
{
Transaction.Current.TransactionCompleted += ((s, e) =>
{
_openConnection = false;
_transactionHooked = false;
});
_transactionHooked = true;
}
}
connection.Open();
return connection;
}
}
Would still value a less hacky solution :)

One of the primary reasons for transaction escalation is when you have multiple (different) connections involved in a transaction. This almost always escalates to a distributed transaction. And it is indeed a pain.
This is why we make sure that all our transactions use a single connection object. There are several ways to do this. For the most part, we use the thread static object to store a connection object, and our classes that do the database persistance work, use the thread static connection object (which is shared of course). This prevents multiple connections objects from being used and has eliminated transaction escalation. You could also achieve this by simply passing a connection object from method to method, but this isn't as clean, IMO.

DBTransactions between stateless calls using GUIDs

I'm looking to add transactional support to my DB engine and providing to Abstract Transaction Handling down to passing in Guids with the DB Action Command. The DB engine would run similar to:
private static Database DB;
public static Dictionary<Guid,DBTransaction> Transactions = new ...()
public static void DoDBAction(string cmdstring,List<Parameter> parameters,Guid TransactionGuid)
{
DBCommand cmd = BuildCommand(cmdstring,parameters);
if(Transactions.ContainsKey(TransactionGuid))
cmd.Transaction = Transactions[TransactionGuid];
DB.ExecuteScalar(cmd);
}
public static BuildCommand(string cmd, List<Parameter> parameters)
{
// Create DB command from EntLib Database and assign parameters
}
public static Guid BeginTransaction()
{
// creates new Transaction adding it to "Transactions" and opens a new connection
}
public static Guid Commit(Guid g)
{
// Commits Transaction and removes it from "Transactions" and closes connection
}
public static Guid Rollback(Guid g)
{
// Rolls back Transaction and removes it from "Transactions" and closes connection
}
The Calling system would run similar to:
Guid g
try
{
g = DBEngine.BeginTransaction()
DBEngine.DoDBAction(cmdstring1, parameters,g)
// do some other stuff
DBEngine.DoDBAction(cmdstring2, parameters2,g)
// sit here and wait for a response from other item
DBEngine.DoDBAction(cmdstring3, parameters3,g)
DBEngine.Commit(g)
}
catch(Exception){ DBEngine.Rollback(g);}
Does this interfere with .NET connection pooling (other than a connection be accidently left open)?
Will EntLib keep the connection open until the commit or rollback?

The connection will be kept open until a commit or rollback. It is the transaction that is keeping the connection open.
It will not affect connection pooling, other than a connection held by a transaction will not be returned to the connection pool.
I would recommend that you look at the .net TransactionScope. This may be able to meet your needs, without you writing any of this custom code.