Develop Reference

SQLiteConnection.Open keeps blocking if pooling enabled

I'm using System.Data.SQLite (v1.0.104) in a multi-threaded C# application. When a thread want's to update the DB, it opens a new connection in a using statement (calling the method below) and executes its queries. This seems to work well with the connection string in the following example:
[MethodImpl(MethodImplOptions.Synchronized)]
private SQLiteConnection CreateSQLiteConnection()
{
var connection = new SQLiteConnection("Data Source=myDatabase.sqlite;Version=3");
connection.Open();
return connection;
}
However if I add Pooling=True to the connection string, I can observe the following: One thread is blocking on connection.Open(); indefinitely while the other threads are waiting to enter CreateSQLiteConnection. As far as I can tell from the debugger, at this point in time no thread is actually performing any kind of update to the db.
I already tried setting busy and default timeouts but that didn't change anything. I also know that the sqlite documentation suggests to avoid multiple threads altogether but that is currently not an option.
I've added the Synchronized attribute to avoid potential issues regarding multiple threads simultaneously calling SQLiteConnection.Open but it did not seem to make a difference.
Does anyone know what might cause SQLiteConnection.Open to behave like that or what I could try to get more details about this?

You can use SQLite Version 3 ==> .s3db,
public SQLiteConnection dbConnection = new SQLiteConnection(#"Data Source=E:\Foldername\myDatabase.s3db;");
SQLiteConnection cnn = new SQLiteConnection(dbConnection);
cnn.Open();
string sql = "SELECT * FROM Tble_UserSetUp";
DataSet ds = new DataSet();
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataAdapter da = new SQLiteDataAdapter();
da.SelectCommand = mycommand;
da.Fill(ds);
cnn.Close();

Using DbConnection without opening it first

I am accessing a DB using the generic DbConnection provided by DbProviderFactories and I have observed that I don´t need to call Open before using the connection. I provide a sample code that is working (either using "System.Data.SqlClient" or "Oracle.DataAccess.Client" as the providerInvariantName parameter). I have performed all CRUD operations with similar code without explicitly calling Open on the connection, without any noticeable error.
I understand that I don´t need to close it, since the using statement takes care of closing and disposing the connection.
But, when is the connection opened in this code, then? Is it automatically opened when I call Fill on the associated DataAdapter? Is there any consequence of not explicitly calling Open on the connection object before using it?
Because if it is unnecesary and I can save myself a couple of lines of code I will sure do. ;-)
DbProviderFactory myFactoryProvider = DbProviderFactories.GetFactory("System.Data.SqlClient");// same with "Oracle.DataAccess.Client"
using (var myConnection = myFactoryProvider.CreateConnection())
using (var myCommand = myFactoryProvider.CreateCommand())
{
try
{
// Set up the connection
myConnection.ConnectionString = _someConnectionString;
myCommand.Connection = myConnection;
myCommand.CommandText = "SELECT * FROM SOME_TABLE";
// Create DataAdapter and fill DataTable
DbDataAdapter dataAdapter = myFactoryProvider.CreateDataAdapter();
dataAdapter.SelectCommand = myCommand;
DataTable myTable = new DataTable();
dataAdapter.Fill(myTable);
// Read the table and do something with the data
foreach (DataRow fila in myTable.Rows)
{
// Do something
}
}
catch (Exception e)
{
string message = e.ToString();
throw;
}
} //using closes and disposes the connection and the command

The connection to the db should be established and opened when the statement
dataAdapter.Fill(myTable);
runs, so your code goes well as is

C# MySqlConnection won't close

I have an application that fires a mysql command (query) "show databases", the query works and returns properly but I can't close my connections. The user I used had 24 connections allowed at the same time so the problem popped up further down my program but reducing the allowed connections to 2 shows me that I can't even close the first query (which isn't in a loop). The code is the following:
protected override Dictionary<string, Jerow_class_generator.Database> loadDatabases()
{
MySqlConnection sqlCon = new MySqlConnection(this.ConnectionString);
sqlCon.Open();
MySqlCommand sqlCom = new MySqlCommand();
sqlCom.Connection = sqlCon;
sqlCom.CommandType = CommandType.Text;
sqlCom.CommandText = "show databases;";
MySqlDataReader sqlDR;
sqlDR = sqlCom.ExecuteReader();
Dictionary<string, Jerow_class_generator.Database> databases = new Dictionary<string, Jerow_class_generator.Database>();
string[] systemDatabases = new string[] { "information_schema", "mysql" };
while (sqlDR.Read())
{
string dbName = sqlDR.GetString(0);
if (!systemDatabases.Contains(dbName))
{
databases.Add(sqlDR.GetString(0), new MySQL.Database(dbName, this));
}
}
sqlCom.Dispose();
sqlDR.Close();
sqlCon.Close();
sqlCon.Dispose();
return databases;
}
P.S. The 'New MySQL.Database(dbName, this));' is my owm made class which only stores the DB structure, could be considered irrelevant.
The exact error I get is 'max_user_connections'. on the connection.open line of the next time a query needs to be fired.

Rather than keeping track of all the Open/Close/Dispose calls all over the place, I'd recommend just replacing all of those with using statements. This will make sure the expected scope of each object is clear and that it will be destroyed/disposed upon exiting that scope.

Close() nor using will help alone with your problem because ADO.NET is using its own connection pooling and connections are by default not closed until program is closed. There are few options to solve this, but consider performance implications and is this really desired behavior for your application.
Add ";Pooling=False" to your connection string.
SqlConnection.ClearPool Method
SqlConnection.ClearAllPools Method
For more information read: SQL Server Connection Pooling (ADO.NET)

Along with the using suggestions above, when creating your sqlDR variable you should use the CloseConnection command behavior to close the actual connection if that is your intended action. As noted in the documentation here.
When the command is executed, the associated Connection object is closed when the associated DataReader object is closed.
So your code to instantiate your reader would look like this:
//to instantiate your variable
MySqlDataReader sqlDR;
sqlDR = sqlCom.ExecuteReader(CommandBehavior.CloseConnection);
//closing your datareader reference here will close the connection as well
sqlDR.Close();
If you wrap all your code in a using block using the above method, you don't need any of those Close() or Dispose() methods other than the sqlDR.Close();

when use "using" key word what happen is.when the garbage collector activate it first dispose objects which was declred in using statement.

I recommend using connection pooling in combination with the MySqlHelper class, passing the connection string as the first argument. That allows MySQL to open the connection if necessary, or keep it open according to the pooling cfg, without you having to know about it.

I changed my code to use 1 connection and keep it open and when testing I came across an error that a datareader should be closed. Now since all my queries didn't close the dataReader object (I used dataTable.Load(cmd.ExecuteReader()).) I think the problem might be there.
Keeping 1 open connection worked perfectly so I don't know what caused the not closing problem. I gues it was the dataReader not closing by itself.

Close() will definitely help you close your.
using (MySqlConnection conn = GetConnection())
{
conn.Open();
using (MySqlCommand cmd = conn.CreateCommand())
{
if (conn.State != ConnectionState.Open)
{
conn.Open();
}
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "UserDetail";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
list.Add(new Album()
{
Id = Convert.ToInt32(reader["UId"]),
Name = reader["FirstName"].ToString(),
ArtistName = reader["LastName"].ToString()
});
}
}
}
}
In the above code, you can see one if condition before opening the connection it will help you to reuse your already open connections check below code.
if (conn.State != ConnectionState.Open)
{
conn.Open();
}

Why is this class member leaving a SQL connection open?

Further to my recent questions, I've now closed most of the connections that our web application was leaving open. However, the connection created by the function below remains open. Can anyone identify why it is not closing?
public DataTable GetSubDepartment(int deptId)
{
DataTable dt = new DataTable();
using (SqlConnection conn = new SqlConnection(Defaults.ConnStr))
{
SqlCommand cmd = new SqlCommand("proc_getDepartmentChild", conn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#dptParent", deptId));
SqlDataAdapter da = new SqlDataAdapter();
da.SelectCommand = cmd;
da.Fill(dt);
}
return dt;
}
* EDIT *
Following #HenkHolterman's comment:
I'm using SQL Server Management Studio Activity log to view the open connections. This one is listed as sleeping. SO what you say makes sense. Is there any way I can tell that this is a pooled connection rather than an open one?

Most probably because it went back to the connection pool.
Call
SqlConnection.ClearAllPools();
to clear the pool, then it should disappear. This could sometimes be useful, but is usually not needed.

I would assume that it's hanging in the connectionpool

Why isn't SqlConnection disposed/closed?

Given the method:
internal static DataSet SelectDataSet(String commandText, DataBaseEnum dataBase)
{
var dataset = new DataSet();
SqlConnection sqlc = dataBase == DataBaseEnum.ZipCodeDb
? new SqlConnection(ConfigurationManager.AppSettings["ZipcodeDB"])
: new SqlConnection(ConfigurationManager.AppSettings["WeatherDB"]);
SqlCommand sqlcmd = sqlc.CreateCommand();
sqlcmd.CommandText = commandText;
var adapter = new SqlDataAdapter(sqlcmd.CommandText, sqlc);
adapter.Fill(dataset);
return dataset;
}
Why is sqlc (the SqlConnection) not disposed/close after the calling method goes out of scope or sqlc has no more references?
EDIT 1:
Even wrapping it in using, I can still seeing the connection using (I have connection pooling turned off):
SELECT DB_NAME(dbid) as 'Database Name',
COUNT(dbid) as 'Total Connections'
FROM sys.sysprocesses WITH (nolock)
WHERE dbid > 0
GROUP BY dbid
EDIT 2:
Have some more debugging with help I got from here - the answer was the someone hard coded a connection string with pooling. Thanks for all the help - if I could, I would mark all the responses as answers.

C#'s garbage collection is non-deterministic but the language does provide a deterministic structure for resource disposal like this:
using (SqlConnection connection = new SqlConnection(...))
{
// ...
}
This will create a try/finally block which will ensure that the connection object is disposed regardless of what happens in the method. You really ought to wrap any instances of types that implement IDisposable in a using block like this as it will ensure responsibly resource management (of unmanaged resources like database connections) and also it gives you the deterministic control you are looking for.

Because c# is a garbage collected language, and garbage collection is not deterministic. The fact of it is that your sqlconnection is disposed. You just don't get to choose when.
Sql connections are a limited resource, and it's easily possible that you might create enough of them to run out. Write it like this instead:
internal static DataSet SelectDataSet(String commandText, DataBaseEnum dataBase)
{
var dataset = new DataSet();
using (SqlConnection sqlc = dataBase == DataBaseEnum.ZipCodeDb
? new SqlConnection(ConfigurationManager.AppSettings["ZipcodeDB"])
: new SqlConnection(ConfigurationManager.AppSettings["WeatherDB"]))
using (SqlCommand sqlcmd = sqlc.CreateCommand())
{
sqlcmd.CommandText = commandText;
var adapter = new SqlDataAdapter(sqlcmd.CommandText, sqlc);
adapter.Fill(dataset);
}
return dataset;
}
Though in this case you might get away with it, because the .Fill() method is a strange beast:
If the IDbConnection is closed before Fill is called, it is opened to retrieve data and then closed.
So that means the Data Adapter should be taking care of it for you, if you start out with a closed connection. I'm much more concerned that you're passing in your sql command as a plain string. There have to be user parameters in your queries from time to time, and this means you're concatenating that data directly into the command string. Don't do that!! Using the SqlCommand's Paramters collection instead.

I agree with all the answers here plus a connection may be a wider scope than just a method. When you need to use your existing connection in different places the scenario changes a bit. Always make sure to call Dispose for all objects that implement IDisposable after you are done using them. This is a good practice so you don't end up with unused objects that the garbage collector can't decide what to do with them.

It will be, after the garbage collection does it's job. Same goes for opening a file stream for writing without closing it. It might get 'locked' even though the codes went out of scope.

I believe it has something to do with SqlConnection pooling. What you can do, and we do often at work is wrap the entire call in a using statement which causes it to call the dispose() method, hense closing the connection and disposing of the object
You could then do something like this instead:
internal static DataSet SelectDataSet(String commandText, DataBaseEnum dataBase)
{
var dataset = new DataSet();
using(SqlConnection sqlc = dataBase == DataBaseEnum.ZipCodeDb
? new SqlConnection(ConfigurationManager.AppSettings["ZipcodeDB"])
: new SqlConnection(ConfigurationManager.AppSettings["WeatherDB"])) {
SqlCommand sqlcmd = sqlc.CreateCommand();
sqlcmd.CommandText = commandText;
var adapter = new SqlDataAdapter(sqlcmd.CommandText, sqlc);
adapter.Fill(dataset);
return dataset;
}
}