In an attempt to close my question on connections remaining open and exceeding the maximum pool, I'm trying tor rewrite the function that is used to connect to our database.
The function exists within a homegrown compiled library. using reflector I can see the code looks like this:
public SqlProvider([Optional, DefaultParameterValue("")] string StrConnection)
{
string str;
if (StrConnection == "")
{
str = ConfigurationSettings.AppSettings["ConStr"];
}
else
{
str = StrConnection;
}
SqlConnection connection = new SqlConnection(str);
connection.Open();
this.MyCommand = new SqlCommand();
SqlCommand myCommand = this.MyCommand;
myCommand.Connection = connection;
myCommand.CommandType = CommandType.Text;
myCommand = null;
this.MyDataAdapter = new SqlDataAdapter(this.MyCommand);
this.MyCommandBuilder = new SqlCommandBuilder(this.MyDataAdapter);
this.MyDataSet = new DataSet();
}
I'm planning on amending this to read
public SqlProvider([Optional, DefaultParameterValue("")] string StrConnection)
{
string str;
if (StrConnection == "")
{
str = ConfigurationSettings.AppSettings["ConStr"];
}
else
{
str = StrConnection;
}
using (SqlConnection connection = new SqlConnection(str))
{
connection.Open();
this.MyCommand = new SqlCommand();
SqlCommand myCommand = this.MyCommand;
myCommand.Connection = connection;
myCommand.CommandType = CommandType.Text;
myCommand = null;
this.MyDataAdapter = new SqlDataAdapter(this.MyCommand);
this.MyCommandBuilder = new SqlCommandBuilder(this.MyDataAdapter);
this.MyDataSet = new DataSet();
}
}
and then recompiling the dll.
Given that an instance of SQLProvider() is typically created at the top of a public class, and then that instance is used within class members (eg:
public class Banner
{
DSLibrary.DataProviders.SqlProvider db = new DSLibrary.DataProviders.SqlProvider(Defaults.ConnStr);
public Banner()
{
}
public DataTable GetBannerImages(string bannerLocation,int DeptId)
{
using (DSLibrary.DataProviders.SqlProvider db = new DSLibrary.DataProviders.SqlProvider(Defaults.ConnStr))
{
DataTable dt = new DataTable();
//Add Parameter #BannerLocation for Banner of Specific Location
//Call proc_getBannerImages Stored procedure for Banner Images
db.AddStoredProcParameter("#BannerLocation", SqlDbType.VarChar, ParameterDirection.Input, 100, bannerLocation);
db.AddStoredProcParameter("#DeptId", SqlDbType.Int, ParameterDirection.Input, 0, DeptId);
dt = db.ExecuteStoredProcedure("proc_getBannerImages");
return dt;
}
}
}
am I going about this the right way? It seems to me the connection will be disposed of before the data has actually been retrieved. Also, Visual Studio tells me that SQLProvider() must be implicitly convertible to System.IDisposable - how would I go about implementing this?
I tried wrapping all the members of class Banner in a using (DSLibrary.DataProviders.SqlProvider db = new DSLibrary.DataProviders.SqlProvider(Defaults.ConnStr)){} statement but intellisense then displays a "Invalid token 'using' in class, struct, or interface member declaration" error.
What is the best way to go about this?
UPDATE
I've tried disassembling adjusting and recompiling the DSLibrary, but as CHris_Lively says, I thinkit's doing nothing for me. Changing the instance in question to what I preceive to be a more standard format works so far:
public DataTable GetBannerImages(string bannerLocation,int DeptId)
{
using (SqlConnection conn = new SqlConnection(Defaults.ConnStr))
{
SqlCommand cmd = new SqlCommand("proc_getBannerImages", conn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#BannerLocation", bannerLocation));
cmd.Parameters.Add(new SqlParameter("#DeptId", DeptId));
SqlDataAdapter da = new SqlDataAdapter();
da.SelectCommand = cmd;
DataTable dt = new DataTable();
da.Fill(dt);
return dt;
}
}
I'm about to look into the Enterprise library, seems like it might be the way forward.
It is not recommended to hold on to connections any longer than required (See also Chapter 14 of Improving .NET Application Performance and Scalability: Patterns and Practices from Microsoft Press).
In practice I'd change your class to instead not have the SqlConnection (or SqlDataAdapter or SqlCommandBuilder) as a member on the class (if you must, then you should implement the IDisposable pattern), but instead create new instances, wrapped in using statements on the class methods that need to use them.
I DO NOT think you are doing it correctly. As soon as you hit the end of the using block, the SqlConnection variable will become unusable . If you want to use it outside the constructor, dont put the using {} around the SqlConnection variable (The sqlcommand variable MyCommand is using it indirectly outside the constructor).
Instead, make you SqlProvider class implement IDisposable, and call Dispose on the MyCommand, MyDataAdapter, MyDataSet etc. variables there.
You probably should have something like this in your SqlProvider class :
public void Dispose()
{
if (MyCommand != null)
{
MyCommand.Dispose();
}
//... Similarly for MyDataAdapter,MyDataSet etc.
}
Your class needs to implement the IDisposable interface if you want to use it in a using block. See http://msdn.microsoft.com/en-us/library/system.idisposable.dispose%28v=VS.100%29.aspx for guidelines on dispose() and IDisposable.
You're close. However, a couple issues.
First, it looks like that whole DSLibrary isn't buying you anything at all.
When doing data access you typically want to structure it where acquiring the connection and executing the command are in the same function. You're methods should only return the result of the operation. This way you can cleanly use the IDisposable interface of the connection, command, and reader.
The following example uses Enterprise Library. Note that the Db doesn't have a using clause. It doesn't implement IDisposable. Instead, the command is responsible for letting go of the connection when it goes out of scope:
public static DataTable GetBannerImages(String bannerLocation, Int32 departmentId)
{
DataTable result = new DataTable();
result.Locale = CultureInfo.CurrentCulture;
Database db = DatabaseFactory.CreateDatabase("NamedConnectionStringFromConfig");
using (DbCommand dbCommand = db.GetStoredProcCommand("proc_getBannerImages"))
{
db.AddInParameter(dbCommand, "BannerLocation", DbType.String, bannerLocation);
db.AddInParameter(dbCommand, "DeptId", DbType.Int32, departmentId);
using (IDataReader reader = db.ExecuteReader(dbCommand))
{
SopDataAdapter dta = new SopDataAdapter(); // descended from DbDataAdapter
dta.FillFromReader(result, reader);
} // using dataReader
} // using dbCommand
return result;
} // method::GetBannerImages
You probably already have something that will convert a reader to a datatable, if not just look into subclassing the System.Data.Common.DbDataAdapter class
I've had tremendous success with the Enterprise Library. It's fast, efficient, and when going this route I've never had memory leaks or db connection issues.
There is no need to set variables to null - anyway they will be deleted by GC.
Also you need to call Dispose() or Close() for all classes that implements IDisposable. e.g. SqlConnection.
You can do that manually:
SqlConnection conn = null;
try
{
// use conn
}
finally
{
if (conn != null)
conn.Close();
}
or automatically using using block:
using (SqlConnection = new SqlConnection())
{
// use conn
}
(as you do)
Also you can decrease your code a bit using operator ?:
string str = String.IsNullOrEmpty(StrConnection) ? ConfigurationSettings.AppSettings["ConStr"] : StrConnection;
or ??:
string str = StrConnection ?? ConfigurationSettings.AppSettings["ConStr"];
Related
First off, prior to asking this question I have been researching connection pooling for the better part of my morning. What I've found is that is turned on by default, handled by the application/system, and C#'s System.Data.X namespaces (where X is something like OLEDBConnection / SqlConnection ) handles it automatically.
But as I'm still new to the whole database access and pooling, I am looking for clarification since I wrote a class to handle connections at a higher level.
public abstract class AbstractDatabase<T> where T: where T : System.Data.Common.DbConnection
{
// Take this class's connection string and return a new connection object
public abstract T GetConnection();
public abstract SqlKata.Compilers.Compiler { get; }
// Compile the query and return a DataTable
// This is actually pointing to a static method that uses the IDBConnection interface and logs any errors, but I copied the relavant code here
public virtual DataTable GetDataTable(SqlKata.Query query)
{
using (var Conn = GetConnection())
{
if (Conn.State == ConnectionState.Closed) Conn.Open();
using (var Cmd = Conn.CreateCommand())
{
Cmd.CommandType = CommandType.Text;
Cmd.Connection = Conn;
Cmd.CommandText = Compiler.Compile(Query).ToString(); // the query is fully formed string here
Cmd.Parameters.AddDummyParameter(); // I noticed that without any parameters, this didn't work, so this method adds a dummy, even if the string doesn't have any '?' characters
using (var DR = Cmd.ExecuteReader())
{
DataTable DT = new DataTable();
DT.Load(DR);
return DT;
}
}
}
}
}
public class SomeDatabase : AbstractDatabase<OleDbConnection>
{
// class implementation
}
here is my question:
If I were to somewhere else in my program do something like this:
DataTable Tbl1 = SomeDatabase.GetDataTable(Query1);
DataTable Tbl2 = SomeDatabase.GetDataTable(Query2);
....
DataTable Tbl10 = SomeDatabase.GetDataTable(Query10);
My understanding is that the method's Using ( var Conn = GetConnection()) statement will automatically issue a Conn.Close() and dispose of the Conn as the method exits. But connection pooling is automatically enabled, so it will actually reuse that connection almost immediately instead of closing it, correct?
With that level of abstraction, does it make sense to write it like this?
using ( var conn = SomeDatabase.GetConnection())
{
DataTable Tbl1 = SomeDatabase.GetDataTable(Query1);
....
DataTable Tbl10 = SomeDatabase.GetDataTable(Query10);
}
Would this make any difference?
Should the method be written to accept the connection that already open ?
Should the class be written to use the same connection object itself, rather than calling GetConnection() every time ? ( for example, a get-only property that initializes the backing field on first request to it, then all subsequent requests actually receive the backing field's object)
I have this code in my SQL Connect class.
public class SqlConnect
{
public string connectionString = ConfigurationManager.ConnectionStrings["cn"].ConnectionString;
private SqlConnection con;
public SqlCommand cmd;
private SqlDataAdapter sda;
private DataTable dt;
private SqlDataReader sdr;
public SqlConnect()
{
con = new SqlConnection(connectionString);
con.Open();
}
public void SqlQuery(string queryTxt)
{
cmd = new SqlCommand(queryTxt, con);
}
public DataTable QueryEx()
{
sda = new SqlDataAdapter(cmd);
dt = new DataTable();
sda.Fill(dt);
con.Close();
return dt;
}
public void NonQueryEx()
{
cmd.ExecuteNonQuery();
con.Close();
}
public void Reader()
{
sdr = cmd.ExecuteReader();
while(sdr.Read())
{
for (int i = 0; i < sdr.FieldCount; i++) ;
}
return;
}
Now, in my button I have this code. I'm having error saying cannot convert from Void to Object.
con = new SqlConnect();
con.SqlQuery("Select Department from Salary");
cboDept.Items.Add(con.Reader());
My question is how can I make this run? when I use the public datatable QueryEx I can only get the column name appear on the combobox (just the Column name). I'm searching here but I cant find any solution. How to make this code run?
You're looking for the DisplayMember and ValueMember properties.
Also, it's generally a bad practice to try to and manage your connection like this as it can potentially lead to leaked connections. In a WinForms environment, it's not as dangerous as it is in a web environment, but it is something to be cognizant of, especially if you have lots of WinForms clients talking to the same database. If you have an exception that is thrown in your NonQueryEx method from your code above and caught at a higher level, your SqlConnection will be left open.
See: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-connection-pooling?redirectedfrom=MSDN
The way you want to make your database calls is to wrap your construction of your SqlConnection object in a using block:
Example:
using (var cn = new SqlConnection(connectionString))
{
var cmd = new SqlCommand(...);
cmd.ExecuteNonQuery();
}
If you're wanting to use an abstraction layer to save yourself from typing, wrap your call within a using block within that layer. This is the #1 mistake that I see with ADO.Net code at new clients (in other words, this is an incredibly common mistake and don't let this criticism discourage you). ADO.Net manages connections behind the scenes. Opening and closing a connection does not actually terminate a connection to a database, it releases it to the pool so the next call to .Open or .Close can reuse that same connection. Holding onto it for any longer than necessary means that new connections may need to be created. Likewise, there is no guarantee when your SqlConnect() instances will be cleaned up by .Net, so if you have a repeated exception that is caught at a higher level, this can result in multiple connections waiting to be released until your SqlConnect class is garbage collected.
Im making a few static classes to avoid rewriting the same code multiple times and to keep all the database related methods in the same place.
the class look like this:
public static class Database_dbSurvey
{
public static DataSet GetQuestionari()
{
SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["default_connection"].ConnectionString);
string query = "[admin].[SRV_Categorie_Lista]";
SqlCommand cmd = new SqlCommand(query, connection);
cmd.CommandType = CommandType.StoredProcedure;
//GetDataSet use the SqlDataAdapter.fill() method
return Utils.GetDataSet(cmd);
}
etc.... (others similar methods)
}
I want to reduce the code and I want to make it more "object oriented", so I started by making a property for the SqlConnection (which is the same for every methods of this class).
private static SqlConnection connection
{
get { return new SqlConnection(ConfigurationManager.ConnectionStrings["default_connection"].ConnectionString); }
}
The problem is that it works perfectly with SqlDataAdapter.fill(), till I use a method like this:
using (connection)
{
connection.Open();
cmd.ExecuteNonQuery();
}
Now, the next usage of the connection will throw the "not istanziated" exception and I can't understand why.
what is the correct way to define the connection property?
p.s.
if you have other suggestion on improving the code it will be higly appreciated
EDIT:
I still dont get it why the "new" keyword do not create another istance of the SQLConnection everytime I call it.
however I made some changes to make the code safer:
private static string connection_string
{
get { return ConfigurationManager.ConnectionStrings["Connection_dbPrysmianSurvey"].ConnectionString; }
}
public static DataSet GetQuestionari(string username)
{
string query = "[dbo].[SRV_Test_Lista]";
using (SqlConnection connection = new SqlConnection(connection_string))
using (SqlCommand cmd = new SqlCommand(query, connection))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#username", username);
return Utils.GetDataSet(cmd);
}
}
public static int CreaTest(string ID_questionario, string username)
{
string query = "[dbo].[srv_test_genera]";
using (SqlConnection connection = new SqlConnection(connection_string))
using (SqlCommand cmd = new SqlCommand(query, connection))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#ID_categoria", ID_questionario);
cmd.Parameters.AddWithValue("#ID_utente", username);
connection.Open();
return (int)cmd.ExecuteScalar();
}
}
But considering the fact that I have 40-50 methods it's still is a pain to rewrite the same lines 40-50 times, any suggestions?
Do not use using keyword in your case.
It will dispose the connection after that using's scope is complete by calling SqlConnection.Dispose method.
using(connection) {...}'s equivalent -
try
{
connection.Open();
cmd.ExecuteNonQuery();
}
catch
{
throw;
}
finally
{
connection.Dispose();
}
As #Parag Meshram mentioned, using keyword disposes everything inside itself when completed.
There are two ways I can suggest:
(1) Don't make connection a static method
private SqlConnection connection() { get { return new
SqlConnection(ConfigurationManager.ConnectionStrings["default_connection"].ConnectionString);}
}
then use it like
SqlConnection newConnection = connection();
using (newConnection) { ... }
Or
(2) Keep your static method as is but use it like this:
try {
connection.Open();
cmd.ExecuteNonQuery(); } catch {
throw; }
This way, your connection is not disposed.
I had the same problem a year ago and that's now I resolved it, using method (2).
I'm working with a DAL object that is written in a layout similar to the following code. I simplified a lot of the code code just to show the setup.
public class UserDatabase : IDisposable
{
private SqlDataAdapter UserDbAdapter;
private SqlCommand UserSelectCommand;
private SqlCommand UserInsertCommand;
private SqlCommand UserUpdateCommand;
private SqlCommand UserDeleteCommand;
private System.Data.SqlClient.SqlConnection SQLConnection;
public UserDatabase()
{
this.SQLConnection = new System.Data.SqlClient.SqlConnection(ConnectionString);
this.UserDbAdapter= new SqlDataAdapter();
this.UserDbAdapter.DeleteCommand = this.UserDeleteCommand;
this.UserDbAdapter.InsertCommand = this.UserInsertCommand;
this.UserDbAdapter.SelectCommand = this.UserSelectCommand;
this.UserDbAdapter.UpdateCommand = this.UserUpdateCommand;
}
private bool FillUsers(DataSet UserDataSet, out int numberOfRecords)
{
bool success = true;
numberOfRecords = 0;
string errorMsg = null;
this.UserDbAdapter.SelectCommand = this.GetUsersSelectCommand();
numberOfRecords = UserDbAdapter.Fill(UserDataSet, UsersTableName);
return success;
}
private SqlCommand GetUserSelectCommand()
{
if (this.UserSelectCommand==null)
this.UserSelectCommand= new System.Data.SqlClient.SqlCommand();
this.UserSelectCommand.CommandText = "dbo.Users_Select";
this.UserSelectCommand.CommandType = System.Data.CommandType.StoredProcedure;
this.UserSelectCommand.Connection = this.SQLConnection;
this.UserSelectCommand.Parameters.Clear();
this.UserSelectCommand.Parameters.AddRange(new System.Data.SqlClient.SqlParameter[] {
new System.Data.SqlClient.SqlParameter("#RETURN_VALUE", System.Data.SqlDbType.Variant, 0, System.Data.ParameterDirection.ReturnValue, false, ((byte)(0)), ((byte)(0)), "", System.Data.DataRowVersion.Current, null)});
return UserSelectCommand;
}
There are multiple other Fill type functions that are written the same way reusing the Connection object, SqlCommands, and SqlDataAdapter. The SqlDataAdapter manages opening and closing of the SqlConnection internally.
So my question is multipart. Is this design bad? If so, why?
If it is bad, should it be changed to keeping things in a more local scope like the following:
public bool FillUsers(DataSet UserDataSet)
{
using (SqlConnection conn = new SqlConnection(ConnectionString))
{
using (SqlCommand command = GetUsersSelectCommand())
{
using (SqlDataAdapter adapter = new SqlDataAdapter(command, conn))
{
adapter.Fill(UserDataSet, UsersTableName);
}
}
}
}
This would have to be done for all the functions which seems like creating, disposing, and then remaking would be worse than keeping the items around. However this seems to be the setup I see everywhere online.
No, there isn't anything wrong with that. You should dispose your objects that implement IDisposable as soon as you are done with them.
Given a SqlConnection, when you dispose of the connection, the underlying connection will simply be returned to the pool. It's not necessarily "closed" as you might think. It's best to let the connection pool do it's job. Here is a link on MSDN to ADO.NET connection pooling. Trying to make it do things it wasn't designed for (some people call this optimizing, surprisingly) is usually a trip down the rabbit hole.
Also, make sure you have actually measured and observed a problem before trying to optimize it. (and I don't mean this in a harsh way, only to save you time).
I'm a big fan of keeping my code simple and trim so it can be re-usable, on thing i'm struggling with is using the data reader for different types of objects, I had it in a method and found there were problems with connections closed or being left open. SO I am being forced, for the mean time to copy and paste the code, which is something I hate!!!
Is there any way I can scale this down so I can put it in a method and make it re-usable and nice?
ENT_AuctionBid ret = new ENT_AuctionBid();
try
{
SqlParameter[] Params = new SqlParameter[]{
new SqlParameter("#ID", ID )
};
using (SqlConnection conn = new SqlConnection(this.ConnectionString))
{
using (SqlCommand command = new SqlCommand("GetItem", conn))
{
SqlDataReader reader;
command.CommandType = CommandType.StoredProcedure;
conn.Open();
command.Parameters.AddRange(Params);
reader = command.ExecuteReader(CommandBehavior.SingleRow);
while (reader.HasRows)
{
while (reader.Read())
{
//
ret = this.Convert(reader);
}
reader.NextResult();
}
reader.Close();
}
}
}
catch (Exception ex)
{
}
return ret;
You should use SQLDataAdapter.
Here's a nice example on how to use it:
http://www.dotnetperls.com/sqldataadapter
Also, you might want to consider switching to Entity Framework, it will make your data access much, much easier, but might be complicated in an existing project.
You can make it using a lot less lines:
// Skipped creating temp variable
try {
using (SqlConnection conn = new SqlConnection(this.ConnectionString))
using (SqlCommand command = new SqlCommand("GetItem", conn) { CommandType = CommandType.StoredProcedure} ) {
command.Parameters.AddWithValue(#ID, ID);
conn.Open();
// reader is IDisposable, you can use using
using (var reader = command.ExecuteReader(CommandBehavior.SingleRow)) {
// Skipped parsing multiple result sets, you return after the first
// otherwise there's no point using SingleRow
// If nothing is read, return default value
return reader.Read() ? this.Convert(reader) : new ENT_AuctionBid();
}
}
}
catch (Exception ex) {
// Handle your exception here
}
// Return default value for error
return new ENT_AuctionBid();
All connections are closed using this code (because using is used). No unneeded loops are created, becuase you only expect a single row. And the temporary variable is not needed, so the abondend object is not created, only when it is used it is created.
This is a bit smaller:-
try
{
using (SqlConnection conn = new SqlConnection(this.ConnectionString))
{
using (SqlCommand command = new SqlCommand("GetItem", conn))
{
command.Paramaters.AddWithValue("#ID",ID);
command.CommandType = CommandType.StoredProcedure;
conn.Open();
reader = command.ExecuteReader();
while (reader.Read())
{
//
ret = this.Convert(reader);
}
}
}
}
catch (Exception ex)
{
}
Create helper methods for creating and returning an object of type SqlCommand. Pass a connection object to this helper method as well as stored procedure name and parameters list (if any). If you have different objects that are created from the data reader, pass the data reader to a constructor and let it generate an object based on that data.
As for closing the connection you should always have try...catch...finally. In the finally section close the connection.
In my projects i usually solve this problem creating an utility class that contains all the methods to access to the DB and manage inside all the stuff related to the db connection and the adapter.
For example a class called DBSql which contains a connection (SqlConnection connection;) as private member and the following methods:
//execute the query passed to the function
public System.Data.DataSet ExecuteQuery(string query)
//returns if a query returns rows or not
public bool HasRows(string query)
//execute commands like update/insert/etc...
public int ExcuteNonQuery(string sql)
In my class, you just pass a string and the class initialize the various DataAdapter and Command to execute it and return a dataset. Obiously you can complicate it to manage parameters/transaction and everything else.
In this way you are sure that the connection and the object are always handled the same way, and, hopefully, in a correct way.
You can use a utility file, such as SqlHelper.cs from Microsoft Data Access Application Block. Then all the code you need is this:
using (SqlDataReader sdr = SqlHelper.ExecuteReader(this.ConnectionString, "GetItem", ID))
{
while (sdr.Read())
{
ret = this .Convert(sdr);
}
}
You could start using LINQ-to-SQL, which has it's own DataClass system in which you just drag-&-drop your database tables and stored procedures. Then you just have to create an instance at the top of your classes -- private MyCustomDataClass _db = new MyCustomDataClass(); and then you can just type in _db.<Here all datatables and SPROCs will appaer for you to choose>.
Example (from when all SPROCs are added to the DataClass)
private MyCustomDataClass _db = new MyCustomDataClass();
public void MethodToRunSPROC(string email, Guid userId)
{
_db.MySPORC_AddEmailToUser(email, userId);
}