I've got a winform with some child forms on it. Now I'm accessing the child forms by a treeview in my main form. Now in that treeview is a list of websites. Since a website is actually a group of pages, this is also shown in the tree. Website is my parent and the pages are the childs.
When you click on one of the pages the statistics are shown in a child form. When clicked on a website a summary of all pages is shown of that website. Since this can have a lot of data I'm using a background worker that fetches the statics in the background after the treeview is loaded.
Now all fine until I'm changing something in the statistics and it saves automatically into the database (sqlserver2008). Then my backgroundworker generates errors saying the sqlreader is closed. So I'm wondering how I can solve this.
Do I have to access the database on a separate connection? Any other ideas?
Thanks
Are you using the same database connection object on several threads? That's not safe. Every thread must have its own database connection, or it must synchronize the usage so that only one thread accesses the database connection object at any given moment.
It sounds like your UPDATE is closing the connection and your background thread is attempting to use it to query. If im reading between the lines correctly, you have created a connection object as a stateful member of your winform which is being opened once and possibly being closed after an UPDATE (or automatically by the connection pool). Either way you need to take a step back here and think about how your doing things.
Your update and select methods (hopefully in separate classes to your forms) should take care of opening and closing connections exclusively rather than in a statefull way (i.e. the scope of the connection remains in the methods). The ADO connection pool will take care of optimising connections for you.
I also recommend you use disconnected data structures such as DataTable and DataSet rather than DataReader as these keep the connection footprint between the client and the database light. You can happily process and pass around these structures without maintaining an open connection which DataReader requires. Ideally you would populate a light-weight object structure rather than passing DataSet around too much as it does carry some memory overhead - there are a bunch of other reasons for this too e.g. type safety.
There is nothing wrong with calling a method which executes queries on multiple threads as long as the scope of the connection is limited to the method. For example:
public static DataSet GetWebStatsData()
{
DataSet StatsData = new DataSet();
using (OleDbConnection conn = new OleDbConnection(ConnString))
{
using (OleDbCommand cm = new OleDbCommand(GetDataProcedure))
{
using (OleDbDataAdapter adap = new OleDbDataAdapter(cm))
{
cm.Connection = conn;
conn.Open();
adap.Fill(StatsData);
conn.Close();
}
}
}
return StatsData;
}
You can happily call the above method from many threads simultaneously and it should work. I have worked on applications making heavy use of the threadpool executing in some cases scores of queries simultaneously with no ill effect. Where this is happening i would recommend setting a maximum size to the ado.net connection pool as it can grow quite large if left un-guarded (there is typically a connection string attribute for this like 'Max Pool Size=n'). Ado.net will marshal the connections on your behalf so there is a lot of plumbing to leverage in the framework here.
I would guess that the navigation event is firing off a new database request using an existing sqlcommand. Can you post up a code snip?
If you're using the same connection string you can open multiple SqlConenction/SqlCommand/SqlDataReader objects and they'll come from the connection pool. There is no reason you can't have multiple connections to the same database! You can read some more information at SQL Server Connection Pooling (ADO.NET).
If you're trying to use the same SqlConnection on multiple threads at the same time, that will definitely produce unpredictable results. Creating a connection for each thread is easiest, or you could use some form of synchronization if you really want to share a single connection. That would only make sense if your worker threads are doing a significant amount of work besides database access though.
Related
In a C# 2008 windows application that use call a web service, there is a large volume of statements that look like the following:
In a C# 2008 application, I am using linq to sql statements that look like the following:
//Code
TDataContext TData = new TDataContext();
var TNumber = (from dw in cms_TData.people
where dw.Organization_Name.ToUpper().Trim() == strOrgnizationName.Trim().
Right before every call that is made to the database, a new data context object is created.
Would this cause some kind of connection pooling problem to the database? If so, can you tell me how to resolve the connection pooling problem?
Connection pooling is not a problem, it is a solution to a problem. It is connection pooling that enables you write
TDataContext TData = new TDataContext();
without a fear of exhausting the limited number of RDBMS connections, or slowing down your system to a crawl due to closing and re-opening connections too often. The only issue that you may run into with the code like that is caching: whatever is cached in TData is gone when it goes out of scope, so you may re-read the same info multiple times unnecessarily. However, the cache on RDBMS side would help you in most cases, so even the caching is not going to be an issue most of the time.
A DataContext is a lightweight object which closes the database connection as soon as it has completed as a task.
Consequently, creating a large number of these objects shouldn't cause a connection pooling problem unless, possibly, they are being created simultaneously on different threads.
I've got a sql server 2k8 database that's being populated with several hundred thousand records per day.
I'm currently writing some code that's going to make a call to the db, retrieve n records, process them, and write some data back to the db.
There are two ways I see to do this (psuedo code):
function xyz() {
conn = conn creation code
conn.open();
while(not last record) {
select next 1000 records
process each record
last record = true
}
conn.close();
xyz();
}
Essentially creating one connection per batch. The second method:
function xyz() {
conn = connection creation code
while(conn.open();) {
select next 1000 records
process each record
last record = true
}
conn.close();
xyz();
}
I'm curious what's better practice. I feel like it's the latter, but it's also going to have a more-or-less permanent/persistent connection to my db. I'm worried about possible memory overruns or some such.
Thoughts?
Scott
ADO.Net Sqlclient provider (which is I assume you will use, since you say is C#) automatically does connection pooling, see SQL Server Connection Pooling (ADO.NET). Pooled connection are not truly closed when you call Close, they are simply returned to the pool. 'Opening' and 'closing' pooled connection is very fast.
unrelated note: you should embed your connection in a using block:
using (SqlConnection conn = new SqlConnection(...))
{
conn.Open ();
...
}
This way you avoid leaking connecitons in exception conditions.
I would handle the connection outside of the function and pass it in. The function is designed to process entries, not to connect to the database and process entries. I would separate those two jobs.
As far as connection opening/closing goes: yes, you should avoid it. It's not too terribly slow on it's own, but if you're just doing processing, there's no reasons to open/close the connection over and over again. If it's due to memory usage (I'm not very familiar with C# by the way, so this part might just be wrong), you should be freeing the result sets. Closing the connection will free the memory associated with it, however, you should be able to get the same effect without having to reconnect.
It depends on what "process each record" means. If that takes a notable amount of time and does not need to maintain the connection, you might want to reconsider keeping the connection open and let the pool handle it.
But if it is constantly reading from/to the connection, you should not artificially disconnect/reconnect.
There is no reason to keep closing and re-opening the connection. It imposes needless load on the database and the network.
I have a GUI where different parts of the information shown is extracted from a database. In order for the GUI not to freeze up I've tried putting the database queries in BackgroundWorkers. Because these access the database asynchronously I get an exception telling me the database connection is already open and used by another.
Is it possible to create a queue for database access?
I've looked into Task and ContinueWith, but since i code against .Net framework 3.5 this is not an option.
What is the DB engine you're using? Most modern databases are optimized for concurrent operations, so there's no need to queue anything.
The thing you're appaently doing wrong is reusing the same IDbConnection instance across different threads. Thats a no-no: each thread has to have its own instance.
I think your problem is in the way you get a connection to the database. If you want to fire separate queries you could use separate connections for separate requests. If you enable connection pooling this does not add a lot of overhead.
Try to use the pool objects. Plus as per your description your trying to open a connection on an unclosed connection object.
If connecting to a database consumes a lot of resources, why should a database connection always be closed in your application if you have to open it again? Can I just make this connection available globally throughout my application so that other classes and methods can reuse it?
For example (in pseudo code):
public class PopulateGridViews()
{
public SqlConnection conn = new SqlConnection(#"Database:DATABASE");
conn.Open();
void PopulateGrid1()
{
SqlCommand cmd = new SqlCommand("SELECT * FROM TABLE1");
cmd.Connection = conn;
cmd.ExecuteNonQuery();
cmd.Dispose();
// Populate Grid1
}
void PopulateGrid2()
{
SqlCommand cmd = new SqlCommand("SELECT * FROM TABLE2");
cmd.Connection = conn;
cmd.ExecuteNonQuery();
cmd.Dispose();
// Populate Grid2
}
}
You should not leave connections open.
You should:
Open connections as late as possible
Close connections as soon as possible
The connection itself is returned to the connection pool. Connections are a limited and relatively expensive resource. Any new connection you establish that has exactly the same connection string will be able to reuse the connection from the pool.
We strongly recommend that you always
close the connection when you are
finished using it so that the
connection will be returned to the
pool. You can do this using either the
Close or Dispose methods of the
Connection object, or by opening all
connections inside a using statement
in C#, or a Using statement in Visual
Basic. Connections that are not
explicitly closed might not be added
or returned to the pool. For more
information, see using Statement (C#
Reference) or How to: Dispose of a
System Resource for Visual Basic. Ref.
You should appropriately wrap anything that implements IDisposable in a using statement block:
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
...
command.ExecuteNonQuery();
}
Because (some) databases also hold open a connection until told by the calling app to close it. If you get hundreds of calls to a database then it's sitting there with 100 open connections tying up resources. It's not uncommon to have thousands or hundreds of thousands of calls to a database in a busy app, and sooner or later the DB performance will kill the app's performance.
It's really just common sense. If you have a valid reason to keep it open, then do so. If not, close it as soon as you're done with it. But it's better to be in the good habit of closing the connections so that you don't just leave them open when you don't intend to. It's a good habit like wearingyour seat belt or closing the refrigerator door when you're not getting food out.
This article states it well (even if it is a bit outdated):
http://www.bewebmaster.com/84.php
A common problem among hosting
companies is that ASP websites do not
close the database connections after
they are opened. This is a basic step
that you should consider to be part of
mandatory code. If you do not close
your database connections, many
problems can occur like web pages
hanging, slow page loads, and more.
Think of it as going through a door to
your house. Maybe the door will shut
by itself, but maybe it won't. If it
doesn't shut, who knows what will
happen. If you live in the country, a
bear could walk in. If you live in the
city, a mugger could walk in. Alright,
well maybe leaving a database
connection open won't lead to anything
that bad, but it will lead to a lot of
unnecessary headaches for both you and
your hosting company.
It is clear if you donot close the connection than it would consume your resource continuously and which would have overall impact on your apllication and it may also not be added or returned to the pool.
What would be a good way to writein C# a .NET3.5 Thread Safe DataSource Class. The class would connect to SQL Server and every method would execute a Stored Procedure.
When the code worked single threaded. I had a Singleton DataSource Class with a private SqlConnection member. Each method opened and closed that connection. When running this with multiple threads it is causing trouble when under certain scenarios the connection has already been opened by another thread. What would be the best way to re-write this class?
Note: By DataSource dont mean any built in .NET class, but a Model class that provides data to the controller.
Why don't you just use built-in ado.net pooling: create and open the connection right before the operation and dispose is asap - for every method.
The problem seems to be from the Singleton design. That can still work, just make sure you do not store the connection as a field. Only use a local variable in each method to hold the connection. That should be thread-safe by design.
And make it Exception-safe as well:
using (var conn = new SqlConnection(...))
{
// call SP
}
SQL Server already coordinates threads through very complex mechanisms. You don't need to do anything specific to achieve thread safety to simply execute a Store Procedure.
You'd have to elaborate more about what your DataSource class should do. You don't need to implement any thread safety code if each of your Create/Read/Update/Delete methods do not alter any state.
UPDATE: In your case, I recommend to just recreate a new SqlConnection instance in every method of your DataSource class, because ADO.NET already handles the pooling for you.