I have a GUI where different parts of the information shown is extracted from a database. In order for the GUI not to freeze up I've tried putting the database queries in BackgroundWorkers. Because these access the database asynchronously I get an exception telling me the database connection is already open and used by another.
Is it possible to create a queue for database access?
I've looked into Task and ContinueWith, but since i code against .Net framework 3.5 this is not an option.
What is the DB engine you're using? Most modern databases are optimized for concurrent operations, so there's no need to queue anything.
The thing you're appaently doing wrong is reusing the same IDbConnection instance across different threads. Thats a no-no: each thread has to have its own instance.
I think your problem is in the way you get a connection to the database. If you want to fire separate queries you could use separate connections for separate requests. If you enable connection pooling this does not add a lot of overhead.
Try to use the pool objects. Plus as per your description your trying to open a connection on an unclosed connection object.
Related
we are building a WinForms desktop application which talks to an SQL Server through NHibernate. After extensive research we settled on the Session / Form strategy using Ninject to inject a new ISession into each Form (or the backing controller to be precise). So far it is working decently.
Unfortunately the main Form holds a lot of data (mostly read-only) which gets stale after some time. To prevent this we implemented a background service (really just a seperate class) which polls the DB for changes and issues an event which lets the main form selectively update the changed rows.
This background service also gets a separate session to minimize interference with the other forms. Our understanding was that it is possible to open a transaction per session in parallel as long as they are not nested.
Sadly this doesn't seem to be the case and we either get an ObjectDisposedException in one of the forms or the service (because the service session used an existing transaction from on of the forms and committed it, which fails the commit in the form or the other way round) or we get an InvalidOperationException stating that "Parallel transactions are not supported by SQL Server".
Is there really no way to open more than one transaction in parallel (across separate sessions)?
And alternatively is there a better way to update stale data in a long running form?
Thanks in advance!
I'm pretty sure you have messed something up, and are sharing either session or connection instances in ways you did not intend.
It can depend a bit on which sort of transactions you use:
If you use only NHibernate transactions (session.BeginTransaction()), each session acts independently. Unless you do something special to insert your own underlying database connections (and made an error there), each session will have their own connection and transaction.
If you use TransactionScope from System.Transactions in addition to the NHibernate transactions, you need to be careful about thread handling and the TransactionScopeOption. Otherwise different parts of your code may unexpectedly share the same transaction if a single thread runs through both parts and you haven't used TransactionScopeOption.RequiresNew.
Perhaps you are not properly disposing your transactions (and sessions)?
I am currently writing WebCrawler, which operates over 8 threads, each thread gets page, scrapes for links and then check if the links have already been captured. If they are new links then they are stored.
This all works, but I’ve since encountered memory problems, so I started migrating the crawler over to store the data in MySQL database.
The problem I’m having, is how I get each thread to independently interact with the database, checking for data and inserting data if required.
It currently works with one thread, but as soon as I scale the thread pool, I get connection is already open errors.
Each thread has its own connection object, created on the thread for connecting to the database. Am I ignorantly concluding that these connections can be separate?
Apologies, it turns out that there was actually an error in my code and I was opening the connection twice in same thread.
For reference if other people have similar problems. It is possible to connect multiple times to the same database across multiple threads from C#. As long as each connection object is thread safe and independent of the others.
I don't know if its safe or possible to share one connections over multiple threads. But as I'm up and running Its not a question or problem I need to test.
Thanks
I am writing c# wrapper class for MySQL and I need to know if I can put a "new MySqlConnection(connstr)" into the constructor and then use the same object between the methods that are going to manipulate the database? Should I create a new connection for every operation (select, insert, delete, update) and destroy it when done. Any ideas or better approaches to write a MySQL wrapper?
I'd recommend not sharing that connection.
It won't scale as well as getting that connection out from a pool when it's needed, performing the SQL operation, closing resources, and then returning the connection to the pool.
I consider it a best practice to restrict SQL objects to the data access layer and using them in the narrowest scope possible. Don't pass them around - the responsibility for cleaning them up isn't clear if you do.
Connections ought to be owned by the service that knows about use cases, units of work, and transactions. Let it check out the connection, make it available to all DAOs that need it, then commit or rollback and close the connection.
DAOs should be given database connections. They should not acquire them on their own, because they can never know if they're part of a larger transaction or not.
For database connection the rule should be Open as Late as possible and close as early as possible,.
So open the connection before executing the query and then close it, rather than sharing a single connection object across different methods.
There is a MySqlHelper class that should do all your connection pooling, opening, closing, disposing, etc for you. "These take a connection string as an argument, and they fully support connection pooling." That is only is you are using the Oracle provided MySQL connector though.
I have a windows service which has five threads.Each thread will pic different Excel file then it reads the excel rows and inserting into data base. Is it possible to INSERT parallelly ? Currently i am using single class with lock for inserting.
If you are inserting and the key is created for you by the DBMS, then there should be no problem, and no need to lock.
This depends on your database. If you database is capable of handling multiple connections (which it should nowaways).
It has nothing to do with your class where you do the insert though. Any locking there is not really necessary (unless, of course, you database does not support multiple connections, which I seriously doubt).
Make sure it's in a transaction, and get rid of the lock! You should be fine...assuming whatever database you're using supports transactions.
Most modern databases will support multiple writes, it's safer to use the transaction incase another goes wrong.
I've got a winform with some child forms on it. Now I'm accessing the child forms by a treeview in my main form. Now in that treeview is a list of websites. Since a website is actually a group of pages, this is also shown in the tree. Website is my parent and the pages are the childs.
When you click on one of the pages the statistics are shown in a child form. When clicked on a website a summary of all pages is shown of that website. Since this can have a lot of data I'm using a background worker that fetches the statics in the background after the treeview is loaded.
Now all fine until I'm changing something in the statistics and it saves automatically into the database (sqlserver2008). Then my backgroundworker generates errors saying the sqlreader is closed. So I'm wondering how I can solve this.
Do I have to access the database on a separate connection? Any other ideas?
Thanks
Are you using the same database connection object on several threads? That's not safe. Every thread must have its own database connection, or it must synchronize the usage so that only one thread accesses the database connection object at any given moment.
It sounds like your UPDATE is closing the connection and your background thread is attempting to use it to query. If im reading between the lines correctly, you have created a connection object as a stateful member of your winform which is being opened once and possibly being closed after an UPDATE (or automatically by the connection pool). Either way you need to take a step back here and think about how your doing things.
Your update and select methods (hopefully in separate classes to your forms) should take care of opening and closing connections exclusively rather than in a statefull way (i.e. the scope of the connection remains in the methods). The ADO connection pool will take care of optimising connections for you.
I also recommend you use disconnected data structures such as DataTable and DataSet rather than DataReader as these keep the connection footprint between the client and the database light. You can happily process and pass around these structures without maintaining an open connection which DataReader requires. Ideally you would populate a light-weight object structure rather than passing DataSet around too much as it does carry some memory overhead - there are a bunch of other reasons for this too e.g. type safety.
There is nothing wrong with calling a method which executes queries on multiple threads as long as the scope of the connection is limited to the method. For example:
public static DataSet GetWebStatsData()
{
DataSet StatsData = new DataSet();
using (OleDbConnection conn = new OleDbConnection(ConnString))
{
using (OleDbCommand cm = new OleDbCommand(GetDataProcedure))
{
using (OleDbDataAdapter adap = new OleDbDataAdapter(cm))
{
cm.Connection = conn;
conn.Open();
adap.Fill(StatsData);
conn.Close();
}
}
}
return StatsData;
}
You can happily call the above method from many threads simultaneously and it should work. I have worked on applications making heavy use of the threadpool executing in some cases scores of queries simultaneously with no ill effect. Where this is happening i would recommend setting a maximum size to the ado.net connection pool as it can grow quite large if left un-guarded (there is typically a connection string attribute for this like 'Max Pool Size=n'). Ado.net will marshal the connections on your behalf so there is a lot of plumbing to leverage in the framework here.
I would guess that the navigation event is firing off a new database request using an existing sqlcommand. Can you post up a code snip?
If you're using the same connection string you can open multiple SqlConenction/SqlCommand/SqlDataReader objects and they'll come from the connection pool. There is no reason you can't have multiple connections to the same database! You can read some more information at SQL Server Connection Pooling (ADO.NET).
If you're trying to use the same SqlConnection on multiple threads at the same time, that will definitely produce unpredictable results. Creating a connection for each thread is easiest, or you could use some form of synchronization if you really want to share a single connection. That would only make sense if your worker threads are doing a significant amount of work besides database access though.