I am writing an application in C# that will connect to a PostgreSQL database via ODBC. The database will be on the same machine as my application. My application will have an event handler that could fire a few times a second. The handler will write two values into a database table. I'm wondering if I should open a database connection once when the application starts for the event handler to use, or if I should open it inside the event handler and close it when the handler is finished with its work.
If connection pooling is used, the question becomes moot, since an existing connection would always be used if one is available, and I could just open and close the connection without worrying about slowing down my application. But it seems that the ODBC driver for PostgreSQL does not support connection pooling. Or does it?
I'm using the PostgreSQL Unicode driver version 9.03.04.00.
Thank you very much.
Never open and close a database connection for every request. It may work as long as there is little traffic, but it is a receipe for disaster otherwise.
You can use an external connection pool like pgBadger, but if that is overkill in your case, just leave the connection open. With several writes per second that is clearly the right thing to do.
Related
I have a need where I will need to check a specific table on a remote SQL Server for any data changes.
And this I will need to do as soon as it is possible e.g almost every second so that as soon as a change happens, I can do something about it on another server where I am running this watcher program.
I think opening a connection every second to remote may be expansive, and keeping a connection open for a very long time is considered a bad practice. What is the best option for me?
Server 1 has a watcher program, which will monitor data change activity on
Server 2 (remote server). I must know about data change as soon as it happens on Server 2 and then perform some actions on Server 1
Thanks
I think opening a connection every second to remote may be expensive,
It's not. Read about Connection Pooling. The network connection and SQL Server session are not closed when you call SqlConnection.Close() or SqlConnection.Dispose(). The underlying connection is simply returned to the pool for later use.
But if the connection is lost or the server fails, you will get a new connection from the pool, so it's vastly more reliable than keeping a single connection open.
I'm developing an application that in the event of losing connectivity restarts the phone.
Before this step we'd like to close any open connections through Connection Manager. I understand we can use ConnMgrReleaseConnection to release a connection but is it possible to get the Connection handle if the connection was established through another application?
If connection manager fails to close the connection we can then perform a RASHangup but we'd like to attempt it through ConnectionManager first.
Nope, there's no way to do it through connection amanger. The handle ConnMgrReleaseConnection wants is the one returned from the call to ConnMgrEstablishConnection. Presumably the "other" application called this and has the handle, but even if that app could give you the handle, it would be invalid in your own process space anyway.
Generally it would be bad form to do something like this anyway, as I'd assume that the app that openened the connection would expect it to always be open once it had asked for it. Forcibly closing it (even through RAS) without that app knowing could lead to unexpected behavior. Probably not a huge issue for you if you're just going to restart the phone, but if you have any sort of control over that other app, I'd add handling where you can tell it to close it's connections.
I am receiving data through a com port continuously and doing some decoding. When decoding is done i have to store the results in a sql database. I am thinking since the decoding is done (in a while loop always running) dozens of times per second and data need to be stored to the database dozen of times every second if it is wise to open and close the connection to the sql server in each while loop or just leave it open and continue to write data to the database.
First of all is this possible? Secondly if the connection remains open can third party applications or computer access the database at the same time and read data as my programm stores data?
A database supports more than one concurrent connection, so yes it is very feasible in this scenario to leave the DB connection open - you will only lock out others if you have i.e. a long running query that results in row/ table locking. Just close the connection when you are done.
Also consider though that most DBs (i.e. SQL Server) use connection pooling internally, so even though you close a DB connection it just goes back to the pool and is not physically closed - the pool manages the physical DB connections - this results in much better performance, so the impact of opening/closing connections rapidly is reduced.
From MSDN:
Connection pooling reduces the number of times that new connections
must be opened. The pooler maintains ownership of the physical
connection. It manages connections by keeping alive a set of active
connections for each given connection configuration. Whenever a user
calls Open on a connection, the pooler looks for an available
connection in the pool. If a pooled connection is available, it
returns it to the caller instead of opening a new connection. When the
application calls Close on the connection, the pooler returns it to
the pooled set of active connections instead of closing it. Once the
connection is returned to the pool, it is ready to be reused on the
next Open call.
I'd open the connection, go through the loop as many times as needed, then close the connection. Opening and closing is very expensive. But, thanks to Mitch Wheat and Dave Markle, I've learned that connection pooling is done for free in the background with .NET, so the expense should be amortized over all your requests.
Even better, I'd batch the requests in the loop and execute the batch after the loop was done. You only require one network round trip that way.
If you agree with that last bit, I'd make sure that the batch INSERT was done in a transaction context so it could be committed or rolled back as a single unit of work. Isolation and thread safety will matter.
I personally would open and close the connection each time. You should be protecting your code in a way that any exception will ultimately close your connection. Will your app be the only application talking to this database?
Actually you must explicitly close SqlConnection.Open()
From MSDN, "Note If the SqlConnection goes out of scope, it is not closed. Therefore, you must explicitly close the connection by calling Close or Dispose."
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.open(v=vs.71).aspx
Cheers!
Background info: I'm coding with C#, using Microsoft SQL Server for databases.
I didn't find much on Google on the subject, so I'm asking here: should I always close a connection to my database after performing a query?
I'm torn between two solutions (maybe better ones exist...):
either open the connection before querying, then close it right after the SQL query
or open the connection at the start of my application, and before each SQL query check if the connection is still up and reopen it if needed.
In the past, I used the first solution but I discovered that opening a new connection can take quite some time (especially over a VPN connection to my LAN opened through 3G), and that it would slow down my application. That's why I decided to go with the second solution (in that case, my connection should be always up if we forget about time-out) and noticed some better performances.
Do I need to close the connection at the end of my application or can I forget about it?
Yes, you should close your connection after each SQL query. The database connection pool will handle the physical network connection, and keep it open for you. You say that you found that opening a connection can take some time - did you find that the application was really doing that multiple times?
(I hope your real application won't be talking directly to the database over 3G, btw... presumably this is just for development purposes...)
One important thing to remember is that there is a unique connection pool for each unique connection string you use... so always use the same connection string unless you need to connect to a different database (or have unique requirements).
Here is a good document on connection pooling with System.Data.SqlClient.SqlConnection.
This will heavily depend on how many clients you anticipate will need to connect to the database. Leaving the connection open, could prevent another user from accessing the DB while they wait for an open connection.
I have written a Windows Service which listens for data from a third party service, holds it in memory for a short time and periodically all the new data is flushed to the database.
I was initially opening a new connection each time I needed to flush the data and closing it again afterwards. (Every 5 seconds or so)
As the server seems to be getting hammered I have changed that so there is a single connection opened and reused for the life of the application.
Just wondering if this is a bad idea?
I usually do web stuff where the connection is open and closed over the life of a single request. What is the best practice for a windows service that needs to do the sort of operation I have described?
I was going to make a fault tolerant connection like this:
private SqlConnection _sqlConnection;
public SqlConnection SqlConnection
{
get
{
if (_sqlConnection == null || !_sqlConnection.State.Equals(ConnectionState.Open))
{
var conn = new SqlConnection(_connectionString);
conn.Open();
return conn;
}
return _sqlConnection;
}
}
so if some reason the existing connection is closed or faulted in some way we would get a new open one
is that bad design for any reason?
If you are the single user of the database, hold onto the connection. If not you can really rely on connection pooling to do that for you.
I personally would go for opening the connection everytime. In .NET 2.0 a new feature was implemented so that if you have an open connection to a sql server and sql server gets restarted, etc... your connection becomes invalid and that is not something I can risk my service with. See my post from some years ago.
Call me conservative but I still think that leaving it up to the connection pool to manage the physical connections to the database is a better choice. So just open and close the connection normally, and leave to the pool to decide what to do. I've done that in web services without any problems, and you will have more connections available to handle the load.
I would not try to maintain an open connection. There will be lots of edge cases where the connection will be become unusable and your code for managing the connection and making sure the old duff connection is correctly disposed would have to be absolutely bullet-proof.
I recommend the more common connection use pattern of open, use, close/dispose. The code will be much easier to write and maintain. Be absolutely sure you are disposing of all command and connection objects once you're done with them. Monitor your app with a profiling tool, and keep a check on the number of open database connections at the server to make sure your code is working the way you intended.
How often you need to dump the data into the database (and therefore open/use/close database connections) depends on a number of factors such as how much data will be in-memory before being dumped, the capability of the database server to consume the data, and the risk of losing data if you've accepted it from the web service, but haven't written it to the database and your service or the server crashes.
If your data is precious, you might want to consider having two processes. One process calls the web service and stores the received data securely in a message queue. Another process reads the messages from the queue and puts the data in the message in the database.
This way of handling this process means you can receive data whilst the database is temporarily down, and all the data will eventually be stored in the database.
Whilst this is a solid solution, it could just as easily be considered overkill, depending on your requirements.