We have a 3-tier application with a C# client, a C# WCF web service layer, and a SQL Server database. The web service connects to the database with ADO.NET. All of our C# code is using the .NET Framework 2.0.
Recently, a customer performed a stress test on our application. During the test, the web server generated a lot of errors like the following:
Could not connect to database for connection string '...'. Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
I am aware there are ways to catch connection errors when the connection pool is full and try acquiring a connection outside of the connection pool. We also found several queries that needed to be tuned, but they dot not explain web server connection timeouts.
We're trying to figure out why the web server was timing out connecting to the database. I have setup the web server to enable all of the ADO.NET performance counters. However, I don't see anything related to the times required to connect or connection timeouts or the like. Ideally, we would be able to graph the connection times in perfmon next to the other ADO.NET counters.
Is there a way to monitor the ADO.NET performance in acquiring connections?
I imagine we could create our own "average connection open time" performance counter by timing attempts to open connections, but I'd rather use something that already exists.
You can use perfmon to get this information. You'll need to attach to the User Connections monitor under SQL Server: General Statistics. Here is the blog post I grabbed that from. This will let you know at what point in time connections are being left open.
You will then need to correlate that with application tasks (i.e. stuff you're doing in the application when you see them continually climb).
Once you've done that you're going to want to get those connections inside a using statement if you don't already:
using (SqlConnection cn = new SqlConnection("some connection string"))
{
cn.Open();
...
}
by doing this you won't need to issue a Close and you won't have to worry about them getting disposed of properly.
In short, the performance counter should help you track down the code in the application that's causing the issue, but it won't be down to the line per say, it will take a bit more effort even from there.
Related
So this issue is NOT one of having too many connection strings or connection strings not being disposed of properly. ASP.NET Sql Server stored session has been working great for months with decent traffic on the website, now I am getting many Unable to connect to SQL Server session database - The wait operation timed out exceptions. There is plenty of memory available for the database, the database has plenty of memory allocated. There is just a few users on the website when it is throwing errors.
I have two blade servers sitting right next to each other and very low traffic to the website so I shouldn't need to increase the connection timeout from the default 15 seconds.
I am using Windows Authentication and I can connect via SSMS and query my ASPSession database that is being used as the persistence layer for ASP.SESSION no problem.
I just have no were to go on this error currently.
Looks like the issue an I/O error with the raid on my database server. Had to swap out a hard drive and all seems well.
I had the same issue, and I didn't know why, because I setted the timeout for 300 seconds(5minutes) at app.config, elsewhise I didn't notice when I was using sqlCommand the timeout of connection was 300 seconds, but for the sqlCommand was 30 seconds, so I setted the sqlCommand.ConnectionTimeout = 300 (was 30) and solved my problem.
Also I read in somewhere about using WITH NOLOCK, that solves for someones, this will speed up your sql, but I suggest to read about at Microsoft because this is not something you should use with you don't understand the concept.
Background info: I'm coding with C#, using Microsoft SQL Server for databases.
I didn't find much on Google on the subject, so I'm asking here: should I always close a connection to my database after performing a query?
I'm torn between two solutions (maybe better ones exist...):
either open the connection before querying, then close it right after the SQL query
or open the connection at the start of my application, and before each SQL query check if the connection is still up and reopen it if needed.
In the past, I used the first solution but I discovered that opening a new connection can take quite some time (especially over a VPN connection to my LAN opened through 3G), and that it would slow down my application. That's why I decided to go with the second solution (in that case, my connection should be always up if we forget about time-out) and noticed some better performances.
Do I need to close the connection at the end of my application or can I forget about it?
Yes, you should close your connection after each SQL query. The database connection pool will handle the physical network connection, and keep it open for you. You say that you found that opening a connection can take some time - did you find that the application was really doing that multiple times?
(I hope your real application won't be talking directly to the database over 3G, btw... presumably this is just for development purposes...)
One important thing to remember is that there is a unique connection pool for each unique connection string you use... so always use the same connection string unless you need to connect to a different database (or have unique requirements).
Here is a good document on connection pooling with System.Data.SqlClient.SqlConnection.
This will heavily depend on how many clients you anticipate will need to connect to the database. Leaving the connection open, could prevent another user from accessing the DB while they wait for an open connection.
I have written a Windows Service which listens for data from a third party service, holds it in memory for a short time and periodically all the new data is flushed to the database.
I was initially opening a new connection each time I needed to flush the data and closing it again afterwards. (Every 5 seconds or so)
As the server seems to be getting hammered I have changed that so there is a single connection opened and reused for the life of the application.
Just wondering if this is a bad idea?
I usually do web stuff where the connection is open and closed over the life of a single request. What is the best practice for a windows service that needs to do the sort of operation I have described?
I was going to make a fault tolerant connection like this:
private SqlConnection _sqlConnection;
public SqlConnection SqlConnection
{
get
{
if (_sqlConnection == null || !_sqlConnection.State.Equals(ConnectionState.Open))
{
var conn = new SqlConnection(_connectionString);
conn.Open();
return conn;
}
return _sqlConnection;
}
}
so if some reason the existing connection is closed or faulted in some way we would get a new open one
is that bad design for any reason?
If you are the single user of the database, hold onto the connection. If not you can really rely on connection pooling to do that for you.
I personally would go for opening the connection everytime. In .NET 2.0 a new feature was implemented so that if you have an open connection to a sql server and sql server gets restarted, etc... your connection becomes invalid and that is not something I can risk my service with. See my post from some years ago.
Call me conservative but I still think that leaving it up to the connection pool to manage the physical connections to the database is a better choice. So just open and close the connection normally, and leave to the pool to decide what to do. I've done that in web services without any problems, and you will have more connections available to handle the load.
I would not try to maintain an open connection. There will be lots of edge cases where the connection will be become unusable and your code for managing the connection and making sure the old duff connection is correctly disposed would have to be absolutely bullet-proof.
I recommend the more common connection use pattern of open, use, close/dispose. The code will be much easier to write and maintain. Be absolutely sure you are disposing of all command and connection objects once you're done with them. Monitor your app with a profiling tool, and keep a check on the number of open database connections at the server to make sure your code is working the way you intended.
How often you need to dump the data into the database (and therefore open/use/close database connections) depends on a number of factors such as how much data will be in-memory before being dumped, the capability of the database server to consume the data, and the risk of losing data if you've accepted it from the web service, but haven't written it to the database and your service or the server crashes.
If your data is precious, you might want to consider having two processes. One process calls the web service and stores the received data securely in a message queue. Another process reads the messages from the queue and puts the data in the message in the database.
This way of handling this process means you can receive data whilst the database is temporarily down, and all the data will eventually be stored in the database.
Whilst this is a solid solution, it could just as easily be considered overkill, depending on your requirements.
I have a website using Microsoft SQL 2008 server over local network. Sometimes, SQL server machine is rebooted, and so the website fails to connect to the database. If the machine is up and running, it will respond fast. If it's down, there is no need to wait for 15 seconds. 3 seconds are ok.
I want to display apologizes on the website when the database is not reachable, and want to do it fast. But setting Connection Timeout=3 in connection string seems having no effect. The page spends 22 seconds to wait before throwing SqlException on SqlConnection.Open();.
What's wrong with it? May it be a hidden configuration which overrides the timeout?
Currently, my connection string is
Data Source=...;
Initial Catalog=...;
Integrated Security=True;
Connection Timeout=3
If I set it to ...;ConnectionTimeout=3 (without space),
System.ArgumentException: Keyword not supported: 'connectiontimeout'.
is thrown (strange, MSDN documentation indicates that we can use both strings).
There is a timeout before the networking hardware reports connection timeout to the network drivers, which in turn notifies the programs waiting for network IO. You can verify transport layer timeouts via telnet servername 1433 (assuming your sql server is listening on port 1433).
But 3 seconds is way too short for a process to initialize the network APIs (assuming your web app is in its own application pool), send request and wait for the hardware to timeout. Updating BIOS/firmware/driver probably won't reduce the response time that much.
It would be better to carry out the connection asynchronously. i do not suggest using EndInvoke to end the asynchronous call as unlucky users may still need to wait 3 full seconds to see any response when the database is down. Maybe an Ajax call is better. If you have a lot of users constantly hitting your web site, you may want to cache the result of connectivity checking and update it in a manner meaningful to your users.
The following blog post assisted me in solving this problem:
http://improve.dk/controlling-sqlconnection-timeouts/
ConnectionTimeout without space is the property name when accessing via code, not for the connection string.
Not sure if this is of use, but when I hit this issue in the past, it was because I also needed to set SqlCommand.CommandTimeout. What happened for me is the connection was opened successfully, then DB server went down, then my next command did not timeout as quickly as I expected based on the Connection Timeout, and this was due to the CommandTimeout needing to be set as well.
When a connection to DB is opened using C# OPEN statement, does that impact the web server performance or only the Database?
So, how does the repeated opening and closing of the database connection impact the web server and the database.
Can somebody please give me some insight on this. Thanks.
Opening a database connection is a relatively expensive operation. Opening database connections can be so expensive that ADO.NET by default enables connection pooling. If you are not using connection pooling then your application is probably going to run slower (decreased response time) and could even hit problems with scalability.
If you are using connection pooling then repeated opening and closing of a SqlConnection does not incur the large overhead of creating a network connection, authenticating with SQL Server, setting any connection specific data (etc.) that occurs without pooling (except on the initial physical connection creation). When Open is called, an existing connection is retrieved from the pool (if available) and when Close is called then connection is returned to the pool.
With connection pooling enabled, I would expect to see a memory increase on both the web and database servers in maintaining the open connections. If you aren't using connection pooling, then you could do some tests to measure what the performance impact is to both servers.
Usually this isn't something you need to worry about — use connection pooling and tune the pool parameters if necessary.
It impacts end-to-end response time, because of stuff going on in both the database and the web server. In short, your web pages will all load more slowly, even under light load.
Throughput-wise, it probably hurts the database more, since it's doing all the authentication work, but that's just a wild guess.