I am working in a small team of about a dozen developers on a project being written in C# WPF as the infrastructure/dba. As I run traces against the SQL Server to see how performance is going, what I am seeing is a constant:
open connection
run some statement
close connection
exec sp_reset_connection
open connection
run some statement
close connection
exec sp_reset_connection
open connection
run some statement
close connection
exec sp_reset_connection
And so on and on and on. I have spoke with the devs about this and some have mentioned possible situations where a foreach loop may be encompassing a using statement so a foreach through a datatable would open and close connections throughout the contents of the datatable.
Question: Is getting better control of the constant opening and closing of connections a worthy goal or are connections really that cheap? My logic is that while opening and closing a connection may relatively be cheap, nothing is cheap when done in sufficiently large numbers.
Details:
.Net Framework 4.5.1
SQL Server 2014
Entity Framework 6
If you use entity framework, you should create the context just before you need it and dispose it as soon as possible:
using (var someContext = new SomeContext())
{
}
The reason is to avoid memory building up and to avoid thread-safety issues.
Of course, don't do this in a loop - this is at the level of a request.
Opening and closing connections to a database are relatively expensive, as can be read in detail here: Performance Considerations (Entity Framework), but I think the same concept mostly applies without EF. During loops, it's usually not recommended to open and close the connection every time, but instead open it, process all rows and close the connection.
The answer would be to let the using encompass the loop, instead of the other way around. If performance is relevant (it almost always is), it definately pays to put effort into efficiënt data access, especially early in the development process.
If performance is an issue, but you don't want to refactor code you should consider setting a ConnectionPooling = true in the connections string.
Connection pooling allows one to keep physical connection, which is generally expensive to setup, while disposing logical connection.
Related
i have a question about closing connection in C#. Company has an application where data flows automatically online from app to DB. I would like to create my own ASP + C# application which will use select from data (DB table which is filled from company app) as source for independent report. My question: can closing of the connection in my app have influence on the second(company, very important app?) - record will miss in db due to close connection? or any other problems?
No, everything will be safe if you close it properly. I recommend you to use using construction always. It will be transformed into try-catch-finally and close resources automatically.
That totally depends on your use-case, if you open and leave open hundreds and hundreds if not thousands and thousands of empty connections, the SQL Server will slowly begin to have some performance degradation.
Think of it as you asking your boss for something, and you say, "Boss-man, I need to ask you a question." But you remind him hundreds and thousands of times a second, "I need to ask you a question." Anything else he tries to do will slowly begin to lose performance, as he has to process the fact that you are going to have to ask him a question. Similarly with Sql Server. Mind you, at this point you haven't actually asked your question yet.
If your DBMS is Microsoft SQL Server, see this article: https://msdn.microsoft.com/en-us/library/ms187030.aspx
SQL Server allows a maximum of 32,767 user connections.
If you open 32k connections to the server, two things will likely happen:
Your DBA will come to you and say "wtf mate?" by the time you get close. A likely argument will ensue in which case you and the DBA will probably end up yelling and creating a scene.
Your DBMS will reach the maximum connection limit and the other all will crap out.
Not saying that any of this will happen, that requires you to open 32,767 concurrent connections, but it just goes to further prove that you should open/close as required. Also, if your Application uses a pool of connections and you open n connections, and the pool limit (separate from SQL Server - mind you) is n, you just stopped your app from opening more.
Generally speaking, you should open your connections as late as possible, and close them as early as possible.
I'm investigating some performance issues in our product. We have several threads (order of tens) that are each doing some form of polling every couple of seconds, and every time we do that we are opening a connection to the DB, querying, then disposing.
So I'm wondering, would it be more efficient for each thread to keep an open connection to the DB so we don't have to go through the overhead of opening a connection every 2 seconds? Or is it better to stick with the textbook using blocks?
First thing to learn about is Connection pooling. You're already using it, don't change your code.
The question becomes: how many connections to claim in my config file?
And that's easy to change and measure.
As mentioned, connection pooling should take care of it but if you are beating on the database with messaging or something like that to check on the status of things every few seconds then you could be filling up the database pool very quickly. If you are on SQL Server, do an SP_WHO2 on the database in a query window and you'll see a lot of information: number of spids (connections open), blocking, etc.
In general, connection setup and teardown is expensive; doing this multiple times in tens of threads might be crippling; note however that the solution you use might already be pooling connections for you (even if you're not aware of it), so this may not be necessary (see your manuals and configuration).
On the other hand, if you decide to implement some sort of connection pooling by yourself, check that your DB server could handle tens of extra connections (it probably should).
I work with Windows-Mobile and Windows-CE using SqlCE and I dont know what better to do.
To open connection when the program open, run any query's... update...delete database and close the connection after the program close?
Or open connection run any query's..update...delete database and close the connection immediately?
Nice. The answers are all over the place. Here's what I know from experience and interacting with the SQL Compact team:
Closing the connection flushes the changes you've made, otherwise the engine waits for the flush period before doing it. It's a good idea to close the connection when you're done using it to ensure that your changes actually go to the store. A power loss after a write and before a flush will lose data.
There is no official connection pool, but opening the first connection is expensive (i.e. slow), all others are quick. The recommendation I got from the team is to actually create a connection when the app starts up and just leave it open. You don't actually need to use it, but keeping it open keeps a lot of connection info cached so that subsequent connections to the same store are quick.
So the answer, actually, is both.
Edit
For those interested, a good example of how this works can be seen in the OpenNETCF ORM library. The library, by default, creates a "maintenance" connection that remains open and is used for doing things like schema queries. All other data operations use their own connection. You also have to option to configure the library to reuse a single connection for the life of the Store, or to use a new connection every time it touches the store. Perfomance and behavior has always been best in all of my projects using the default (which is why I made it the default).
Always keep a connection open for the lifetime of your Windows Mobile app. Opening a SQL Server Compact database is a costly operation.
You should close your connection each time you have completed an sql transaction to free connection ports. Always a good practice to avoid security breach.
Connection establishment is a slow operation, so, creating and closing it can slow down the application. On the opposite hand, if you have a lot of clients, the connection pool will be filled very quickly and other clients won't be able to connect.
There are already some conflicting answers here.
To be honest, I'm not enirely sure how WinCE deals with connections. I don't think there is a ConnectionPool.
But the general pattern in .NET is to keep connections open as short as possible. This improves reliability and prevents resource leaks. Make sure you know about the using (var conn = ...) { ... } pattern.
So I would say: go with your second option, and only keep connections longer if you really experience a performance problem, and if opening the connection is the cause. I don't think it will be with SqlCE
On a single-user platform such as wince, there's no harm in keeping the connection open, and you may get better performance.
If worry about data lost because you are not calling Close() frequently, you can execute your code within a transaction that commits changes to disk immediately:
using (SqlCeTransaction transaction = this.connection.BeginTransaction())
{
using (SqlCeCommand command = new SqlCeCommand(query, connection))
{
command.Transaction = transaction;
command.ExecuteNonQuery();
}
transaction.Commit(CommitMode.Immediate);
}
Of course, there is still some performance lost when using CommitMode.Immediate too frequently.
I was wondering if it is a good idea to maintain a database connection ( System.Data.SqlClient.SqlConnection() ) open or is it recommended to close the connection after using it and open it again when needed ? ( Note : My app will run continuously for days/months ) . I am kind of pushed towards leaving it open. Which solution is better ?
In general, dispose of it when you're done with it, and don't worry about it.
ADO.NET implements connection pooling by default, so the connection is kept open behind the scenes so as to spare you the performance penalty of opening new connections all of the time.
Another reason to close the connections in your code -- if you lose connectivity to your database server when you're not using the connection, you won't encounter an error, which could happen if you keep the connection open.
You absolutely should open your connections as late as possible, and close them as soon as possible.
Not only should you close them when you are done, though: you should close them even between closely-related commands, if there is any other code running in between at all. In fact, let me put it like this: Close your ADO.NET Connection objects as quickly and frequently as you practically can. (that is, don't do obviously stupid things to close connections that obviously should not be closed)
By default, the ADO.NET provider for SQL Server (as well as most of the other prominent providers) provide connection pooling. This will actually manage the creation and destruction of these connections for you - but only if you close them when you are done.
If you wrap the creation and use of your connection objects in using blocks, this is easy to do...
using(SqlConnection conn = new SqlConnection(...))
{
///Open and use the connection
} //the 'using' causes it to automatically be closed.
Open the connection if you needed, don't waste resources ;-)
Is it smart to keep the connection open throughout the entire session?
I made a C# application that connects to a MySql database, the program both reads and writes to it and the application has to be running about 10 hours a day non-stop.
Are there any risk attached to keeping the connection open instead of calling the close() function every time after you've plucked something from the database and opening it again when you need something new?
Leaving a connection open for a while is fine, as long as:
you don't have so many concurrently idle connections that you hit the MySQL connection limit;
you don't leave it open for hours without doing anything. The default MySQL connection wait_timeout is 8 hours; leave a connection inactive for that long and when you next come to use it you'll get a “MySQL server has gone away” error.
Since you're using ADO.NET, you can use ADO.NET's inbuilt connection pooling capabilities. Actually, let me refine that: you must always use ADO.NET's inbuilt connection pooling capabilities. By doing so you will get the .NET runtime to transparently manage your connections for you in the background. It will keep the connections open for a while even if you closed them and reuse them if you open a new connection. This is really fast stuff.
Make sure to mention in your connection string that you want pooled connections as it might not be the default behaviour.
You only need to create connections locally when you need them, since they're pooled in the backrgound so there's no overhead in creating a new connection:
using (var connection = SomeMethodThatCreatesAConnectionObject())
{
// do your stuff here
connection.Close(); // this is not necessary as
// Dispose() closes it anyway
// but still nice to do.
}
That's how you're supposed to do it in .NET.
Yes you can, provided:
You will reconnect if you lose the connection
You can reset the connection state if something strange happens
You will detect if the connection "goes quiet", for example if a firewall timeout occurs
Basically it requires a good deal of attention to failure cases and correct recovery; connecting and disconnecting often is a lot easier.
I think, if there is a connection pooling mechanism, you'd better close the connection.
One reason for it is that you do not need to re-check if your connection is still alive or not.
If the application is using the connection there is no reason to close it. If you don't need the connection you should close it. If you were to have multiple applications connect to the database, you have a fixed number of connections to that database. That's why it's better to close when you are done and reopen when you need it.
From a security point of view, I'd say its better to close it after a query, just to be sure that no other program can inject it's own things into the opened connection.
As performance is conered, it is clearly better to have the connection opened through the whole time.
Your choice^^
No, I don't see any reason why not to leave a connection open and re-use it: after all, this is the whole point behind the various connection-pool technologies that are about (although these are generally reserved for multi-threaded situations where works are all operating on the same data source).
But, to expand on the answer by bobince, - just beacause you are not closing the connection, don't assume that something else won't: the connection could timeout, there could be connection issues or a hundred and one other reasons why your connection dies. You need to assume that the connection may not be there, and add logic to code for this exception-case.
It is not good practise in my opinion to keep the connections open.
Another aspect that speaks for closing connections every time is scaleability. It might be fine now to leave it open but what if you app is used by twice 3-times the amount of users. It's a pain in the neck to go back and change all the code. (i know i've done it :-)
Your problem will be solved if you use connection pooling in your code. You don't need to open and close connection so you save precious resources which are used while opening a connection. You just return the connection to a pool which when requested for a connection returns back a idle connection.
Of course I am of the opinion, get an instance of the connection, use it, commit/rollback your work and return it to the pool. I would not suggest keeping the connection open for so long.
One thing I didn't see in the other answers, yet: In case you have prepared statements or temporary tables they might block server resources till the connection is closed. But on the other hand it can be useful to keep the connection around for some time instead of recreating them every few moments.
You'll pay a performance penalty if you're constantly opening and closing connections. It might be wise to use connection pooling and a short wait_timeout if you are concerned that too many running copies of your app will eat up too many database connections.