Develop Reference

Redis Timeout optimization. Need suggestions

I am getting this exception -Redis Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use.
I have set ServiceStack.Redis.PoolSizeMultiplier to 100 and ServiceStack.Redis.PoolTimeOutSeconds to 50.
But i am still getting errors. What do you suggest. To increase or decrease this timeout value? I need some explanation about this Redis.TimeOutValue. What are advantages of increasing these values?

This error message indicates that all Redis connections are currently in use and that the client timed out waiting for a free connection from the pool to be available.
This can also happen if your redis client connections are not properly disposed of after usage. Make sure all your Redis Client instances are disposed after usage, e.g. by using a using {} statement:
using (var redis = redisManager.GetClient())
{
//...
}
Another solution to avoid this error is to switch to using the RedisManagerPool which will create a new connection outside of the pool after the pool size has been exceeded, however this could be masquerading an issue that your Redis clients are not being properly disposed.

Proper way to open and close connection in loop

I have a code like below :
public void Do
{
using (var myConnection = new SqlConnection(connectionString))
{
for (int i = 0; i < 4; i++)
{
MyProcess.Execute(myConnection);
}
myConnection.Close();
myConnection.dispose();
}
}
public class MyProcess
{
public void Execute(SqlConnection myConnection)
{
if (myConnection.State == ConnectionState.Closed)
myConnection.Open();
//long running code
}
}
Execute methods sometimes take 5-10 minutes,sometimes run in 1-2 minutes for each iteration.
Now here i am confused that whether i shall open and close connection for each iteration and this will be efficient or whether i open and close connection 1 time only.
But with opening and closing connection once, this will hold resource for each of the iteration and will consume resource.
So i am not getting what should be the proper way to handle this
Can anybody please give me some advice on this?

ADO.NET uses Connection Pooling under the hood. That is why opening a new connection each time should not be an issue. Your call to myConnection.Open() will not actually result in opening of physical connection to the Database each time.
Check also this question. It's author made a measure test for connection opening. I have shown him that time of subsequent calls to DbConnection.Open() approaches to 0.
The most preferred way of using DbConnection is to open it for the minimal time period. If your MyProcess.Execute() has a lot of other non-DB related logic that takes considerable time, you should not keep an opened connection during this execution.
Check also Best Practices for Using ADO.NET article. It has following explicit statement:
High performance applications keep connections to the data source in
use for a minimal amount of time, as well as take advantage of
performance enhancing technology such as connection pooling.
So when does Ado.net removes the connection from connection pool?
This article provides some internal details on connection pooling:
The connection pooler removes a connection from the pool after it has
been idle for approximately 4-8 minutes, or if the pooler detects that
the connection with the server has been severed. Note that a severed
connection can be detected only after attempting to communicate with
the server. If a connection is found that is no longer connected to
the server, it is marked as invalid. Invalid connections are removed
from the connection pool only when they are closed or reclaimed.
This article also provides some details on pooling tuning if it's required. However you should consider such manual configuration only if you experience some performance issues caused by the pooling.

C# SqlConnections using up entire connection pool

I've written a service that occasionally has to poll a database very often. Usually I'd create a new SqlConnection with the SqlDataAdapter and fire away like this:
var table = new DataTable();
using(var connection = new SqlConnection(connectionString))
{
connection.Open();
using(var adapter = new SqlDataAdapter(selectStatement, connection))
{
adapter.Fill(table);
}
}
However in a heavy load situation (which occurs maybe once a week), the service might actually use up the entire connection pool and the service records the following exception.
System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Multiple threads in the service have to access the SQL server for various queries and I'd like as much of them to run in parallel as possible (and that obviously works too well sometimes).
I thought about several possible solutions:
I thought about increasing the connection pool size, however that might just delay the problem.
Then I thought about using a single connection for the service and keep that open for the remainder of the service running, which might a simple option, however it will keep the connection open even if there is no workload to be done and would have to handle connection resets by the server etc. which I do not know the effect of.
Lastly I thought about implementing my own kind of pool that manages the number of concurrent connections and keeps the threads on hold until there is a free slot.
What would be the recommended procedure or is there a best practice way of handling this?

Well the solution in the end was not exactly ideal (fixing the issue on the SQL Server side) so I ended up checking the number of concurrent connections in the job queuing system.
The service will now not create another thread for document generation unless it can guarantee that the connection pool is actually available. The bad bottleneck on the SQL server is still in place, however the service now no longer generates exceptions.
The downside of course is, that the queue gets longer while there is some blocking query executing on the SQL Server, which might delay document generation for a minute or two. So it isn't an ideal solution but a workable one, since the delay isn't critical as the documents aren't needed directly but stored for archival purpose.
The better solution would have been to fix it SQL Server side.

Connection pool issues on shared hosting with proper disposal

I know that there is a similar question like this on Stack Overflow, but it doesn't quite fit what I have tried. I believe the reason for this problem is different, although the end-result is the same.
Whenever my site has been running for about two or three days, I get the following error (which is obviously caused by either connections/queries hanging, or not being disposed properly).
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
At first, I did some investigation and realized that using using, using a finalize call and other solutions weren't enough. The error would still eventually occur.
So I decided to figure out wether or not the Server Explorer (which I use often) in Visual Studio leaks connections. I had already changed the maximum execution and connection timeout to 3 seconds, and had reduced the pool size to 10 connections, so the Server Explorer could be the cause. No luck!
So what do I try now?

Here's the code I solved the problem with. When connecting, catch the exception and clear all pools, and then try again.
_connection =
new SqlConnection(connectionString);
try
{
_connection.Open();
}
catch (InvalidOperationException)
{
SqlConnection.ClearAllPools();
_connection.Open();
}
Also, when disposing the connection, make sure to clear the pools.
if (_connection != null)
{
SqlConnection.ClearAllPools();
_connection.Close();
_connection.Dispose();
}

Connection.open for hangs indefinitely, no exception is thrown

When I try to do the following code, the program hangs indefinitely. I don't know why and there seems to be other unanswered topics on the matter. Although, if the IP\website cannot be reached, then it works as intended.
private void DoStuff()
{
string connectionString = "Data Source=www.google.com;Connection Timeout=5";
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open(); //Hangs here indefinitely
Console.WriteLine("Test");
}
}
For example, if I set the connection string to
connectionString = "Data Source=www.nonexistentsite.com;Connection Timeout=5";
then it will throw an exception. How do I get it to throw an exception for an active site? ... Also google is just for testing purposes, obviously.
EDIT :
If I try to connect to an unreachable server name or IP address I WILL get this exception...
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
UPDATE :
After letting the program run for quite a while, it usually times out finally after 3-5 minutes and gives me the error I posted above. How can I get it to timeout quicker?

If you have set an FQDN (Fully Qualified Domain Name) for your Data Source such as example.com and the DNS server is unable to resolve this FQDN for a long time it is pretty obvious that your request will hang out. Make sure that the machine from which you are running your application can reach the SQL server and resolve it without any issues. Also you probably want to make sure that there is no firewall that might be blocking the request.
Another possible cause for those symptoms is if you have exhausted the connection pool of ADO.NET. This could happen if you have many slow SQL queries running in parallel, each of them taking a physical connection to the database. There is a limit in the number of available connections on this pool and when this limit is reached the next call to connection.Open() might wait for an available connection to be returned to the pool.
Remark: you might also need to specify in your connection string how you want to authenticate against the SQL server. Checkout connectionstrings.com for more examples.
All this is to say that there is absolutely nothing wrong in the C# code you have posted in your question. It looks more like a network related problem that you could bring to the attention of your network administrators.

To get the connection to exit after a specified amount of time without success, you can use the Connection Timeout parameter in the connection string. The number you specify is in seconds, so for example, Connection Timeout=240 is equal to 240 seconds\60 seconds = 4 minutes.
Sample connection string:
<add name="MyConnectionString"
connectionString="
Data Source=MyServer\MSSQL2017;
Initial Catalog=MyDatabase;
Integrated Security=True;
Connection Timeout=10;"/>
In the above connection string, the Open() command will timeout after 10 seconds.