Develop Reference

Adding Keepalive option to MySQL connection causes error "Unable to connect to any of the specified MySQL hosts"

I am working on a project where a thread is run and opens a permanent database connection. The reason for this, is potentially as the project grows it could receive more and more requests so it's more efficient to keep the database open and usable instead of opening and closing the database.
Basically what the thread does is look for events in a queue, and if there is an event it starts working on the database to store and process the event. At its peak this thread could receive 50,000-100,000 requests a day potentially a lot more, as more and more users (hopefully) use the service.
However, because it's new, there are times where this thread doesn't have anything to do, so I end up hitting the exception "The connection must be valid and open" and I believe this is because the connection to the database is automatically dropped over 8 hours of inactivity. At the moment this can happen so I am trying to add a Keepalive option to the connection so this doesn't happen but for some reason, as soon as I add this, I then get the error "Unable to connect to any of the specified MySQL hosts".
I am using a MysqlConnectionStringBuilder as follows:
MySqlConnectionStringBuilder connectionBuilder = new MySqlConnectionStringBuilder();
connectionBuilder.Server = server;
connectionBuilder.UserID = username;
connectionBuilder.Password = password;
connectionBuilder.Port = 3306;
//Open the connection
MySqlConnection conn = new MySqlConnection(connectionBuilder.ConnectionString);
conn.Open();
This above works perfectly fine until I add the following:
connectionBuilder.Keepalive = 60;
When the above line is added is when I then get the error.
I am using a TCP connection as it's a remote connection from my Dev PC to a dev server - is there a setting on the server to enable this? as everything I've found on Google this option is all that's required.

C# Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. Reading networkstream

I have looked at
unable to read data from the transport connection the connection was closed
Unable to read data from the transport connection : An existing connection was forcibly closed by the remote host
Unable to read data from the transport connections
I am new to this and I have been told the issue is not the server. Is there anyway the issue can be client side or the client is causing an issue that forces the server to disconnect?
I make my connection
_client.BeginConnect(host, port, ConnectCallback, null);
and then in ConnectCallback
_client.EndConnect(result);`
_client.NoDelay = true;`
_client.ReceiveBufferSize = 65535;`
NetworkStream ns = _client.GetStream();`
byte[] buffer = new byte[_client.ReceiveBufferSize];`
ns.BeginRead(buffer, 0, buffer.Length, ReadCallback, buffer);`
and then in ReadCallback
ns = _client.GetStream();`
read = ns.EndRead(result);`
Which is where it sometimes fails. There are other connections in the app and they have no problems. One of the connections is to the same server that I am having issues, but on a different port.
All the machines are on the same network. This is the only client connecting to this particular server/port. We run the app for about 3 hours and this error happens anytime.
Edit
I appreciate the comments and input. Being new to programming and networking, etc, I still have the question: Is it possible the client is causing the issue?

The remote endpoint or a device on the network is closing the TCP connection.
You always need to test that the TCP connection is available and connected before reading from it.
You have no control over when the remote endpoint closes the connection. Even if you wrote the code on the remote endpoint it could still close due to a power outage.
The ReadCallback will be fired when the connection is closed, as well as when data is available. Your code is responsible for detecting that the connection has been closed and taking the necessary action in your code to deal with this scenario.
The easiest way to deal with this is to catch the exception, stop reading/writing from/to the socket, do any clean up tasks and reconnect if necessary etc etc..

Test connection to the IBM MQ server every couple of minutes

We are using the IBM.XMS 8.0.0.5 .NET library to connect to the IBM MQ server and create our listeners.
But sometimes the VPN tunnel goes in sleep mode (this happens if one of our servers restarts for example). To prevent this, it is necessary to keep the VPN tunnel 'awake' by sending a network packet through the tunnel.
I looked around but IBM MQ does not have any implementation to test the connection to the server. I need some kind of 'ping' which will keep the tunnel up. But pinging is not allowed, I think they reject ICMP echos.
I am planning to create an async Task which will test the connection regularly depending on a configured interval.
Any advice on this one please?
PS Sometimes the connection falls asleep as I mentioned above. And without knowing anything the system engineer restarts the service which sends a dispose to the IBM server, even a dispose message leads to the connection being back up. And all items on the queue start being consumed by us while the service is being stopped... I need to solve this but I don't have a clue how I can do this.
EDIT
Connection is made and consumers are set up as follows:
var factoryFactory = XMSFactoryFactory.GetInstance(XMSC.CT_WMQ);
// Create WMQ Connection Factory.
var cf = factoryFactory.CreateConnectionFactory();
// Set the properties
cf.SetStringProperty(XMSC.WMQ_HOST_NAME, _parameters.IbmMqHost);
cf.SetIntProperty(XMSC.WMQ_PORT, _parameters.IbmMqPort);
cf.SetStringProperty(XMSC.WMQ_CHANNEL, _parameters.IbmMqChannel);
cf.SetIntProperty(XMSC.WMQ_CONNECTION_MODE, XMSC.WMQ_CM_CLIENT);
cf.SetStringProperty(XMSC.WMQ_QUEUE_MANAGER, _parameters.IbmMqQueueManager);
cf.SetStringProperty(XMSC.USERID, username);
cf.SetStringProperty(XMSC.PASSWORD, pw);
cf.SetIntProperty(XMSC.WMQ_CLIENT_RECONNECT_OPTIONS, XMSC.WMQ_CLIENT_RECONNECT);
// Create connection.
connection = cf.CreateConnection();
connection.ExceptionListener = ExceptionCallback;
session = connection.CreateSession(false, AcknowledgeMode.AutoAcknowledge);
destination = session.CreateQueue(queuename);
consumer = session.CreateConsumer(destination);
consumer.MessageListener = listener;
connection.Start();

It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail. SocketFailure on PING

I am trying to make a simple example of reading and writing from azure redis cache and I get this error
An exception of type 'StackExchange.Redis.RedisConnectionException' occurred in StackExchange.Redis.dll but was not handled in user code
Additional information: It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail. SocketFailure on PING
The code I am using is this, I changed dns and password
// Get Connection instance
ConnectionMultiplexer connection = ConnectionMultiplexer
.Connect("xx.redis.cache.windows.net,ssl=false,password=...");
// Get database
IDatabase databaseCache = connection.GetDatabase();
// Add items
databaseCache.StringSet("foo1", "1");
databaseCache.StringSet("foo2", "2");
// Add items with experation value
databaseCache.StringSet("foo3", "3", TimeSpan.FromMinutes(20));
Stopwatch sw = new Stopwatch();
sw.Start();
// Get item value
string foo1Value = databaseCache.StringGet("foo1");
sw.Stop();
Console.WriteLine("Elapsed={0}", sw.Elapsed);
return View();

Azure Redis Cache only enables the SSL endpoint by default. The most secure approach is to set "ssl=true" when calling ConnectionMultiplexer.Connect().
Alternatively, you can use the Azure Portal to enable the non-SSL endpoint on your Azure Redis Cache, but then your password and all data will be sent in clear text.

I had exact same exception and it turned out to be corporate firewall, which is blocking port 6379, 6380.
I copied my test console app in an environment outside company network and connection was successful.
So if Redis server is running on the internet and your network is behind a firewall make sure the ports are open.

Permanent connection to cassandra or connection per request?

I'm right now implementing my own UserStore for WebApi 2 so it works with cassandra. My question is, if i should close the connection after a request and reconnect to cassandra for the next request.
Right now i am establishing the connection at startup of the application and passing a cassandra Context to the UserStore to work with. the connection is closed when i shut down the application.
I'm wondering if e.g. 10 people register at the same, is this possible with only one connection?
static Startup()
{
PublicClientId = "self";
//Connecting to Cassandra
Cluster cluster = Cluster.Builder().AddContactPoint("127.0.0.1").Build();
Session session = cluster.Connect();
Context context = new Context(session);
//passing context
UserManagerFactory = () => new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(context));
OAuthOptions = new OAuthAuthorizationServerOptions
{
TokenEndpointPath = new PathString("/Token"),
Provider = new ApplicationOAuthProvider(PublicClientId, UserManagerFactory),
AuthorizeEndpointPath = new PathString("/api/Account/ExternalLogin"),
AccessTokenExpireTimeSpan = TimeSpan.FromDays(14),
AllowInsecureHttp = true
};
}
//UserStore Method for Registration
public virtual Task CreateAsync(TUser user)
{
if (user == null)
{
throw new ArgumentNullException("user");
}
var usertable = _context.GetTable<TUser>("users");
//insert user into table
}

When you are deciding if you should have connection pooling or not, there is typically 2 questions you need to answer:
What is the cost of establishing connection every time?
Establishing connection on some networks like EC2 is more expensive depending on the type of network and machines and your architecture setup. It can take several milliseconds and that adds up to your query time if you are establishing connections every time. If you care about saving those milliseconds, then pooling connections is a better option for you.
Connections to databases are resources managed by your OS that your Application Server and DB server are holding onto while being consumed or sleeping. If your hardware resources are low, the connections should be treated like files. You open them, read from, or write to them, then close them. If you don't have hardware resource constraints then don't worry about pooling resource.
On the Cassandra side if you set your rpc_server_type to sync, then each connection will have its own thread which takes minimum of 180K and if you have a lot of clients, then memory will be your limiting factor. If you chose hsha as rpc_server_type, then this won't be a problem.
Read this: https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L362
About your application performance:
Ideally if you are using Cassandra with a multi-node setup, you want to distribute your requests to multiple nodes (coordinators) so that it scales better. Because of this it is better if you don't stick with the same connection as you will always be talking to the same co-ordinator.
Secondly, if you are multi-threading, you want to make sure your connection is sticking with the same thread while being used for the duration of each query to Cassandra, otherwise, you may end up with race conditions where one thread is updating resources of the connection which is already being used (e.g. trying to sent query packets to server, when it previously was waiting for a response from server).
I recommend implementing a thread-safe connection pool and open several connections on startup of your application and randomly use them for each of your requests, and kill them when your application server stops. Make sure you consider changing the rpc_server_type in Cassandra if you have hardware constraints.

The DataStax C# driver already provides connection pooling internally, thus the recommended way to use the driver is to use a Cluster instance per your C* cluster and a Session per keyspace.Basically initialize these at the startup of your app (nb you can also prepare the PreparedStatements too), reuse these across request, and close them for cleanup when stopping the app (for upgrading, etc).
I'd strongly recommend you to give a quick read of the C# driver docs. It shouldn't take you long and you'll know more about what's included in the driver.