Develop Reference

Stream CopyToAsync - Detect client disconnection and set a timeout

I am writing a ConnectionHandler as a part of Kestrel. The idea is that when a client connects, the ConnectionHandler opens a socket with another server in the network, gets a continuous stream of data and forwards them back to the client. In the meantime, the client can also send data to the ConnectionHandler that the latter is constantly forwarding to the other server in the network (opened socket).
public override async Task OnConnectedAsync(ConnectionContext connection)
{
TcpClient serverSocket = TcpClient(address, port);
serverSocket.ReceiveTimeout = 10000;
serverSocket.SendTimeout = 10000;
NetworkStream dataStream = serverSocket.GetStream();
dataStream.ReadTimeout = 10000;
dataStream.WriteTimeout = 10000;
Stream clientStreamOut = connection.Transport.Output.AsStream();
Stream clientStreamIn = connection.Transport.Input.AsStream();
Task dataTask = Task.Run(async () =>
{
try
{
await dataStream.CopyToAsync(clientStreamOut);
}
catch
{
await LogsHelper.Log(logStream, LogsHelper.BROKEN_CLIENT_STREAM);
return;
}
}, connection.ConnectionClosed);
Task clientTask = Task.Run(async () =>
{
try
{
await clientStreamIn.CopyToAsync(dataStream);
}
catch
{
await LogsHelper.Log(logStream, LogsHelper.BROKEN_DATA_STREAM);
return;
}
}, connection.ConnectionClosed);
await Task.WhenAny(dataTask, clientTask);
}
I am encountering 3 issues:
For the socket with the other server, I am using a TcpClient and I use a NetworkStream. Even though I am setting both ReadTimeout and WriteTimeout to 10 seconds, for both TcpClient and NetworkStream, the opened socket is waiting forever, even if the other server in the network does not send any data for 5 minutes.
Setting timeout for clientStreamOut and clientStreamIn (e.g: clientStreamIn.ReadTimeout = 10000;) is also failing with an exception that it's not supported for that particular stream. I was wondering, is it possible somehow to provide a timeout?
When a client connects to the ConnectionHandler, OnConnectedAsync is triggered. The issue with the code comes when a client disconnects (either due to network drop or for whatever reason). Sometimes disconnection of the client is being detected and the session terminates, while other times it hangs forever, even if the client has actually been disconnected. I was expecting that CopyToAsync will throw an exception in case of a disconnection since I assume that CopyToAsync is trying to write, but that's not always the case.
connection.ConnectionClosed is a CancellationToken that comes from OnConnectedAsync, I read here https://github.com/dotnet/runtime/issues/23207 that it can be used in CopyToAsync. However, I am not sure how I can use it. Also, it is worth to mention that I have zero control over the client code.
I am running the app using Docker
FROM mcr.microsoft.com/dotnet/core/sdk:3.1
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1

The ReadTimeout and WriteTimeout properties only apply to synchronous reads/writes, not asynchronous ones.
For asynchronous code, you'll need to implement your own read timeouts (write timeouts are generally unnecessary). E.g., use Task.Delay and kill the connection if data isn't received in that time.

Too slow socket response (21 seconds) using TCP protocol

I am trying to create socket and connect to a device. The problem occurs when the device is not in the network and I try to connect. I get false as response (which is ok and on which base I update status of my device), but the problem is that I get the response in 21 seconds. I am using this function to connect:
public bool Connect(IPEndPoint ipEndPoint, IPAddress ipAddress, Guid id, bool isAlive)
{
try
{
clSocket = new Socket(ipAddress.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
clSocket.NoDelay = false;
clSocket.ReceiveTimeout = 1000;
clSocket.SendTimeout = 1000;
clSocket.Connect(ipEndPoint);
return true;
}
catch (Exception ex)
{
return false;
}
}
As much as I read about "21 seconds timeout" I found out that I have to change the register:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters
Value Name: TcpMaxDataRetransmissions
Data Type: REG_DWORD - Number
Valid Range: 0 - 0xFFFFFFFF
Default: 3
I am reffering to this articles:
http://getin2me.blogspot.com/2010/08/are-you-also-facing-windows-sockets-21.html and this post:
https://social.msdn.microsoft.com/Forums/en-US/fd586377-34b3-4cb8-a3af-0a24c608e399/connectex-winsock-question?forum=vcgeneral
Why in Windows 8 I can not find this register? Is there any other solution so I can get quick (by quick I mean normal <1ms to <20ms i.e.) response true(connected) or false(not connected)?

by quick I mean normal <1ms to <20ms i.e.)
If the device is not there you can not get a response about the failed connection that fast. A TCP connections gets established by the 3-way-handshake. In this handshake the client first sends a SYN packet to the server and waits for the SYN+ACK back.
If the server is there but the port is closed it will send a RST back (instead of SYN+ACK) and the client knows that that the server is unreachable. This can be done in RTT time, which is a few ms in a local network but can be easily more than 20ms if you are on the internet.
If the server is not there it can not send a response back. In this case the client will retry to send the SYN multiple times because the packet might simply be lost. How long it will wait for the response depends on the OS and tuning, but often the first retry is done after 1 second, the second after 3 seconds... until after some time the client simply gives up and assumes the server is not there. This means in this case it will take multiple seconds and there is nothing you can do about it.

You could try pinging the address first and see if you get a response, if you dont, dont attempt to try to connect to it
Ping pinger = new Ping();
try
{
PingReply reply = pinger.Send(ip);
if (reply.Status == IPStatus.Success)
{
}
else
{
}
}

Slow Http Get Requests With C#

I currently have a python server running that handles many different kinds of requests and I'm trying to do this using C#. My code is as follows:
try
{
ServicePointManager.DefaultConnectionLimit = 10;
System.Net.ServicePointManager.Expect100Continue = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Proxy = null;
request.Method = "GET";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
response.Close();
}
}
catch (WebException e)
{
Console.WriteLine(e);
}
My first get request is almost instant but after that, the time it takes for one request to go through is almost 30 seconds to 1 min. I have researched everywhere online and tried to change the settings to make it run faster but it can't seem to work. Are there anymore settings that I could change to make it faster?

By using my psychic debugging skills I guess your server only accepts one connection at the time. You do not close your request, so you connection is kept alive. Keep alive attribute. The server will accept new connection when you current one is closed, which is default 100000ms or the server timeout. In your case I guess 30 to 60 seconds. You can start by setting the keepalive attribe to false.

C# Create 10 parallel web requests at a time [duplicate]

I have an program where I need to create some large number of folders to an external sharepoint site (external meaning I can't use the sharepoint object model). Web requests work well for this, but simply doing them one at a time (send request, wait for response, repeat) is rather slow. I decided to multithread the requests, to try and speed it up. The program has sped up considerably, but after some amount of time (between 1-2 minutes or so), concurrency exceptions start getting thrown.
Code is below, is this the best way to go about this?
Semaphore Lock = new Semaphore(10, 10);
List<string> folderPathList = new List<string>();
//folderPathList populated
foreach (string folderPath in folderPathList)
{
Lock.WaitOne();
new Thread(delegate()
{
WebRequest request = WebRequest.Create(folderPath);
request.Credentials = DefaultCredentials;
request.Method = "MKCOL";
WebResponse response = request.GetResponse();
response.Close();
Lock.Release();
}).Start();
}
for(int i = 1;i <= 10;i++)
{
Lock.WaitOne();
}
The exception is something along the lines of
Unhandled Exception: System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: Only one useage of each socket address is normally permitted 192.0.0.1:81
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddre
ss socketAddress)
at System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state,
IAsyncResult asyncResult, Int32 timeout, Exception& exception)

You might create too many connections, thus using up all the local ports you can use.
There's a timeout period for when a port can be reused after you close it.
WebRequest hides all the low level socket handling for you, but I am guessing it eventually runs out of ports, or tries to (re)bind to a socket already in a TIME_WAIT state.
You should make sure you read the response stream, even if you don't care about the response. This should help not producing too many lingering connections.
WebResponse response = request.GetResponse();
new StreamReader(response.GetResponseStream()).ReadToEnd();
I'll paste some relevant info from here:
When a connection is closed, on the side that is closing the connection the 5 tuple
{ Protocol, Local IP, Local Port, Remote IP, Remote Port} goes into a TIME_WAIT state for 240 seconds by default.
In this case, the protocol is fixed - TCP
the local IP, remote IP and remote PORT are also typically fixed. So the variable is the local port.
What happens is that when you don't bind, a port in the range 1024-5000 is used.
So roughly you have 4000 ports. If you use all of them in 4 minutes - meaning roughly you
make 16 web service calls per second for 4 minutes you will exhaust all the ports. That is the cause of this exception.
OK now how can this be fixed?
One of the ways is to increase the dynamic port range. The max by default is 5000. You can set this up to 65534.
HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort is the key to use.
The second thing you can do is once the connection does get into an TIME_WAIT state you can reduce the time it is
in that state, Default is 4 minutes, but you can set this to 30 seconds
HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\TCPTimedWaitDelay is the key to use.
Set this to 30 seconds

You're not closing the webrequest which might cause the connection to be open unecessarily long. This sounds like a perfect job for Parallel.Net's Parallel.Foreach, just be sure to indicate how many threads you want it running on
ParallelOptions parallelOptions = new ParallelOptions();
parallelOptions.MaxDegreeOfParallelism = 10;
Parallel.ForEach(folderPathList, parallelOptions, folderPathList =>
{
using(WebRequest request = WebRequest.Create(folderPath))
{
request.Credentials = DefaultCredentials;
request.Method = "MKCOL";
GetResponse request = WebRequest.Create(folderPath);
request.Credentials = DefaultCredentials;
request.Method = "MKCOL";
using (WebResponse response = request.GetResponse());
}
});
Another thing to keep in mind is maxConnections, be sure to set it in your app.config:
<configuration>
<system.net>
<connectionManagement>
<add address = "*" maxconnection = "100" />
</connectionManagement>
</system.net>
</configuration>
Of couse in a real-world scenario you would have to add try-catch to and retrying connections that might time out leading to more complicated code

For this kind of IO intensive tasks, asynchronous programming model is very useful. However, it is a little hard to use in C#.C# also has language level support for async now, you can try the CTP release.

try this
folderPathList.ToList().ForEach(p =>
{
ThreadPool.QueueUserWorkItem((o) =>
{
WebRequest request = WebRequest.Create(p);
request.Credentials = DefaultCredentials;
request.Method = "MKCOL";
WebResponse response = request.GetResponse();
response.Close();
});
EDIT - different webrequest approach
folderPathList.ToList().ForEach(p =>
{
ThreadPool.QueueUserWorkItem((o) =>
{
using (WebClient client = new WebClient())
{
client.Credentials = DefaultCredentials;
client.UploadString(p, "MKCOL", "");
}
});
});

HttpWebRequest keep alive - how is the connection reused by .net?

Hy,
I am using HttpWebRequest in 10 concurent threads to download a list of images. I sorted the images after the hostName so each of this threads are getting an image from the same Hostname.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.ServicePoint.ConnectionLimit = 10;
myReq.Timeout = 2000;
myReq.KeepAlive = true;
HttpWebResponse myResp = (HttpWebResponse )myReq.GetResponse();
After the program is running for a while I keep getting timeout exception.
My thoughts are that I get exception because maybe the Host server has some limitation regarding the concurent connections from the same user.
So how is a connection reused in .net ?
In my program each thread is creating a new connection to a hostname, or is reusing the existing one because of the KeepAlive property ??

It seems that the problem was some servers that were using http/1.0.
HttpWebRequest is using 100 Continue behaviour but the server doesn't support it.
So i changed the property System.Net.ServicePointManager.Expect100Continue to false and then everything worked fine.