Ok, I have a strange exception thrown from my code that's been bothering me for ages.
System.Net.Sockets.SocketException: A blocking operation was interrupted by a call to WSACancelBlockingCall
at System.Net.Sockets.Socket.Accept()
at System.Net.Sockets.TcpListener.AcceptTcpClient()
MSDN isn't terribly helpful on this : http://msdn.microsoft.com/en-us/library/ms741547(VS.85).aspx and I don't even know how to begin troubleshooting this one. It's only thrown 4 or 5 times a day, and never in our test environment. Only in production sites, and on ALL production sites.
I've found plenty of posts asking about this exception, but no actual definitive answers on what is causing it, and how to handle or prevent it.
The code runs in a separate background thread, the method starts :
public virtual void Startup()
{
TcpListener serverSocket= new TcpListener(new IPEndPoint(bindAddress, port));
serverSocket.Start();
then I run a loop putting all new connections as jobs in a separate thread pool. It gets more complicated because of the app architecture, but basically:
while (( socket = serverSocket.AcceptTcpClient()) !=null) //Funny exception here
{
connectionHandler = new ConnectionHandler(socket, mappingStrategy);
pool.AddJob(connectionHandler);
}
}
From there, the pool has it's own threads that take care of each job in it's own thread, separately.
My understanding is that AcceptTcpClient() is a blocking call, and that somehow winsock is telling the thread to stop blocking and continue execution.. but why? And what am I supposed to do? Just catch the exception and ignore it?
Well, I do think some other thread is closing the socket, but it's certainly not from my code.
What I would like to know is: is this socket closed by the connecting client (on the other side of the socket) or is it closed by my server. Because as it is at this moment, whenever this exception occurs, it shutsdown my listening port, effectively closing my service. If this is done from a remote location, then it's a major problem.
Alternatively, could this be simply the IIS server shutting down my application, and thus cancelling all my background threads and blocking methods?
Is it possible that the serverSocket is being closed from another thread? That will cause this exception.
This is my example solution to avoid WSAcancelblablabla:
Define your thread as global then you can use invoke method like this:
private void closinginvoker(string dummy)
{
if (InvokeRequired)
{
this.Invoke(new Action<string>(closinginvoker), new object[] { dummy });
return;
}
t_listen.Abort();
client_flag = true;
c_idle.Close();
listener1.Stop();
}
After you invoke it, close the thread first then the forever loop flag so it block further waiting (if you have it), then close tcpclient then stop the listener.
This could happen on a serverSocket.Stop(). Which I called whenever Dispose was called.
Here is how my exception handling for the listen thread looked like:
try
{
//...
}
catch (SocketException socketEx)
{
if (_disposed)
ar.SetAsCompleted(null, false); //exception because listener stopped (disposed), ignore exception
else
ar.SetAsCompleted(socketEx, false);
}
Now what happened was, every so often the exception would occur before _disposed was set to true. So the solution for me was to make everything thread safe.
Same here!
But i figured out, that the ReceiveBuffer on 'server-side' was flooded from the clients!
(In my case a bunch of RFID-Scanners, who kept spamming the TagCode, instead of stop sending until next TagCode arrives)
It helped to raise the ReceiveBuffers and reconfigure the scanners...
More recently I saw this exception when using HttpWebRequest to PUT a large file and the Timeout period was passed.
Using the following code as long as your upload time > 3 seconds it will cause this error as far as I could see.
string path = "Reasonably large file.dat";
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
System.Net.HttpWebRequest req = (HttpWebRequest)System.Net.HttpWebRequest.Create("Some URL");
req.Method = "PUT";
req.Timeout = 3000; //3 seconds, small timeout to demonstrate
long length = new System.IO.FileInfo(path).Length;
using (FileStream input = File.OpenRead(path))
{
using (Stream output = req.GetRequestStream())
{
long remaining = length;
int bytesRead = 0;
while ((bytesRead = input.Read(buffer, 0, (int)Math.Min(remaining, (decimal)bufferSize))) > 0)
{
output.Write(buffer, 0, bytesRead);
remaining -= bytesRead;
}
output.Close();
}
input.Close();
}
Related
I have a simple C# Windows Presentation Framework program, which spawns a TCP Listener. This TCP Listener accepts a socket from a connection, which I then want to be able to receive and send data on.
I wasn't sure how to do that with the same socket, so I setup this loop.
Essentially at the start of every loop, checking with a timeout (I think is 10 ms) for new data, then regardless of new data proceding to send data if applicable. If I didn't have the timeout, I couldn't send data without first recieving data, as calling socket.recieve would block until we had something.
However, having this timeout causes horrible lag with my application. Despite operating in a different frame! From testing, as soon as I remove the timeout, the lag goes away. So I am fairly confident that is the issue.
This leads me to thinking, is this the wrong way to be doing this? Should I be utilizing that Async Receive call instead? I didn't think that was the solution but I am not sure. Is there a better way to do this? I guess purposefully throwing this exception is very bad for performance.
System.Net.IPAddress localAddr = System.Net.IPAddress.Parse("127.0.0.1);
TcpListener mListener = new TcpListener(localAddr, 5000);
var thread = new System.ThreadingThread(() => {
mListener.Start();
//Loop Forever
mListener.Start();
while(true){
Socket client = mListener.AcceptSocket();
client.ReceiveTimeout = 10;
while(true){
byte[] data = new byte[5000];
int returnsize = 0;
try{
returnSize = client.Recieve(data);
}catch(Exception e){
//Client Timeout!
//We will catch a data on the next loop
}
//now here I would check here if a queue has data to send
//if so use client.send
//proceed to next loop
}
}
});
thread.Start();
Thanks for the help!
I have a c# application with the following requirements.
Periodically(700ms) send data to a cgi using a GET.
The server will respond with only the GET string sent as confirmation.
Values sent are constantly changing and are not stored, ordered, or retried.
Only one connection made to the server at a time.
Timer events that find a previous connection still in progress should simply exit quietly and let the next event carry on.
The client machine is Windows Server 2012. The HTTP server is a black box appliance(believed to be running linux). Environment is low-latency production with extremely tight security(network/policies/etc).
The problem:
Periodically, the client will begin throwing exceptions
"The operation timed out"
and the drip feed stops. Simultaneously, Wireshark shows
no connections being made from the client to the server.
However, VisualStudio shows that
every 700ms the client is attempting to connect and is timing out.
When it's not throwing the above exception, it seems to work fine. At no time does the HTTP server appear to be rejecting connections. Sometimes the client will recover, most of the time it will continue to throw exceptions until restarted. AppDomain thread pool is stable. Memory consumption is fine.
To confuse matters, I have identical hardware/software/network without the security in a lab where everything runs flawlessly for days at a time. My code is being fingered as "malfunctory".
I have recently discovered that the HttpWebRequest.Timeout property bounds the entire life of the transaction, not just the response wait. I am increasing this and it's currently under test.
Anyone see anything glaring(or otherwise) that would cause this problem in the below code?
// timer call back at 700ms interval
void m_postTimer_Elapsed(object sender, ElapsedEventArgs e)
{
try
{
if (Interlocked.Read(ref this.m_isLocked) == 0)
{
if (Monitor.TryEnter(this.m_lock, 10))
{
Interlocked.Exchange(ref this.m_isLocked, 1);
try
{
// value1 & value2 set elsewhere
string url = String.Format("http://1.1.1.1/set.cgi?value1={0}&value2={1}", value1, value2);
this.m_post = (HttpWebRequest)WebRequest.Create(url);
this.m_post.Timeout = 500;
this.m_post.Credentials = new NetworkCredential(this.user, this.pass);
WebResponse response = this.m_post.GetResponse();
response.Close();
Monitor.Exit(this.m_lock);
Interlocked.Exchange(ref this.m_isLocked, 0);
}
catch (WebException ex)
{
// handle web exceptions
// if we've locked earlier and hit an exception,
// the unlock has been skipped. unlock
if (Interlocked.Read(ref this.m_isLocked) > 0)
{
Monitor.Exit(this.m_lock);
Interlocked.Exchange(ref this.m_isLocked, 0);
}
}
}
}
else
{
// indicate that a pre-existing connection is still in progress
}
}
catch (Exception ex)
{
// handle generic exception
// if we've locked earlier and hit an exception,
// the unlock has been skipped. unlock
if (Interlocked.Read(ref this.m_isLocked) > 0)
{
Monitor.Exit(this.m_lock);
Interlocked.Exchange(ref this.m_isLocked, 0);
}
}
}
I'm trying to educate myself on the intricacies of reading from a NetworkStream, and understanding the various ways in which problems can occur. I have the following code:
public async Task ReceiveAll()
{
var ns = this.tcp.GetStream();
var readBuffer = new byte[1000];
while (true)
{
int bytesRead;
try
{
bytesRead = await ns.ReadAsync(readBuffer, 0, readBuffer.Length);
if (bytesRead == 0)
{
// Remote disconnection A?
break;
}
}
catch (IOException)
{
// Remote disconnection B?
break;
}
catch (ObjectDisposedException)
{
// Local disconnection?
break;
}
/*Do something with readBuffer */
}
}
I've marked three points in the code where the program says 'something has gone awry, there is no point continuing'.
The 'Local disconnection' isn't exactly something wrong, it will happen when I locally close the socket which is the only way to exit the loop under normal circumstances. I don't think anything else can cause this, so I think I'm safe to just swallow the exception.
The two 'Remote disconnection' points are what I'm not sure about. I know ReadAsync will return 0 if the connection is terminated remotely (A), but the IOException also seems to fire in some circumstances. If my remote client is a C# console, then closing the socket seems to make 'A' happen, and closing the console window seems to be make 'B' happen. I'm not sure I understand what the difference is between these scenarios?
Finally, a bit of a general question, but is there anything glaringly wrong with this bit of code or my above assumptions?
Thanks.
EDIT: In response to my use of ObjectDisposedException to abort out of the loop:
This is what my 'Stop' method looks like (from the same class as above):
public void Stop()
{
this.tcp.Close();
}
This causes the pending 'ReadAsync' to except with ObjectDisposedException. AFAIK there isn't any other way to abort this. Changing this to:
public void Stop()
{
this.tcp.Client.Shutdown(SocketShutdown.Both);
}
Doesn't appear to actually do anything to the pending call, it just continues waiting.
When NetworkStream returns 0, this means that the socket has received a disconnect packet from the remote party. This is how network connections are supposed to end.
The correct way to shut down the connection (especially if you are in a full-duplex conversation) is to call socket.Shutdown(SocketShutdown.Send) and give the remote party some time to close their send channel. This ensures that you receive any pending data instead of slamming the connection shut. ObjectDisposedException should never be part of the normal application flow.
Any exceptions thrown indicate that something went wrong, and I think it's safe to say you can no longer rely on the current connection.
TL;DR
I don't see anything wrong with your code, but (especially in full-duplex communication) I'd shut down the send channel and wait for a 0-byte packet to prevent receiving ObjectDisposedExceptions by default:
use tcp.Shutdown(SocketShutdown.Send) to tell the remote party you want to disconnect
your loop may still receive data that the remote party was sending
your loop will, if everything went right, then receive a 0-byte packet, indicating that the remote party is disconnecting
loop terminates the right way
you may want to decide to dispose the socket after a certain amount of time, if you haven't received the 0-byte packet
I have a problem on a hand held device that should be listening for messages from my server application. When the device loses connection to the network my server is on, this background worker (implemented from OpenNetCF) stops responding. I've placed messages in the ProgressChanged and RunWorkerCompleted events to see when they are raised, and in the following code after the RecieveFrom in the while loop, and after the loop termination, as well as in all of the exceptions. I don't see the messages from the exceptions, or after the while loop at all, and the messages stop after the connection is lost. All of the messages are shown by setting the text in a visible label, with the background colour of the label changing so that I can see if the loop is running. The loop seems to stop running, even after the connection is re-gained, and attempting to re-run the backgroundworker generates an "Already in use" exception. So, why would the worker stop responding while continuing to run?
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker wWorker = (BackgroundWorker)sender;
byte[] wBytes = new Byte[4096];
string wsReceive;
EndPoint wRemoteEP = (EndPoint)new IPEndPoint(IPAddress.Any, 0);
try
{
while (true)
{
if (wWorker.CancellationPending)
{
break;
}
mSocket.ReceiveFrom(wBytes, ref wRemoteEP);
if (wBytes.Length < 1)
{
continue;
}
wsReceive = Encoding.ASCII.GetString(wBytes, 0, wBytes.Length);
wWorker.ReportProgress(0, wsReceive);
}
mSocket.Close();
}
catch (ThreadAbortException)
{
}
catch (ThreadStateException)
{
}
catch (Exception E)
{
MessageBox.Show(E.Message, "Communication Error");
}
}
Right from the docs on Socket.ReceiveFrom:
If no data is available for reading, the ReceiveFrom method will block until data is available
So when communication is lost, you're unable to receive and your ReceiveFrom call blocks indefinitely. It's going to stop reporting anything while it waits for data, but the thread is still alive and therefore the BackgroundWorker can't be re-run.
Just my 2 cents:
Whenever I implement a socket communication, I treat 0 bytes received as an indication that the remote server has closed gracefully. I think that you should also treat this to start a reconnection attempt, instead of just "continuing" the loop.
I am using the following code (built from answers to my previous questions on SO):
public void Start()
{
listener = new TcpListener(IPAddress.Any, 9030);
listener.Start();
Console.WriteLine("Listening...");
StartAccept();
}
private void StartAccept()
{
listener.BeginAcceptTcpClient(HandleAsyncConnection, listener);
}
private void HandleAsyncConnection(IAsyncResult res)
{
StartAccept();
TcpClient client = listener.EndAcceptTcpClient(res);
StringBuilder sb = new StringBuilder();
var data = new byte[client.ReceiveBufferSize];
using (NetworkStream ns = client.GetStream())
{
int readCount;
while ((readCount = ns.Read(data, 0, client.ReceiveBufferSize)) != 0)
{
sb.Append(Encoding.UTF8.GetString(data, 0, readCount));
}
// Do work
// Test reply
Byte[] replyData = System.Text.Encoding.ASCII.GetBytes(DateTime.Now.ToString());
ns.Write(replyData, 0, replyData.Length);
ns.Flush();
ns.Close();
}
client.Close();
}
The line "Do work" represents where I will do the required processing for my client.
However I can't see how to use this code to read the client's data and then reply to it. When using this code I can read perfectly what is sent by my client, however once that occurs the client locks up and eventually complains that the connection was terminated. It does not receive my reply.
Any ideas on how to fix this?
Okay, first of all, you are mixing asynchronous calls (BeginAcceptTcpClient) and synchronous (Read and Write) calls. That completely kills the purpose of asynchronous code. Second, maybe this is why your socket gets closed ? Performing a sync op on an async socket. I'm not sure, but without the client code it's impossible to tell.
Anyway, this is NOT how you build an asynchronous, multi-client server.
Here is a fully asynchronous server implementation : http://msdn.microsoft.com/en-us/library/fx6588te.aspx
i think you should use a length byte to alloc your buffer. ReceiveBufferSize could be called multiple times, i think there is no gurantee you receive everything in one block.
You have misunderstood how Read works. It blocks until something is received from the other end point. The only time it returns 0 is when the other side have disconnected, hence you will continue reading until the other side disconnects.
When using TCP you need to know when a message ends. You can do that either by sending the message length first as a header or by using a suffix (as a line feed) after each message. Then you should keep reading until the complete message has arrived.