I have a TCPServer running for file receiving, and it works 99% of the time, but it will sometimes(randomly it seems) hang on the AcceptSocket() line even though I wait for the Pending() Flag to be true. Here is the code I am using for the server. (I am aware Thread.Sleep() is bleh, but that is not the issue at hand... I don't think).
// wait for the file connection
int elapsed = 0; bool timeout = false; int overtime = 60000;
while (fileServer.Pending() == false && (elapsed < overtime))
{
Thread.Sleep(1000);
elapsed += 1000;
if (elapsed > overtime)
timeout = true;
}
if (timeout)
{
Program.EventMessage("FILE SERVER TIMEOUT, NO FILE RECIEVED");
incoming_file = false;
continue;
}
Program.EventMessage("OPENING SOCKET");
fileSocketForClient = fileServer.AcceptSocket();
Basically, since this code does work. What would cause the issue of AcceptSocket() hanging even though it has a pending connection?
Is there anyway to check to see if the Socket Accepting is blocking that I could implement a timeout?
I'd like for the socket to remain blocking as the rest of the application does need this operation to be completed to move forward, it just needs a timeout or something so if it does crap out, can recover without having to manually restart the entire program.
Issue found, and I know what I need to do to fix it (timeout AcceptSocket()), but scrounging around SO I have not found a viable way to do so yet.
As a temporary band-aid I have moved this to multithreaded, so at least when it dies the server can still go... there will just be the forever blocking thread to deal with.
A connection was pending. Does that mean a connection will still be pending later? No. Status functions don't predict the future.
"I'm smart enough to think of every possible way a socket operation could block and make sure that doesn't happen, so I can use blocking socket operations and still be assured they'll never block." No, you aren't. Don't even try. It won't work. Use non-blocking socket operations.
Related
I connect to a client using TcpClient.BeginConnect where i have a loop that is running as long as a CancellationToken isn't requested cancelled.
while (!_token.IsCancellationRequested)
{
var s = reader.ReadLine() ?? string.Empty;
}
I also reconnect and handle exceptions with logging.
When I start this application everything works as expected. However, when I put my computer to sleep and wake it up again it seems all the threads have terminated.
The workflow is as follows:
I start (from the main thread) a new Task which executes DoWork. I execute like this: Task.Run(()=>DoWork(),_token);
DoWork instantiates a new TcpClient and initiates a BeginConnect like this:
(_client = new TcpClient()).BeginConnect(Address, Port, ConnectCallback, _client);
Within ConnectCallback I have a while statement which keeps reading data from the stream (see above).
Any idea what happens to the threads when the computer goes to sleep?
When your computer wakes from a sleep, the connections will sometimes (but not always) be severed. It depends on a number of factors, some of which might not be under your control.
In terms of your callback method, there is no exception triggered but reader.EndOfStream will be true when the connection has ended and all prior data has been read. Edit: However, if the TCP stack is unaware that the remote has disconnected, this call will block until data arrives, the TcpClient.ReceiveTimeout period has passed, or until the TCP session idle timeout, whichever happens first.
(Edit: If no data is sent, the connection will only end automatically if the TCP stack is aware (based on its own network state detection, or TCP packets) that the remote has disconnected; otherwise it will wait until session idle timeout, which can take up to an hour or so, depending on the client. One solution is for the client to send data regularly (if it has not received data recently) to act as a kind of early disconnection detection.)
In fact when the connection ends (e.g. after your computer wakes up), the while loop you posted would go into a tight loop, maxing out a CPU core, because reader.ReadLine() keeps returning null. You can check for that and break out of the loop:
if (reader.EndOfStream)
{
break;
}
The solution to this problem (although not a nice one) was to use a timer which explicitly closed the connection after a given amount of time.
var timeoutTimer = new Timer(state =>
{
var temp = (TcpClient)state;
Log.Error("Read timeout. Closing connection.");
temp.Close();
}, client, Timeout.Infinite, Timeout.Infinite);
Prior to accessing the stream I activate this timer:
timeoutTimer.Change(20000, Timeout.Infinite); // Handle timeouts.
And reset it afterwards:
timeoutTimer.Change(Timeout.Infinite, Timeout.Infinite); // Reset timeout.
Both using reader.EndOfStream or reader.ReadLine causes the thread to stop at that point, which isn't resolved unless the connetion is forcefully terminated.
EDIT:
Setting TcpClient.ReceiveTimeout does the same thing as above - probably better. Throws an IOException when the reciever does not receive any data for the specified amount of time (in ms).
There are 5 threads running in an infinite loop.
2 of them will send messages when queue is not empty.
4 of them will keep sending heartbeat within 5 minutes.
1 of them is to request data from another source.
When it utilizes 100% of the CPU, I can not use any other applications in the window. The whole window becomes very slow.
EDIT: can sleep be put after WaitOne?
if(autoEvent.WaitOne())
{
}
else
{
}
Thread.Sleep(100);
Can sleep be put after subscriber.Recv() which is ZeroMQ ?
all threads i put a sleep if no Recv(), however there is one thread i do not dare to put a sleep in realtime datafeed thread which has only client.Send, will just one thread cause 100% ?
Q: How to make a program not utilize 100% CPU?
A: Don't create a busy loop!!!!
Blocking is Good. There are lots of ways to accomplish "block until there's something to do". Including using an alarm signal or timer (poor, but a definite improvement), doing a socket read with a timeout (if you happen to be notified with a network socket) or using a Windows Event object with a timeout.
Failing all else, you can always use a "Sleep()". I would discourage using "Sleep" if you can avoid it - there are almost always much better design strategies. But it will keep you from a 100% CPU busy loop ;)
=======================================
Addendum: you posted some code (thank you!)
You're using xxx.WaitOne().
Just use WaitOne() (a blocking call), with a timeout. This is an IDEAL solution: no busy loop, no "Sleep" required!
http://msdn.microsoft.com/en-us/library/aa332441%28v=vs.71%29.aspx
Put System.Threading.Thread.Sleep(100) (100 milliseconds sleep = time for system to do something else) in your infinite loops.
For the threads that send messages, when the queue is emtpy, use a ResetEvent
DeliverMessageThread_DoWork
{
while(true)
{
if(GetNextMessage() == null)
MyAutoResetEvent.WaitOne(); // The thread will suspend here until the ARE is signalled
else
{
DeliverMessage();
Thread.Sleep(10); // Give something else a chance to do something
}
}
}
MessageGenerator_NewMessageArrived(object sender, EventArgs e)
{
MyAutoResetEvent.Set(); // If the deliver message thread is suspended, it will carry on now until there are no more messages to send
}
This way, you won't have 2 threads chewing up all of the CPU cycles all of the time
Is this bad programming ?
DateTime dtExpire = DateTime.Now.AddSeconds(90);
while (client.Connected && DateTime.Now < dtExpire)
{
if (client.Available == 0) continue;
//or can also use: if (!networkStream.DataAvailable) continue;
dtExpire = DateTime.Now.AddSeconds(30);
//now do stuff with client via stream
}
The goal being to insure that the client does not take too mor time than the server is willing to wait to pocess incoming messages. Of course, this code is inside of a Try/Catch block, as well as a Using Stream block, so the server would gracefully handle dropped connections or any other socket exceptions.
Basically, I just want to know if there's a better way to handle this. Thanks.
Use the ReceiveTimeout property to specify how long to wait for an incoming message. When you use the Receive method (or its family of methods) and a timeout occurs, a SocketException will be thrown.
client.ReceiveTimeout = 90;
Your code will be more complex if you have to accomplish this asynchronously, but it doesn't look like you are. Receive by itself should do the job as it will block on the current thread.
This is called busy waiting.
You are essentially clogging the CPU even when there is no "real" work to be done (i.e. when you are just waiting on client.Available to become different from 0). Fortunately, your busy waiting has a timeout so at least it won't clog the CPU forever.
Whether you can do it more efficiently really only depends on what the client is and whether it implements a more efficient waiting strategy.
If it doesn't then you'll be stuck with some form of busy waiting, but not all is lost - if you can tolerate a slight delay in detecting the change in client.Available, then doing...
if (client.Available == 0) {
Thread.Sleep(max_delay_you_can_tolerate);
continue;
}
...would go a long way taking the pressure off the CPU.
--- EDIT ---
If client is in fact a Socket, take a look at Blocking and ReceiveTimeout properties.
I have a multi thread application written by c#, my max thread number is 256 and this application gets the performance counters of the computers in an Ip interval(192.168.1.0 -192.168.205.255)
it works fine and turns many times in a day. because I have to get reports.
But the problem is some times one machine keeps a thread and never finishes its work so my loop doesnt turn...
Are there any way to create threads with a countdown parameter. when I start the threads in foreach?
foreach(Thread t in threads)
{
t.start(); -----> t.start(countdownParameter) etc....
}
coundown parameter is the max life of each threads. This mean if a thread cant reach a machine it have to be abort. for example 60 seconds.. no not 256 machines, I meant 256 threads... there are about 5000 ip and 600 of them are alive. soo I am using 256 threads to read their values. and the other thing is loop. my loop is working as while all off the ipies finish it starts from beginning.
You can't specify a timeout for thread execution. However, you can try to Join each thread with a timeout, and abort it if it doesn't exit.
foreach(Thread t in threads)
{
t.Start();
}
TimeSpan timeOut = TimeSpan.FromSeconds(10);
foreach(Thread t in threads)
{
if (!t.Join(timeOut))
{
// Still not complete after 10 seconds, abort
t.Abort();
}
}
There are of course more elegant ways to do it, like using WaitHandles with the WaitAll method (note that WaitAll is limited to 64 handles at a time on most implementations, and doesn't work on STA threads, like the UI thread)
You should not terminate the thread from the outside. (Never kill a thread, make it commit suicide). Killing a thread can easily corrupt the state of an appdomain if you're not very careful.
You should rewrite the network code in the threads to either time out once the time-limit has been reached, or use asynchronous network code.
Usually a thread gets stuck on a blocking call (unless of course you have a bug causing an infinite loop). You need to identify which call is blocking and "poke" it to get it to unblock. It could be that your thread is waiting inside one of the .NET BCL waiting calls (WaitHandle.WaitOne, etc.) in which case you could use Thread.Interrupt to unblock it. But, in your case it is more likely that the API managing the communication with the remote computers is hung. Sometimes you can simply close the connection from a separate thread and that will unblock the hung method (as is the case with the Socket class). If all else fails then you really might have to fall back on the method of last of calling Thread.Abort. Just keep in mind that if you abort a thread it might corrupt the state of the app domain in which the abort originated or even the entire process itself. There were a lot of provisions added in .NET 2.0 that make aborts a lot safer than they were before, but there is still some risk.
You can use smth like this:
public static T Exec<T>(Func<t> F, int Timeout, out bool Completed)
{
T result = default(T);
Thread thread = new Thread(() => result = F());
thread.Start();
Completed = thread.Join(Timeout);
if(!Completed) thread.Abort();
return result;
}
I am using the TcpClient class in C#.
Each time there is a new tcp connection request, the usual practice is to create a new thread to handle it. And it should be possible for the main thread to terminate these handler threads anytime.
My solution for each of these handler thread is as follows:
1 Check NetworkStream's DataAvailable method
1.1 If new data available then read and process new data
1.2 If end of stream then self terminate
2 Check for terminate signal from main thread
2.1 If terminate signal activated then self terminate
3 Goto 1.
The problem with this polling approach is that all of these handler threads will be taking up significant processor resources and especially so if there is a huge number of these threads. This makes it highly inefficient.
Is there a better way of doing this?
See Asynchronous Server Socket Example to learn how to do this the ".NET way", without creating new threads for each request.
Believe it or not that 1000 tick sleep will really keep things running smooth.
private readonly Queue<Socket> sockets = new Queue<Socket>();
private readonly object locker = new object();
private readonly TimeSpan sleepTimeSpan = new TimeSpan(1000);
private volatile Boolean terminate;
private void HandleRequests()
{
Socket socket = null;
while (!terminate)
{
lock (locker)
{
socket = null;
if (sockets.Count > 0)
{
socket = sockets.Dequeue();
}
}
if (socket != null)
{
// process
}
Thread.Sleep(sleepTimeSpan);
}
}
I remember working on a similar kind of Windows Service. It was a NTRIP Server that can take around 1000 TCP connections and route the data to a NTRIP Caster.
If you have a dedicated server for this application then it will not be a problem unless you add more code to each thread (File IO, Database etc - although in my case I also had Database processing to log the in/out for each connection).
The things to watch out for:
Bandwidth when the threads goes up to 600 or so. You will start seeing disconnections when the TCP Buffer window is choked for some reason or the available bandwidth falls short
The operating system on which you are running this application might have some restrictions, which can cause disconnections
The above might not be applicable in your case but I just wanted it put it here because I faced then during development.
You're right that you do not want all of your threads "busy waiting" (i.e. running a small loop over and over). You either want them blocking, or you want to use asynchronous I/O.
As John Saunders mentioned, asynchronous I/O is the "right way" to do this, since it can scale up to hundreds of connections. Basically, you call BeginRead() and pass it a callback function. BeginRead() returns immediately, and when data arrives, the callback function is invoked on a thread from the thread pool. The callback function processes the data, calls BeginRead() again, and then returns, which releases the thread back into the pool.
However, if you'll only be holding a handful of connections open at a time, it's perfectly fine to create a thread for each connection. Instead of checking the DataAvailable property in a loop, go ahead and call Read(). The thread will block, consuming no CPU, until data is available to read. If the connection is lost, or you close it from another thread, the Read() call will throw an exception, which you can handle by terminating your reader thread.