Develop Reference

C#/.NET application design with multi-threading/tasks

I want to design an application that provides some bi-directional communcation between two otherwise completely separate systems. (bridge)
One of them can call my application via web services - the other one is a piece of 3rd party hardware. It speaks RS232. Using a RS232<->ETH transceiver we manage to talk to the piece of hardware using TCP.
The program has the following requirements.
There is a main thread running the "management service". This might be a WCF endpoint or a self-hosted webapi REST service for example. It provides methods to start new worker instances or get a list of all worker instances and their respective states.
There are numerous "worker" threads. Each of them has a state model with 5 states.
In one state for example a TCP listener must be spawned to accept incoming connections from a connected hardware device (socket based programming is mandatory). Once it gets the desired information it sends back a response and transitions into the next state.
It should be possible from the main (manager) thread to (gracefully) end single worker threads (for example if the worker thread is stuck in a state where it cannot recover from)
This is where I am coming from:
I considered WCF workflow services (state model activity) however I wasn't sure how to spawn a TcpListener there - and keep it alive. I do not need any workflow "suspend/serialize and resume/deserialize" like behavior.
The main thread is probably not that much of a concern - it just has to be there and running. It's the child (background) threads and their internal state machine that worry me.
I tried to wrap my mind around how Tasks might help here but I ended up thinking threads are actually a better fit for the task
Since there has been a lot of development in .NET (4+) I am not sure which approach to follow... the internet is full of 2005 to 2010 examples which are probably more than just outdated. It is very difficult to separate the DOs from the DONTs.
I'm glad for any hints.
UPDATE: Okay I'll try to clarify what my question is...
I think the easiest way is to provide some pseudo code.
public static void Main()
{
// Start self-hosted WCF service (due to backwards compatibility, otherwise I'd go with katana/owin) on a worker thread
StartManagementHeadAsBackgroundThread();
// Stay alive forever
while(running)
{
// not sure what to put here. Maybe Thread.Sleep(500)?
}
// Ok, application is shutting down => somehow "running" is not true anymore.
// One possible reason might be: The management service's "Shutdown()" method is being called
// Or the windows service is being stopped...
WaitForAllChildrenToReachFinalState();
}
private static void StartManagementHeadAsBackgroundThread()
{
ThreadStarter ts = new ThreadStarter(...);
Thread t = new Thread(ts);
t.Start();
}
The management head (= wcf service) offers a few methods
StartCommunicator() to start new worker threads doing the actual work with 5 states
Shutdown() to shut down the whole application, letting all worker threads finish gracefully (usually a question of minutes)
GetAllCommunicatorInstances() to show a summary of all worker threads and the current state they are in.
DestroyCommunicatorInstance(port) to forcefully end a worker thread - for example if communicator is stuck in a state where it cannot recover from.
Anyway I need to spawn new background threads from the "management" service (StartCommunicator method).
public class Communicator
{
private MyStateEnum _state;
public Communicator(int port)
{
_state = MyStateEnum.Initializing;
// do something
_state = MyStateEnum.Ready;
}
public void Run()
{
while(true)
{
// again a while(true) loop?!
switch(_state):
{
case MyStateEnum.Ready:
{
// start TcpListener - wait for TCP packets to arrive.
// parse packets. If "OK" set next good state. Otherwise set error state.
}
}
if(_state == MyStateEnum.Error) Stop();
break;
}
}
public void Stop()
{
// some cleanup.. disposes maybe. Not sure yet.
}
}
public enum MyStateEnum
{
Initializing, Ready, WaitForDataFromDevice, SendingDataElsewhere, Done, Error
}
So the question is whether my approach will take me anywhere or if I'm completely on the wrong track.
How do I implement this best? Threads? Tasks? Is while(true) a valid thing to do? How do I interact with the communicator instances from within my "management service"? What I am looking for is an annotated boiler plate kind of solution :)

My suggestion would be to use a ASP.NET Web API service and mark the Controller actions as async. From that point on, use Tasks as much as possible and when you end up with blocking IO the HTTP server will not be blocked. Personally, I would avoid Threads until you are absolutely sure that you can't achieve the same thing with Tasks.

I would recommend looking at using a thread pool. This will help with managing resources and make for more efficient use of the resources.
As far as terminating threads, thread pool threads are background workers and will be terminated when your service stops, however, from your description above that is not sufficient. Your threads should always have the ability to receive a message asking them to terminate.

One server many clients: Threads or classes

I'm doing an application in C#, with a server and some clients (not more than 60), and I would like to be able to deal with each client independently. The communication between server and client is simple but I have to wait for some ack's and I don't want to block any query.
So far, I've done two versions of the server side, one it's based on this:
http://aviadezra.blogspot.com.es/2008/07/code-sample-net-sockets-multiple.html
and in the other one, I basically create a new thread for each client. Both versions work fine...but I would like to know pros and cons of the two methods.
Any programming pattern to follow in this sort of situation?

To answer your question it's both. You have threads and classes running in those threads. Whether you use WCF, async, sockets, or whatever, you will be running some object in a thread (or shuffled around a threadpool like with async). With WCF you can configure the concurrency model, and if you have to wait for ack's or other acknowledgement you'd be best to set it to multiple threads so you don't block other requests.
In the example you linked to the author is using AsyncCallback as the mechanism for telling you that a socket has data. But, from the MSDN you can see:
Use an AsyncCallback delegate to process the results of an asynchronous operation in a separate thread
So it's really no different for small scale apps. Using async like this can help you avoid allocating stack space for each thread, if you were to do a large application this would matter. But for a small app I think it just adds complexity. C# 4.5+ and F# do a cleaner job with async, so if you can use something like that then maybe go for it.
Doing it the way you have, you have a single thread that is responsible for socket management. It'll sit and accept new connections. When it gets a request it hands that socket to a new dedicated thread that will then sit on that socket and read from it. This thread is your client connection. I like to encapsulate the socket client reading into a base class that can do the low level io required and then act as a router for requests. I.e. when I get request XYZ I'll do request ABC. You can even have it dispatch events and subscribe to those events elsewhere (like in the async example). Now you've decoupled your client logic from your socket reading logic.
If you do things with WCF you don't need sockets and all that extra handling, but you should still be aware that calls are multi-threaded and properly synchronize your application when applicable.
For 60 clients I think you should choose whatever works best for you. WCF is easy to set up and easy to work with, I'd use that, but sockets are fine too. If you are concerned about the number of threads running, don't be. While it's bad to have too many threads running, most of your threads will actually be blocked while they are waiting on IO. Threads that are in a wait state aren't scheduled by the OS and don't really matter. Not to mention the waiting is most likely is using io completion ports under the hood so the wait overhead is pretty much negligible for a small application like yours.
In the end, I'd go with whatever is easiest to write, maintain, and extend.

Is it necessary to use the async Begin/End methods if already on a separate thread?

Trying to figure out whether or not I should use async methods or not such as:
TcpListener.BeginAcceptTcpClient
TcpListener.EndcceptTcpClient
and
NetworkStream.BeginRead
NetworkStream.EndRead
as opposed to their synchronous TcpListener.AcceptTcpClient and NetworkStream.Read versions. I've been looking at related threads but I'm still a bit unsure about one thing:
Question: The main advantage of using an asynchronous method is that the GUI is not locked up. However, these methods will be called on separate Task threads as it is so there is no threat of that. Also, TcpListener.AcceptTcpClient blocks the thread until a connection is made so there is no wasted CPU cycles. Since this is the case, then why do so many always recommend using the async versions? It seems like in this case the synchronous versions would be superior?
Also, another disadvantage of using asynchronous methods is the increased complexity and constant casting of objects. For example, having to do this:
private void SomeMethod()
{
// ...
listener.BeginAcceptTcpClient(OnAcceptConnection, listener);
}
private void OnAcceptConnection(IAsyncResult asyn)
{
TcpListener listener = (TcpListener)asyn.AsyncState;
TcpClient client = listener.EndAcceptTcpClient(asyn);
}
As opposed to this:
TcpClient client = listener.AcceptTcpClient();
Also it seems like the async versions would have much more overhead due to having to create another thread. (Basically, every connection would have a thread and then when reading that thread would also have another thread. Threadception!)
Also, there is the boxing and unboxing of the TcpListener and the overhead associated with creating, managing, and closing these additional threads.
Basically, where normally there would just be individual threads for handling individual client connections, now there is that and then an additional thread for each type of operation performed (reading/writing stream data and listening for new connections on the server's end)
Please correct me if I am wrong. I am still new to threading and I'm trying to understand this all. However, in this case it seems like using the normal synchronous methods and just blocking the thread would be the optimal solution?

TcpListener.AcceptTcpClient blocks the thread until a connection is made so there is no wasted CPU cycles.
But there is also no work getting done. A Thread is a very expensive operating system object, about the most expensive there is. Your program is consuming a megabyte of memory without it being used while the thread blocks on connection request.
However, these methods will be called on separate Task threads as it is so there is no threat of that
A Task is not a good solution either, it uses a threadpool thread but the thread will block. The threadpool manager tries to keep the number of running TP threads equal to the number of cpu cores on the machine. That won't work well when a TP thread blocks for a long time. It prevents other useful work from being done by other TP threads that are waiting to get their turn.
BeginAcceptTcpClient() uses a so-called I/O completion callback. No system resources are consumed while the socket is listening. As soon as a connection request comes in, the operating system runs an APC (asynchronous procedure call) which grabs a threadpool thread to make the callback. The thread itself is in use for, typically, a few microseconds. Very efficient.
This kind of code will get a lot simpler in the next version of C# with the next async and await keywords. End of the year, maybe.

If you call AcceptTcpClient() on any thread, that thread is useless until you get a connection.
If you call BeginAcceptTcpClient(), the calling thread can stop immediately, without wasting the thread.
This is particularly important when using the ThreadPool (or the TPL), since they use a limited number of pool threads.
If you have too many threads waiting for operations, you can run out of threadpool threads, so that new work items will have to wait until one of the other threads finish.

What are scalable methods of queueing work items in asynchronous TCP server callbacks in C#?

According to this post:
How to write a scalable Tcp/Ip based server
jerrylvl states:
----------*
Processing
When you get the callback from the Begin call you made, it is very important to realise that the code in the callback will execute on the low-level IOCP thread. It is absolutely essential that you avoid lengthy operations in this callback. Using these threads for complex processing will kill your scalability just as effectively as using 'thread-per-connection'.
The suggested solution is to use the callback only to queue up a work item to process the incoming data, that will be executed on some other thread. Avoid any potentially blocking operations inside the callback so that the IOCP thread can return to its pool as quickly as possible. In .NET 4.0 I'd suggest the easiest solution is to spawn a Task, giving it a reference to the client socket and a copy of the first byte that was already read by the BeginReceive call. This task is then responsible for reading all data from the socket that represent the request you are processing, executing it, and then making a new BeginReceive call to queue the socket for IOCP once more. Pre .NET 4.0, you can use the ThreadPool, or create your own threaded work-queue implementation.
----------*
My question is how exactly would I be doing this in .Net 4.0? Could someone please provide me with a code example which would work well in a scalable environment?
Thanks!

My question was answered in more depth here: C# - When to use standard threads, ThreadPool, and TPL in a high-activity server
This goes into specifics on using each the TPL, ThreadPool, and standard threads to perform work items, and when to use each method.

Nonblocking Tcp server

It's not a question really, i'm just looking for some guidelines :)
I'm currently writing some abstract tcp server which should use as low number of threads as it can.
Currently it works this way. I have a thread doing listening and some worker threads. Listener thread is just sits and wait for clients to connect I expect to have a single listener thread per server instance. Worker threads are doing all read/write/processing job on clients socket.
So my problem is in building efficient worker process. And I came to some problem I can't really solve yet. Worker code is something like that(code is really simple just to show a place where i have my problem):
List<Socket> readSockets = new List<Socket>();
List<Socket> writeSockets = new List<Socket>();
List<Socket> errorSockets = new List<Socket>();
while( true ){
Socket.Select( readSockets, writeSockets, errorSockets, 10 );
foreach( readSocket in readSockets ){
// do reading here
}
foreach( writeSocket in writeSockets ){
// do writing here
}
// POINT2 and here's the problem i will describe below
}
it works all smothly accept for 100% CPU utilization because of while loop being cycling all over again, if I have my clients doing send->receive->disconnect routine it's not that painful, but if I try to keep alive doing send->receive->send->receive all over again it really eats up all CPU. So my first idea was to put a sleep there, I check if all sockets have their data send and then putting Thread.Sleep in POINT2 just for 10ms, but this 10ms later on produces a huge delay of that 10ms when I want to receive next command from client socket.. For example if I don't try to "keep alive" commands are being executed within 10-15ms and with keep alive it becomes worse by atleast 10ms :(
Maybe it's just a poor architecture? What can be done so my processor won't get 100% utilization and my server to react on something appear in client socket as soon as possible? Maybe somebody can point a good example of nonblocking server and architecture it should maintain?

Take a look at the TcpListener class first. It has a BeginAccept method that will not block, and will call one of your functions when someone connects.
Also take a look at the Socket class and its Begin methods. These work the same way. One of your functions (a callback function) is called whenever a certain event fires, then you get to handle that event. All the Begin methods are asynchronous, so they will not block and they shouldn't use 100% CPU either. Basically you want BeginReceive for reading and BeginSend for writing I believe.
You can find more on google by searching for these methods and async sockets tutorials. Here's how to implement a TCP client this way for example. It works basically the same way even for your server.
This way you don't need any infinite looping, it's all event-driven.

Are you creating a peer-to-peer application or a client server application? You got to consider how much data you are putting through the sockets as well.
Asynchronous BeginSend and BeginReceive is the way to go, you will need to implement the events but it's fast once you get it right.
Probably don't want to set your Send and Receive timeouts too high as well, but there should be a timeout so that if nothing is receive after a certain time, it will come out of the block and you can handle it there.

Microsoft has a nice async TCP server example. It takes a bit to wrap your head around it. It was a few hours of my own time before I was able to create the basic TCP framework for my own program based on this example.
http://msdn.microsoft.com/en-us/library/fx6588te.aspx
The program logic goes kind of like this. There is one thread that calls listener.BeginAccept and then blocks on allDone.WaitOne. The BeginAccept is an async call which gets offloaded to the threadpool and handled by the OS. When a new connection comes in, the OS calls the callback method passed in from BeginAccept. That method flips allDone to let the main listening thread to know it can listen once again. The callback method is just a transitionary method and continues on to call yet another async call to receive data.
The callback method supplied, ReadCallback, is the primary work "loop"(effectively recursive async calls) for the async calls. I use the term "loop" loosely because each method calls actually finishes, but not before calling the next async method. Effectively, you have a bunch of async calls all calling each other and you pass around your "state" object. This object is your own object and you can do whatever you want with it.
Every callback method will only get two things returned when the OS calls your method:
1) Socket Object representing the connection
2) State object with which you use for your logic
With your state object and socket object, you can effectively handle your "connections" asynchronously. The OS is VERY good at this.
Also, because your main loop blocks waiting for a connection to come it and off-loads those connections to the thread pool via async calls, it remains idle most of the time. The thread pool for your sockets is handled by the OS via completion ports, so they don't do any real work until data comes in. Very little CPU is used and it's effectively threaded via the thread pool.
P.S. From what I understand, you don't want to do any hard work with these methods, just handling the movement of the data. Since the thread pool is the pool for your Network IO and is shared by other programs, you should offload any hard work via threads/tasks/async as to not cause the socket thread pool to get bogged down.
P.P.S. I haven't found a way of closing the listening connection other than just disposing "listener". Because the async call for beginListen is called, that method will never return until a connection comes in, which means, I can't tell it to stop until it returns. I think I'll post a question on MSDN about it. and link if I get a good response.

Everything is fine is your code exept timeout value. You set it to 10 microseconds (10*10^-6) so your while routine iterates very often. You should set and adequate value (10 seconds for example) and your code will not eat 100% CPU.
List<Socket> readSockets = new List<Socket>();
List<Socket> writeSockets = new List<Socket>();
List<Socket> errorSockets = new List<Socket>();
while( true ){
Socket.Select( readSockets, writeSockets, errorSockets, 10*1000*1000 );
foreach( readSocket in readSockets ){
// do reading here
}
foreach( writeSocket in writeSockets ){
// do writing here
}
// POINT2 and here's the problem i will describe below
}