Develop Reference

Receive concurrent asynchronous requests and process them one at a time

Background
We have a service operation that can receive concurrent asynchronous requests and must process those requests one at a time.
In the following example, the UploadAndImport(...) method receives concurrent requests on multiple threads, but its calls to the ImportFile(...) method must happen one at a time.
Layperson Description
Imagine a warehouse with many workers (multiple threads). People (clients) can send the warehouse many packages (requests) at the same time (concurrently). When a package comes in a worker takes responsibility for it from start to finish, and the person who dropped off the package can leave (fire-and-forget). The workers' job is to put each package down a small chute, and only one worker can put a package down a chute at a time, otherwise chaos ensues. If the person who dropped off the package checks in later (polling endpoint), the warehouse should be able to report on whether the package went down the chute or not.
Question
The question then is how to write a service operation that...
can receive concurrent client requests,
receives and processes those requests on multiple threads,
processes requests on the same thread that received the request,
processes requests one at a time,
is a one way fire-and-forget operation, and
has a separate polling endpoint that reports on request completion.
We've tried the following and are wondering two things:
Are there any race conditions that we have not considered?
Is there a more canonical way to code this scenario in C#.NET with a service oriented architecture (we happen to be using WCF)?
Example: What We Have Tried?
This is the service code that we have tried. It works though it feels like somewhat of a hack or kludge.
static ImportFileInfo _inProgressRequest = null;
static readonly ConcurrentDictionary<Guid, ImportFileInfo> WaitingRequests =
new ConcurrentDictionary<Guid, ImportFileInfo>();
public void UploadAndImport(ImportFileInfo request)
{
// Receive the incoming request
WaitingRequests.TryAdd(request.OperationId, request);
while (null != Interlocked.CompareExchange(ref _inProgressRequest, request, null))
{
// Wait for any previous processing to complete
Thread.Sleep(500);
}
// Process the incoming request
ImportFile(request);
Interlocked.Exchange(ref _inProgressRequest, null);
WaitingRequests.TryRemove(request.OperationId, out _);
}
public bool UploadAndImportIsComplete(Guid operationId) =>
!WaitingRequests.ContainsKey(operationId);
This is example client code.
private static async Task UploadFile(FileInfo fileInfo, ImportFileInfo importFileInfo)
{
using (var proxy = new Proxy())
using (var stream = new FileStream(fileInfo.FullName, FileMode.Open, FileAccess.Read))
{
importFileInfo.FileByteStream = stream;
proxy.UploadAndImport(importFileInfo);
}
await Task.Run(() => Poller.Poll(timeoutSeconds: 90, intervalSeconds: 1, func: () =>
{
using (var proxy = new Proxy())
{
return proxy.UploadAndImportIsComplete(importFileInfo.OperationId);
}
}));
}
It's hard to write a minimum viable example of this in a Fiddle, but here is a start that give a sense and that compiles.
As before, the above seems like a hack/kludge, and we are asking both about potential pitfalls in its approach and for alternative patterns that are more appropriate/canonical.

Simple solution using Producer-Consumer pattern to pipe requests in case of thread count restrictions.
You still have to implement a simple progress reporter or event. I suggest to replace the expensive polling approach with an asynchronous communication which is offered by Microsoft's SignalR library. It uses WebSocket to enable async behavior. The client and server can register their callbacks on a hub. Using RPC the client can now invoke server side methods and vice versa. You would post progress to the client by using the hub (client side). In my experience SignalR is very simple to use and very good documented. It has a library for all famous server side languages (e.g. Java).
Polling in my understanding is the totally opposite of fire-and-forget. You can't forget, because you have to check something based on an interval. Event based communication, like SignalR, is fire-and-forget since you fire and will get a reminder (cause you forgot). The "event side" will invoke your callback instead of you waiting to do it yourself!
Requirement 5 is ignored since I didn't get any reason. Waiting for a thread to complete would eliminate the fire and forget character.
private BlockingCollection<ImportFileInfo> requestQueue = new BlockingCollection<ImportFileInfo>();
private bool isServiceEnabled;
private readonly int maxNumberOfThreads = 8;
private Semaphore semaphore = new Semaphore(numberOfThreads);
private readonly object syncLock = new object();
public void UploadAndImport(ImportFileInfo request)
{
// Start the request handler background loop
if (!this.isServiceEnabled)
{
this.requestQueue?.Dispose();
this.requestQueue = new BlockingCollection<ImportFileInfo>();
// Fire and forget (requirement 4)
Task.Run(() => HandleRequests());
this.isServiceEnabled = true;
}
// Cache multiple incoming client requests (requirement 1) (and enable throttling)
this.requestQueue.Add(request);
}
private void HandleRequests()
{
while (!this.requestQueue.IsCompleted)
{
// Wait while thread limit is exceeded (some throttling)
this.semaphore.WaitOne();
// Process the incoming requests in a dedicated thread (requirement 2) until the BlockingCollection is marked completed.
Task.Run(() => ProcessRequest());
}
// Reset the request handler after BlockingCollection was marked completed
this.isServiceEnabled = false;
this.requestQueue.Dispose();
}
private void ProcessRequest()
{
ImportFileInfo request = this.requestQueue.Take();
UploadFile(request);
// You updated your question saying the method "ImportFile()" requires synchronization.
// This a bottleneck and will significantly drop performance, when this method is long running.
lock (this.syncLock)
{
ImportFile(request);
}
this.semaphore.Release();
}
Remarks:
BlockingCollection is a IDisposable
TODO: You have to "close" the BlockingCollection by marking it completed:
"BlockingCollection.CompleteAdding()" or it will loop indeterminately waiting for further requests. Maybe you introduce a additional request methods for the client to cancel and/ or to update the process and to mark adding to the BlockingCollection as completed. Or a timer that waits an idle time before marking it as completed. Or make your request handler thread block or spin.
Replace Take() and Add(...) with TryTake(...) and TryAdd(...) if you want cancellation support
Code is not tested
Your "ImportFile()" method is a bottleneck in your multi threading environment. I suggest to make it thread safe. In case of I/O that requires synchronization, I would cache the data in a BlockingCollection and then write them to I/O one by one.

The problem is that your total bandwidth is very small-- only one job can run at a time-- and you want to handle parallel requests. That means that queue time could vary wildly. It may not be the best choice to implement your job queue in-memory, as it would make your system much more brittle, and more difficult to scale out when your business grows.
A traditional, scaleable way to architect this would be:
An HTTP service to accept requests, load balanced/redundant, with no session state.
A SQL Server database to persist the requests in a queue, returning a persistent unique job ID.
A Windows service to process the queue, one job at a time, and mark jobs as complete. The worker process for the service would probably be single-threaded.
This solution requires you to choose a web server. A common choice is IIS running ASP.NET. On that platform, each request is guaranteed to be handled in a single-threaded manner (i.e. you don't need to worry about race conditions too much), but due to a feature called thread agility the request might end with a different thread, but in the original synchronization context, which means you will probably never notice unless you are debugging and inspecting thread IDs.

Given the constraints context of our system, this is the implementation we ended up using:
static ImportFileInfo _importInProgressItem = null;
static readonly ConcurrentQueue<ImportFileInfo> ImportQueue =
new ConcurrentQueue<ImportFileInfo>();
public void UploadAndImport(ImportFileInfo request) {
UploadFile(request);
ImportFileSynchronized(request);
}
// Synchronize the file import,
// because the database allows a user to perform only one write at a time.
private void ImportFileSynchronized(ImportFileInfo request) {
ImportQueue.Enqueue(request);
do {
ImportQueue.TryPeek(out var next);
if (null != Interlocked.CompareExchange(ref _importInProgressItem, next, null)) {
// Queue processing is already under way in another thread.
return;
}
ImportFile(next);
ImportQueue.TryDequeue(out _);
Interlocked.Exchange(ref _importInProgressItem, null);
}
while (ImportQueue.Any());
}
public bool UploadAndImportIsComplete(Guid operationId) =>
ImportQueue.All(waiting => waiting.OperationId != operationId);
This solution works well for the loads we are expecting. That load involves a maximum of about 15-20 concurrent PDF file uploads. The batch of up to 15-20 files tends to arrive all at once and then to go quiet for several hours until the next batch arrives.
Criticism and feedback is most welcome.

Understanding fire and forget when using infinite loops

Can someone tell me what the best practice/proper way of doing this is?
I'm also using WPF, not a console or ASP.NET.
Using Listener to accept clients and spin off a new "thread" for each client that handles all the I/O and Exception catching for that client.
Method 1: Fire and forget, and just throw it into a variable to get rid of the warning.
public static async Task Start(CancellationToken token)
{
m_server = TcpListener.Create(33777);
m_server.Start();
running = true;
clientCount = 0;
// TODO: Add try... catch
while (!token.IsCancellationRequested)
{
var client = await m_server.AcceptTcpClientAsync().ConfigureAwait(false);
Client c = new Client(client);
var _ = HandleClientAsync(c);
}
}
Here's the Client Handler code:
public static async Task HandleClientAsync(Client c)
{
// TODO: add try...catch
while (c.connected)
{
string data = await c.reader.ReadLineAsync();
// Now we will parse the data and update variables accordingly
// Just Regex and some parsing that updates variables
ParseAndUpdate(data);
}
}
Method 2: The same thing... but with Task.Run()
var _ = Task.Run(() => HandleClientAsync());
Method 3: an intermediate non async function (doubt this is good. Should be Async all the way)
But this at least gets rid of the squiggly line without using the variable trick which kinda feels dirty.
while (!token.IsCancellationRequested)
{
var client = await m_server.AcceptTcpClientAsync().ConfigureAwait(false);
Client c = new Client(client);
NonAsync(c);
}
public static void NonAsync(VClient vc)
{
Task.Run(() => HandleClientAsync(vc));
}
Method 4: Make HandleClientAsync an Async void instead of Async Task (really bad)
public static async Task HandleClientAsync(Client c)
// Would change to
public static async Void HandleClientAsync(Client c)
Questions:
Is it any better to use Task.Run() When doing a fire and forget task?
Is it just accepted that you need to use the var _ = FireAndForget() trick to do fire and forget? I could just ignore the warning but something feels wrong about it.
If I wanted to update my UI from a Client, how would I do that? Would I just use a dispatcher?
Thanks guys

I've never been a fan of background workers which you expect to run for a long time, being run in a task. Tasks get scheduled to run on threads drawn from a pool. As you schedule these long running tasks, the thread pool gets smaller and smaller. Eventually all of the threads from the pool are busy running your tasks, and things get really slow and unmanageable.
My recommendation here? Use the Thread class and manage them yourselves. In this way, you keep your thread pool and the overhead for for tasks out of the picture.
Addendum - Producer Consumer Model
Another interesting question to consider: Do you really need a thread for every client? Threads are reasonably costly to create and maintain in terms of memory overhead, and if your client interaction is such that the client threads spend the vast majority of their time waiting around on something to do, then perhaps a producer consumer model is more suited to your use case.
Example:
Client connects on listening thread, gets put in a client queue
Worker thread responsible for checking to see if the clients need anything comes along through that queue every so often and checks - does the client have a new message to service? If so, it services all messages the client has, then moves on
In this way, you limit the number of threads working to just the number needed to manage the message queue. You can even get fancy and add worker threads dynamically based on how long its been since all the clients have been serviced.
If you insist
If you really like what you have going, I suggest refactoring what youre doing a bit so that rather than HandleClientAsync you do something more akin to CreateServiceForClient(c);
This could be a synchronous method that returns something like a ClientService. ClientService could then create the task that does what your HandleClientAsync does now, and store that task as a member. It could also provide methods like
ClientService.WaitUntilEnd()
and
ClientService.Disconnect() (which could set a cancellation token, also stored as a member variable)

FIFO programmed main thread dispatcher?

To begin with, I'm using unity. Which makes me stuck with .NET 3.5. I'm currently working on a server program which uses the Socket object's asynchronous methods (E.G. BeginReceive, BeginAccept, BeginReceiveFrom etc.). When the server receives a packet from a client, this packet is received on a worker thread. Now I'm left with some data on a worker thread, and I want the main thread to process this data using a function that I specify. I implemented that:
using System;
using System.Threading;
using System.Collections;
using System.Collections.Generic;
public class MyDispatcherClass
{
public delegate void MyDel();
private readonly Queue<MyDel> commands = new Queue<MyDel>();
Object lockObj = new object ();
public void Add(MyDel dc)
{
lock (lockObj)
{
commands.Enqueue (dc);
}
}
public void Invoke()
{
lock (lockObj)
{
while (commands.Count > 0)
{
commands.Dequeue().Invoke();
}
}
}
}
Then I would use it this way:
// As a global variable:
MyDispatcherClass SomeDispatcher = new MyDispatcherClass ();
//The function that I want to call:
public void MyFunction (byte[] data)
{
// Do some stuff on the main thread
}
//When I receive a message on a worker thread I do that:
SomeDispatcher.Add (()=> MyFunction (byte[] data)); //Asuume that "data" is the message I received from a client
//Each frame on the main thread I call:
SomeDispatcher.Invoke ();
After some research, I found that the lock statement does not guarantee a %100 FIFO implementation. Which is not what I wanted, sometimes this may cause a total server breakdown! I want to achieve the same result with a %100 guarantee that data will be processed in the same order it was received from a client. How could I accomplish that?

Threads will run in whatever order they want, so you can't force the order going into the queue. But you can put in more data into the queue than just what you will eventually be processing.
If you add a DateTime, (or even just an int with a specified order) to the data being sent you can sort your queue on that when you pull data from it, (and possibly not pull any data less than 0.5 seconds old to give time for other threads to write their data.)
Normally when dealing with client-server relationships each thread represents one client so you don't have to worry about this as commands are FIFO within the thread, (although they might not be when 2 different clients are sending messages.)
Do you close and re-open the socket on the same client? that could make it use different threads. If you need a specific order and are sending things fairly soon after each other it might be better to leave the socket open.

Understanding async/await to manage multiple clients

I'm getting confused by await/async as I may still not get the point of its correct usage.
I have a simple WPF-UI and a ViewModel-method to start listening for clients which want to connect.
The following method is executed when the user clicks the button to start listening:
public void StartListening()
{
_tcpListener.Start(); // TcpListener
IsListening = true; // bool
Task.Factory.StartNew(DoStartListeningAsync, TaskCreationOptions.LongRunning);
}
The method DoStartListeningAsync which is called is defined like
private async Task DoStartListeningAsync()
{
while (IsListening)
{
using (var newClient = await _tcpListener.AcceptTcpClientAsync() /*.WithWaitCancellation(_cts.Token)*/)
{
apiClient = new ApiClient();
if(await apiClient.InitClientAsync()) // <-- here is the problem
{
// ... apiClient is now initialized
}
// ... do more and go back to await _tcpListener.AcceptTcpClientAsync()
}
}
}
The ApiClient class' InitClientAsync method is defined like:
public async Task<bool> InitClientAsync()
{
using (var requestStream = await _apiWebRequest.GetRequestStreamAsync())
{
_apiStreamWriter = new StreamWriter(requestStream);
}
// ... do somehing with the _apiStreamWriter
return true;
}
However, sometimes the InitClientAsync-call will get stuck at await _apiWebRequest.GetRequestStreamAsync() which then will freeze the execution of the DoStartListeningAsync-method at // <-- here is the problem.
In case the DoStartListeningAsync is stuck, no new connections will be handled which destroys my whole concept of handling multiple clients asynchronously.

Since you are using "await" keyword along the code path, you won't actually serve
multiple clients asynchronously.
The thing is, your code in the background thread will serve clients one by one. Take a deeper look - in the while loop you are getting request stream, wait till it is loaded, serve it, and then wait for other request stream.
async/await principle doesn't itself provide ability to serve multiple actions at the time. The only thing it is doing - prevents blocking current thread from being reusable by other code. So if going with async/await, you allow system yo use your current task thread, while it is waiting to other async action to complete (like _apiWebRequest.GetRequestStreamAsync()).
But since you are having one task, and you are waiting on every iteration of while loop - your code will work the same way, if you wrote it completely synchronous. The only profit is that you are using Task, and so .Net can reuse it's thread from thread pool while you are waiting for async actions to complete.
If you wan't to serve multiple clients asynchronously, you should either start multiple tasks, or don't wait till request is completely served - so actually remove some awaits from your code.
So you should move towards design, there you have one listening task/thread, that does nothing exept reading requests and putting it to the some queue. And having other tasks, that serve requests, reading it from the queue.
If I understood you correctly, you are using TcpListener under the hood. So what you need, is the loop where you accept new clients, and start serving them in the different thread/task without any waiting, so going directly to accepting other clients. But you can and should use async/await inside those handlers that serve clients.
Take a look at this answer - not completely your case (since I don't know all details of implementation), but just to get the idea.

What is the async/await equivalent of a ThreadPool server?

I am working on a tcp server that looks something like this using synchronous apis and the thread pool:
TcpListener listener;
void Serve(){
while(true){
var client = listener.AcceptTcpClient();
ThreadPool.QueueUserWorkItem(this.HandleConnection, client);
//Or alternatively new Thread(HandleConnection).Start(client)
}
}
Assuming my goal is to handle as many concurrent connections as possible with the lowest resource usage, this seems that it will be quickly limited by the number of available threads. I suspect that by using Non-blocking Task apis, I will be able to handle much more with fewer resources.
My initial impression is something like:
async Task Serve(){
while(true){
var client = await listener.AcceptTcpClientAsync();
HandleConnectionAsync(client); //fire and forget?
}
}
But it strikes me that this could cause bottlenecks. Perhaps HandleConnectionAsync will take an unusually long time to hit the first await, and will stop the main accept loop from proceeding. Will this only use one thread ever, or will the runtime magically run things on multiple threads as it sees fit?
Is there a way to combine these two approaches so that my server will use exactly the number of threads it needs for the number of actively running tasks, but so that it will not block threads unnecessarily on IO operations?
Is there an idiomatic way to maximize throughput in a situation like this?

I'd let the Framework manage the threading and wouldn't create any extra threads, unless profiling tests suggest I might need to. Especially, if the calls inside HandleConnectionAsync are mostly IO-bound.
Anyway, if you like to release the calling thread (the dispatcher) at the beginning of HandleConnectionAsync, there's a very easy solution. You can jump on a new thread from ThreadPool with await Yield(). That works if you server runs in the execution environment which does not have any synchronization context installed on the initial thread (a console app, a WCF service), which is normally the case for a TCP server.
The following illustrate this (the code is originally from here). Note, the main while loop doesn't create any threads explicitly:
using System;
using System.Collections.Generic;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;
class Program
{
object _lock = new Object(); // sync lock
List<Task> _connections = new List<Task>(); // pending connections
// The core server task
private async Task StartListener()
{
var tcpListener = TcpListener.Create(8000);
tcpListener.Start();
while (true)
{
var tcpClient = await tcpListener.AcceptTcpClientAsync();
Console.WriteLine("[Server] Client has connected");
var task = StartHandleConnectionAsync(tcpClient);
// if already faulted, re-throw any error on the calling context
if (task.IsFaulted)
await task;
}
}
// Register and handle the connection
private async Task StartHandleConnectionAsync(TcpClient tcpClient)
{
// start the new connection task
var connectionTask = HandleConnectionAsync(tcpClient);
// add it to the list of pending task
lock (_lock)
_connections.Add(connectionTask);
// catch all errors of HandleConnectionAsync
try
{
await connectionTask;
// we may be on another thread after "await"
}
catch (Exception ex)
{
// log the error
Console.WriteLine(ex.ToString());
}
finally
{
// remove pending task
lock (_lock)
_connections.Remove(connectionTask);
}
}
// Handle new connection
private async Task HandleConnectionAsync(TcpClient tcpClient)
{
await Task.Yield();
// continue asynchronously on another threads
using (var networkStream = tcpClient.GetStream())
{
var buffer = new byte[4096];
Console.WriteLine("[Server] Reading from client");
var byteCount = await networkStream.ReadAsync(buffer, 0, buffer.Length);
var request = Encoding.UTF8.GetString(buffer, 0, byteCount);
Console.WriteLine("[Server] Client wrote {0}", request);
var serverResponseBytes = Encoding.UTF8.GetBytes("Hello from server");
await networkStream.WriteAsync(serverResponseBytes, 0, serverResponseBytes.Length);
Console.WriteLine("[Server] Response has been written");
}
}
// The entry point of the console app
static async Task Main(string[] args)
{
Console.WriteLine("Hit Ctrl-C to exit.");
await new Program().StartListener();
}
}
Alternatively, the code might look like below, without await Task.Yield(). Note, I pass an async lambda to Task.Run, because I still want to benefit from async APIs inside HandleConnectionAsync and use await in there:
// Handle new connection
private static Task HandleConnectionAsync(TcpClient tcpClient)
{
return Task.Run(async () =>
{
using (var networkStream = tcpClient.GetStream())
{
var buffer = new byte[4096];
Console.WriteLine("[Server] Reading from client");
var byteCount = await networkStream.ReadAsync(buffer, 0, buffer.Length);
var request = Encoding.UTF8.GetString(buffer, 0, byteCount);
Console.WriteLine("[Server] Client wrote {0}", request);
var serverResponseBytes = Encoding.UTF8.GetBytes("Hello from server");
await networkStream.WriteAsync(serverResponseBytes, 0, serverResponseBytes.Length);
Console.WriteLine("[Server] Response has been written");
}
});
}
Updated, based upon the comment: if this is going to be a library code, the execution environment is indeed unknown, and may have a non-default synchronization context. In this case, I'd rather run the main server loop on a pool thread (which is free of any synchronization context):
private static Task StartListener()
{
return Task.Run(async () =>
{
var tcpListener = TcpListener.Create(8000);
tcpListener.Start();
while (true)
{
var tcpClient = await tcpListener.AcceptTcpClientAsync();
Console.WriteLine("[Server] Client has connected");
var task = StartHandleConnectionAsync(tcpClient);
if (task.IsFaulted)
await task;
}
});
}
This way, all child tasks created inside StartListener wouldn't be affected by the synchronization context of the client code. So, I wouldn't have to call Task.ConfigureAwait(false) anywhere explicitly.
Updated in 2020, someone just asked a good question off-site:
I was wondering what is the reason for using a lock here? This is not
necessary for exception handling. My understanding is that a lock is
used because List is not thread safe, therefore the real question
is why add the tasks to a list (and incur the cost of a lock under
load).
Since Task.Run is perfectly able to keep track of the tasks it
started, my thinking is that in this specific example the lock is
useless, however you put it there because in a real program, having
the tasks in a list allows us to for example, iterate currently
running tasks and terminate the tasks cleanly if the program receives
a termination signal from the operating system.
Indeed, in a real-life scenario we almost always want to keep track of the tasks we start with Task.Run (or any other Task objects which are "in-flight"), for a few reasons:
To track task exceptions, which otherwise might be silently swallowed if go unobserved elsewhere.
To be able to wait asynchronously for completion of all the pending tasks (e.g., consider a Start/Stop UI button or handling a request to start/stop a inside a headless Windows service).
To be able to control (and throttle/limit) the number of tasks we allow to be in-flight simultaneously.
There are better mechanisms to handle a real-life concurrency workflows (e.g., TPL Dataflow Library), but I did include the tasks list and the lock on purpose here, even in this simple example. It might be tempting to use a fire-and-forget approach, but it's almost never is a good idea. In my own experience, when I did want a fire-and-forget, I used async void methods for that (check this).

The existing answers have correctly proposed to use Task.Run(() => HandleConnection(client));, but not explained why.
Here's why: You are concerned, that HandleConnectionAsync might take some time to hit the first await. If you stick to using async IO (as you should in this case) this means that HandleConnectionAsync is doing CPU-bound work without any blocking. This is a perfect case for the thread-pool. It is made to run short, non-blocking CPU work.
And you are right, that the accept loop would be throttled by HandleConnectionAsync taking a long time before returning (maybe because there is significant CPU-bound work in it). This is to be avoided if you need a high frequency of new connections.
If you are sure that there is no significant work throttling the loop you can save the additional thread-pool Task and not do it.
Alternatively, you can have multiple accepts running at the same time. Replace await Serve(); by (for example):
var serverTasks =
Enumerable.Range(0, Environment.ProcessorCount)
.Select(_ => Serve());
await Task.WhenAll(serverTasks);
This removes the scalability problems. Note, that await will swallow all but one error here.

Try
TcpListener listener;
void Serve(){
while(true){
var client = listener.AcceptTcpClient();
Task.Run(() => this.HandleConnection(client));
//Or alternatively new Thread(HandleConnection).Start(client)
}
}

According to the Microsoft http://msdn.microsoft.com/en-AU/library/hh524395.aspx#BKMK_VoidReturnType, the void return type shouldn't be used because it is not able to catch exceptions. As you have pointed out you do need "fire and forget" tasks, so my conclusion is to that you must always return Task (as Microsoft have said), but you should catch the error using:
TaskInstance.ContinueWith(i => { /* exception handler */ }, TaskContinuationOptions.OnlyOnFaulted);
An example I used as proof is below:
public static void Main()
{
Awaitable()
.ContinueWith(
i =>
{
foreach (var exception in i.Exception.InnerExceptions)
{
Console.WriteLine(exception.Message);
}
},
TaskContinuationOptions.OnlyOnFaulted);
Console.WriteLine("This needs to come out before my exception");
Console.ReadLine();
}
public static async Task Awaitable()
{
await Task.Delay(3000);
throw new Exception("Hey I can catch these pesky things");
}

Is there any reason you need to accept connections async? I mean, does awaiting any client connection give you any value? The only reason for doing it would be because there are some other work going on in the server while waiting for a connection. If there is you could probably do something like this:
public async void Serve()
{
while (true)
{
var client = await _listener.AcceptTcpClientAsync();
Task.Factory.StartNew(() => HandleClient(client), TaskCreationOptions.LongRunning);
}
}
This way the accepting will release the current thread leaving option for other things to be done, and the handling is run on a new thread. The only overhead would be spawning a new thread for handling the client before it would go straight back to accepting a new connection.
Edit:
Just realized it's almost the same code you wrote. Think I need to read your question again to better understand what you're actually asking :S
Edit2:
Is there a way to combine these two approaches so that my server will use exactly the
number of threads it needs for the number of actively running tasks, but so that it will
not block threads unnecessarily on IO operations?
Think my solution actually answer this question. Is it really necessary though?
Edit3:
Made Task.Factory.StartNew() actually create a new thread.