Develop Reference

Making more remoting calls than threads by making synchronous methods async

I have a bunch of remoting calls that are all synchronous (3rd party library). Most of them take a lot of time so I'm not able to use them more often then about 5 to 10 times per second. This is too slow because I need to call them at least 3000 times every couple of minutes and many more if the services was stopped for some time. There is virtually no CPU work on the client. It gets the data, checks some simple conditions and makes another call that it has to wait for.
What would be the best way to make them async (call them in an async fashion - I guess I need some async wrapper) so that I can make more requests at the same time? Currently It's limited by the number of threads (which is four).
I was thinking about calling them with Task.Run but every article I read says it's for CPU bound work and that it uses thread-pool threads. If I get it correctly, with this approach I won't be able to break the thread limit, will I?. So which approach would actually fit best here?
What about Task.FromResult? Can I await such methods asynchronously in a greater number than there are threads?
public async Task<Data> GetDataTakingLotsOfTime(object id)
{
var data = remoting.GetData(id);
return await Task.FromResult(data);
}

I was thinking about calling them with Task.Run but every article I read says it's for CPU bound work and that it uses thread-pool threads.
Yes, but when you're stuck with a sync API then Task.Run() might be your lesser evil, especially on a Client.
Your current version of GetDataTakingLotsOfTime() is not really async. The FromResult() merely helps to suppress the Warning about that.
What about Task.FromResult? Can I await such methods asynchronously in a greater number than there are threads?
Not clear where your "number of threads" idea comes from but yes, starting a Task method and awaiting it later essentially runs it on the ThreadPool. But Task.Run is clearer in that respect.
Note that that does not depend on the async modifier of the method - async is an implementation detail, the caller only cares that it returns a Task.
Currently It's limited by the number of threads (which is four).
This needs some explaining. I don't get it.

You are executing a remote call, and your thread needs to wait idly for the result of the remote call. During this wait your thread could do useful things, like executing other remote calls.
Times when your thread is idly waiting for other processes to finish, like writing to a disk, querying a database or fetching information from the internet are typically situations where you'll see an async function next to a non-async function: Write and WriteAsync, Send and SendAsync.
If at the deepest level of your synchronous call you have access to an async version of the call, then your life would be easy. Alas it seems that you don't have such an async version.
Your proposed solution using Task.Run has the disadvantage of the overhead in starting a new thread (or running one from the thread pool).
You could lower this overhead by creating a workshop object. In the workshop, a dedicated thread (a worker), or several dedicated threads are waiting at one input point for an order to do something. The threads performs the task and posts the result at the output point.
Users of the workshop have one access point (front office?) where they post the request to do something, and await for the result.
For this I used System.Threading.Tasks.Dataflow.BufferBlock. Install Nuget package TPL Dataflow.
You can dedicate your workshop to accept only work to GetDataTakingLotsOfTime; I made my workshop generic: I accept every job that implements interface IWork:
interface IWork
{
void DoWork();
}
The WorkShop has two BufferBlocks: one to input work requests and one to output finished work. The workshop has a thread (or several threads) that wait at the input BufferBlock until a job arrives. Does the Work, and when finished posts the job to the output BufferBlock
class WorkShop
{
public WorkShop()
{
this.workRequests = new BufferBlock<IWork>();
this.finishedWork = new BufferBlock<IWork>();
this.frontOffice = new FrontOffice(this.workRequests, this.finishedWork);
}
private readonly BufferBlock<IWork> workRequests;
private readonly BufferBlock<IWork> finishedWork;
private readonly FrontOffice frontOffice;
public FrontOffice {get{return this.frontOffice;} }
public async Task StartWorkingAsync(CancellationToken token)
{
while (await this.workRequests.OutputAvailableAsync(token)
{ // some work request at the input buffer
IWork requestedWork = this.workRequests.ReceiveAsync(token);
requestedWork.DoWork();
this.FinishedWork.Post(requestedWork);
}
// if here: no work expected anymore:
this.FinishedWork.Complete();
}
// function to close the WorkShop
public async Task CloseShopAsync()
{
// signal that no more work is to be expected:
this.WorkRequests.Complete();
// await until the worker has finished his last job for the day:
await this.FinishedWork.Completion();
}
}
TODO: proper reaction on CancellationToken.CancellationRequested
TODO: proper reaction on exceptions thrown by work
TODO: decide whether to use several threads doing the work
FrontOffice has one async function, that accepts work, sends the work to the WorkRequests and awaits for the work to finish:
public async Task<IWork> OrderWorkAsync(IWork work, CancellationToken token)
{
await this.WorkRequests.SendAsync(work, token);
IWork finishedWork = await this.FinishedWork.ReceivedAsync(token);
return finishedWork;
}
So your process created a WorkShop object and starts one or more threads that will StartWorking.
Whenever any thread (inclusive your main thread) needs some work to be performed in async-await fashion:
Create An object that holds the input parameters and the DoWork function
Ask the WorkShop for the FrontOffice
await OrderWorkAsync
.
class InformationGetter : IWork
{
public int Id {get; set;} // the input Id
public Data FetchedData {get; private set;} // the result from Remoting.GetData(id);
public void DoWork()
{
this.FetchedData = remoting.GetData(this.Id);
}
}
Finally the Async version of your remote
async Task<Data> RemoteGetDataAsync(int id)
{
// create the job to get the information:
InformationGetter infoGetter = new InformationGetter() {Id = id};
// go to the front office of the workshop and order to do the job
await this.MyWorkShop.FrontOffice.OrderWorkAsync(infoGetter);
return infoGetter.FetchedData;
}

Receive concurrent asynchronous requests and process them one at a time

Background
We have a service operation that can receive concurrent asynchronous requests and must process those requests one at a time.
In the following example, the UploadAndImport(...) method receives concurrent requests on multiple threads, but its calls to the ImportFile(...) method must happen one at a time.
Layperson Description
Imagine a warehouse with many workers (multiple threads). People (clients) can send the warehouse many packages (requests) at the same time (concurrently). When a package comes in a worker takes responsibility for it from start to finish, and the person who dropped off the package can leave (fire-and-forget). The workers' job is to put each package down a small chute, and only one worker can put a package down a chute at a time, otherwise chaos ensues. If the person who dropped off the package checks in later (polling endpoint), the warehouse should be able to report on whether the package went down the chute or not.
Question
The question then is how to write a service operation that...
can receive concurrent client requests,
receives and processes those requests on multiple threads,
processes requests on the same thread that received the request,
processes requests one at a time,
is a one way fire-and-forget operation, and
has a separate polling endpoint that reports on request completion.
We've tried the following and are wondering two things:
Are there any race conditions that we have not considered?
Is there a more canonical way to code this scenario in C#.NET with a service oriented architecture (we happen to be using WCF)?
Example: What We Have Tried?
This is the service code that we have tried. It works though it feels like somewhat of a hack or kludge.
static ImportFileInfo _inProgressRequest = null;
static readonly ConcurrentDictionary<Guid, ImportFileInfo> WaitingRequests =
new ConcurrentDictionary<Guid, ImportFileInfo>();
public void UploadAndImport(ImportFileInfo request)
{
// Receive the incoming request
WaitingRequests.TryAdd(request.OperationId, request);
while (null != Interlocked.CompareExchange(ref _inProgressRequest, request, null))
{
// Wait for any previous processing to complete
Thread.Sleep(500);
}
// Process the incoming request
ImportFile(request);
Interlocked.Exchange(ref _inProgressRequest, null);
WaitingRequests.TryRemove(request.OperationId, out _);
}
public bool UploadAndImportIsComplete(Guid operationId) =>
!WaitingRequests.ContainsKey(operationId);
This is example client code.
private static async Task UploadFile(FileInfo fileInfo, ImportFileInfo importFileInfo)
{
using (var proxy = new Proxy())
using (var stream = new FileStream(fileInfo.FullName, FileMode.Open, FileAccess.Read))
{
importFileInfo.FileByteStream = stream;
proxy.UploadAndImport(importFileInfo);
}
await Task.Run(() => Poller.Poll(timeoutSeconds: 90, intervalSeconds: 1, func: () =>
{
using (var proxy = new Proxy())
{
return proxy.UploadAndImportIsComplete(importFileInfo.OperationId);
}
}));
}
It's hard to write a minimum viable example of this in a Fiddle, but here is a start that give a sense and that compiles.
As before, the above seems like a hack/kludge, and we are asking both about potential pitfalls in its approach and for alternative patterns that are more appropriate/canonical.

Simple solution using Producer-Consumer pattern to pipe requests in case of thread count restrictions.
You still have to implement a simple progress reporter or event. I suggest to replace the expensive polling approach with an asynchronous communication which is offered by Microsoft's SignalR library. It uses WebSocket to enable async behavior. The client and server can register their callbacks on a hub. Using RPC the client can now invoke server side methods and vice versa. You would post progress to the client by using the hub (client side). In my experience SignalR is very simple to use and very good documented. It has a library for all famous server side languages (e.g. Java).
Polling in my understanding is the totally opposite of fire-and-forget. You can't forget, because you have to check something based on an interval. Event based communication, like SignalR, is fire-and-forget since you fire and will get a reminder (cause you forgot). The "event side" will invoke your callback instead of you waiting to do it yourself!
Requirement 5 is ignored since I didn't get any reason. Waiting for a thread to complete would eliminate the fire and forget character.
private BlockingCollection<ImportFileInfo> requestQueue = new BlockingCollection<ImportFileInfo>();
private bool isServiceEnabled;
private readonly int maxNumberOfThreads = 8;
private Semaphore semaphore = new Semaphore(numberOfThreads);
private readonly object syncLock = new object();
public void UploadAndImport(ImportFileInfo request)
{
// Start the request handler background loop
if (!this.isServiceEnabled)
{
this.requestQueue?.Dispose();
this.requestQueue = new BlockingCollection<ImportFileInfo>();
// Fire and forget (requirement 4)
Task.Run(() => HandleRequests());
this.isServiceEnabled = true;
}
// Cache multiple incoming client requests (requirement 1) (and enable throttling)
this.requestQueue.Add(request);
}
private void HandleRequests()
{
while (!this.requestQueue.IsCompleted)
{
// Wait while thread limit is exceeded (some throttling)
this.semaphore.WaitOne();
// Process the incoming requests in a dedicated thread (requirement 2) until the BlockingCollection is marked completed.
Task.Run(() => ProcessRequest());
}
// Reset the request handler after BlockingCollection was marked completed
this.isServiceEnabled = false;
this.requestQueue.Dispose();
}
private void ProcessRequest()
{
ImportFileInfo request = this.requestQueue.Take();
UploadFile(request);
// You updated your question saying the method "ImportFile()" requires synchronization.
// This a bottleneck and will significantly drop performance, when this method is long running.
lock (this.syncLock)
{
ImportFile(request);
}
this.semaphore.Release();
}
Remarks:
BlockingCollection is a IDisposable
TODO: You have to "close" the BlockingCollection by marking it completed:
"BlockingCollection.CompleteAdding()" or it will loop indeterminately waiting for further requests. Maybe you introduce a additional request methods for the client to cancel and/ or to update the process and to mark adding to the BlockingCollection as completed. Or a timer that waits an idle time before marking it as completed. Or make your request handler thread block or spin.
Replace Take() and Add(...) with TryTake(...) and TryAdd(...) if you want cancellation support
Code is not tested
Your "ImportFile()" method is a bottleneck in your multi threading environment. I suggest to make it thread safe. In case of I/O that requires synchronization, I would cache the data in a BlockingCollection and then write them to I/O one by one.

The problem is that your total bandwidth is very small-- only one job can run at a time-- and you want to handle parallel requests. That means that queue time could vary wildly. It may not be the best choice to implement your job queue in-memory, as it would make your system much more brittle, and more difficult to scale out when your business grows.
A traditional, scaleable way to architect this would be:
An HTTP service to accept requests, load balanced/redundant, with no session state.
A SQL Server database to persist the requests in a queue, returning a persistent unique job ID.
A Windows service to process the queue, one job at a time, and mark jobs as complete. The worker process for the service would probably be single-threaded.
This solution requires you to choose a web server. A common choice is IIS running ASP.NET. On that platform, each request is guaranteed to be handled in a single-threaded manner (i.e. you don't need to worry about race conditions too much), but due to a feature called thread agility the request might end with a different thread, but in the original synchronization context, which means you will probably never notice unless you are debugging and inspecting thread IDs.

Given the constraints context of our system, this is the implementation we ended up using:
static ImportFileInfo _importInProgressItem = null;
static readonly ConcurrentQueue<ImportFileInfo> ImportQueue =
new ConcurrentQueue<ImportFileInfo>();
public void UploadAndImport(ImportFileInfo request) {
UploadFile(request);
ImportFileSynchronized(request);
}
// Synchronize the file import,
// because the database allows a user to perform only one write at a time.
private void ImportFileSynchronized(ImportFileInfo request) {
ImportQueue.Enqueue(request);
do {
ImportQueue.TryPeek(out var next);
if (null != Interlocked.CompareExchange(ref _importInProgressItem, next, null)) {
// Queue processing is already under way in another thread.
return;
}
ImportFile(next);
ImportQueue.TryDequeue(out _);
Interlocked.Exchange(ref _importInProgressItem, null);
}
while (ImportQueue.Any());
}
public bool UploadAndImportIsComplete(Guid operationId) =>
ImportQueue.All(waiting => waiting.OperationId != operationId);
This solution works well for the loads we are expecting. That load involves a maximum of about 15-20 concurrent PDF file uploads. The batch of up to 15-20 files tends to arrive all at once and then to go quiet for several hours until the next batch arrives.
Criticism and feedback is most welcome.

Understanding fire and forget when using infinite loops

Can someone tell me what the best practice/proper way of doing this is?
I'm also using WPF, not a console or ASP.NET.
Using Listener to accept clients and spin off a new "thread" for each client that handles all the I/O and Exception catching for that client.
Method 1: Fire and forget, and just throw it into a variable to get rid of the warning.
public static async Task Start(CancellationToken token)
{
m_server = TcpListener.Create(33777);
m_server.Start();
running = true;
clientCount = 0;
// TODO: Add try... catch
while (!token.IsCancellationRequested)
{
var client = await m_server.AcceptTcpClientAsync().ConfigureAwait(false);
Client c = new Client(client);
var _ = HandleClientAsync(c);
}
}
Here's the Client Handler code:
public static async Task HandleClientAsync(Client c)
{
// TODO: add try...catch
while (c.connected)
{
string data = await c.reader.ReadLineAsync();
// Now we will parse the data and update variables accordingly
// Just Regex and some parsing that updates variables
ParseAndUpdate(data);
}
}
Method 2: The same thing... but with Task.Run()
var _ = Task.Run(() => HandleClientAsync());
Method 3: an intermediate non async function (doubt this is good. Should be Async all the way)
But this at least gets rid of the squiggly line without using the variable trick which kinda feels dirty.
while (!token.IsCancellationRequested)
{
var client = await m_server.AcceptTcpClientAsync().ConfigureAwait(false);
Client c = new Client(client);
NonAsync(c);
}
public static void NonAsync(VClient vc)
{
Task.Run(() => HandleClientAsync(vc));
}
Method 4: Make HandleClientAsync an Async void instead of Async Task (really bad)
public static async Task HandleClientAsync(Client c)
// Would change to
public static async Void HandleClientAsync(Client c)
Questions:
Is it any better to use Task.Run() When doing a fire and forget task?
Is it just accepted that you need to use the var _ = FireAndForget() trick to do fire and forget? I could just ignore the warning but something feels wrong about it.
If I wanted to update my UI from a Client, how would I do that? Would I just use a dispatcher?
Thanks guys

I've never been a fan of background workers which you expect to run for a long time, being run in a task. Tasks get scheduled to run on threads drawn from a pool. As you schedule these long running tasks, the thread pool gets smaller and smaller. Eventually all of the threads from the pool are busy running your tasks, and things get really slow and unmanageable.
My recommendation here? Use the Thread class and manage them yourselves. In this way, you keep your thread pool and the overhead for for tasks out of the picture.
Addendum - Producer Consumer Model
Another interesting question to consider: Do you really need a thread for every client? Threads are reasonably costly to create and maintain in terms of memory overhead, and if your client interaction is such that the client threads spend the vast majority of their time waiting around on something to do, then perhaps a producer consumer model is more suited to your use case.
Example:
Client connects on listening thread, gets put in a client queue
Worker thread responsible for checking to see if the clients need anything comes along through that queue every so often and checks - does the client have a new message to service? If so, it services all messages the client has, then moves on
In this way, you limit the number of threads working to just the number needed to manage the message queue. You can even get fancy and add worker threads dynamically based on how long its been since all the clients have been serviced.
If you insist
If you really like what you have going, I suggest refactoring what youre doing a bit so that rather than HandleClientAsync you do something more akin to CreateServiceForClient(c);
This could be a synchronous method that returns something like a ClientService. ClientService could then create the task that does what your HandleClientAsync does now, and store that task as a member. It could also provide methods like
ClientService.WaitUntilEnd()
and
ClientService.Disconnect() (which could set a cancellation token, also stored as a member variable)

Wrapping synchronous code into asynchronous call

I have a method in ASP.NET application, that consumes quite a lot of time to complete. A call to this method might occur up to 3 times during one user request, depending on the cache state and parameters that user provides. Each call takes about 1-2 seconds to complete. The method itself is synchronous call to the service and there is no possibility to override the implementation.
So the synchronous call to the service looks something like the following:
public OutputModel Calculate(InputModel input)
{
// do some stuff
return Service.LongRunningCall(input);
}
And the usage of the method is (note, that call of method may happen more than once):
private void MakeRequest()
{
// a lot of other stuff: preparing requests, sending/processing other requests, etc.
var myOutput = Calculate(myInput);
// stuff again
}
I tried to change the implementation from my side to provide simultaneous work of this method, and here is what I came to so far.
public async Task<OutputModel> CalculateAsync(InputModel input)
{
return await Task.Run(() =>
{
return Calculate(input);
});
}
Usage (part of "do other stuff" code runs simultaneously with the call to service):
private async Task MakeRequest()
{
// do some stuff
var task = CalculateAsync(myInput);
// do other stuff
var myOutput = await task;
// some more stuff
}
My question: Do I use the right approach to speed up the execution in ASP.NET application or am I doing unnecessary job trying to run synchronous code asynchronously?
Can anyone explain why the second approach is not an option in ASP.NET (if it is really not)?
Also, if such approach is applicable, do I need to call such method asynchronously if it is the only call we might perform at the moment (I have such case, when no other stuff there is to do while waiting for completion)?
Most of the articles in the net on this topic covers using async-await approach with the code, that already provides awaitable methods, but that's not my case. Here is the nice article describing my case, which doesn't describe the situation of parallel calls, declining the option to wrap sync call, but in my opinion my situation is exactly the occasion to do it.

It's important to make a distinction between two different types of concurrency. Asynchronous concurrency is when you have multiple asynchronous operations in flight (and since each operation is asynchronous, none of them are actually using a thread). Parallel concurrency is when you have multiple threads each doing a separate operation.
The first thing to do is re-evaluate this assumption:
The method itself is synchronous call to the service and there is no possibility to override the implementation.
If your "service" is a web service or anything else that is I/O-bound, then the best solution is to write an asynchronous API for it.
I'll proceed with the assumption that your "service" is a CPU-bound operation that must execute on the same machine as the web server.
If that's the case, then the next thing to evaluate is another assumption:
I need the request to execute faster.
Are you absolutely sure that's what you need to do? Are there any front-end changes you can make instead - e.g., start the request and allow the user to do other work while it's processing?
I'll proceed with the assumption that yes, you really do need to make the individual request execute faster.
In this case, you'll need to execute parallel code on your web server. This is most definitely not recommended in general because the parallel code will be using threads that ASP.NET may need to handle other requests, and by removing/adding threads it will throw the ASP.NET threadpool heuristics off. So, this decision does have an impact on your entire server.
When you use parallel code on ASP.NET, you are making the decision to really limit the scalability of your web app. You also may see a fair amount of thread churn, especially if your requests are bursty at all. I recommend only using parallel code on ASP.NET if you know that the number of simultaneous users will be quite low (i.e., not a public server).
So, if you get this far, and you're sure you want to do parallel processing on ASP.NET, then you have a couple of options.
One of the easier methods is to use Task.Run, very similar to your existing code. However, I do not recommend implementing a CalculateAsync method since that implies the processing is asynchronous (which it is not). Instead, use Task.Run at the point of the call:
private async Task MakeRequest()
{
// do some stuff
var task = Task.Run(() => Calculate(myInput));
// do other stuff
var myOutput = await task;
// some more stuff
}
Alternatively, if it works well with your code, you can use the Parallel type, i.e., Parallel.For, Parallel.ForEach, or Parallel.Invoke. The advantage to the Parallel code is that the request thread is used as one of the parallel threads, and then resumes executing in the thread context (there's less context switching than the async example):
private void MakeRequest()
{
Parallel.Invoke(() => Calculate(myInput1),
() => Calculate(myInput2),
() => Calculate(myInput3));
}
I do not recommend using Parallel LINQ (PLINQ) on ASP.NET at all.

I found that the following code can convert a Task to always run asynchronously
private static async Task<T> ForceAsync<T>(Func<Task<T>> func)
{
await Task.Yield();
return await func();
}
and I have used it in the following manner
await ForceAsync(() => AsyncTaskWithNoAwaits())
This will execute any Task asynchronously so you can combine them in WhenAll, WhenAny scenarios and other uses.
You could also simply add the Task.Yield() as the first line of your called code.

this is probably the easiest generic way in your case
return await new Task(
new Action(
delegate () {
// put your synchronous code here
}
)
);

What do I do with async Tasks I don't want to wait for?

I am writing a multi player game server and am looking at ways the new C# async/await features can
help me. The core of the server is a loop which updates all the actors in the game as fast as it
can:
while (!shutdown)
{
foreach (var actor in actors)
actor.Update();
// Send and receive pending network messages
// Various other system maintenance
}
This loop is required to handle thousands of actors and update multiple times per second to keep the
game running smoothly. Some actors occasionally perform slow tasks in their update functions, such
as fetching data from a database, which is where I'd like to use async. Once this data is retrieved
the actor wants to update the game state, which must be done on the main thread.
As this is a console application, I plan to write a SynchronizationContext which can dispatch
pending delegates to the main loop. This allows those tasks to update the game once they complete
and lets unhandled exceptions be thrown into the main loop. My question is, how do write the async
update functions? This works very nicely, but breaks the recommendations not to use async void:
Thing foo;
public override void Update()
{
foo.DoThings();
if (someCondition) {
UpdateAsync();
}
}
async void UpdateAsync()
{
// Get data, but let the server continue in the mean time
var newFoo = await GetFooFromDatabase();
// Now back on the main thread, update game state
this.foo = newFoo;
}
I could make Update() async and propogate the tasks back to the main loop, but:
I don't want to add overhead to the thousands of updates that will never use it.
Even in the main loop I don't want to await the tasks and block the loop.
Awaiting the task would cause a deadlock anyway as it needs to complete on the awaiting thread.
What do I do with all these tasks I can't await? The only time I might want to know they've all
finished is when I'm shutting the server down, but I don't want to collect every task generated by
potentially weeks worth of updates.

My understanding is that the crux of it is that you want:
while (!shutdown)
{
//This should happen immediately and completions occur on the main thread.
foreach (var actor in actors)
actor.Update(); //includes i/o bound database operations
// The subsequent code should not be delayed
...
}
Where the while loop is running in your main console thread. This is a tight single-threaded loop. You could run the foreach in parallel, but then you would still be waiting for the longest running instance (the i/o bound operation to get the data from the database).
await async is not the best option within this loop, you need to run these i/o database tasks on a thread pool. On the thread pool async await would be useful to free up pool threads.
So, the next question is how to get these completions back to your main thread. Well, it seems like you need something equivalent to a message pump on your main thread. See this post for information on how to do that, though that may be a bit heavy handed. You could just have a completion queue of sorts that you check on the main thread in each pass through your while Loop. You would use one of the concurrent data structures to do this so that it is all thread safe then set Foo if it needs to be set.
It seems that there is some room to rationalise this polling of actors and threading, but without knowing the details of the app it is hard to say.
A couple of points: -
If you do not have a Wait higher up on a task, your main console thread will exit and so will your application. See here for details.
As you have pointed out, await async does not block the current thread, but it does mean that the code subsequent to the await will only execute on completion of the await.
The completion may or may not be completed on the calling thread. You have already mentioned Synchronization Context, so I won't go into the details.
Synchronization Context is null on a Console app. See here for information.
Async isn't really for fire-and-forget type operations.
For fire and forget you can use one of these options depending on your scenario:
Use Task.Run or Task.StartNew. See here for differences.
Use a producer/consumer type pattern for the long running scenarios running under your own threadpool.
Be aware of the following: -
That you will need to handle the exceptions in your spawned tasks / threads. If there are any exceptions that you do not observe, you may want to handle these, even just to log their occurence. See the information on unobserved exceptions.
If your process dies while these long running tasks are on the queue or starting they will not be run, so you may want some kind of persistence mechanism (database, external queue, file) that keeps track of the state of these operations.
If you want to know about the state of these tasks, then you will need to keep track of them in some way, whether it is an in memory list, or by querying the queues for your own thread pool or by querying the persistence mechanism. The nice thing about the persistence mechanism is that it is resilient to crashes and during shutdown you could just close down immediately, then pick up where you ended up when you restart (this of course depends on how critical it is that the tasks are run within a certain timeframe).

First, I recommend that you do not use your own SynchronizationContext; I have one available as part of my AsyncEx library that I commonly use for Console apps.
As far as your update methods go, they should return Task. My AsyncEx library has a number of "task constants" that are useful when you have a method that might be asynchronous:
public override Task Update() // Note: not "async"
{
foo.DoThings();
if (someCondition) {
return UpdateAsync();
}
else {
return TaskConstants.Completed;
}
}
async Task UpdateAsync()
{
// Get data, but let the server continue in the mean time
var newFoo = await GetFooFromDatabase();
// Now back on the main thread, update game state
this.foo = newFoo;
}
Returning to your main loop, the solution there isn't quite as clear. If you want every actor to complete before continuing to the next actor, then you can do this:
AsyncContext.Run(async () =>
{
while (!shutdown)
{
foreach (var actor in actors)
await actor.Update();
...
}
});
Alternatively, if you want to start all actors simultaneously and wait for them all to complete before moving to the next "tick", you can do this:
AsyncContext.Run(async () =>
{
while (!shutdown)
{
await Task.WhenAll(actors.Select(actor => actor.Update()));
...
}
});
When I say "simultaneously" above, it is actually starting each actor in order, and since they all execute on the main thread (including the async continuations), there's no actual simultaneous behavior; each "chuck of code" will execute on the same thread.

I highly recommend watching this video or just taking a look at the slides:
Three Essential Tips for Using Async in Microsoft Visual C# and Visual Basic
From my understanding what you should probably be doing in this scenario is returning Task<Thing> in UpdateAsync and possibly even Update.
If you are performing some async operations with 'foo' outside the main loop what happens when the async part completes during a future sequential update? I believe you really want to wait on all your update tasks to complete and then swap your internal state over in one go.
Ideally you would start all the slow (database) updates first and then do the other faster ones so that the entire set is ready as soon as possible.