The last couple of days I've been reading about async/await. Yesterday I found this video on Channel 9 what made wonder about some things. Please consider the slide below.
Aside from the issues Lucian Wischik addresses, I wondered about variable assignment. Say we changed the async void into async Task, and added await before the SendData call. This enables us to get the stream, assign the variable m_GetResponse, wait for two seconds and print it. But what happens to the variable? It could be written by a different thread than it is read. Do we need some sort of a memory barrier here, make the variable volatile, or perhaps something else? Could it still be null when we print it?
In the above example, it is safe to read the m_GetResponse because assignment will happen in same UI thread given that this is being called from UI.
This is because SynchronizationContext will be captured and continued when the async method resumes. So it is the same UI thread which writes the field and reading it. This isn't a problem here. Refer my related answer here
If called from non UI context, there is no guarantee that continuations will be ran in same thread. Usually it will be ran in ThreadPool thread. Given that the field read isn't volatile it is possible that you could get the previous value if necessary barriers are not inserted. But you don't need to worry about it because TPL already does this for you.
From the above link
Yes, TPL includes the appropriate barriers when tasks are queued and
at the beginning/end of task execution so that values are
appropriately made visible
So with TPL, you don't need to worry about memory barriers given that the Tasks are already completed. But if you're creating threads manually(which you shouldn't be doing) and dealing with threads directly --you'll have to insert necessary memory barriers.
Btw, ReadToEnd is a blocking call. I would not call it in UI thread. I'd use ReadToEndAsync instead to make your UI thread free. And I'll not use field here; I'll return the value from async method because every method call is just dependent on the parameter so it makes sense to return the value from the method.
So, your method will become something like the following
private async Task<string> SendDataAsync(string url)
{
var request = WebRequest.Create(url);
using(var response = await request.GetResponseAsync());
using(var reader = new StreamReader(request.GetResponseStream());
return await reader.ReadToEndAsync();
}
But what happens to the variable? It could be written by a different
thread than it is read.
If m_GetResponse is a private field, and this class gets invoked multiple times by different threads, then yes, it is possible for the value to be "dirty" once someone else tries to read it. In order to make it thread-safe, you could lock around it. It seems that the authors intention was to invoke this from the UI thread only, hence him making SendData a private method. In that case, it is safe for m_GetResponse to be a private field, as the continuation of the async method which is responsible for the variable assignment will happen inside the UI message loop.
Could it still be null when we print it?
It could be null if somewhere else in the code, someone sets that variable to null, as it's a class level variable. If you're talking about "could it be that we try to print m_GetResponse before await finishes the state-machine execution, then no. Again, i'm not sure the authors intentions were made around concurrent execution, but rather to show you async-await features.
or perhaps something else?
In order to make it thread safe, you can simply remove the global variable, and return a local variable instead. SendData shouldn't be async void anyway, as it isn't used for event handler delegate assignment like Button1_Click.
You could do it better like this (i'll use HttpClient for simplicity):
public async Task<string> SendDataAsync(string url)
{
var httpClient = new HttpClient();
var response = await httpClient.GetAsync();
return response.Content.ReadAsStringAsync();
}
Note you should remember that async-await isn't meant to address parallelism, it's more about concurrency and easing the use of naturally async IO operations.
Related
Consider the following async function that modifies a non-thread-safe list:
async Task AddNewToList(List<Item> list)
{
// Suppose load takes a few seconds
Item item = await LoadNextItem();
list.Add(item);
}
Simply put: Is this safe?
My concern is that one may invoke the async method, and then while it's loading (either on another thread, or as an I/O operation), the caller may modify the list.
Suppose that the caller is partway through the execution of list.Clear(), for example, and suddenly the Load method finishes! What will happen?
Will the task immediately interrupt and run the list.Add(item); code? Or will it wait until the main thread is done with all scheduled CPU tasks (ie: wait for Clear() to finish), before running the code?
Edit: Since I've basically answered this for myself below, here's a bonus question: Why? Why does it immediately interrupt instead of waiting for CPU bound operations to complete? It seems counter-intuitive to not queue itself up, which would be completely safe.
Edit: Here's a different example I tested myself. The comments indicate the order of execution. I am disappointed!
TaskCompletionSource<bool> source;
private async void buttonPrime_click(object sender, EventArgs e)
{
source = new TaskCompletionSource<bool>(); // 1
await source.Task; // 2
source = null; // 4
}
private void buttonEnd_click(object sender, EventArgs e)
{
source.SetResult(true); // 3
MessageBox.Show(source.ToString()); // 5 and exception is thrown
}
No, its not safe. However also consider that the caller might also have spawned a thread and passed the List to its child thread before calling your code, even in a non async environment, which will have the same detrimental effect.
So; although not safe, there is nothing inherently thread-safe about receiving a List from a caller anyway - there is no way of knowing whether the list is actually being processed from other threads that your own.
Short answer
You always need to be careful using async.
Longer answer
It depends on your SynchronizationContext and TaskScheduler, and what you mean by "safe."
When your code awaits something, it creates a continuation and wraps it in a task, which is then posted to the current SynchronizationContext's TaskScheduler. The context will then determine when and where the continuation will run. The default scheduler simply uses the thread pool, but different types of applications can extend the scheduler and provide more sophisticated synchronization logic.
If you are writing an application that has no SynchronizationContext (for example, a console application, or anything in .NET core), the continuation is simply put on the thread pool, and could execute in parallel with your main thread. In this case you must use lock or synchronized objects such as ConcurrentDictionary<> instead of Dictionary<>, for anything other than local references or references that are closed with the task.
If you are writing a WinForms application, the continuations are put in the message queue, and will all execute on the main thread. This makes it safe to use non-synchronized objects. However, there are other worries, such as deadlocks. And of course if you spawn any threads, you must make sure they use lock or Concurrent objects, and any UI invocations must be marshaled back to the UI thread. Also, if you are nutty enough to write a WinForms application with more than one message pump (this is highly unusual) you'd need to worry about synchronizing any common variables.
If you are writing an ASP.NET application, the SynchronizationContext will ensure that, for a given request, no two threads are executing at the same time. Your continuation might run on a different thread (due to a performance feature known as thread agility), but they will always have the same SynchronizationContext and you are guaranteed that no two threads will access your variables at the same time (assuming, of course, they are not static, in which case they span across HTTP requests and must be synchronized). In addition, the pipeline will block parallel requests for the same session so that they execute in series, so your session state is also protected from threading concerns. However you still need to worry about deadlocks.
And of course you can write your own SynchronizationContext and assign it to your threads, meaning that you specify your own synchronization rules that will be used with async.
See also How do yield and await implement flow of control in .NET?
Assuming the "invalid acces" occures in LoadNextItem(): The Task will throw an exception. Since the context is captured it will pass on to the callers thread so list.Add will not be reached.
So, no it's not thread-safe.
Yes I think that could be a problem.
I would return item and add to the list on the main tread.
private async void GetIntButton(object sender, RoutedEventArgs e)
{
List<int> Ints = new List<int>();
Ints.Add(await GetInt());
}
private async Task<int> GetInt()
{
await Task.Delay(100);
return 1;
}
But you have to call from and async so I do not this this would work either.
I just encountered this code. I immediately started cringing and talking to myself (not nice things). The thing is I don't really understand why and can't reasonably articulate it. It just looks really bad to me - maybe I'm wrong.
public async Task<IHttpActionResult> ProcessAsync()
{
var userName = Username.LogonName(User.Identity.Name);
var user = await _user.GetUserAsync(userName);
ThreadPool.QueueUserWorkItem((arg) =>
{
Task.Run(() => _billing.ProcessAsync(user)).Wait();
});
return Ok();
}
This code looks to me like it's needlessly creating threads with ThreadPool.QueueUserWorkItem and Task.Run. Plus, it looks like it has the potential to deadlock or create serious resource issues when under heavy load. Am I correct?
The _billing.ProcessAsync() method is awaitable(async), so I would expect that a simple "await" keyword would be the right thing to do and not all this other baggage.
I believe Scott is correct with his guess that ThreadPool.QueueUserWorkItem should have been HostingEnvironment.QueueBackgroundWorkItem. The call to Task.Run and Wait, however, are entirely nonsensical - they're pushing work to the thread pool and blocking a thread pool thread on it, when the code is already on the thread pool.
The _billing.ProcessAsync() method is awaitable(async), so I would expect that a simple "await" keyword would be the right thing to do and not all this other baggage.
I strongly agree.
However, this will change the behavior of the action. It will now wait until Billing.ProcessAsync is completed, whereas before it would return early. Note that returning early on ASP.NET is almost always a mistake - I would say any "billing" processing would be even more certainly a mistake. So, replacing this mess with await will make the app more correct, but it will cause the ProcessAsync action to take longer to return to the client.
It's strange, but depending on what the author is trying to achieve, it seems ok to me to queue a work item in the thread pool from inside an async method.
This is not as starting a thread, it's just queueing an action to be done in a ThreadPool's thread when there is a free one. So the async method (ProcessAsync) can continue and don't need to care about the result.
The weird thing is the code inside the lambda to be enqueued in the ThreadPool. Not only the Task.Run() (which is superflous and just causes unnecessary overhead), but to call an async method without waiting for it to finish is bad inside a method that should be run by the ThreadPool, because it returns the control flow to the caller when awaiting something.
So the ThreadPool eventually thinks this method is finished (and the thread free for the next action in the queue), while actually the method wants to be resumed later.
This may lead to very undefined behaviour. This code may have been working (in certain circumstances), but I would not rely on it and use it as productive code.
(The same goes for calling a not-awaited async method inside Task.Run(), as the Task "thinks" it's finished while the method actually wants to be resumed later).
As solution I'd propose to simply await that async method, too:
await _billing.ProcessAsync(user);
But of course without any knowledge about the context of the code snippet I can't guarantee anything. Note that this would change the behaviour: while until now the code did not wait for _billing.ProcessAsync() to finsih, it would now do. So maybe leaving out await and just fire and forget
_billing.ProcessAsync(user);
maybe good enough, too.
I am going to use this method in a Load Test which means thousands of calls may happen very quickly from different threads. I am wondering if I have to consider what would happen on subsequent call, where a new WebClient is created but before the prior await is complete?
public static async Task<string> SendRequest(this string url)
{
using (var wc = new WebClient())
{
var bytes = await wc.DownloadDataTaskAsync(url);
using (var reader = new StreamReader(new MemoryStream(bytes)))
{
return await reader.ReadToEndAsync();
}
}
}
I use the term reentrant to describe the fact that this method will be called by one or more threads.
So we want to know what potential problems could arise from using this method in a multithreaded context, either through a single call in an environment that has multiple threads, or where multiple calls are being made from one or more threads.
The first thing to look at is what does this method expose externally. If we're designing this method, we can control what it does, but not what the callers do. We need to assume that anyone can do anything with whatever they pass into our method, what they do with the returned value, and what they do with the type/object instance that the class is called on. Let's look at each of these in turn.
The URL:
Obviously the caller can pass in an invalid URL, but that's not an issue that's specific to asynchrony or multithreading. They can't really do anything else with this parameter. They can't mutate the string from another thread after passing it to us, because string is immutable (or at least observably immutable externally).
The return value:
So at first glance, this in fact may appear to be a problem. We're returning an object instance (a Task); that object is being mutated by this method that we're writing (to mark it as faulted, excepted, completed) and it is also likely to be mutated by the caller of this method (to add continuations). It's also quite plausible for this Task to end up being mutated from multiple different threads (the task could be passed to any number of other threads, which could mutate it by adding continuations, or be reading values while we're mutating it).
Fortunately, Task was very specifically designed to support all of these situations, and it will function properly due to the synchronization that it performs internally. As authors of this method, we don't need to care who adds what continuations to our task, from what thread, whether or not different people are adding them at the same time, what order things happen in, whether continuations are added before or after we mark the task as completed, or any of that. While the task can be mutated externally, even from other threads, there's nothing that they could do that would be observable to us, from this method. Likewise, their continuations are going to function appropriately regardless of what we do. Their continuations will always fire some time after the task is marked as completed, or immediately if it was already completed. It doesn't have the possible race conditions that an event based model has of adding an event handler after the event is fired to signal completion.
Finally, we have state of the type/instance.
This one is easy. It's a static method, so there are no instance fields that we could access even if we wanted to. There are also no static fields that this method accesses, so no state is shared between threads that way that we need to be concerned about.
Other than the string input and task output, the state that this method uses is entirely local variables that are never accessible outside of this method. Since this method does everything in a single thread (if there is a synchronization context, or it at least does everything sequentially even if thread pool threads are used), we don't need to worry about any threading issues internally, only what could be happening externally by the caller.
When you're concerned about methods being called multiple times before previous calls have finished, the primary concern here is around access to fields. If the method was accessing instance/static fields, then one would need to consider the implications not only of a method being called with any given input state, but also with what's going on if other methods are accessing those fields at the same time. Since we access none, this is moot for this method.
I'm just beginning to learn C# threading and concurrent collections, and am not sure of the proper terminology to pose my question, so I'll describe briefly what I'm trying to do. My grasp of the subject is rudimentary at best at this point. Is my approach below even feasible as I've envisioned it?
I have 100,000 urls in a Concurrent collection that must be tested--is the link still good? I have another concurrent collection, initially empty, that will contain the subset of urls that an async request determines to have been moved (400, 404, etc errors).
I want to spawn as many of these async requests concurrently as my PC and our bandwidth will allow, and was going to start at 20 async-web-request-tasks per second and work my way up from there.
Would it work if a single async task handled both things: it would make the async request and then add the url to the BadUrls collection if it encountered a 4xx error? A new instance of that task would be spawned every 50ms:
class TestArgs args {
ConcurrentBag<UrlInfo> myCollection { get; set; }
System.Uri currentUrl { get; set; }
}
ConcurrentQueue<UrlInfo> Urls = new ConncurrentQueue<UrlInfo>();
// populate the Urls queue
<snip>
// initialize the bad urls collection
ConcurrentBag<UrlInfo> BadUrls = new ConcurrentBag<UrlInfo>();
// timer fires every 50ms, whereupon a new args object is created
// and the timer callback spawns a new task; an autoEvent would
// reset the timer and dispose of it when the queue was empty
void SpawnNewUrlTask(){
// if queue is empty then reset the timer
// otherwise:
TestArgs args = {
myCollection = BadUrls,
currentUrl = getNextUrl() // take an item from the queue
};
Task.Factory.StartNew( asyncWebRequestAndConcurrentCollectionUpdater, args);
}
public async Task asyncWebRequestAndConcurrentCollectionUpdater(TestArgs args)
{
//make the async web request
// add the url to the bad collection if appropriate.
}
Feasible? Way off?
The approach seems fine, but there are some issues with the specific code you've shown.
But before I get to that, there have been suggestions in the comments that Task Parallelism is the way to go. I think that's misguided. There's a common misconception that if you want to have lots of work going on in parallel, you necessarily need lots of threads. That's only true if the work is compute-bound. But the work you're doing will be IO bound - this code is going to spend the vast majority of its time waiting for responses. It will do very little computation. So in practice, even if it only used a single thread, your initial target of 20 requests per second doesn't seem like a workload that would cause a single CPU core to break into a sweat.
In short, a single thread can handle very high levels of concurrent IO. You only need multiple threads if you need parallel execution of code, and that doesn't look likely to be the case here, because there's so little work for the CPU in this particular job.
(This misconception predates await and async by years. In fact, it predates the TPL - see http://www.interact-sw.co.uk/iangblog/2004/09/23/threadless for a .NET 1.1 era illustration of how you can handle thousands of concurrent requests with a tiny number of threads. The underlying principles still apply today because Windows networking IO still basically works the same way.)
Not that there's anything particularly wrong with using multiple threads here, I'm just pointing out that it's a bit of a distraction.
Anyway, back to your code. This line is problematic:
Task.Factory.StartNew( asyncWebRequestAndConcurrentCollectionUpdater, args);
While you've not given us all your code, I can't see how that will be able to compile. The overloads of StartNew that accept two arguments require the first to be either an Action, an Action<object>, a Func<TResult>, or a Func<object,TResult>. In other words, it has to be a method that either takes no arguments, or accepts a single argument of type object (and which may or may not return a value). Your 'asyncWebRequestAndConcurrentCollectionUpdater' takes an argument of type TestArgs.
But the fact that it doesn't compile isn't the main problem. That's easily fixed. (E.g., change it to Task.Factory.StartNew(() => asyncWebRequestAndConcurrentCollectionUpdater(args));) The real issue is what you're doing is a bit weird: you're using Task.StartNew to invoke a method that already returns a Task.
Task.StartNew is a handy way to take a synchronous method (i.e., one that doesn't return a Task) and run it in a non-blocking way. (It'll run on the thread pool.) But if you've got a method that already returns a Task, then you didn't really need to use Task.StartNew. The weirdness becomes more apparent if we look at what Task.StartNew returns (once you've fixed the compilation error):
Task<Task> t = Task.Factory.StartNew(
() => asyncWebRequestAndConcurrentCollectionUpdater(args));
That Task<Task> reveals what's happening. You've decided to wrap a method that was already asynchronous with a mechanism that is normally used to make non-asynchronous methods asynchronous. And so you've now got a Task that produces a Task.
One of the slightly surprising upshots of this is that if you were to wait for the task returned by StartNew to complete, the underlying work would not necessarily be done:
t.Wait(); // doesn't wait for asyncWebRequestAndConcurrentCollectionUpdater to finish!
All that will actually do is wait for asyncWebRequestAndConcurrentCollectionUpdater to return a Task. And since asyncWebRequestAndConcurrentCollectionUpdater is already an async method, it will return a task more or less immediately. (Specifically, it'll return a task the moment it performs an await that does not complete immediately.)
If you want to wait for the work you've kicked off to finish, you'll need to do this:
t.Result.Wait();
or, potentially more efficiently, this:
t.Unwrap().Wait();
That says: get me the Task that my async method returned, and then wait for that. This may not be usefully different from this much simpler code:
Task t = asyncWebRequestAndConcurrentCollectionUpdater("foo");
... maybe queue up some other tasks ...
t.Wait();
You may not have gained anything useful by introducing `Task.Factory.StartNew'.
I say "may" because there's an important qualification: it depends on the context in which you start the work. C# generates code which, by default, attempts to ensure that when an async method continues after an await, it does so in the same context in which the await was initially performed. E.g., if you're in a WPF app and you await while on the UI thread, when the code continues it will arrange to do so on the UI thread. (You can disable this with ConfigureAwait.)
So if you're in a situation in which the context is essentially serialized (either because it's single-threaded, as will be the case in a GUI app, or because it uses something resembling a rental model, e.g. the context of an particular ASP.NET request), it may actually be useful to kick an async task off via Task.Factory.StartNew because it enables you to escape the original context. However, you just made your life harder - tracking your tasks to completion is somewhat more complex. And you might have been able to achieve the same effect simply by using ConfigureAwait inside your async method.
And it may not matter anyway - if you're only attempting to manage 20 requests a second, the minimal amount of CPU effort required to do that means that you can probably manage it entirely adequately on one thread. (Also, if this is a console app, the default context will come into play, which uses the thread pool, so your tasks will be able to run multithreaded in any case.)
But to get back to your question, it seems entirely reasonable to me to have a single async method that picks a url off the queue, makes the request, examines the response, and if necessary, adds an entry to the bad url collection. And kicking the things off from a timer also seems reasonable - that will throttle the rate at which connections are attempted without getting bogged down with slow responses (e.g., if a load of requests end up attempting to talk to servers that are offline). It might be necessary to introduce a cap for the maximum number of requests in flight if you hit some pathological case where you end up with tens of thousands of URLs in a row all pointing to a server that isn't responding. (On a related note, you'll need to make sure that you're not going to hit any per-client connection limits with whichever HTTP API you're using - that might end up throttling the effective throughput.)
You will need to add some sort of completion handling - just kicking off asynchronous operations and not doing anything to handle the results is bad practice, because you can end up with exceptions that have nowhere to go. (In .NET 4.0, these used to terminate your process, but as of .NET 4.5, by default an unhandled exception from an asynchronous operation will simply be ignored!) And if you end up deciding that it is worth launching via Task.Factory.StartNew remember that you've ended up with an extra layer of wrapping, so you'll need to do something like myTask.Unwrap().ContinueWith(...) to handle it correctly.
Of course you can. Concurrent collections are called 'concurrent' because they can be used... concurrently by multiple threads, with some warranties about their behaviour.
A ConcurrentQueue will ensure that each element inserted in it is extracted exactly once (concurrent threads will never extract the same item by mistake, and once the queue is empty, all the items have been extracted by a thread).
EDIT: the only thing that could go wrong is that 50ms is not enough to complete the request, and so more and more tasks cumulate in the task queue. If that happens, your memory could get filled, but the thing would work anyway. So yes, it is feasible.
Anyway, I would like to underline the fact that a task is not a thread. Even if you create 100 tasks, the framework will decide how many of them will be actually executed concurrently.
If you want to have more control on the level of parallelism, you should use asynchronous requests.
In your comments, you wrote "async web request", but I can't understand if you wrote async just because it's on a different thread or because you intend to use the async API.
If you were using the async API, I'd expect to see some handler attached to the completion event, but I couldn't see it, so I assumed you're using synchronous requests issued from an asynchronous task.
If you're using asynchronous requests, then it's pointless to use tasks, just use the timer to issue the async requests, since they are already asynchronous.
When I say "asynchronous request" I'm referring to methods like WebRequest.GetResponseAsync and WebRequest.BeginGetResponse.
EDIT2: if you want to use asynchronous requests, then you can just make requests from the timer handler. The BeginGetResponse method takes two arguments. The first one is a callback procedure, that will be called to report the status of the request. You can pass the same procedure for all the requests. The second one is an user-provided object, which will store status about the request, you can use this argument to differentiate among different requests. You can even do it without the timer. Something like:
private readonly int desiredConcurrency = 20;
struct RequestData
{
public UrlInfo url;
public HttpWebRequest request;
}
/// Handles the completion of an asynchronous request
/// When a request has been completed,
/// tries to issue a new request to another url.
private void AsyncRequestHandler(IAsyncResult ar)
{
if (ar.IsCompleted)
{
RequestData data = (RequestData)ar.AsyncState;
HttpWebResponse resp = data.request.EndGetResponse(ar);
if (resp.StatusCode != 200)
{
BadUrls.Add(data.url);
}
//A request has been completed, try to start a new one
TryIssueRequest();
}
}
/// If urls is not empty, dequeues a url from it
/// and issues a new request to the extracted url.
private bool TryIssueRequest()
{
RequestData rd;
if (urls.TryDequeue(out rd.url))
{
rd.request = CreateRequestTo(rd.url); //TODO implement
rd.request.BeginGetResponse(AsyncRequestHandler, rd);
return true;
}
else
{
return false;
}
}
//Called by a button handler, or something like that
void StartTheRequests()
{
for (int requestCount = 0; requestCount < desiredConcurrency; ++requestCount)
{
if (!TryIssueRequest()) break;
}
}
I have a question about how customizable the new async/await keywords and the Task class in C# 4.5 are.
First some background for understanding my problem: I am developing on a framework with the following design:
One thread has a list of "current things to do" (usually around 100 to 200 items) which are stored as an own data structure and hold as a list. It has an Update() function that enumerates the list and look whether some "things" need to execute and does so. Basically its like a big thread sheduler. To simplify things, lets assume the "things to do" are functions that return the boolean true when they are "finished" (and should not be called next Update) and false when the sheduler should call them again next update.
All the "things" must not run concurrently and also must run in this one thread (because of thread static variables)
There are other threads which do other stuff. They are structured in the same way: Big loop that iterates a couple of hundret things to do in a big Update() - function.
Threads can send each other messages, including "remote procedure calls". For these remote calls, the RPC system is returning some kind of future object to the result value. In the other thread, a new "thing to do" is inserted.
A common "thing" to do are just sequences of RPCs chained together. At the moment, the syntax for this "chaining" is very verbose and complicated, since you manually have to check for the completion state of previous RPCs and invoke the next ones etc..
An example:
Future f1, f2;
bool SomeThingToDo() // returns true when "finished"
{
if (f1 == null)
f1 = Remote1.CallF1();
else if (f1.IsComplete && f2 == null)
f2 = Remote2.CallF2();
else if (f2 != null && f2.IsComplete)
return true;
return false;
}
Now this all sound awefull like async and await of C# 5.0 can help me here. I haven't 100% fully understand what it does under the hood (any good references?), but as I get it from some few talks I've watched, it exactly does what I want with this nicely simple code:
async Task SomeThingToDo() // returning task is completed when this is finished.
{
await Remote1.CallF1();
await Remote2.CallF2();
}
But I can't find a way how write my Update() function to make something like this happen. async and await seem to want to use the Task - class which in turn seems to need real threads?
My closest "solution" so far:
The first thread (which is running SomeThingToDo) calls their functions only once and stores the returned task and tests on every Update() whether the task is completed.
Remote1.CallF1 returns a new Task with an empty Action as constructor parameter and remembers the returned task. When F1 is actually finished, it calls RunSynchronously() on the task to mark it as completed.
That seems to me like a pervertion of the task system. And beside, it creates shared memory (the Task's IsComplete boolean) between the two threads which I would like to have replaced with our remote messanging system, if possible.
Finally, it does not solve my problem as it does not work with the await-like SomeThingToDo implementation above. It seems the auto-generated Task objects returned by an async function are completed immediately?
So finally my questions:
Can I hook into async/await to use my own implementations instead of Task<T>?
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Any good reference what exactly happens when I write async and await?
I haven't 100% fully understand what it does under the hood - any good references?
Back when we were designing the feature Mads, Stephen and I wrote some articles at a variety of different levels for MSDN magazine. The links are here:
http://blogs.msdn.com/b/ericlippert/archive/2011/10/03/async-articles.aspx
Start with my article, then Mads's, then Stephen's.
It seems the auto-generated Task objects returned by an async function are completed immediately?
No, they are completed when the code in the method body returns or throws, same as any other code.
Can I hook into async/await to use my own implementations instead of Task<T>?
A method which contains an await must return void, Task or Task<T>. However, the expression that is awaited can return any type so long as you can call GetAwaiter() on it. That need not be a Task.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Absolutely. A Task just represents work that will complete in the future. Though that work is typically done on another thread, there is no requirement.
To answer your questions:
Can I hook into async/await to use my own implementations instead of Task?
Yes. You can await anything. However, I do not recommend this.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
The Task type represents a future. It does not necessarily "run" on a thread; it can represent the completion of a download, or a timer expiring, etc.
Any good reference what exactly happens when I write async and await?
If you mean as far as code transformations go, this blog post has a nice side-by-side. It's not 100% accurate in its details, but it's enough to write a simple custom awaiter.
If you really want to twist async to do your bidding, Jon Skeet's eduasync series is the best resource. However, I seriously do not recommend you do this in production.
You may find my async/await intro helpful as an introduction to the async concepts and recommended ways to use them. The official MSDN documentation is also unusually good.
I did write the AsyncContext and AsyncContextThread classes that may work for your situation; they define a single-threaded context for async/await methods. You can queue work (or send messages) to an AsyncContextThread by using its Factory property.
Can I hook into async/await to use my own implementations instead of Task?
Yes.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Yes.
Any good reference what exactly happens when I write async and await?
Yes.
I would discourage you from asking yes/no questions. You probably don't just want yes/no answers.
async and await seem to want to use the Task - class which in turn seems to need real threads?
Nope, that's not true. A Task represents something that can be completed at some point in the future, possibly with a result. It's sometimes the result of some computation in another thread, but it doesn't need to be. It can be anything that is happening at some point in the future. For example, it could be the result of an IO operation.
Remote1.CallF1 returns a new Task with an empty Action as constructor parameter and remembers the returned task. When F1 is actually finished, it calls RunSynchronously() on the task to mark it as completed.
So what you're missing here is the TaskCompletionSource class. With that missing puzzle piece a lot should fit into place. You can create the TCS object, pass the Task from it's Task property around to...whomever, and then use the SetResult property to signal it's completion. Doing this doesn't result in the creation of any additional threads, or use the thread pool.
Note that if you don't have a result and just want a Task instead of a Task<T> then just use a TaskCompletionSource<bool> or something along those lines and then SetResult(false) or whatever is appropriate. By casting the Task<bool> to a Task you can hide that implementation from the public API.
That should also provide the "How" variations of the first two questions that you asked instead of the "can I" versions you asked. You can use a TaskCompletionSource to generate a task that is completed whenever you say it is, using whatever asynchronous construct you want, which may or may not involve the use of additional threads.