Develop Reference

How to wrap Webclient.DownloadFileAsync into an awaitable method?

I'd like to wrap the WebClient.DownloadFileAsync method, which is actually non blocking, into a method which returns a Task and can be awaited until the download is completed.
I understand the logic behind suscribing to the DownloadFileCompleted event, as explained here (and in plenty other posts) but I'd like to wrap it into something more user friendly. And my knowledge of asynchronous programming is quite superficial.
The best I can think of is to wrap DownloadFile method (synchronous) into a Task and returns it , but I've read many times that wrapping a synchronous method onto a an asynchronous one is not a good practice
return Task.Run(() =>client.DownloadFile(fileUri, localPath));
It's actually the first time I meet an async method which is non awaitable, is there a reason why it has been thought this way?
Thanks

As per #JonSkeet's comment above, the answer is :
The method DownloadFileAsync has been thought long before await/async was implemented
The method I was looking for is called DownloadFileTaskAsync, a straightforward async file downloader, and there is no need to rewrite a wrapper

async and await without "threads"? Can I customize what happens under-the-hood?

I have a question about how customizable the new async/await keywords and the Task class in C# 4.5 are.
First some background for understanding my problem: I am developing on a framework with the following design:
One thread has a list of "current things to do" (usually around 100 to 200 items) which are stored as an own data structure and hold as a list. It has an Update() function that enumerates the list and look whether some "things" need to execute and does so. Basically its like a big thread sheduler. To simplify things, lets assume the "things to do" are functions that return the boolean true when they are "finished" (and should not be called next Update) and false when the sheduler should call them again next update.
All the "things" must not run concurrently and also must run in this one thread (because of thread static variables)
There are other threads which do other stuff. They are structured in the same way: Big loop that iterates a couple of hundret things to do in a big Update() - function.
Threads can send each other messages, including "remote procedure calls". For these remote calls, the RPC system is returning some kind of future object to the result value. In the other thread, a new "thing to do" is inserted.
A common "thing" to do are just sequences of RPCs chained together. At the moment, the syntax for this "chaining" is very verbose and complicated, since you manually have to check for the completion state of previous RPCs and invoke the next ones etc..
An example:
Future f1, f2;
bool SomeThingToDo() // returns true when "finished"
{
if (f1 == null)
f1 = Remote1.CallF1();
else if (f1.IsComplete && f2 == null)
f2 = Remote2.CallF2();
else if (f2 != null && f2.IsComplete)
return true;
return false;
}
Now this all sound awefull like async and await of C# 5.0 can help me here. I haven't 100% fully understand what it does under the hood (any good references?), but as I get it from some few talks I've watched, it exactly does what I want with this nicely simple code:
async Task SomeThingToDo() // returning task is completed when this is finished.
{
await Remote1.CallF1();
await Remote2.CallF2();
}
But I can't find a way how write my Update() function to make something like this happen. async and await seem to want to use the Task - class which in turn seems to need real threads?
My closest "solution" so far:
The first thread (which is running SomeThingToDo) calls their functions only once and stores the returned task and tests on every Update() whether the task is completed.
Remote1.CallF1 returns a new Task with an empty Action as constructor parameter and remembers the returned task. When F1 is actually finished, it calls RunSynchronously() on the task to mark it as completed.
That seems to me like a pervertion of the task system. And beside, it creates shared memory (the Task's IsComplete boolean) between the two threads which I would like to have replaced with our remote messanging system, if possible.
Finally, it does not solve my problem as it does not work with the await-like SomeThingToDo implementation above. It seems the auto-generated Task objects returned by an async function are completed immediately?
So finally my questions:
Can I hook into async/await to use my own implementations instead of Task<T>?
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Any good reference what exactly happens when I write async and await?

I haven't 100% fully understand what it does under the hood - any good references?
Back when we were designing the feature Mads, Stephen and I wrote some articles at a variety of different levels for MSDN magazine. The links are here:
http://blogs.msdn.com/b/ericlippert/archive/2011/10/03/async-articles.aspx
Start with my article, then Mads's, then Stephen's.
It seems the auto-generated Task objects returned by an async function are completed immediately?
No, they are completed when the code in the method body returns or throws, same as any other code.
Can I hook into async/await to use my own implementations instead of Task<T>?
A method which contains an await must return void, Task or Task<T>. However, the expression that is awaited can return any type so long as you can call GetAwaiter() on it. That need not be a Task.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Absolutely. A Task just represents work that will complete in the future. Though that work is typically done on another thread, there is no requirement.

To answer your questions:
Can I hook into async/await to use my own implementations instead of Task?
Yes. You can await anything. However, I do not recommend this.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
The Task type represents a future. It does not necessarily "run" on a thread; it can represent the completion of a download, or a timer expiring, etc.
Any good reference what exactly happens when I write async and await?
If you mean as far as code transformations go, this blog post has a nice side-by-side. It's not 100% accurate in its details, but it's enough to write a simple custom awaiter.
If you really want to twist async to do your bidding, Jon Skeet's eduasync series is the best resource. However, I seriously do not recommend you do this in production.
You may find my async/await intro helpful as an introduction to the async concepts and recommended ways to use them. The official MSDN documentation is also unusually good.
I did write the AsyncContext and AsyncContextThread classes that may work for your situation; they define a single-threaded context for async/await methods. You can queue work (or send messages) to an AsyncContextThread by using its Factory property.

Can I hook into async/await to use my own implementations instead of Task?
Yes.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Yes.
Any good reference what exactly happens when I write async and await?
Yes.
I would discourage you from asking yes/no questions. You probably don't just want yes/no answers.
async and await seem to want to use the Task - class which in turn seems to need real threads?
Nope, that's not true. A Task represents something that can be completed at some point in the future, possibly with a result. It's sometimes the result of some computation in another thread, but it doesn't need to be. It can be anything that is happening at some point in the future. For example, it could be the result of an IO operation.
Remote1.CallF1 returns a new Task with an empty Action as constructor parameter and remembers the returned task. When F1 is actually finished, it calls RunSynchronously() on the task to mark it as completed.
So what you're missing here is the TaskCompletionSource class. With that missing puzzle piece a lot should fit into place. You can create the TCS object, pass the Task from it's Task property around to...whomever, and then use the SetResult property to signal it's completion. Doing this doesn't result in the creation of any additional threads, or use the thread pool.
Note that if you don't have a result and just want a Task instead of a Task<T> then just use a TaskCompletionSource<bool> or something along those lines and then SetResult(false) or whatever is appropriate. By casting the Task<bool> to a Task you can hide that implementation from the public API.
That should also provide the "How" variations of the first two questions that you asked instead of the "can I" versions you asked. You can use a TaskCompletionSource to generate a task that is completed whenever you say it is, using whatever asynchronous construct you want, which may or may not involve the use of additional threads.

Custom awaitables for dummies

In Async/Await FAQ, Stephen Toub says:
An awaitable is any type that exposes a GetAwaiter method which returns a valid awaiter.
...
An awaiter is any type returned from an awaitable’s GetAwaiter method and that conforms to a particular pattern.
So in order to be an awaiter, a type should:
Implement the INotifyCompletion interface.
Provide a boolean property called IsCompleted.
Provide a parameterless GetResult method that returns void or TResult.
(I'm ignoring ICriticalNotifyCompletion for now.)
I know the page I mentioned has a sample that shows how the compiler translates await operations but I'm stil having a hard time understanding.
When I await an awaitable,
When is IsCompleted checked? Where should I set it?
When is OnCompleted called?
Which thread calls OnCompleted?
I saw examples of both directly invoking the continuation parameter of OnCompleted and using Task.Run(continuation) in different examples, which should I go for and why?

Why would you want a custom awaiter?
You can see the compiler's interpretation of await here. Essentially:
var temp = e.GetAwaiter();
if (!temp.IsCompleted)
{
SAVE_STATE()
temp.OnCompleted(&cont);
return;
cont:
RESTORE_STATE()
}
var i = temp.GetResult();
Edit from comments: OnCompleted should schedule its argument as a continuation of the asynchronous operation.

In the vast majority of cases, you as a developer need not worry about this. Use the async and await keywords and the compiler and runtime handle all this for you.
To answer your questions:
When does the code checks IsCompleted? Where should I set it?
The task should set IsCompleted when the task has finished doing what it was doing. For example, if the task was loading data from a file, IsCompleted should return true when the data is loaded and the caller can access it.
When does it calls OnCompleted?
OnCompleted usually contains a delegate supplied by the caller to execute when the task has completed.
Does it call OnCompleted in parallel or should the code
inside OnCompleted be asynchronous?
The code in OnCompleted should be thread neutral (not care which thread it is called from). This may be problematic for updating COM objects in Single Threaded Apartments (like any UI classes in Metro/Windows8/Windows Store apps). It does not have to be asynchronous but may contain asynchronous code.
I saw examples of both directly invoking the continuation parameter of OnCompleted
and using Task.Run(continuation) in different examples, which should I go for and when?
Use async/await when you can. Otherwise, use Task.Run() or Task.Wait() because they follow the sequential programming model most people are used to. Using continuations may still be required, particularly in Metro apps where you have apartment issues.

Is it necessary to call EndInvoke in the callback from an EventHandler.BeginInvoke (C# .Net 3.5)

I'm setting up a WCF client and server scenario that needs to be reliable and scalable and as part of that I'm trying to make it multithreaded. Admittedly I don't have extensive multithreading experience.
On the client, the callback from the server (in this case "OnMessage") fires a client-side event that should be passed off in the background. I have the client's CallbackBehavior ConcurrencyMode set to Single (at least, for now). So in order to do a reply to the callback, rather than calling the handler as normal (either handler(sender, eventargs, or handler.Invoke...) I'm calling handler.BeginInvoke.
Everything works fine, but in the callback I'm wondering if I need to explicitly call EndInvoke or if I can basically do nothing (here's my lack of multithreading experience showing).
public void OnMessage(Message message)
{
EventHandler<MessageEventArgs> handler = OnServerMessage;
if (handler != null)
handler.BeginInvoke(this, new MessageEventArgs(message), CompleteHandler, handler);
}
public void CompleteHandler(IAsyncResult result)
{
((EventHandler<MessageEventArgs>)result.AsyncState).EndInvoke(result);
}
Can I replace CompleteHandler with a direct reference to handler.EndInvoke, or an empty method, or perhaps is there something more I should be doing?
Also, at the moment, the event subscriber blocks waiting on user input from the console. When the application is complete, it would either block waiting on the database, or be asynchronous bottom-to-top (so this may not be necessary, but I'd still like to know).

Yes you have to. Only way to find out if the invoked method threw an exception. And to cleanup the remoting state of the call so it can be garbage collected instead of letting it linger for another 10 minutes.

You need to wrap your CompleteHandler code in a try/catch block and then properly handle any exception thrown by the EndInvoke. Right now the service works under 'lab' conditions, but when things go wrong you won't even know what goes wrong, because you don't handle/log errors properly.
As for the issue of getting the state from the global state, or from the IAsyncResult, or from a callback closure: IASyncResult.AsyncState is the most generic solution and will continue to work correctly after somebody else refactors the calling code beyond any recognition.

In general, BeginInvoke starts an asynchronous operation, whereas EndInvoke waits for it to finish. So it depends on your intention: if you want to guarantee that your asynchronous code is finished, you need EndInvoke, otherwise no.
(Beware however that if your BeginInvoke calls into the same thread where you actually are (it's a valid usage, just the code will be run on the next Dispatcher loop), you must not use EndInvoke, as this would create a deadlock. But this seems to be not your case.)

Using BeginInvoke/EndInvoke in a multithreaded fashion. How do AsyncCallback, AsyncWaitHandle and IsCompleted interact?

Andreas Huber's answer to this question gave me an idea to implement Concurrent<T> with async delegates instead of the ThreadPool. However, I am finding it harder to understand what's going on when an AsyncCallback is passed to BeginInvoke, especially when several threads have access to IAsyncResult. Unfortunately, this case doesn't seem to be covered at MSDN or anywhere I could find. Moreover, all articles I could find were either written before closures and generics were available or just seem that way. There are several questions (and the answers which I hope are true, but I am ready to be disappointed):
1) Would using a closure as an AsyncCallback make any difference?
(Hopefully not)
2) If a thread waits on the AsyncWaitHandle, will it be signaled
a) before the callback starts or
b) after it finishes?
(Hopefully b)
3) While the callback is running, what will IsCompleted return? Possibilities I can see:
a) true;
b) false;
c) false before the callback calls EndInvoke, true after.
(Hopefully b or c)
4) Will DisposedObjectException be thrown if some thread waits on the AsyncWaitHandle after EndInvoke is called?
(Hopefully not, but I expect yes).
Provided the answers are as I hope, this seems like it should work:
public class Concurrent<T> {
private IAsyncResult _asyncResult;
private T _result;
public Concurrent(Func<T> f) { // Assume f doesn't throw exceptions
_asyncResult = f.BeginInvoke(
asyncResult => {
// Assume assignment of T is atomic
_result = f.EndInvoke(asyncResult);
}, null);
}
public T Result {
get {
if (!_asyncResult.IsCompleted)
// Is there a race condition here?
_asyncResult.AsyncWaitHandle.WaitOne();
return _result; // Assume reading of T is atomic
}
...
If the answers to the questions 1-3 are the ones I hope for, there should be no raace condition here, as far as I can see.

Question 1
I think part of the problem is misconception. IAsyncResult is not accessed from multiple threads unless you explicitly pass it to one. If you look at the implementation for mos Begin*** style API's in the BCL, you'll notice the IAsyncResult is only ever created and destroyed from the thread where the Begin*** or End*** call actually occur.
Question 2
AsyncWaitHandle should be signaled after the operation is 100% complete.
Question 3
IsCompleted should return true once the underlying operation is complete (no more work to do). The best way to view IsComplete is that if the value is
true -> Calling End*** will return immediately
false -> Callind End*** will block for some period of time
Question 4
This is implementation dependent. There is no way to really give a blanket answer here.
Samples
If you are interested in an API which allows you to easily run a delegate on another thread and access the result when finished, check out my RantPack Utility Library. It's available in source and binary form. It has a fully fleshed out Future API which allows for the concurrent running of delegates.
Additionally there is an implementation of IAsyncResult which covers most of the questions in this post.

I've been looking into async calls just recently. I found a pointer to an article with an example implementation of an IAsyncResult by respected author Jeffrey Richter. I learned a lot about how async calls work by studying this implementation.
You might also see if you can download and examine the source code for the System.Runtime.Remoting.Messaging.AsyncResult you're specifically concerned with. Here's a link to instructions on how to do this in Visual Studio.
To add a bit to JaredPar's good answers...
1: I believe if you define a closure which can be assigned to a variable of type AsyncCallback (takes an IAsyncResult and returns void) it should work as you would expect a closure to work as that delegate, but I'm not sure if there could be scope issues. The originating local scope should have returned long before the callback gets invoked (that's what makes it an asynchronous operation), so bear that in mind with respect to references to local (stack) variables and how that will behave. Referencing member variables should be fine, I would think.
2: I think from your comment that you may have misunderstood the answer to this one. In Jeffrey Richter's example implementation the wait handle is signaled before the callback is invoked. If you think about it, it has to be this way. Once it invokes the callback it loses control of the execution. Suppose the callback method throws an exception.... execution could unwind back past the method which invoked the callback and thus prevent it from ever later signaling the wait handle! So the wait handle needs to be signalled before callback is invoked. They're also much closer in time if done in that order than if it signals the wait handle only after the callback has returned.
3: As JaredPar says, IsCompleted should be returning true before the callback and before the wait handle is signaled. This makes sense because if IsCompleted is false you would expect the call to EndInvoke to block, and the whole point of the wait handle (as with the callback) is to know when the result is ready and it won't block. So, first IsCompleted is set to true, then the wait handle is signalled, and then the callback is called. See how Jeffrey Richter's example does it. However, you probably should try to avoid assumptions about the order in which these three methods (polling, wait handle, callback) might detect the completion, because it's possible to implement them in a different order than expected.
4: I can't help you there, except that you might find the answer by debugging into the framework source code for the implementation you're curious about. Or you could probably come up with an experiment to find out... or set up a good experiment and debug into the framework source to be really sure.