Develop Reference

In .NET, are async methods exclusively those with the async keyword?

I've been learning about HttpClient (consuming API's in .NET in general), and therefore about async programming. I'm still pretty lost right now, so trying to clear some things up. One question I can't seem to find an answer to - are asynchronous methods implemented exclusively using the async keyword?
I understand that you could (theoretically) create synchronous methods using async programming, but how do you recognize that? For example, from this link and this example:
public string GetReleases(string url)
{
using (var httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.Add(RequestConstants.UserAgent, RequestConstants.UserAgentValue);
var response = httpClient.GetStringAsync(new Uri(url)).Result;
return response;
}
}
The author says:
For the simplicity’s sake, I implemented it synchronously
But how do I recognize it is synchronous? Is it solely because the method is not defined with async Task, for example:
public async Task<string> GetReleases(string url)
Does that mean that in this HttpClient tutorial, this example is also not asynchronous:
// GET: Student
public ActionResult Index()
{
IEnumerable<StudentViewModel> students = null;
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:64189/api/");
//HTTP GET
var responseTask = client.GetAsync("student");
responseTask.Wait();
var result = responseTask.Result;
if (result.IsSuccessStatusCode)
{
var readTask = result.Content.ReadAsAsync<IList<StudentViewModel>>();
readTask.Wait();
students = readTask.Result;
}
else //web api sent error response
{
//log response status here..
students = Enumerable.Empty<StudentViewModel>();
ModelState.AddModelError(string.Empty, "Server error. Please contact administrator.");
}
}
return View(students);
}
So to sum up, my questions are:
Is async programming (in .NET), and async methods, implemented exclusively using async and void/Task/Task?
If not, how else, and if so, how do I recognize "true" asynchronous methods compared to synchronous methods implemented using asychronous principles (like the example above?)
Why then did the above examples use "sync through async", since from everything I've read everyone says to NEVER do that? Are they just bad examples, or for simplicity sake (even so, shouldn't "the correct way" > "simplicity"? Would it be ok to use this in situations where, e.g., I have very simple code and want to avoid threading/deadlock issues, since I'm not comfortable with async programming just yet?

how can I recognize truely async/sync methods?
You can't. Not really. You can spot methods that are potentially async, but little else can be learned without consulting documentation or the implementation of those methods.
So, you can examine a method's return type. If it's void, you don't know much. If it's Task, Task<T>, ValueTask<T> or any other awaitable type1, then the method may be asynchronous. But bear in mind, the method signature may be fixed because the type inherited the method from a base class or it's implementing an interface; So whilst the method has the potential to be async, the actual implementation of that method may be completely synchronous.
Or, the method may have the potential to be asynchronous but may have particular control flows which lead to it behaving synchronously2. These may be e.g. that if certain conditions are true, the method already has the answer, so it returns it straight away. Otherwise it goes off and does something async - as one example.
If the return type isn't awaitable and it's non-void, all you can actually reason about the method is that, at the point at which it returns, it's supplied some value for that return type3. There's no way to reason about any other work that may have been started by that method - only that if it's done something like that, it doesn't intend for you to be able to discover when that work has completed.
If you're looking at the implementation of a method and asking yourself "is this implementation async" then the important thing is to work out what this code makes happen after control is returned back to the caller.
When is control returned back to the caller?
When we hit a return
When we hit an await, maybe
When we hit an await, we only return control back to the caller4 if the awaitable that we're awaiting isn't complete yet. So we have to find out where that awaitable came from and, if it came from calling another method, we have to start again from scratch in considering what that method does.
If the method contains awaits then it's usually safest to say that it's potentially async - but bear in mind the above possibilities about already completed awaitables and early returns.
If it's not async/await, what else might it have done? Well, if it's working with Tasks, it may have created one or more of those tasks to represent it's ongoing work. It may have scheduled more code to run via ContinueWith.
If it's not working with Tasks directly, hasn't called something else that is potentially async, hasn't cause a new Thread to be created and isn't needlessly obfuscated, it's probably synchronous.
Would it be ok to use this in situations where, e.g., I have very simple code and want to avoid threading/deadlock issues, since I'm not comfortable with async programming just yet?
The sync over async patterns shown in the examples in your question are more prone to deadlocks than working with async/await. Why? Because they block the current thread waiting for other things to happen. But the current thread or resources that it has locked may be special - and the actual async tasks that it's invoked may need to gain access to that same thread/resource before they can complete. Classic deadlock.
1Awaitable is one of a number of places where C# uses "duck typing". If there's a method available (instance or extension) called GetAwaiter that returns a type with the right shape, it's awaitable. So despite the fact that you'll probably never see one, be aware that custom awaitables are possible in the language.
2"Hot path" optimizations come to mind.
3And out/ref parameters. Of course, if it has those it won't be an async method implemented via the async/await keywords but it may still have some asynchronous behaviour.
4If we've already yielded control back to the caller during an earlier await, we won't return control back to the caller at later awaits, but we'll be returning it to "something" that isn't our code.

If a method returns a Task or Task<T> (with exception to void in case of event handler) then it can be awaited and hence it can be asynchronous. async keyword is only an indication that it may await somewhere for another awaitable. Based on the control flow, async may return without actually awaiting anything.
Asynchronous programming is not a new thing, it has existed in many forms like callbacks, Invokes etc.
Examples you have provided are not using async await pattern properly. Microsoft has provided naming convention (Async Suffix) and Task<T>, Task as types for async programming. So if you see some method returning Task<T> or Task and Method name has suffix "Async" then you can consider it asynchronous. Although suffix thing is not required by compiler, it helps in differentiating it from its synchronous counterpart. (Read vs ReadAsync)
They are bad examples, those action should be marked as async and all the Async methods should be awaited for result. There can be an exception in some console program where main can't be async.
Please read Stephen Cleary blog on async await

An asynchronous method in C# can return void, Task or Task<T>, where void should be avoided in general because it cannot be awaited.
As a convention asynchronous methods should be called DoSomethingAsync()
The async keyword, however, is an implementation detail and does not belong to the API. This keyword is only required if you use an await in the method body. But that need not be the case. You could simply delegate to another asynchronous method, without the need to mark the method as async and using an await.
So 'true' asynchronous methods should be recognizable by the Async suffix of the method name, however, you can't be sure the implementor actually uses naturally asynchronous operations or even runs synchronously some parts or the whole method.
In the example he made the method synchronously by putting .Result at the end of GetStringAsync

Use await for background threads or not?

I'm currently getting into the async/await keywords, and went through the following questions: When correctly use Task.Run and when just async-await and Async/Await vs Threads
However even the second link doesn't answer my question, which is when to simply use
Task.Run(...)
versus
await Task.Run(...)
Is it situational or is there something to be gained by using await (and thus returning to the caller)?

The code Task.Run(...) (in both examples) sends a delegate to the thread pool and returns a task that will contain the results of that delegate. These "results" can be an actual return value, or it can be an exception. You can use the returned task to detect when the results are available (i.e., when the delegate has completed).
So, you should make use of the Task if any of the following are true:
You need to detect when the operation completed and respond in some way.
You need to know if the operation completed successfully.
You need to handle exceptions from the operation.
You need to do something with the return value of the operation.
If your calling code needs to do any of those, then use await:
await Task.Run(...);
With long-running background tasks, sometimes you want to do those (e.g., detect exceptions), but you don't want to do it right away; in this case, just save the Task and then await it at some other point in your application:
this.myBackgroundTask = Task.Run(...);
If you don't need any of the above, then you can do a "fire and forget", as such:
var _ = Task.Run(...); // or just "Task.Run(...);"
Note that true "fire and forget" applications are extremely rare. It's literally saying "run this code, but I don't care whether it completes, when it completes, or whether it was successful".

You can use Task.Run() when handling logic with fire and forget type, similar to invoking events if someone is subscribed to them. You can use this for logging, notifying, etc.
If you depend of the result or actions executed in your method, you need to use await Task.Run() as it pauses current execution until your task is finished.

async and await without "threads"? Can I customize what happens under-the-hood?

I have a question about how customizable the new async/await keywords and the Task class in C# 4.5 are.
First some background for understanding my problem: I am developing on a framework with the following design:
One thread has a list of "current things to do" (usually around 100 to 200 items) which are stored as an own data structure and hold as a list. It has an Update() function that enumerates the list and look whether some "things" need to execute and does so. Basically its like a big thread sheduler. To simplify things, lets assume the "things to do" are functions that return the boolean true when they are "finished" (and should not be called next Update) and false when the sheduler should call them again next update.
All the "things" must not run concurrently and also must run in this one thread (because of thread static variables)
There are other threads which do other stuff. They are structured in the same way: Big loop that iterates a couple of hundret things to do in a big Update() - function.
Threads can send each other messages, including "remote procedure calls". For these remote calls, the RPC system is returning some kind of future object to the result value. In the other thread, a new "thing to do" is inserted.
A common "thing" to do are just sequences of RPCs chained together. At the moment, the syntax for this "chaining" is very verbose and complicated, since you manually have to check for the completion state of previous RPCs and invoke the next ones etc..
An example:
Future f1, f2;
bool SomeThingToDo() // returns true when "finished"
{
if (f1 == null)
f1 = Remote1.CallF1();
else if (f1.IsComplete && f2 == null)
f2 = Remote2.CallF2();
else if (f2 != null && f2.IsComplete)
return true;
return false;
}
Now this all sound awefull like async and await of C# 5.0 can help me here. I haven't 100% fully understand what it does under the hood (any good references?), but as I get it from some few talks I've watched, it exactly does what I want with this nicely simple code:
async Task SomeThingToDo() // returning task is completed when this is finished.
{
await Remote1.CallF1();
await Remote2.CallF2();
}
But I can't find a way how write my Update() function to make something like this happen. async and await seem to want to use the Task - class which in turn seems to need real threads?
My closest "solution" so far:
The first thread (which is running SomeThingToDo) calls their functions only once and stores the returned task and tests on every Update() whether the task is completed.
Remote1.CallF1 returns a new Task with an empty Action as constructor parameter and remembers the returned task. When F1 is actually finished, it calls RunSynchronously() on the task to mark it as completed.
That seems to me like a pervertion of the task system. And beside, it creates shared memory (the Task's IsComplete boolean) between the two threads which I would like to have replaced with our remote messanging system, if possible.
Finally, it does not solve my problem as it does not work with the await-like SomeThingToDo implementation above. It seems the auto-generated Task objects returned by an async function are completed immediately?
So finally my questions:
Can I hook into async/await to use my own implementations instead of Task<T>?
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Any good reference what exactly happens when I write async and await?

I haven't 100% fully understand what it does under the hood - any good references?
Back when we were designing the feature Mads, Stephen and I wrote some articles at a variety of different levels for MSDN magazine. The links are here:
http://blogs.msdn.com/b/ericlippert/archive/2011/10/03/async-articles.aspx
Start with my article, then Mads's, then Stephen's.
It seems the auto-generated Task objects returned by an async function are completed immediately?
No, they are completed when the code in the method body returns or throws, same as any other code.
Can I hook into async/await to use my own implementations instead of Task<T>?
A method which contains an await must return void, Task or Task<T>. However, the expression that is awaited can return any type so long as you can call GetAwaiter() on it. That need not be a Task.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Absolutely. A Task just represents work that will complete in the future. Though that work is typically done on another thread, there is no requirement.

To answer your questions:
Can I hook into async/await to use my own implementations instead of Task?
Yes. You can await anything. However, I do not recommend this.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
The Task type represents a future. It does not necessarily "run" on a thread; it can represent the completion of a download, or a timer expiring, etc.
Any good reference what exactly happens when I write async and await?
If you mean as far as code transformations go, this blog post has a nice side-by-side. It's not 100% accurate in its details, but it's enough to write a simple custom awaiter.
If you really want to twist async to do your bidding, Jon Skeet's eduasync series is the best resource. However, I seriously do not recommend you do this in production.
You may find my async/await intro helpful as an introduction to the async concepts and recommended ways to use them. The official MSDN documentation is also unusually good.
I did write the AsyncContext and AsyncContextThread classes that may work for your situation; they define a single-threaded context for async/await methods. You can queue work (or send messages) to an AsyncContextThread by using its Factory property.

Can I hook into async/await to use my own implementations instead of Task?
Yes.
If that's not possible, can I use Task without anything that relates to "blocking" and "threads"?
Yes.
Any good reference what exactly happens when I write async and await?
Yes.
I would discourage you from asking yes/no questions. You probably don't just want yes/no answers.
async and await seem to want to use the Task - class which in turn seems to need real threads?
Nope, that's not true. A Task represents something that can be completed at some point in the future, possibly with a result. It's sometimes the result of some computation in another thread, but it doesn't need to be. It can be anything that is happening at some point in the future. For example, it could be the result of an IO operation.
Remote1.CallF1 returns a new Task with an empty Action as constructor parameter and remembers the returned task. When F1 is actually finished, it calls RunSynchronously() on the task to mark it as completed.
So what you're missing here is the TaskCompletionSource class. With that missing puzzle piece a lot should fit into place. You can create the TCS object, pass the Task from it's Task property around to...whomever, and then use the SetResult property to signal it's completion. Doing this doesn't result in the creation of any additional threads, or use the thread pool.
Note that if you don't have a result and just want a Task instead of a Task<T> then just use a TaskCompletionSource<bool> or something along those lines and then SetResult(false) or whatever is appropriate. By casting the Task<bool> to a Task you can hide that implementation from the public API.
That should also provide the "How" variations of the first two questions that you asked instead of the "can I" versions you asked. You can use a TaskCompletionSource to generate a task that is completed whenever you say it is, using whatever asynchronous construct you want, which may or may not involve the use of additional threads.

Hook when async method is started

Is there an integration point into the async/await mechanisms to hook in and know when an async method is started?
Can a custom TaskScheduler provide this hook? A custom SynchronizationContext?
I want to be able to track certain method calls and know which Task (or code) they're associated with.
Thanks!

According to the Task-based Asynchronous Pattern in section "Task Status":
All tasks returned from TAP methods must be “hot” [...] meaning that the asynchronous operations they represent have already been initiated and their TaskStatus is an enumeration value other than Created.
That means that you will always start the Task as soon as you create it (at least if the factory method implements the TAP, which is true for all methods provided by the .Net framework).
If you create and return a "cold" (non-running) Task using new Task(), there is no clean way to find out if the Task.Start() method was called. You can only check the TaskStatus property periodically.

Async/Await - is it *concurrent*?

I've been considering the new async stuff in C# 5, and one particular question came up.
I understand that the await keyword is a neat compiler trick/syntactic sugar to implement continuation passing, where the remainder of the method is broken up into Task objects and queued-up to be run in order, but where control is returned to the calling method.
My problem is that I've heard that currently this is all on a single thread. Does this mean that this async stuff is really just a way of turning continuation code into Task objects and then calling Application.DoEvents() after each task completes before starting the next one?
Or am I missing something? (This part of the question is rhetorical - I'm fully aware I'm missing something :) )

It is concurrent, in the sense that many outstanding asychronous operations may be in progress at any time. It may or may not be multithreaded.
By default, await will schedule the continuation back to the "current execution context". The "current execution context" is defined as SynchronizationContext.Current if it is non-null, or TaskScheduler.Current if there's no SynchronizationContext.
You can override this default behavior by calling ConfigureAwait and passing false for the continueOnCapturedContext parameter. In that case, the continuation will not be scheduled back to that execution context. This usually means it will be run on a threadpool thread.
Unless you're writing library code, the default behavior is exactly what's desired. WinForms, WPF, and Silverlight (i.e., all the UI frameworks) supply a SynchronizationContext, so the continuation executes on the UI thread (and can safely access UI objects). ASP.NET also supplies a SynchronizationContext that ensures the continuation executes in the correct request context.
Other threads (including threadpool threads, Thread, and BackgroundWorker) do not supply a SynchronizationContext. So Console apps and Win32 services by default do not have a SynchronizationContext at all. In this situation, continuations execute on threadpool threads. This is why Console app demos using await/async include a call to Console.ReadLine/ReadKey or do a blocking Wait on a Task.
If you find yourself needing a SynchronizationContext, you can use AsyncContext from my Nito.AsyncEx library; it basically just provides an async-compatible "main loop" with a SynchronizationContext. I find it useful for Console apps and unit tests (VS2012 now has built-in support for async Task unit tests).
For more information about SynchronizationContext, see my Feb MSDN article.
At no time is DoEvents or an equivalent called; rather, control flow returns all the way out, and the continuation (the rest of the function) is scheduled to be run later. This is a much cleaner solution because it doesn't cause reentrancy issues like you would have if DoEvents was used.

The whole idea behind async/await is that it performs continuation passing nicely, and doesn't allocate a new thread for the operation. The continuation may occur on a new thread, it may continue on the same thread.

The real "meat" (the asynchronous) part of async/await is normally done separately and the communication to the caller is done through TaskCompletionSource. As written here http://blogs.msdn.com/b/pfxteam/archive/2009/06/02/9685804.aspx
The TaskCompletionSource type serves two related purposes, both alluded to by its name: it is a source for creating a task, and the source for that task’s completion. In essence, a TaskCompletionSource acts as the producer for a Task and its completion.
and the example is quite clear:
public static Task<T> RunAsync<T>(Func<T> function)
{
if (function == null) throw new ArgumentNullException(“function”);
var tcs = new TaskCompletionSource<T>();
ThreadPool.QueueUserWorkItem(_ =>
{
try
{
T result = function();
tcs.SetResult(result);
}
catch(Exception exc) { tcs.SetException(exc); }
});
return tcs.Task;
}
Through the TaskCompletionSource you have access to a Task object that you can await, but it isn't through the async/await keywords that you created the multithreading.
Note that when many "slow" functions will be converted to the async/await syntax, you won't need to use TaskCompletionSource very much. They'll use it internally (but in the end somewhere there must be a TaskCompletionSource to have an asynchronous result)

The way I like to explain it is that the "await" keyword simply waits for a task to finish but yields execution to the calling thread while it waits. It then returns the result of the Task and continues from the statement after the "await" keyword once the Task is complete.
Some people I have noticed seem to think that the Task is run in the same thread as the calling thread, this is incorrect and can be proved by trying to alter a Windows.Forms GUI element within the method that await calls. However, the continuation is run in the calling thread where ever possible.
Its just a neat way of not having to have callback delegates or event handlers for when the Task completes.

I feel like this question needs a simpler answer for people. So I'm going to oversimplify.
The fact is, if you save the Tasks and don't await them, then async/await is "concurrent".
var a = await LongTask1(x);
var b = await LongTask2(y);
var c = ShortTask(a, b);
is not concurrent. LongTask1 will complete before LongTask2 starts.
var a = LongTask1(x);
var b = LongTask2(y);
var c = ShortTask(await a, await b);
is concurrent.
While I also urge people to get a deeper understanding and read up on this, you can use async/await for concurrency, and it's pretty simple.