Develop Reference

Proper method of creating, not awaiting, and ensuring completion of, Tasks

Preface: I don't have a good understanding of the underlying implementation of tasks in C#, only their usage. Apologies for anything I butcher below:
I'm unable to find a good answer to the question of "How can I start a task but not await it?" in C#. More specifically, how can I guarantee that the task completes even if the context of the task is finalized/destroyed?
_ = someFunctionAsync(); is satisfactory for launching and forgetting about a task, but what if the parent is transient? What if the task cannot complete before the parent task? This is a frequent occurrence in controller methods, and tasks written in the fashion _ = someFunctionAsync(); are subject to cancellation.
Example Code:
[HttpGet]
public IActionResult Get()
{
_ = DoSomethingAsync();
return StatusCode(204);
}
In order to combat this cancellation, I created a (fairly stupid, imo) static class to hold onto the tasks so that they have time to complete, but it does not work, as the tasks are cancelled when the parent controller is destroyed:
public static class IncompleteTaskManager
{
private static ConcurrentBag<Task> _incompleteTasks = new();
private static event EventHandler<Task>? _onTaskCompleted;
public static void AddTask(Task t)
{
_onTaskCompleted += (sender, task) =>
{
_incompleteTasks = new ConcurrentBag<Task>(_incompleteTasks.Where(task => task != t));
};
_incompleteTasks.Add(CreateTaskWithRemovalEvent(t));
}
private static async Task CreateTaskWithRemovalEvent(Task t)
{
await t;
_onTaskCompleted?.Invoke(null, t);
}
}
Plus, this seems convoluted and feels like a bad solution to a simple problem. So, how the heck do I handle this? What is the proper way of starting a task, forgetting about it, but guaranteeing it runs to completion?
Edit 1, in case anyone suggests it: I've read posts suggesting that _ = Task.Run(async () => await someFunctionAsync()); may serve my needs, but this is not the case either. Though another thread runs the method, its context is lost as well and the task is cancelled, cancelling the child task.
Edit 2: I realize that the controller example is not necessarily the best, as I could simply write the code differently to respond immediately, then wait for the method to complete before disposing of the controller:
[HttpGet]
public async Task Get()
{
base.Response.StatusCode = 204;
await base.Response.CompleteAsync(); //Returns 204 to caller here.
await DoSomethingAsync();
}

There's a lot to unpack here. I'll probably miss a few details, but let me share a few things that should set up a pretty good foundation.
Fundamentally, what it sounds like you're asking about is how to create background tasks in ASP.NET. In .NET 4.x, there was a QueueBackgroundWorkItem method created for this purpose: it gave your task a new cancellation token to use instead of the one provided by the controller action, and it switched you to a different context for the action you provided.
In asp.net core, there are more powerful (but more complicated) IHostedService implementations, including the BackgroundService, but there's nothing quite as simple as QueueBackgroundWorkItem. However, the docs include an example showing how you can use a BackgroundService to basically write your own implementation of the same thing. If you use their code, you should be able to inject an IBackgroundTaskQueue into your controller and call QueueBackgroundWorkItemAsync to enqueue a background task.
Both of these approaches take care of the need to have something await the tasks that get started. You can never truly "guarantee" completion of any given tasks, but they can at least handle the common use cases more gracefully. For example, they let your hosting environment (e.g. IIS) know that something is still running, so it doesn't automatically shut down just because no requests are coming in. And if the hosting environment is being instructed to shut down, it can signal that fact through the cancellation tokens and you can hopefully quickly get your task into a safe state for shutting down rather than being unceremoniously aborted.
They also handle the problem of uncaught exceptions in the background tasks: the exceptions are caught and logged instead of either being silently eaten or completely killing the application.
Neither of these do anything to help you maintain context about things like the current request or user. This is sensible, because the whole point is to allow an action to extend beyond the scope of any given request. So you'll need to write any code you call in these actions to not rely on HttpContext/IHttpContextAccessor or anything stateful like that. Instead, gather what information you need from the context prior to enqueueing the background task, and pass that information along as variables and parameters to downstream code. This is usually good practice anyway, since the HTTP Context is a responsibility that should stay in controller-level code, while most of your business logic should think in terms of business-level models instead. And relying on State is usually best avoided where possible, to create software that's more reliable, testable, etc.
For other types of applications, there are other approaches you'd need to take. Usually it's best to do an internet search for [framework] background tasks where [framework] is the framework you're working in (WPF, e.g.). Different frameworks will have different restrictions. For example, if you write a console app that expects to run without any interaction beyond the command-line arguments, the Task returned from your Main function will probably need to await all the tasks that you start therein. A WPF app, on the other hand, might kick off several background tasks when events like button clicks are invoked, but there are tricks to make sure you do CPU-intensive work on background threads while only interacting with UI elements while on the UI thread.

how can I guarantee that the task completes even if the context of the task is finalized/destroyed?
...
the tasks are cancelled when the parent controller is destroyed
...
Though another thread runs the method, its context is lost as well and the task is cancelled, cancelling the child task.
Your core question is about how to run a task that continues running after the request completes. So there is no way to preserve the request context. Any solution you use must copy any necessary information out of the request context before the request completes.
Plus, this seems convoluted and feels like a bad solution to a simple problem. So, how the heck do I handle this? What is the proper way of starting a task, forgetting about it, but guaranteeing it runs to completion?
That last part is the stickler: "guaranteeing it runs to completion". Discarding tasks, using Task.Run, and using an in-memory collection of in-progress tasks are all incorrect solutions in this case.
The only correct solution is even more convoluted than these relatively simple approaches: you need a basic distributed architecture (explained in detail on my blog). Specifically:
A durable queue (e.g., Azure Queue). This holds the serialized representation of the work to be done - including any values from the request context.
A background processor (e.g., Azure Function). I prefer independent background processors, but it's also possible to use BackgroundService for this.
The durable queue is the key; it's the only way to guarantee the tasks will be executed.

Can you use ConfigureAwait(false) without being thread-safe?

I see people all over the place recommend using ConfigureAwait(false) where you can, and it is a must for library authors, and so on.
But since the continuation of ConfigureAwait(false) can run on any thread from thread pool, then how can you safely protect against multiple threads accessing the same state in your library?
Say you have the following API for your library:
async Task FooAsync()
{
// Do something
//barAsync and saveToFileAsync are private methods.
await barAsync().ConfigureAwait(false);
// counter is a private field
counter++;
await saveToFileAsync().ConfigureAwait(false);
// Do other things
}
If a UI thread keeps calling this FooAsync (e.g. because of user pressing button), wouldn't this code corrupt the value of counter and the file saved? Since multiple threads might be executing?
I find it hard to fanthom using ConfigureAwait(false) without being thread-safe, except for the simplist cases that do not modify state.
Update
I might not have been clear, but in our team, we decided we are going single-threaded. And so, from the answers below, it seems we can't use ConfigureAwait(false) then, since it introduces the possibility of parallelism, which needs to be controlled using locks and so on.

But since the continuation of ConfigureAwait(false) can run on any thread from thread pool, then how can you safely protect against multiple threads accessing the same state in your library?
await does introduce the possibility of reentrancy, but having it actually cause a problem is rare. Asynchronous code by its nature encourages a more functional kind of structure (inputs to a method are its parameters, and outputs are its return values). It's possible to have asynchronous methods have side effects and depend on state, but it's not terribly common.
Note that it is the await that causes accidental reentrancy. ConfigureAwait(false) resumes on the thread pool, but that doesn't cause the issue here.
If a UI thread keeps calling this FooAsync (e.g. because of user pressing button), wouldn't this code corrupt the value of counter and the file saved? Since multiple threads might be executing?
Yes and sort of. Yes, the counter may get an unexpected value, but it's not necessarily because of multiple threads. Consider the same code without ConfigureAwait(false): you still have multiple invocations of that function running, just on a single thread. They're still fighting over the counter and any other shared state. In that case, because of the single thread, counter++ is atomic, but because it's shared, a single invocation of that function may see the value unexpectedly change when resuming from an await.
With ConfigureAwait(false), you do have the additional concern of accidental parallelism (with await you have accidental reentrancy), so if you have non-threadsafe shared state, things can get worse. Reentrancy can cause unexpected states, but parallelism can cause invalid states.

ConfigureAwait is not about thread-safety. It's about avoiding capturing the context.
If you want your code to be thread-safe, then you should implement it to be. This usually involves using some kind of synchronization construct(s), such as for example a lock.
As already pointed out, your FooAsync() is not thread-safe even if you remove the calls to ConfigureAwait(false). Two or more threads can still call it simultaneously, even in a UI application where there is a SynchronizationContext available.
how can you safely protect against multiple threads accessing the same state in your library?
By synchronizing the access to any shared resource. Assuming counter is the only critical section in your code, you could make the method thread-safe using the Interlocked.Increment API:
async Task FooAsync()
{
...
Interlocked.Increment(ref counter);
...
}
This will increment counter and store the new result as an atomic operation.
There are a bunch of other synchronization constructs as well. Which one to use depends on what you are doing basically. Avoid calling ConfigureAwait(false) is not a way to make code thread-safe though.

C# Async await deadlock problem gone in .NetCore?

In .NetFramework there was a high risk of a deadlock occuring when synchronizing to the synchronization context using:
var result = asyncMethod().Result;
var result = asyncMethod().GetAwaiter().GetResult();
instead of
var result = await asyncMethod();
(read Stephen Cleary blogpost for more info)
Since the synchronization context has been removed in .NetCore. Does this mean that the above methods are now safe to use?

Yes and no. It's true that there's no synchronization context in .NET Core, and thereby, one of the major sources of deadlock issues has been eliminated. However, this doesn't mean that deadlocks are totally impossible. Regardless, you should not let good programming practices slide, just because it may not be a big issue in one circumstance any more. ASP.NET Core, in particular, is fully async, so there is no reason to ever use a sync version of a method or simply block on an async task. Use await as you always would and should.

You Can Block on Async Code - But You Shouldn’t
The first and most obvious consequence is that there’s no context captured by await. This means that blocking on asynchronous code won’t cause a deadlock. You can use Task.GetAwaiter().GetResult() (or Task.Wait or Task.Result) without fear of deadlock.
However, you shouldn’t. Because the moment you block on asynchronous code, you’re giving up every benefit of asynchronous code in the first place. The enhanced scalability of asynchronous handlers is nullified as soon as you block a thread.
There were a couple of scenarios in (legacy) ASP.NET where blocking was unfortunately necessary: ASP.NET MVC filters and child actions. However, in ASP.NET Core, the entire pipeline is fully asynchronous; both filters and view components execute asynchronously.
In conclusion, ideally you should strive to use async all the way; but if your code needs to, it can block without danger.
-Extract from blogpost by Stephen Cleary
Credit to GSerg for finding the post
However, you might encounter thread pool starvation
http://labs.criteo.com/2018/10/net-threadpool-starvation-and-how-queuing-makes-it-worse/

Async when continuing on another thread

I know that a Task may continue the execution on another thread, proven by this code.
public async Task Test()
{
var id1 = System.Environment.CurrentManagedThreadId;
await Task.Delay(1000);
var id2 = System.Environment.CurrentManagedThreadId;
Console.Write($"First {id1} then {id2}");
}
I expect the framework to handle memory barrieres, so that id1 is visible when accessed in the last statement.
But what if you are using some kind of framework e.g. NHibernate where the ISession is not thread safe. Frameworks may even check that the thread ID is still the same. How does this mix?
When building website, I'll to use a IOC container with a nested container per request, but when then thread could change within the same request, wouldn't this lead to all kind of problems? ThreadStatic would not work as expected

Not being thread safe usually means don't use it from multiple threads at the same time, rather than don't use it from one thread and then from another thread later.
I don't know about NHibernate specifically, but if it's a problem, consider using EF Core.
In general with async, don't use variables attached to specific threads, like thread local or thread static.
However, local variables, class members, logical call contexts, HttpContext if you're on asp.net etc continue to work. If you do have something that will be lost after an await, you can usually just save it to a local variable first.
The default ConfigureAwait(true) also restores some context on the continuation, but it can be hard to know what is restored and what isn't. Later versions of .net do a better job of this, like restoring the culture so that resources keep working after an await.
There's a good article by Stephen Toub here:
https://blogs.msdn.microsoft.com/pfxteam/2012/06/15/executioncontext-vs-synchronizationcontext/

Is async and await exclusively for GUI-based asynchronous programming?

I've been reading about the new async and await operators in C# and tried to figure out in which circumstances they would possibly be useful to me. I studied several MSDN articles and here's what I read between the lines:
You can use async for Windows Forms and WPF event handlers, so they can perform lengthy tasks without blocking the UI thread while the bulk of the operation is being executed.
async void button1_Click(object sender, EventArgs e)
{
// even though this call takes a while, the UI thread will not block
// while it is executing, therefore allowing further event handlers to
// be invoked.
await SomeLengthyOperationAsync();
}
A method using await must be async, which means that the usage of any async function somewhere in your code ultimately forces all methods in the calling sequence from the UI event handlers up until the lowest-level async method to be async as well.
In other words, if you create a thread with an ordinary good old ThreadStart entry point (or a Console application with good old static int Main(string[] args)), then you cannot use async and await because at one point you would have to use await, and make the method that uses it async, and hence in the calling method you also have to use await and make that one async and so on. But once you reach the thread entry point (or Main()), there's no caller to which an await would yield control to.
So basically you cannot use async and await without having a GUI that uses the standard WinForms and WPF message loop. I guess all that makes indeed sense, since MSDN states that async programming does not mean multithreading, but using the UI thread's spare time instead; when using a console application or a thread with a user defined entry point, multithreading would be necessary to perform asynchronous operations (if not using a compatible message loop).
My question is, are these assumptions accurate?

So basically you cannot use async and await without having a GUI that uses the standard WinForms and WPF message loop.
That's absolutely not the case.
In Windows Forms and WPF, async/await has the handy property of coming back to the UI thread when the asynchronous operation you were awaiting has completed, but that doesn't mean that's the only purpose to it.
If an asynchronous method executes on a thread-pool thread - e.g. in a web service - then the continuation (the rest of the asynchronous method) will simply execute in any thread-pool thread, with the context (security etc) preserved appropriately. This is still really useful for keeping the number of threads down.
For example, suppose you have a high traffic web service which mostly proxies requests to other web services. It spends most of its time waiting for other things, whether that's due to network traffic or genuine time at another service (e.g. a datbase). You shouldn't need lots of threads for that - but with blocking calls, you naturally end up with a thread per request. With async/await, you'd end up with very few threads, because very few requests would actually need any work performed for them at any one point in time, even if there were a lot of requests "in flight".
The trouble is that async/await is most easily demonstrated with UI code, because everyone knows the pain of either using background threads properly or doing too much work in the UI thread. That doesn't mean it's the only place the feature is useful though - far from it.
Various server-side technologies (MVC and WCF for example) already have support for asynchronous methods, and I'd expect others to follow suit.

A method using await must be async, which means that the usage of any async function somewhere in your code ultimately forces all methods in the calling sequence from the UI event handlers up until the lowest-level async method to be async as well.
Not true - methods marked async just mean they can use await, but callers of those methods have no restrictions. If the method returns Task or Task<T> then they can use ContinueWith or anything else you could do with tasks in 4.0
A good non-UI example is MVC4 AsyncController.
Ultimately, async/await is mostly about getting the compiler rewriting so you can write what looks like synchronous code and avoid all the callbacks like you had to do before async/await was added. It also helps with the SynchronizationContext handling, useful for scenarios with thread affinity (UI frameworks, ASP.NET), but even without those, it's still useful. Main can always do DoStuffAsync().Wait(); for instance. :)

My question is, are these assumptions accurate?
No.
You can use async for Windows Forms and WPF event handlers, so they can perform lengthy tasks without blocking the UI thread while the bulk of the operation is being executed.
True. Also true for other UI applications including Silverlight and Windows Store.
And also true for ASP.NET. In this case, it's the HTTP request thread that is not blocked.
A method using await must be async, which means that the usage of any async function somewhere in your code ultimately forces all methods in the calling sequence from the UI event handlers up until the lowest-level async method to be async as well.
This is a best practice ("async all the way down"), but it's not strictly required. You can block on the result of an asynchronous operation; many people choose to do this in Console applications.
an ordinary good old ThreadStart entry point
Well... I do have to take issue with "ordinary good old". As I explain on my blog, Thread is pretty much the worst option you have for doing background operations.
I recommend you review my introduction to async and await, and follow up with the async / await FAQ.

async-await is only wrapper for Task class manipulations, which is part of so named Tasks Parallel Library - TPL(published before async-await auto code generation tech.)
So fact is you may not use any references to UI controls within async - await.
Typically async-await is powerfull tool for any web and server relations, loading resources, sql. It works with smart waiting data with alive UI.
Typically TPL application: from simple big size loop till multi stages parallel calculations in complex calculations based on shared data (ContinueWith and so on)