Develop Reference

Waiting for all jobs to be finished with async await

I'm trying to understand the usage of async-await in C#5. If I have 2 jobs started in a method, is there a best way to wait for their completion in C#5+ ? I've done the example below but I fail to see what the async await keywork brings here besides free documentation with async keyword.
I made the following example, I want "FINISHED !" to be printed last. It is not the case however. What did I miss ? How can I make the async method wait until all jobs are finished ? Is there a point using async-await here ? I could just do Task.WaitAll with a non-async method here. I don't really understand what async brings in case you want to wait.
class Program
{
static void Main(string[] args)
{
var fooWorker = new FooWorker();
var barWorker = new BarWorker();
var test = new Class1(fooWorker, barWorker);
test.SomeWork();
Console.ReadLine();
}
}
public class Foo
{
public Foo(string bar) => Bar = bar;
public string Bar { get; }
}
public class Class1
{
private IEnumerable<Foo> _foos;
private readonly FooWorker _fooWorker;
private readonly BarWorker _barWorker;
public Class1(FooWorker fooWorker, BarWorker barWorker)
{
_fooWorker = fooWorker;
_barWorker = barWorker;
}
public void SomeWork()
{
_foos = ProduceManyFoo();
MoreWork();
Console.WriteLine("FINISHED !");
}
private async void MoreWork()
{
if (_foos == null || !_foos.Any()) return;
var fooList = _foos.ToList();
Task fooWorkingTask = _fooWorker.Work(fooList);
Task barWorkingTask = _barWorker.Work(fooList);
await Task.WhenAll(fooWorkingTask, barWorkingTask);
}
private IEnumerable<Foo> ProduceManyFoo()
{
int i = 0;
if (++i < 100) yield return new Foo(DateTime.Now.ToString(CultureInfo.InvariantCulture));
}
}
public abstract class AWorker
{
protected virtual void DoStuff(IEnumerable<Foo> foos)
{
foreach (var foo in foos)
{
Console.WriteLine(foo.Bar);
}
}
public Task Work(IEnumerable<Foo> foos) => Task.Run(() => DoStuff(foos));
}
public class FooWorker : AWorker { }
public class BarWorker : AWorker { }

You are firing off tasks and just forgetting them, while the thread continues running. This fixes it.
Main:
static async Task Main(string[] args)
{
var fooWorker = new FooWorker();
var barWorker = new BarWorker();
var test = new Class1(fooWorker, barWorker);
await test.SomeWork();
Console.ReadLine();
}
SomeWork:
public async Task SomeWork()
{
_foos = ProduceManyFoo();
await MoreWork();
Console.WriteLine("FINISHED !");
}
MoreWork signature change:
private async Task MoreWork()
The obvious code smell which should help make the problem clear is using async void. Unless required this should always be avoided.
When using async and await you'll usually want to chain the await calls to the top-level (in this case Main).
await is non-blocking, so anything that calls an async method should really care about the Task being returned.

How to create never ending DataFlow Mesh with exception handling?

I am creating a Task processor which uses TPL DataFlow. I will follow a producer consumer model where in Producer produces some items to be processed once in a while and consumers keep waiting for new items to arrive. Here is my code:
async Task Main()
{
var runner = new Runner();
CancellationTokenSource cts = new CancellationTokenSource();
Task runnerTask = runner.ExecuteAsync(cts.Token);
await Task.WhenAll(runnerTask);
}
public class Runner
{
public async Task ExecuteAsync(CancellationToken cancellationToken) {
var random = new Random();
ActionMeshProcessor processor = new ActionMeshProcessor();
await processor.Init(cancellationToken);
while (!cancellationToken.IsCancellationRequested)
{
await Task.Delay(TimeSpan.FromSeconds(1)); // wait before enqueuing more
int[] items = GetItems(random.Next(3, 7));
await processor.ProcessBlockAsync(items);
}
}
private int[] GetItems(int count)
{
Random randNum = new Random();
int[] arr = new int[count];
for (int i = 0; i < count; i++)
{
arr[i] = randNum.Next(10, 20);
}
return arr;
}
}
public class ActionMeshProcessor
{
private TransformBlock<int, int> Transformer { get; set; }
private ActionBlock<int> CompletionAnnouncer { get; set; }
public async Task Init(CancellationToken cancellationToken)
{
var options = new ExecutionDataflowBlockOptions
{
CancellationToken = cancellationToken,
MaxDegreeOfParallelism = 5,
BoundedCapacity = 5
};
this.Transformer = new TransformBlock<int, int>(async input => {
await Task.Delay(TimeSpan.FromSeconds(1)); //donig something complex here!
if (input > 15)
{
throw new Exception($"I can't handle this number: {input}");
}
return input + 1;
}, options);
this.CompletionAnnouncer = new ActionBlock<int>(async input =>
{
Console.WriteLine($"Completed: {input}");
await Task.FromResult(0);
}, options);
this.Transformer.LinkTo(this.CompletionAnnouncer);
await Task.FromResult(0); // what do I await here?
}
public async Task ProcessBlockAsync(int[] arr)
{
foreach (var item in arr)
{
await this.Transformer.SendAsync(item); // await if there are no free slots
}
}
}
I added a condition check above to throw an exception to mimic an exceptional case.
Here are my questions:
What is the best way I can handle exceptions in the above mesh without bringing the whole mesh down?
Is there a better way to initialize/start/continue a never ending DataFlow mesh?
Where do I await Completion?
I have looked in to this similar question

Exceptions
There's nothing asynchronous in your init it could be a standard synchronous constructor. You can handle exceptions in your mesh without taking the mesh down with a simple try catch in the lamda you provide to the block. You can then handle that case by either filtering the result from your mesh or ignoring the result in the following blocks. Below is an example of filtering. For the simple case of an int you can use an int? and filter out any value that was null or of course you could set any type of magic indicator value if you like. If your actually passing around a reference type you can either push out null or mark the data item as dirty in way that can be examined by the predicate on your link.
public class ActionMeshProcessor {
private TransformBlock<int, int?> Transformer { get; set; }
private ActionBlock<int?> CompletionAnnouncer { get; set; }
public ActionMeshProcessor(CancellationToken cancellationToken) {
var options = new ExecutionDataflowBlockOptions {
CancellationToken = cancellationToken,
MaxDegreeOfParallelism = 5,
BoundedCapacity = 5
};
this.Transformer = new TransformBlock<int, int?>(async input => {
try {
await Task.Delay(TimeSpan.FromSeconds(1)); //donig something complex here!
if (input > 15) {
throw new Exception($"I can't handle this number: {input}");
}
return input + 1;
} catch (Exception ex) {
return null;
}
}, options);
this.CompletionAnnouncer = new ActionBlock<int?>(async input =>
{
if (input == null) throw new ArgumentNullException("input");
Console.WriteLine($"Completed: {input}");
await Task.FromResult(0);
}, options);
//Filtering
this.Transformer.LinkTo(this.CompletionAnnouncer, x => x != null);
this.Transformer.LinkTo(DataflowBlock.NullTarget<int?>());
}
public async Task ProcessBlockAsync(int[] arr) {
foreach (var item in arr) {
await this.Transformer.SendAsync(item); // await if there are no free slots
}
}
}
Completion
You can expose Complete() and Completion from your processor and use those to await the completion when your app shutsdown, assuming thats the only time you'd shutdown the mesh. Also, make sure you propagate completion through your links properly.
//Filtering
this.Transformer.LinkTo(this.CompletionAnnouncer, new DataflowLinkOptions() { PropagateCompletion = true }, x => x != null);
this.Transformer.LinkTo(DataflowBlock.NullTarget<int?>());
}
public void Complete() {
Transformer.Complete();
}
public Task Completion {
get { return CompletionAnnouncer.Completion; }
}
Then, based on your sample the most likely place for completion is outside the loop driving your processing:
public async Task ExecuteAsync(CancellationToken cancellationToken) {
var random = new Random();
ActionMeshProcessor processor = new ActionMeshProcessor();
await processor.Init(cancellationToken);
while (!cancellationToken.IsCancellationRequested) {
await Task.Delay(TimeSpan.FromSeconds(1)); // wait before enqueuing more
int[] items = GetItems(random.Next(3, 7));
await processor.ProcessBlockAsync(items);
}
//asuming you don't intend to throw from cancellation
processor.Complete();
await processor.Completion();
}

How to make different calls to an async method waiting for the result of first call?

Let's say I have a method it gets data from server
Task<Result> GetDataFromServerAsync(...)
If there is an ongoing call in progress, I don't want to start a new request to server but wait for the original to finish.
Let's say I have
var result = await objet.GetDataFromServerAsync(...);
and in a different place, called almost at the same time I have a second call
var result2 = await objet.GetDataFromServerAsync(...);
I don't want the second to start a new request to server if the first didn't finish. I want both calls to get the same result as soon as first call finish. This is a proof of concept, I have options but I wanted to see how easy it's to do this.

Here is a quick example using Lazy<Task<T>>:
var lazyGetDataFromServer = new Lazy<Task<Result>>
(() => objet.GetDataFromServerAsync(...));
var result = await lazyGetDataFromServer.Value;
var result2 = await lazyGetDataFromServer.Value;
It doesn't matter if these 2 awaits are done from separate threads as Lazy<T> is thread-safe, so result2 if ran second will still wait and use the same output from result.
Using the code from here you can wrap this up in a class called AsyncLazy<T>, and add a custom GetAwaiter so that you can just await it without the need to do .Value, very tidy =)

You can use anything for syncrhonization in your method.
For example, I used SemaphoreSlim:
public class PartyMaker
{
private bool _isProcessing;
private readonly SemaphoreSlim _slowStuffSemaphore = new SemaphoreSlim(1, 1);
private DateTime _something;
public async Task<DateTime> ShakeItAsync()
{
try
{
var needNewRequest = !_isProcessing;
await _slowStuffSemaphore.WaitAsync().ConfigureAwait(false);
if (!needNewRequest) return _something;
_isProcessing = true;
_something = await ShakeItSlowlyAsync().ConfigureAwait(false);
return _something;
}
finally
{
_isProcessing = false;
_slowStuffSemaphore.Release();
}
}
private async Task<DateTime> ShakeItSlowlyAsync()
{
await Task.Delay(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
return DateTime.UtcNow;
}
}
Usage:
var maker = new PartyMaker();
var tasks = new[] {maker.ShakeItAsync(), maker.ShakeItAsync()};
Task.WaitAll(tasks);
foreach (var task in tasks)
{
Console.WriteLine(task.Result);
}
Console.WriteLine(maker.ShakeItAsync().Result);
Here is result:
17.01.2017 22:28:39
17.01.2017 22:28:39
17.01.2017 22:28:41
UPD
With this modification you can call async methods with args:
public class PartyMaker
{
private readonly SemaphoreSlim _slowStuffSemaphore = new SemaphoreSlim(1, 1);
private readonly ConcurrentDictionary<int, int> _requestCounts = new ConcurrentDictionary<int, int>();
private readonly ConcurrentDictionary<int, DateTime> _cache = new ConcurrentDictionary<int, DateTime>();
public async Task<DateTime> ShakeItAsync(Argument argument)
{
var key = argument.GetHashCode();
DateTime result;
try
{
if (!_requestCounts.ContainsKey(key))
{
_requestCounts[key] = 1;
}
else
{
++_requestCounts[key];
}
var needNewRequest = _requestCounts[key] == 1;
await _slowStuffSemaphore.WaitAsync().ConfigureAwait(false);
if (!needNewRequest)
{
_cache.TryGetValue(key, out result);
return result;
}
_cache.TryAdd(key, default(DateTime));
result = await ShakeItSlowlyAsync().ConfigureAwait(false);
_cache[key] = result;
return result;
}
finally
{
_requestCounts[key]--;
if (_requestCounts[key] == 0)
{
int temp;
_requestCounts.TryRemove(key, out temp);
_cache.TryRemove(key, out result);
}
_slowStuffSemaphore.Release();
}
}
private async Task<DateTime> ShakeItSlowlyAsync()
{
await Task.Delay(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
return DateTime.UtcNow;
}
}
public class Argument
{
public Argument(int value)
{
Value = value;
}
public int Value { get; }
public override int GetHashCode()
{
return Value.GetHashCode();
}
}

How to use ContinueWith with condition on existing Task<TResult>

I'm trying to get some help on the .ContinueWith() method.
I know something like thisTask<Task<bool>> t2 = nextTask.ContinueWith(t => T.FirstLevel("GG"));
As you see below, this is a demo of what I was trying to do. My first level task returns a bool that will determine which task to continue with, also my second level task will return a bool to determine whether to go back execute the first level again or exit. This is where I'm stuck, I wouldn't want to make it recursive though.
Can anyone help?
I know one can use event handler to resolve this matter but the actual application code is really complex to change at this moment.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Continuewith
{
class Program
{
static void Main(string[] args)
{
tester();
Console.ReadKey();
}
static async void tester()
{
TaskFunctions T = new TaskFunctions();
List<string> p = new List<string>() { "111"};
List<Task<bool>> CocurrentTasks = new List<Task<bool>>();
foreach (string s in p)
{
CocurrentTasks.Add(T.FirstLevel(s));
}
while (CocurrentTasks.Count > 0)
{
Task<bool> nextTask = await Task.WhenAny(CocurrentTasks);
if(await nextTask)
{
//do second level
}
else
{
//do first level
}
CocurrentTasks.Remove(nextTask);
}
}
}
class TaskFunctions
{
public async Task<bool> FirstLevel(string gg)
{
Console.WriteLine(gg);
Random random = new Random();
int randomNumber = random.Next(0, 10);
await Task.Delay(500 * randomNumber); //other real useful Task Function will be in place to take up the time
if (randomNumber > 5)
return true;
else
return false;
}
public async Task<bool> SecondLevel(string jj)
{
Console.WriteLine(jj);
Random random = new Random();
int randomNumber = random.Next(0, 10);
await Task.Delay(500 * randomNumber); //other real useful Task Function will be in place to take up the time
if (randomNumber > 5)
return true;
else
return false;
}
}
}
Here is my solution, not perfect but I can use it alright.
static async void tester()
{
TaskFunctions T = new TaskFunctions();
List<string> p = new List<string>() { "111"};
List<Task<bool>> CocurrentTasks = new List<Task<bool>>();
foreach (string s in p)
{
CocurrentTasks.Add(T.FirstLevel(s));
}
while (CocurrentTasks.Count > 0)
{
Task<bool> nextTask = await Task.WhenAny(CocurrentTasks);
if(await nextTask)
{
Task<Task<bool>> t2 = nextTask.ContinueWith(t => T.SecondLevel("GG"));
if(!await t2.Unwarp())
{
CocurrentTasks.Add(T.FirstLevel("111"));
}
}
else
{
CocurrentTasks.Add(T.FirstLevel("111"));
}
CocurrentTasks.Remove(nextTask);
}
}
This, however, requires the first level task returns "111" so that the program will be execute that specific data again. "111" can be anything one can pass not just limited to a string.

Why do you need FirstLevel and SecondLevel to be separate Tasks? Maybe it would be easier to control those calls as part of one, sequential Task? Basically, you would want to move code in your while to a method:
static async void tester()
{
TaskFunctions T = new TaskFunctions();
List<string> p = new List<string>() { "111" };
List<Task> CocurrentTasks = new List<Task>();
foreach (string s in p)
{
CocurrentTasks.Add(CallConcurrentTasks(T, s));
}
while (CocurrentTasks.Count > 0)
{
var nextTask = await Task.WhenAny(CocurrentTasks);
CocurrentTasks.Remove(nextTask);
}
}
static async Task CallConcurrentTasks(TaskFunctions t, string arg)
{
do
{
if (await t.FirstLevel(arg))
{
if (!await T.SecondLevel("GG"))
{
return;
}
}
}
while (true); //you should probably make some additional end condition here?
}
I hope I got the refacotring right. Alternatively you could use a thread-safe list for ConcurrentTasks collection and use your while code without any changes (collection passed as an argument to CallConcurrentTasks method).

Encapsulate a task to record duration of an async function

If I start with a full console app:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
namespace ConsoleApplication3
{
internal class Program
{
private static void Main(string[] args)
{
var tas = new List<TaskAuditor<string>>();
// I want to await these when they actually run so another thread can use it while
// wait for (example) my EF/SQL to run Async
tas.Add(new TaskAuditor<string>(await GetName(string.Empty, 1)));
tas.Add(new TaskAuditor<string>(await GetName(string.Empty, 2)));
var running = new Task<String>[2];
foreach (var ta in tas)
{
running[0] = ta.Start();
}
var runningCount = tas.Count;
while (runningCount > 0)
{
var idx = Task.WaitAny(running);
runningCount--;
var task = running[idx];
var ta = tas.FirstOrDefault(t => t.Task == task);
Console.WriteLine(ta.Duration.ToString());
}
foreach (var ta in tas)
{
Console.WriteLine(ta.Task.Result);
}
Console.WriteLine("Press any key to exit.");
Console.ReadKey();
}
public async Task<string> GetName(string constr, int id)
{
string result = id.ToString();
// EF/SQL Async goes here
if (string.IsNullOrWhiteSpace(constr))
{
await Task.Delay(1000 * id);
}
return result;
}
}
public class TaskAuditor<T>
{
private Task<T> _task;
private Stopwatch _sw = new Stopwatch();
public TaskAuditor(Task<T> task)
{
_task = task;
}
public Task<T> Start()
{
_sw.Start();
_task.Start();
_sw.Stop();
return _task;
}
public TimeSpan? Duration()
{
TimeSpan? result = null;
if (!_sw.IsRunning)
{
result = _sw.Elapsed;
}
return result;
}
public Task<T> Task
{
get
{
return _task;
}
}
}
}
DotNetFiddle Sample.
The problem is that I need to await the method and it turns into an Async nightmare I can't quite figure out.

First, your auditor needs some work. As I describe on my blog, you can't call Start on a Promise Task. If you want to represent an operation that results in a task, then use Func<Task<T>>. The next problem is that you're stopping the stopwatch before the task actually completes:
public class TaskAuditor<T>
{
private Func<Task<T>> _func;
private Stopwatch _sw = new Stopwatch();
public TaskAuditor(Func<Task<T>> func)
{
_func = func;
}
public async Task<T> StartAsync()
{
_sw.Start();
try
{
return await _func();
}
finally
{
_sw.Stop();
}
}
public TimeSpan? Duration()
{
TimeSpan? result = null;
if (!_sw.IsRunning)
{
result = _sw.Elapsed;
}
return result;
}
}
Next, your calling method should use modern conveniences like Task.WhenAll, and a separate method to handle the result as they each complete instead of trying to pull the results out of a list:
private static async Task<string> ProcessAsync(TaskAuditor<string> auditor)
{
try
{
return await auditor.StartAsync();
}
finally
{
Console.WriteLine(auditor.Duration().ToString());
}
}
private static async Task MainAsync()
{
var tas = new List<TaskAuditor<string>>();
tas.Add(new TaskAuditor<string>(() => GetName(string.Empty, 1)));
tas.Add(new TaskAuditor<string>(() => GetName(string.Empty, 2)));
var running = tas.Select(ta => ProcessAsync(ta));
var results = await Task.WhenAll(running);
foreach (var result in results)
{
Console.WriteLine(result);
}
}
This also keeps the code asynchronous up to the point where it can't be asynchronous, namely, Main:
private static void Main()
{
MainAsync().Wait();
Console.WriteLine("Press any key to exit.");
Console.ReadKey();
}