Develop Reference

How to convert a list of generic tasks of different types that are stored in a List<Task>, to a Task<List<object>>?

I am looking to create a Task<List> that when invoked with a set of methods executes in parallel, runs and returns the result in the same order of the tasks in an array. The Tasks can return different types. I tried below. Not sure if I am heading in right direction.
private async Task<IList<object>> RunTasks<T>(IList<Task> taskList)
{
var allTasks = Task.WhenAll(taskList);
var ret = new object[taskList.Count];
await allTasks;
for (int i=0;i<taskList.Count;i++)
{
ret[i] = taskList[i].IsFaulted ?
default : ((Task<T>)taskList[i]).Result;
}
//otherPolicies.AppsPermissionsPolicy = teamsAppPermissionDocFromAAE
// .ToMTTeamsAppPermissionPolicy().ToMTPolicyDocument();
//Wrap AAE TeamsApp doc response into other Policies
return ret;
}
If Task1 & Task2 returns different types in taskList do we need T for RunTasks ? if so, What type do we pass in to Invoke RunTasks?. If we dont need it, then how do we convert the Tasks return type to its corresponding object in the for loop immediately after tasks completed before we return the object array with results?

I think that converting a List<Task> to a List<Task<object>> cannot be done without reflection. Or without the dynamic keyword, like in the implementation below:
public static Task<object[]> WhenAllToObject(IEnumerable<Task> tasks)
{
ArgumentNullException.ThrowIfNull(tasks);
return Task.WhenAll(tasks.Select(async task =>
{
// First await the task, to ensure that it ran successfully.
await task.ConfigureAwait(false);
// Then try to get its result, if it's a generic Task<T>.
try
{
return await (dynamic)task;
}
catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException)
{
throw new InvalidOperationException("Non-generic task found.");
}
}));
}
Usage example:
List<Task> heterogeneousListOfTasks = new()
{
Task.FromResult(13),
Task.FromResult("Hello"),
Task.FromResult(true),
};
object[] results = await WhenAllToObject(heterogeneousListOfTasks);
Alternative: This one is inspired by IS4's ingenious type-matching trick. The advantage over the above implementation is that the validation of the tasks argument happens synchronously. That's because the pattern matching for the Task case is a non-async method (it elides async and await).
public static Task<object[]> WhenAllToObject(IEnumerable<Task> tasks)
{
ArgumentNullException.ThrowIfNull(tasks);
return Task.WhenAll(tasks.Select(task =>
{
if (task == null) throw new ArgumentException(
$"The {nameof(tasks)} argument included a null value.", nameof(tasks));
Task<object> taskOfObject = ToTaskOfObject((dynamic)task);
if (taskOfObject == null) throw new ArgumentException(
$"The {nameof(tasks)} argument included a non-generic Task.", nameof(tasks));
return taskOfObject;
}));
}
private static Task<object> ToTaskOfObject(Task task) // Not async
=> null;
private static async Task<object> ToTaskOfObject<T>(Task<T> task)
=> await task.ConfigureAwait(false);
Both implementations have similar behavior with the Task.WhenAll method, but not identical. The Task.WhenAll propagates all exceptions of all tasks. On the contrary the WhenAllToObject propagates only the first exception of each task.

If the number of tasks is 2 (you said task1 and task2) you may consider doing this without a loop:
var t1 = DoOneThingAsync();
var t2 = DoAnotherThingAsync();
await Task.WhenAll(t1, t2); // Without this you can miss exceptions
var result1 = await t1;
var result2 = await t2;
For a dynamic number of tasks I would still suggest separating them by type:
List<Task<int>> xs = ...;
List<Task<string>> ys = ...;
var groupX = await Task.WhenAll(xs);
var groupY = await Task.WhenAll(ys);

I'd argue that in most cases, you shouldn't need this, as once you get the result as object, you'll most likely want to cast it back to some concrete type anyway, so why not work with the tasks directly until you are sure what the types are?
Anyway, the non-generic Task class doesn't expose the Result property like Task<T> does, and there aren't any simple workarounds. We'll have to use some tricks!
By far my favourite way is to use dynamic combined with overloads, resulting in a sort of runtime type-based pattern matching:
public static async Task<IEnumerable<object>> TasksResults(IEnumerable<Task> tasks)
{
await Task.WhenAll(tasks);
return tasks.Select(t => TaskResult((dynamic)t));
}
static object TaskResult<T>(Task<T> task)
{
return task.Result;
}
static object TaskResult(Task task)
{
return null;
}
Using (dynamic) makes the expression behave as if t had actually the most concrete type of the object inside. If it is actually Task<T> (with a result), the first overload is chosen, otherwise the second overload is picked. No exception handling necessary.
I've also taken the liberty to use IEnumerable for the argument and return type. Feel free to use .ToList() if you need to.

How to return a Task<T> without await

Is it possible to return a task from a method which first calls multiple Task<T> returning methods and then returns some type that includes the results from previous calls without using await?
For example, the below is straight forward:
public Task<SomeType> GetAsync() => FirstOrDefaultAsync();
However, I would like to do something like this:
public Task<SomeType> GetAsync()
{
var list = GetListAsync(); // <-- Task<List<T>>
var count = GetCountAsync(); // <-- Task<int>
return new SomeType // <-- Obviously compiler error
{
List /* <-- List<T> */ = list, // <-- Also compiler error
Count /* <-- int */ = count, // <-- Also compiler error
};
}
Is it possible to do this without having to write:
public async Task<SomeType> GetAsync()
{
return new Type2
{
List = await GetListAsync(),
Count = await GetCountAsync(),
};
}

Frankly, the version already in the question is correct:
public async Task<SomeType> GetAsync()
{
return new Type2
{
List = await GetListAsync(),
Count = await GetCountAsync(),
};
}
I realize you asked "without using await", but: the hacks to avoid the await are suboptimal; in particular, you should almost never use ContinueWith - that is the legacy API, and the Task implementation is now optimized for await, not ContinueWith.
As for:
Because I read that having multiple await is bad for performance. I try to await on last call
No; once you have one incomplete await, it pretty much doesn't matter how many more you have - they're effectively free. The issue of having one vs zero incomplete await is comparable to the ContinueWith, so : you're not gaining anything by avoiding the await.
Conclusion: just use the await. It is simpler and more direct, and the internals are optimized for it.
As a minor optimization, you might want to add ConfigureAwait(false), i.e.
public async Task<SomeType> GetAsync()
{
return new Type2
{
List = await GetListAsync().ConfigureAwait(false),
Count = await GetCountAsync().ConfigureAwait(false),
};
}
Or if they should run concurrently, and the implementation supports it:
public Task<SomeType> GetAsync()
{
var list = GetListAsync();
var count = GetCountAsync();
return new SomeType
{
List = await list.ConfigureAwait(false),
Count = await count.ConfigureAwait(false),
};
}

You can use Task.WhenAll together with Task.ContinueWith.
public Task<SomeType> GetAsync()
{
var list = GetListAsync();
var count = GetCountAsync();
return Task.WhenAll(list, count).ContinueWith(_ => new Type2
{
List = list.Result,
Count = count.Result,
});
}
Edit
As suggested in comments, you're better off just using await. I also advice to read the post linked by GSerg - Performance of Task.ContinueWith in non-async method vs. using async/await

The problem is that the Task.WhenAll method does not accept tasks with different result types. All tasks must be of the same type. Fortunately this is easy to fix. The WhenAll variant bellow waits for two tasks with different types, and returns a task with the combined results.
public static Task<TResult> WhenAll<T1, T2, TResult>(
Task<T1> task1, Task<T2> task2, Func<T1, T2, TResult> factory)
{
return Task.WhenAll(task1, task2).ContinueWith(t =>
{
var tcs = new TaskCompletionSource<TResult>();
if (t.IsFaulted)
{
tcs.SetException(t.Exception.InnerExceptions);
}
else if (t.IsCanceled)
{
tcs.SetCanceled();
}
else
{
tcs.SetResult(factory(task1.Result, task2.Result));
}
return tcs.Task;
}, default, TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default).Unwrap();
}
It can be used like this:
public Task<SomeType> GetAsync()
{
return WhenAll(GetListAsync(), GetCountAsync(),
(list, count) => new SomeType { List = list, Count = count });
}
The advantage over the other solutions is at the handling of exceptions. If both GetListAsync and GetCountAsync fail, the task returned from GetAsync will preserve both exceptions in a shallow AggregateException (not nested in another AggregateException).
Btw this answer is inspired by a Stephen Cleary's answer here.

Understanding Task vs Task<T> as a return type

Apologies if this has been addressed somewhere, but it is quite overwhelming the amount of resources on the topic.
I wrote the following method that awaits on an async call
public async Task GetAllObjectsNames(string bucketName)
{
if (string.IsNullOrEmpty(bucketName))
{
throw new ArgumentNullException(nameof(bucketName), "bucket name can't be null or empty string");
}
var request = new ListObjectsRequest
{
BucketName = bucketName
};
await s_client.ListObjectsAsync(request);
}
Working on wrapping my head around the difference between returning a Task and returning a Task<T>
public async Task<IEnumerable<string>> GetAllObjectsNames(string bucketName)
{
if (string.IsNullOrEmpty(bucketName))
{
throw new ArgumentNullException(nameof(bucketName), "bucket name can't be null or empty string");
}
var request = new ListObjectsRequest
{
BucketName = bucketName
};
var response = await s_client.ListObjectsAsync(request);
return response.S3Objects.Select(o => o.Key);
}
Could someone chime in?

Task represents an operation which might be in progress, or might have been cancelled, faulted, or completed. A completed Task does not contain any sort of result, it just represents the fact that the operation has finished.
Task<T> also represents an operation which might be in progress, or might have been cancelled, faulted, or might have completed. However, a completed Task<T> will contain the result of that operation, which you can then read.
You can write:
public async Task<IEnumerable<string>> GetAllObjectsNames(string bucketName)
// ...
var objects = await GetAllObjectsNames("Foo");
await GetAllObjectsNames("Foo"); // Throws away the result of the Task
However you can only write:
public async Task GetAllObjectsNames(string bucketName)
// ...
await GetAllObjectsNames("Foo");
The Task returned from this version of GetAllObjectsNames cannot contain a result, so you can only tell when that operation has finished - you cannot get a result out of it.
You might use Task as the return value from an UploadObject method - where you care when the upload operation finishes, but there's no result for you to read. You would probably use Task<T> for GetAllObjectsNames, since the point of that operation is to fetch some values for you to then use.
Conceptually it's the same as the difference between a method which returns void (does not return a result), or a method which returns anything else. Or the difference between Action and Func<T>.

System.Threading.Tasks.Task<TResult> represents an async operation which can return a value (of type TResult).
(If it helps, this is akin to System.Func<…, TResult>, which basically is an action that returns a value).
System.Threading.Tasks.Task represents an async operation (which does not return a value).
(Again, this is akin to System.Action<…>.)

Avoiding async/await if I know most of the time the result will be cached

Which of these is the best option when working with caching in C#?
I am interested at the compiler level which of these is the most elegant / performant solution.
E.g does the .net compiler use any tricks to know that when code will run synchronously and avoid creating/running unnecessary async await code?
Option 1, Use async/await and use Task.FromResult for cached values;
public async Task<T> GetValue<T>(string key)
{
if (_cache.containsKey(key))
{
// 99% of the time will hit this
return Task.FromResult(_cache.GetItem(key));
}
return await _api.GetValue(key);
}
Option 2, Avoid async/await and use something like GetAwaiter().GetResult() for the few times the API Endpoint will be hit.
public T GetValue<T>(string key)
{
if (_cache.containsKey(key))
{
// 99% of the time will hit this
return _cache.GetItem(key);
}
return _api.GetValue(key).GetAwaiter().GetResult();
}
Any insights would be very much appreciated.

The official approach is to cache the Task<T>, and not the T.
This also has the advantage that if someone requests the value, you can kick off the request to fetch the value and then cache the resulting, in-progress Task<T>. If someone else requests the cached value before the request has completed, they're also given the same in-progress Task<T>, and you don't end up making two requests.
For example:
public Task<T> GetValue<T>(string key)
{
// Prefer a TryGet pattern if you can, to halve the number of lookups
if (_cache.containsKey(key))
{
return _cache.GetItem(key);
}
var task = _api.GetValue(key);
_cache.Add(key, task);
return task;
}
Note that you need to think about failure in this case: if the request to the API fails, then you'll be caching a Task which contains an exception. This might be what you want, but it might not be.
If for some reason you can't do this, then the official advice is to use ValueTask<T> for high-performance scenarios. This type has some gotchas (such as you can't await it twice), so I recommend reading this. If you don't have high performance requirements, Task.FromResult is fine.

Your first won't work. The simplest, and the one to go for most of the time is:
public async Task<T> GetValueAsync<T>(string key)
{
if (_cache.ContainsKey(key))
{
return _cache.GetItem(key);
}
T result = await _api.GetValueAysnc(key);
_cache.Add(key, result);
return result;
}
Or better still if possible:
public async Task<T> GetValueAsync<T>(string key)
{
if (_cache.TryGet(key, out T result))
{
return result;
}
result = await _api.GetValueAysnc(key);
_cache.Add(key, result);
return result;
}
This works fine and will return an already-completed task when the value was in the cache, so awaiting it will continue immediately.
However if the value is in the cache much of the time and the method is called often enough for the extra apparatus around async to make a difference then you can avoid it entirely in such a case:
public Task<T> GetValueAsync<T>(string key)
{
if (_cache.TryGet(key, out Task<T> result))
{
return result;
}
return GetAndCacheValueAsync(string key);
}
private async Task<T> GetAndCacheValueAsync<T>(string key)
{
var task = _api.GetValueAysnc(key);
result = await task;
_cache.Add(key, task);
return result;
}
Here if the value is cached we avoid both the state-machine around async and also the creation of a new Task<T> since we have stored an actual Task. Each of these are only done in the first case.

What you are looking for probably is memoization.
A implementation might be something like this:
public static Func<T, TResult> Memoize<T, TResult>(this Func<T, TResult> f)
{
var cache = new ConcurrentDictionary<T, TResult>();
return a => cache.GetOrAdd(a, f);
}
Measure(() => slowSquare(2)); // 00:00:00.1009680
Measure(() => slowSquare(2)); // 00:00:00.1006473
Measure(() => slowSquare(2)); // 00:00:00.1006373
var memoizedSlow = slowSquare.Memoize();
Measure(() => memoizedSlow(2)); // 00:00:00.1070149
Measure(() => memoizedSlow(2)); // 00:00:00.0005227
Measure(() => memoizedSlow(2)); // 00:00:00.0004159
Source

First of all, this calls for linking the speed rant:
https://ericlippert.com/2012/12/17/performance-rant/
Microoptimisations like these are usually left to the JiT. My rule of thumb is that if you really need that difference, then you are propably dealing with realtime programming. And for Realtime Proramming a Garbage Collected runtime like .NET was propably the wrong environment to begin with. Something with direct memory management like unsafe code - even native C++ or Assembler - would have been better.
Secondly, a task might just be the wrong tool here. Maybe what you actually want is something like Lazy[T]? Or any of the 5 different Chache classes? (as with timer, there is about one for specific User Interface technology).
It is possible to use any tool for many purposes. But tasks are for Multitasking and there are better tools for caching and lazy initialisation. And Lazy[T] is even inherently Thread save.

Delay and recurse

I'm trying to write a function that will test some condition and return a value, but if the condition isn't satisfied, it will wait and then try again. I could, of course, just throw in a Thread.Sleep, but I thought I ought to be able to pull this off with tasks and asynch / await, but I can't quite hit of the right syntax to make this work. For example:
public async Task<T> Get<T>(TimeSpan waittime)
{
if (someCondition)
{
return SomeFunctionThatReturnsValue<T>();
}
else
{
return await Get<T>(waitime);
}
}
Works, but doesn't have any delay (obviously), so I've tried multiple variations like this:
public async Task<T> Get<T>(TimeSpan waittime)
{
if (someCondition)
{
return SomeFunctionThatReturnsValue<T>();
}
else
{
return await Task.Delay(waittime).ContinueWith(t => Get<T>(waitime));
}
}
But this give me the compile time error:
Error 52 Since this is an async method, the return expression must be of type 'T' rather than 'Task<T>'
I can change the last return to this:
return await Task.Delay(waittime).ContinueWith(t => Get<T>(waittime).Result);
And it compiles, but that doesn't seem exactly right either.

Since you want your method to be asyncrhonous, when you want to get the result of a Task you should use await, not Result, so that the action is performed asynchronously, so you can write that operation as:
return await await Task.Delay(waittime).ContinueWith(t => Get<T>(waitime));
Alternatively, whenever you have two awaits you can use Unwrap instead; it's not really better or worse; it's equivalent:
return await Task.Delay(waittime).ContinueWith(t => Get<T>(waitime)).Unwrap();
Note that by using Unwrap you could also make the method not be async and not await it because Unwrap is already doing the job of turning your Task<Task<T>> into a Task<T>.
Of course, you generally shouldn't be using ContinueWith in an async method for the most part, you should simply be using await to attach continuations to your tasks:
await Task.Delay(waittime)
return Get<T>(waitime);
It's also worth noting that you should really be using a while loop here, rather than using recursion, particularly because you have an async method which means building another state machine for every single recursive invocation.