Develop Reference

Async generator, previous iterations await a future iteration?

I want to generate an enumerable of tasks, the tasks will complete at different times.
How can I make a generator in C# that:
yields tasks
every few iterations, resolves previously yielded tasks with results that are only now known
The reason I want to do this is because I am processing a long iterable of inputs, and every so often I accumulate enough data from these inputs to send a batch API request and finalise my outputs.
Pseudocode:
IEnumerable<Task<Output>> Process(IEnumerable<Input> inputs)
{
var queuedInputs = Queue<Input>();
var cumulativeLength = 0;
foreach (var input in inputs)
{
yield return waiting task for this input
queuedInputs.Enqueue(input);
cumulativeLength += input.Length;
if (cumulativeLength > 10)
{
cumulativeLength = 0
GetFromAPI(queue).ContinueWith((apiTask) => {
Queue<BatchResult> batchResults = apiTask.Result;
while (queuedInputs.Count > 0)
{
batchResult = batchResults.Dequeue();
historicalInput = queuedInputs.Dequeue();
var output = MakeOutput(historicalInput, batchResult);
resolve earlier input's task with this output
}
});
}
}
}

The shape of your solution is going to be driven by the shape of your problem. There's a couple of questions I have because your problem domain seems odd:
Are all your inputs known at the outset? The (synchronous) IEnumerable<Input> implies they are.
Are you sure you want to wait for a batch of inputs before sending any query? What about the "remainder" if you're batching by 10 but have 55 inputs?
Assuming you do have synchronous inputs, and that you want to batch with remainders, you can just accumulate all your inputs immediately, batch them, and walk the batches, asynchronously providing outputs:
async IAsyncEnumerable<Output> Process(IEnumerable<Input> inputs)
{
foreach (var batchedInput in inputs.Batch(10))
{
var batchResults = await GetFromAPI(batchedInput);
for (int i = 0; i != batchedInput.Count; ++i)
yield return MakeOutput(batchedInput[i], batchResults[i]);
}
}
public static IEnumerable<IReadOnlyList<TSource>> Batch<TSource>(this IEnumerable<TSource> source, int size)
{
List<TSource>? batch = null;
foreach (var item in source)
{
batch ??= new List<TSource>(capacity: size);
batch.Add(item);
if (batch.Count == size)
{
yield return batch;
batch = null;
}
}
if (batch?.Count > 0)
yield return batch;
}
Update:
If you want to start the API calls immediately, you can move those out of the loop:
async IAsyncEnumerable<Output> Process(IEnumerable<Input> inputs)
{
var batchedInputs = inputs.Batch(10).ToList();
var apiCallTasks = batchedInputs.Select(GetFromAPI).ToList();
foreach (int i = 0; i != apiCallTasks.Count; ++i)
{
var batchResults = await apiCallTasks[i];
var batchedInput = batchedInputs[i];
for (int j = 0; j != batchedInput.Count; ++j)
yield return MakeOutput(batchedInput[j], batchResults[j]);
}
}

One approach is to use the TPL Dataflow library. This library offers a variety of components named "blocks" (TransformBlock, ActionBlock etc), where each block is processing its input data, and then propagates the results to the next block. The blocks are linked together so that the completion of the previous block in the pipeline triggers the completion of the next block etc, until the final block which is usually an ActionBlock<T> with no output. Here is an example:
var block1 = new TransformBlock<int, string>(item =>
{
Thread.Sleep(1000); // Simulate synchronous work
return item.ToString();
}, new()
{
MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded,
EnsureOrdered = false
});
var block2 = new BatchBlock<string>(batchSize: 10);
var block3 = new ActionBlock<string[]>(async batch =>
{
await Task.Delay(1000); // Simulate asynchronous work
}); // The default MaxDegreeOfParallelism is 1
block1.LinkTo(block2, new() { PropagateCompletion = true });
block2.LinkTo(block3, new() { PropagateCompletion = true });
// Provide some input in the pipeline
block1.Post(1);
block1.Post(2);
block1.Post(3);
block1.Post(4);
block1.Post(5);
block1.Complete(); // Mark the first block as completed
await block3.Completion; // Await the completion of the last block
The TPL Dataflow library is powerful and flexible, but is has a weak point in the propagation of exceptions. There is no built-in way to instruct the block1 to stop working, if the block3 fails. You can read more about this issue here. It might not be a serious issue, if you don't expect your blocks to fail very often.

Assuming MyGenerator() returns List<Task<T>>, and the number of tasks is relatively small (even in the hundreds is probably fine) then you can use Task.WhenAny(), which returns the first Task that completes. Then remove that Task from the list, process the result, and move on to the next:
var tasks = MyGenerator();
while (tasks.Count > 0) {
var t = Task.WhenAny(tasks);
tasks.Remove(t);
var result = await t; // this won't actually wait since the task is already done
// Do something with result
}
There is a good discussion of this in an article by Stephen Toub, which explains in more detail, and gives alternatives if your task list is in the thousands: Processing tasks as they complete
There's also this article, but I think Stephen's is better written: Process asynchronous tasks as they complete (C#)

Using TaskCompletionSource:
IEnumerable<Task<Output>> Process(IEnumerable<Input> inputs)
{
var tcss = new List<TaskCompletionSource<Output>>();
var queue = new Queue<(Input, TaskCompletionSource<Output>)>();
var cumulativeLength = 0;
foreach (var input in inputs)
{
var tcs = new TaskCompletionSource<Output>();
queue.Enqueue((input, tcs));
tcss.Add(tcs);
cumulativeLength += input.Length;
if (cumulativeLength > 10)
{
cumulativeLength = 0
var queueClone = Queue<(Input, TaskCompletionSource<Input>)>(queue);
queue.Clear();
GetFromAPI(queueClone.Select(x => x.Item1)).ContinueWith((apiTask) => {
Queue<BatchResult> batchResults = apiTask.Result;
while (queueClone.Count > 0)
{
var batchResult = batchResults.Dequeue();
var (queuedInput, queuedTcs) = queueClone.Dequeue();
var output = MakeOutput(queuedInput, batchResult);
queuedTcs.SetResult(output)
}
});
}
}
GetFromAPI(queue.Select(x => x.Item1)).ContinueWith((apiTask) => {
Queue<BatchResult> batchResults = apiTask.Result;
while (queue.Count > 0)
{
var batchResult = batchResults.Dequeue();
var (queuedInput, queuedTcs) = queue.Dequeue();
var output = MakeOutput(queuedInput, batchResult);
queuedTcs.SetResult(output)
}
});
foreach (var tcs in tcss)
{
yield return tcs.Task;
}
}

C# Multithreading with slots

I have this function which checks for proxy servers and currently it checks only a number of threads and waits for all to finish until the next set is starting. Is it possible to start a new thread as soon as one is finished from the maximum allowed?
for (int i = 0; i < listProxies.Count(); i+=nThreadsNum)
{
for (nCurrentThread = 0; nCurrentThread < nThreadsNum; nCurrentThread++)
{
if (nCurrentThread < nThreadsNum)
{
string strProxyIP = listProxies[i + nCurrentThread].sIPAddress;
int nPort = listProxies[i + nCurrentThread].nPort;
tasks.Add(Task.Factory.StartNew<ProxyAddress>(() => CheckProxyServer(strProxyIP, nPort, nCurrentThread)));
}
}
Task.WaitAll(tasks.ToArray());
foreach (var tsk in tasks)
{
ProxyAddress result = tsk.Result;
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
}
tasks.Clear();
}

This seems much more simple:
int numberProcessed = 0;
Parallel.ForEach(listProxies,
new ParallelOptions { MaxDegreeOfParallelism = nThreadsNum },
(p)=> {
var result = CheckProxyServer(p.sIPAddress, s.nPort, Thread.CurrentThread.ManagedThreadId);
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
Interlocked.Increment(numberProcessed);
});
With slots:
var obj = new Object();
var slots = new List<int>();
Parallel.ForEach(listProxies,
new ParallelOptions { MaxDegreeOfParallelism = nThreadsNum },
(p)=> {
int threadId = Thread.CurrentThread.ManagedThreadId;
int slot = slots.IndexOf(threadId);
if (slot == -1)
{
lock(obj)
{
slots.Add(threadId);
}
slot = slots.IndexOf(threadId);
}
var result = CheckProxyServer(p.sIPAddress, s.nPort, slot);
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
});
I took a few shortcuts there to guarantee thread safety. You don't have to do the normal check-lock-check dance because there will never be two threads attempting to add the same threadid to the list, so the second check will always fail and isn't needed. Secondly, for the same reason, I don't believe you need to ever lock around the outer IndexOf either. That makes this a very highly efficient concurrent routine that rarely locks (it should only lock nThreadsNum times) no matter how many items are in the enumerable.

Another solution is to use a SemaphoreSlim or the Producer-Consumer Pattern using a BlockinCollection<T>. Both solution support cancellation.
SemaphoreSlim
private async Task CheckProxyServerAsync(IEnumerable<object> proxies)
{
var tasks = new List<Task>();
int currentThreadNumber = 0;
int maxNumberOfThreads = 8;
using (semaphore = new SemaphoreSlim(maxNumberOfThreads, maxNumberOfThreads))
{
foreach (var proxy in proxies)
{
// Asynchronously wait until thread is available if thread limit reached
await semaphore.WaitAsync();
string proxyIP = proxy.IPAddress;
int port = proxy.Port;
tasks.Add(Task.Run(() => CheckProxyServer(proxyIP, port, Interlocked.Increment(ref currentThreadNumber)))
.ContinueWith(
(task) =>
{
ProxyAddress result = task.Result;
// Method call must be thread-safe!
UpdateProxyDbRecord(result.IPAddress, result.OnlineStatus);
Interlocked.Decrement(ref currentThreadNumber);
// Allow to start next thread if thread limit was reached
semaphore.Release();
},
TaskContinuationOptions.OnlyOnRanToCompletion));
}
// Asynchronously wait until all tasks are completed
// to prevent premature disposal of semaphore
await Task.WhenAll(tasks);
}
}
Producer-Consumer Pattern
// Uses a fixed number of same threads
private async Task CheckProxyServerAsync(IEnumerable<ProxyInfo> proxies)
{
var pipe = new BlockingCollection<ProxyInfo>();
int maxNumberOfThreads = 8;
var tasks = new List<Task>();
// Create all threads (count == maxNumberOfThreads)
for (int currentThreadNumber = 0; currentThreadNumber < maxNumberOfThreads; currentThreadNumber++)
{
tasks.Add(
Task.Run(() => ConsumeProxyInfo(pipe, currentThreadNumber)));
}
proxies.ToList().ForEach(pipe.Add);
pipe.CompleteAdding();
await Task.WhenAll(tasks);
}
private void ConsumeProxyInfo(BlockingCollection<ProxyInfo> proxiesPipe, int currentThreadNumber)
{
while (!proxiesPipe.IsCompleted)
{
if (proxiesPipe.TryTake(out ProxyInfo proxy))
{
int port = proxy.Port;
string proxyIP = proxy.IPAddress;
ProxyAddress result = CheckProxyServer(proxyIP, port, currentThreadNumber);
// Method call must be thread-safe!
UpdateProxyDbRecord(result.IPAddress, result.OnlineStatus);
}
}
}

If I'm understanding your question properly, this is actually fairly simple to do with await Task.WhenAny. Basically, you keep a collection of all of the running tasks. Once you reach a certain number of tasks running, you wait for one or more of your tasks to finish, and then you remove the tasks that were completed from your collection and continue to add more tasks.
Here's an example of what I mean below:
var tasks = new List<Task>();
for (int i = 0; i < 20; i++)
{
// I want my list of tasks to contain at most 5 tasks at once
if (tasks.Count == 5)
{
// Wait for at least one of the tasks to complete
await Task.WhenAny(tasks.ToArray());
// Remove all of the completed tasks from the list
tasks = tasks.Where(t => !t.IsCompleted).ToList();
}
// Add some task to the list
tasks.Add(Task.Factory.StartNew(async delegate ()
{
await Task.Delay(1000);
}));
}

I suggest changing your approach slightly. Instead of starting and stopping threads, put your proxy server data in a concurrent queue, one item for each proxy server. Then create a fixed number of threads (or async tasks) to work on the queue. This is more likely to provide smooth performance (you aren't starting and stopping threads over and over, which has overhead) and is a lot easier to code, in my opinion.
A simple example:
class ProxyChecker
{
private ConcurrentQueue<ProxyInfo> _masterQueue = new ConcurrentQueue<ProxyInfo>();
public ProxyChecker(IEnumerable<ProxyInfo> listProxies)
{
foreach (var proxy in listProxies)
{
_masterQueue.Enqueue(proxy);
}
}
public async Task RunChecks(int maximumConcurrency)
{
var count = Math.Max(maximumConcurrency, _masterQueue.Count);
var tasks = Enumerable.Range(0, count).Select( i => WorkerTask() ).ToList();
await Task.WhenAll(tasks);
}
private async Task WorkerTask()
{
ProxyInfo proxyInfo;
while ( _masterList.TryDequeue(out proxyInfo))
{
DoTheTest(proxyInfo.IP, proxyInfo.Port)
}
}
}

Should this work as expected when controlling the amount of tasks running [duplicate]

Let's say I have 100 tasks that do something that takes 10 seconds.
Now I want to only run 10 at a time like when 1 of those 10 finishes another task gets executed till all are finished.
Now I always used ThreadPool.QueueUserWorkItem() for such task but I've read that it is bad practice to do so and that I should use Tasks instead.
My problem is that I nowhere found a good example for my scenario so could you get me started on how to achieve this goal with Tasks?

SemaphoreSlim maxThread = new SemaphoreSlim(10);
for (int i = 0; i < 115; i++)
{
maxThread.Wait();
Task.Factory.StartNew(() =>
{
//Your Works
}
, TaskCreationOptions.LongRunning)
.ContinueWith( (task) => maxThread.Release() );
}

TPL Dataflow is great for doing things like this. You can create a 100% async version of Parallel.Invoke pretty easily:
async Task ProcessTenAtOnce<T>(IEnumerable<T> items, Func<T, Task> func)
{
ExecutionDataflowBlockOptions edfbo = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 10
};
ActionBlock<T> ab = new ActionBlock<T>(func, edfbo);
foreach (T item in items)
{
await ab.SendAsync(item);
}
ab.Complete();
await ab.Completion;
}

You have several options. You can use Parallel.Invoke for starters:
public void DoWork(IEnumerable<Action> actions)
{
Parallel.Invoke(new ParallelOptions() { MaxDegreeOfParallelism = 10 }
, actions.ToArray());
}
Here is an alternate option that will work much harder to have exactly 10 tasks running (although the number of threads in the thread pool processing those tasks may be different) and that returns a Task indicating when it finishes, rather than blocking until done.
public Task DoWork(IList<Action> actions)
{
List<Task> tasks = new List<Task>();
int numWorkers = 10;
int batchSize = (int)Math.Ceiling(actions.Count / (double)numWorkers);
foreach (var batch in actions.Batch(actions.Count / 10))
{
tasks.Add(Task.Factory.StartNew(() =>
{
foreach (var action in batch)
{
action();
}
}));
}
return Task.WhenAll(tasks);
}
If you don't have MoreLinq, for the Batch function, here's my simpler implementation:
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int batchSize)
{
List<T> buffer = new List<T>(batchSize);
foreach (T item in source)
{
buffer.Add(item);
if (buffer.Count >= batchSize)
{
yield return buffer;
buffer = new List<T>();
}
}
if (buffer.Count >= 0)
{
yield return buffer;
}
}

You can create a method like this:
public static async Task RunLimitedNumberAtATime<T>(int numberOfTasksConcurrent,
IEnumerable<T> inputList, Func<T, Task> asyncFunc)
{
Queue<T> inputQueue = new Queue<T>(inputList);
List<Task> runningTasks = new List<Task>(numberOfTasksConcurrent);
for (int i = 0; i < numberOfTasksConcurrent && inputQueue.Count > 0; i++)
runningTasks.Add(asyncFunc(inputQueue.Dequeue()));
while (inputQueue.Count > 0)
{
Task task = await Task.WhenAny(runningTasks);
runningTasks.Remove(task);
runningTasks.Add(asyncFunc(inputQueue.Dequeue()));
}
await Task.WhenAll(runningTasks);
}
And then you can call any async method n times with a limit like this:
Task task = RunLimitedNumberAtATime(10,
Enumerable.Range(1, 100),
async x =>
{
Console.WriteLine($"Starting task {x}");
await Task.Delay(100);
Console.WriteLine($"Finishing task {x}");
});
Or if you want to run long running non async methods, you can do it that way:
Task task = RunLimitedNumberAtATime(10,
Enumerable.Range(1, 100),
x => Task.Factory.StartNew(() => {
Console.WriteLine($"Starting task {x}");
System.Threading.Thread.Sleep(100);
Console.WriteLine($"Finishing task {x}");
}, TaskCreationOptions.LongRunning));
Maybe there is a similar method somewhere in the framework, but I didn't find it yet.

I would love to use the simplest solution I can think of which as I think using the TPL:
string[] urls={};
Parallel.ForEach(urls, new ParallelOptions() { MaxDegreeOfParallelism = 2}, url =>
{
//Download the content or do whatever you want with each URL
});

Why do async await without being told

I am trying to make part of my system run in parallel, but some some reason do it wait for each element before it starts the next even though i have not told it to await. I would like to start executing ExecuteItem for each this.Items and then continue when they are all done.
bool singleThread = false;
public async Task Execute()
{
if (!this.singleThread)
{
var tasks = this.Items.Select(x => this.ExecuteItem(x)).ToArray();
Task.WaitAll(tasks);
}
else
{
foreach (var item in this.Items)
{
await this.ExecuteItem(item);
}
}
}
private async Task ExecuteItem(IMyItem item)
{
MappedDiagnosticsContext.Set("itemRef", item.ItemRef);
try
{
await this.HandelItem(item);
}
catch (Exception exp)
{
Logger.ErrorException(string.Format("Execution for {0} failed.", item.ItemName), exp);
Logger.Error("Error Message: ", exp.Message);
}
MappedDiagnosticsContext.Remove("itemRef");
}
To make clear my problem my code behaves as if had wrote the following
var tasks = this.Items.Select(x => await this.ExecuteItem(x)).ToArray();
To make sure it was not some kind of linq problem have i rewriten the problem code to the following, however the code still blocks tasks[i] = this.ExecuteItem(this.Items[i]);
Task[] tasks = new Task[this.Items.Count];
for (int i = 0; i < this.Items.Count; i++)
{
Console.WriteLine("Adding " + this.Items[i].ItemName);
tasks[i] = this.ExecuteItem(this.Items[i]);
}
Console.WriteLine("Waiting!!!");
Task.WaitAll(tasks);

Something in HandelItem is blocking.
async methods don't run completely asynchronously, they execute synchronously up until the point they hit an await. So all of ExecuteItem, up to HandelItem will run before the tasks list is built. This synchronous behavior would continue into HandelItem if it is an async method, so likely HandelItem is executing while building up the tasks list.
This is easily seen with this example program:
static void Main(string[] args)
{
var items = Enumerable.Range(1, 2);
Console.WriteLine("Start");
var tasks = items.Select(i => AsyncMethod(i)).ToArray();
Console.WriteLine("Got tasks");
Task.WaitAll(tasks);
Console.WriteLine("Done!");
}
static async Task AsyncMethod(int i)
{
Console.WriteLine("Enter {0}", i);
await AsyncMethod2(i);
await Task.Delay(1000);
Console.WriteLine("Exit {0}", i);
}
static async Task AsyncMethod2(int i)
{
Console.WriteLine("Enter2 {0}", i);
await Task.Delay(2000);
Console.WriteLine("Exit2 {0}", i);
}
It's output is:
Start
Enter 1
Enter2 1
Enter 2
Enter2 2
Got tasks
Exit2 2
Exit2 1
Exit 1
Exit 2
Done!
So both async methods run while building the task list, up until the point that they have to wait. So if HandelItem does something non-asynchronous, it will cause blocking.

If you want the tasks to execute in parallel; and wait until all are complete:
await Task.WhenAll(this.Items.Select(item=>this.ExecuteItem(item)));

Have a set of Tasks with only X running at a time

Let's say I have 100 tasks that do something that takes 10 seconds.
Now I want to only run 10 at a time like when 1 of those 10 finishes another task gets executed till all are finished.
Now I always used ThreadPool.QueueUserWorkItem() for such task but I've read that it is bad practice to do so and that I should use Tasks instead.
My problem is that I nowhere found a good example for my scenario so could you get me started on how to achieve this goal with Tasks?

SemaphoreSlim maxThread = new SemaphoreSlim(10);
for (int i = 0; i < 115; i++)
{
maxThread.Wait();
Task.Factory.StartNew(() =>
{
//Your Works
}
, TaskCreationOptions.LongRunning)
.ContinueWith( (task) => maxThread.Release() );
}

TPL Dataflow is great for doing things like this. You can create a 100% async version of Parallel.Invoke pretty easily:
async Task ProcessTenAtOnce<T>(IEnumerable<T> items, Func<T, Task> func)
{
ExecutionDataflowBlockOptions edfbo = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 10
};
ActionBlock<T> ab = new ActionBlock<T>(func, edfbo);
foreach (T item in items)
{
await ab.SendAsync(item);
}
ab.Complete();
await ab.Completion;
}

You have several options. You can use Parallel.Invoke for starters:
public void DoWork(IEnumerable<Action> actions)
{
Parallel.Invoke(new ParallelOptions() { MaxDegreeOfParallelism = 10 }
, actions.ToArray());
}
Here is an alternate option that will work much harder to have exactly 10 tasks running (although the number of threads in the thread pool processing those tasks may be different) and that returns a Task indicating when it finishes, rather than blocking until done.
public Task DoWork(IList<Action> actions)
{
List<Task> tasks = new List<Task>();
int numWorkers = 10;
int batchSize = (int)Math.Ceiling(actions.Count / (double)numWorkers);
foreach (var batch in actions.Batch(actions.Count / 10))
{
tasks.Add(Task.Factory.StartNew(() =>
{
foreach (var action in batch)
{
action();
}
}));
}
return Task.WhenAll(tasks);
}
If you don't have MoreLinq, for the Batch function, here's my simpler implementation:
public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> source, int batchSize)
{
List<T> buffer = new List<T>(batchSize);
foreach (T item in source)
{
buffer.Add(item);
if (buffer.Count >= batchSize)
{
yield return buffer;
buffer = new List<T>();
}
}
if (buffer.Count >= 0)
{
yield return buffer;
}
}

You can create a method like this:
public static async Task RunLimitedNumberAtATime<T>(int numberOfTasksConcurrent,
IEnumerable<T> inputList, Func<T, Task> asyncFunc)
{
Queue<T> inputQueue = new Queue<T>(inputList);
List<Task> runningTasks = new List<Task>(numberOfTasksConcurrent);
for (int i = 0; i < numberOfTasksConcurrent && inputQueue.Count > 0; i++)
runningTasks.Add(asyncFunc(inputQueue.Dequeue()));
while (inputQueue.Count > 0)
{
Task task = await Task.WhenAny(runningTasks);
runningTasks.Remove(task);
runningTasks.Add(asyncFunc(inputQueue.Dequeue()));
}
await Task.WhenAll(runningTasks);
}
And then you can call any async method n times with a limit like this:
Task task = RunLimitedNumberAtATime(10,
Enumerable.Range(1, 100),
async x =>
{
Console.WriteLine($"Starting task {x}");
await Task.Delay(100);
Console.WriteLine($"Finishing task {x}");
});
Or if you want to run long running non async methods, you can do it that way:
Task task = RunLimitedNumberAtATime(10,
Enumerable.Range(1, 100),
x => Task.Factory.StartNew(() => {
Console.WriteLine($"Starting task {x}");
System.Threading.Thread.Sleep(100);
Console.WriteLine($"Finishing task {x}");
}, TaskCreationOptions.LongRunning));
Maybe there is a similar method somewhere in the framework, but I didn't find it yet.

I would love to use the simplest solution I can think of which as I think using the TPL:
string[] urls={};
Parallel.ForEach(urls, new ParallelOptions() { MaxDegreeOfParallelism = 2}, url =>
{
//Download the content or do whatever you want with each URL
});