I have a loop that creates 5 Tasks. How can I insert a Delay of 5 seconds between each Task. I don't know how to fit Task.Delay(5000) in there.
var tasks = new List<Task<int>>();
for (var i = 0; i < 5; i++)
{
tasks.Add(ProcessQueueAsync());
}
await Task.WhenAll(tasks);
My ProcessQueAsync method calls a server, retrieves data and returns and int.
private async Task<int> ProcessQueAsync()
{
var result = await CallToServer();
return result.Count;
}
for (var i = 0; i < 5; i++)
{
tasks.Add(ProcessQueueAsync());
await Task.Delay(5000);
}
Or:
for (var i = 0; i < 5; i++)
{
await ProcessQueueAsync();
await Task.Delay(5000);
}
Depending on that you want.
If you want the tasks to run one after the other, with a 5 second delay, you should perhaps look at Task.ContinueWith instead of using Task.WhenAll. This would allow you to run tasks in serial rather than in parallel.
Related
I have this function which checks for proxy servers and currently it checks only a number of threads and waits for all to finish until the next set is starting. Is it possible to start a new thread as soon as one is finished from the maximum allowed?
for (int i = 0; i < listProxies.Count(); i+=nThreadsNum)
{
for (nCurrentThread = 0; nCurrentThread < nThreadsNum; nCurrentThread++)
{
if (nCurrentThread < nThreadsNum)
{
string strProxyIP = listProxies[i + nCurrentThread].sIPAddress;
int nPort = listProxies[i + nCurrentThread].nPort;
tasks.Add(Task.Factory.StartNew<ProxyAddress>(() => CheckProxyServer(strProxyIP, nPort, nCurrentThread)));
}
}
Task.WaitAll(tasks.ToArray());
foreach (var tsk in tasks)
{
ProxyAddress result = tsk.Result;
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
}
tasks.Clear();
}
This seems much more simple:
int numberProcessed = 0;
Parallel.ForEach(listProxies,
new ParallelOptions { MaxDegreeOfParallelism = nThreadsNum },
(p)=> {
var result = CheckProxyServer(p.sIPAddress, s.nPort, Thread.CurrentThread.ManagedThreadId);
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
Interlocked.Increment(numberProcessed);
});
With slots:
var obj = new Object();
var slots = new List<int>();
Parallel.ForEach(listProxies,
new ParallelOptions { MaxDegreeOfParallelism = nThreadsNum },
(p)=> {
int threadId = Thread.CurrentThread.ManagedThreadId;
int slot = slots.IndexOf(threadId);
if (slot == -1)
{
lock(obj)
{
slots.Add(threadId);
}
slot = slots.IndexOf(threadId);
}
var result = CheckProxyServer(p.sIPAddress, s.nPort, slot);
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
});
I took a few shortcuts there to guarantee thread safety. You don't have to do the normal check-lock-check dance because there will never be two threads attempting to add the same threadid to the list, so the second check will always fail and isn't needed. Secondly, for the same reason, I don't believe you need to ever lock around the outer IndexOf either. That makes this a very highly efficient concurrent routine that rarely locks (it should only lock nThreadsNum times) no matter how many items are in the enumerable.
Another solution is to use a SemaphoreSlim or the Producer-Consumer Pattern using a BlockinCollection<T>. Both solution support cancellation.
SemaphoreSlim
private async Task CheckProxyServerAsync(IEnumerable<object> proxies)
{
var tasks = new List<Task>();
int currentThreadNumber = 0;
int maxNumberOfThreads = 8;
using (semaphore = new SemaphoreSlim(maxNumberOfThreads, maxNumberOfThreads))
{
foreach (var proxy in proxies)
{
// Asynchronously wait until thread is available if thread limit reached
await semaphore.WaitAsync();
string proxyIP = proxy.IPAddress;
int port = proxy.Port;
tasks.Add(Task.Run(() => CheckProxyServer(proxyIP, port, Interlocked.Increment(ref currentThreadNumber)))
.ContinueWith(
(task) =>
{
ProxyAddress result = task.Result;
// Method call must be thread-safe!
UpdateProxyDbRecord(result.IPAddress, result.OnlineStatus);
Interlocked.Decrement(ref currentThreadNumber);
// Allow to start next thread if thread limit was reached
semaphore.Release();
},
TaskContinuationOptions.OnlyOnRanToCompletion));
}
// Asynchronously wait until all tasks are completed
// to prevent premature disposal of semaphore
await Task.WhenAll(tasks);
}
}
Producer-Consumer Pattern
// Uses a fixed number of same threads
private async Task CheckProxyServerAsync(IEnumerable<ProxyInfo> proxies)
{
var pipe = new BlockingCollection<ProxyInfo>();
int maxNumberOfThreads = 8;
var tasks = new List<Task>();
// Create all threads (count == maxNumberOfThreads)
for (int currentThreadNumber = 0; currentThreadNumber < maxNumberOfThreads; currentThreadNumber++)
{
tasks.Add(
Task.Run(() => ConsumeProxyInfo(pipe, currentThreadNumber)));
}
proxies.ToList().ForEach(pipe.Add);
pipe.CompleteAdding();
await Task.WhenAll(tasks);
}
private void ConsumeProxyInfo(BlockingCollection<ProxyInfo> proxiesPipe, int currentThreadNumber)
{
while (!proxiesPipe.IsCompleted)
{
if (proxiesPipe.TryTake(out ProxyInfo proxy))
{
int port = proxy.Port;
string proxyIP = proxy.IPAddress;
ProxyAddress result = CheckProxyServer(proxyIP, port, currentThreadNumber);
// Method call must be thread-safe!
UpdateProxyDbRecord(result.IPAddress, result.OnlineStatus);
}
}
}
If I'm understanding your question properly, this is actually fairly simple to do with await Task.WhenAny. Basically, you keep a collection of all of the running tasks. Once you reach a certain number of tasks running, you wait for one or more of your tasks to finish, and then you remove the tasks that were completed from your collection and continue to add more tasks.
Here's an example of what I mean below:
var tasks = new List<Task>();
for (int i = 0; i < 20; i++)
{
// I want my list of tasks to contain at most 5 tasks at once
if (tasks.Count == 5)
{
// Wait for at least one of the tasks to complete
await Task.WhenAny(tasks.ToArray());
// Remove all of the completed tasks from the list
tasks = tasks.Where(t => !t.IsCompleted).ToList();
}
// Add some task to the list
tasks.Add(Task.Factory.StartNew(async delegate ()
{
await Task.Delay(1000);
}));
}
I suggest changing your approach slightly. Instead of starting and stopping threads, put your proxy server data in a concurrent queue, one item for each proxy server. Then create a fixed number of threads (or async tasks) to work on the queue. This is more likely to provide smooth performance (you aren't starting and stopping threads over and over, which has overhead) and is a lot easier to code, in my opinion.
A simple example:
class ProxyChecker
{
private ConcurrentQueue<ProxyInfo> _masterQueue = new ConcurrentQueue<ProxyInfo>();
public ProxyChecker(IEnumerable<ProxyInfo> listProxies)
{
foreach (var proxy in listProxies)
{
_masterQueue.Enqueue(proxy);
}
}
public async Task RunChecks(int maximumConcurrency)
{
var count = Math.Max(maximumConcurrency, _masterQueue.Count);
var tasks = Enumerable.Range(0, count).Select( i => WorkerTask() ).ToList();
await Task.WhenAll(tasks);
}
private async Task WorkerTask()
{
ProxyInfo proxyInfo;
while ( _masterList.TryDequeue(out proxyInfo))
{
DoTheTest(proxyInfo.IP, proxyInfo.Port)
}
}
}
Suppose I kick off 5 async tasks, and I want to print the results in the order they were requested:
public async void RunTasks()
{
var tasks = new List<Task<int>>();
for(int i=1; i<=5; i++)
{
tasks.Add(DoSomething(i));
}
var results = await Task.WhenAll(tasks);
Console.WriteLine(String.Join(',', results));
}
public async Task<int> DoSomething(int taskNumber)
{
var random = new Random();
await Task.Delay(random.Next(5000));
return taskNumber;
}
This will always print "1,2,3,4,5" - because Task.WhenAll() orders the results by the order requested, not by the order in which they finished.
Unfortunately this means I have to wait for ALL Tasks to finish until I can print anything.
How might I instead print the result of each task as soon as it's finished, but still respecting the order they were requested?
So I should always see "1,2,3,4,5" - but it may arrive gradually:
"1"
"1,2,3"
"1,2,3,4"
"1,2,3,4,5"
(no need to worry about the actual reasoning for doing this, treat it as a fun problem)
var tasks = new List<Task<int>>();
for(int i=1; i<=5; i++)
{
tasks.Add(DoSomething(i));
}
foreach (var task in tasks)
{
var result = await task;
Console.WriteLine(result);
}
We kick off all of the tasks first, then loop over them in order, awaiting each in turn. If the task being awaited has previously completed, the await just returns its result. Otherwise we wait until it completes.
Try a TransformBlock it will output the items it processes one by one in the order the were received by default even if the elements are processed in parallel.
public async Task Order()
{
var tBlock = new TransformBlock<int, string>(async x =>
{
await Task.Delay(100);
return x.ToString();
}, new ExecutionDataflowBlockOptions() { MaxDegreeOfParallelism = 10 });
var sub = tBlock.AsObservable().Subscribe(x => Console.Write(x));
foreach (var num in Enumerable.Range(0, 10))
{
tBlock.Post(num);
}
tBlock.Complete();
await tBlock.Completion;
sub.Dispose();
}
Output:
0123456789
C# 8 adds support for asynchronuous iterator blocks, so you can await things and return an IAsyncEnumarator instead of an IEnumerable:
public async IAsyncEnumerable<int> EnumerateAsync() {
for (int i = 0; i < 10; i++) {
yield return i;
await Task.Delay(1000);
}
}
With a non-blocking consuming code that looks like this:
await foreach (var item in EnumerateAsync()) {
Console.WriteLine(item);
}
This will result in my code running for about 10 seconds. However, sometimes I want to break out of the await foreach before all elements are consumed. With an breakhowever, we would need to wait until the current awaited Task.Delay has finished. How can we break immediately out of that loop without waiting for any dangling async tasks?
The use of a CancellationToken is the solution since that is the only thing that can cancel the Task.Delay in your code. The way we get it inside your IAsyncEnumerable is to pass it as a parameter when creating it, so let's do that:
public async IAsyncEnumerable<int> EnumerateAsync(CancellationToken cancellationToken = default) {
for (int i = 0; i < 10; i++) {
yield return i;
await Task.Delay(1000, cancellationToken);
}
}
With the consuming side of:
// In this example the cancellation token will be caneled after 2.5 seconds
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(2.5));
await foreach (var item in EnumerateAsync(cts.Token)) {
Console.WriteLine(item);
}
Sure, this will cancel the enumeration after 3 elements were returned, but will end in an TaskCanceledException thrown out of Task.Delay. To gracefully exit the await foreach we have to catch it and break on the producing side:
public async IAsyncEnumerable<int> EnumerateAsync(CancellationToken cancellationToken = default) {
for (int i = 0; i < 10; i++) {
yield return i;
try {
await Task.Delay(1000, cancellationToken);
} catch (TaskCanceledException) {
yield break;
}
}
}
Note
As of now this is still in preview and is subject to possible change. If you are interested in this topic you can watch a discussion of the C# language team about CancellationToken in an IAsyncEnumeration.
I have a Parallel.For loop which I use to peform a lot of HTTP request at a certain point when a scheduled task occurs like this:
Parallel.For(0, doc.GetElementsByTagName("ItemID").Count, i => {
var xmlResponse = PerformHttpRequestMethod();
});
Is there any way for me to set the loop to pause after the counter value hits 2,4,6,8,10 and so on...
so every 2 method calls it performs, sleep for 2 minutes lets say..
Is there any way I could achieve this ?
I recommend you to use Task.Delay.
Now your method is asyncronous using async/await.
public async Task DoSomething()
{
int i = 0;
while (i < doc.GetElementsByTagName("ItemID").Count)
{
Task.Run(() => PerformHttpRequestMethod());
if(i%2==0){
await Task.Delay(TimeSpan.FromMinutes(2));
//or simply:
await Task.Delay(120000);//120000 represents 2 minutes.
}
i++;
}
}
OR simply if you want to use for loop.
public async Task DoSomething()
{
for (int i = 0; i < doc.GetElementsByTagName("ItemID").Count; i++)
{
Task.Run(() => PerformHttpRequestMethod());
if(i%2==0){
await Task.Delay(TimeSpan.FromMinutes(2));
}
}
}
How would this 2nd example look if I'd want to do iterations from 0 to 4 then sleep 5 to 9 and so on... ?
public async Task DoSomething()
{
for (int i = 0; i < doc.GetElementsByTagName("ItemID").Count; i=i+5)
{
if( i%10 == 0 ){
for( int j=i;j<=i+4;j++){
Task.Run(() => PerformHttpRequestMethod());
}
}
else{
for(int j=i;j<=i+4;j++){
await Task.Delay(TimeSpan.FromMinutes(2));
}
}
}
}
Let's test the correctitude of algorithm.
i=0 -> (0%10==0 ->true) ,then will execute Task.Run(() => PerformHttpRequestMethod()) for i=(0,4)
i=5 -> (5%10==0 ->false), then will execute await Task.Delay(TimeSpan.FromMinutes(2)); for i=(5,9).
And so on...
I don't really see the point of using a Parallel.For if you want to sleep for x number of minutes or seconds every other iteration...how about using a plain old for loop?:
for(int i = 0; i < doc.GetElementsByTagName("ItemID").Count; ++i)
{
var xmlResponse = PerformHttpRequestMethod();
if (i % 2 == 0)
{
System.Threading.Thread.Sleep(TimeSpan.FromMinutes(2));
}
}
Or maybe you want to keep track of the how many iterations that are currently in flight?:
int inFlight = 0;
Parallel.For(0, doc.GetElementsByTagName("ItemID").Count, i => {
System.Threading.Interlocked.Increment(ref inFlight);
if (inFlight % 2 == 0)
System.Threading.Thread.Sleep(TimeSpan.FromMinutes(2));
var xmlResponse = PerformHttpRequestMethod();
System.Threading.Interlocked.Decrement(ref inFlight);
});
You can do that by combining a Parallel.For and a normal for loop:
for(var i = 0;i<doc.GetElementsByTagName("ItemID").Count;i = i+2)
{
Parallel.For(0, 2, i => {
var xmlResponse = PerformHttpRequestMethod();
});
Thread.Sleep(2000);
}
I want to perform multiple loops in the same time using async Task (I don't want use Parallel.ForEach)
I do :
static async void Run()
{
await MultiTasks();
}
static async Task MultiTasks()
{
var Loop1 = Loop1Async();
var Loop2 = Loop2Async();
await Loop1;
await Loop2;
}
static async Task Loop1Async()
{
for (int i = 0; i < 500; i++)
{
Console.WriteLine("Loop 1 : " + i);
}
}
static async Task Loop2Async()
{
for (int i = 0; i < 500; i++)
{
Console.WriteLine("Loop 2 : " + i);
}
}
Run() is called in my Main method.
But the two loop is not executed in the same time. Once the first lap is completed, the second begins
Why and how to do this ?
You have the fundamental misunderstanding common to beginner users of await.
Let me be very clear about this. Await does not make something asynchronous. It asynchronously waits for an asynchronous operation to complete.
You are awaiting a synchronous operation. There is absolutely nothing asynchronous about the methods you are calling; await does not make them asynchronous. Rather, if they were asynchronous, then the await would return control to the caller immediately, so that the caller could continue to do work. When the asynchronous work finishes, then at some point in the future the remainder of the method is executed, with the awaited result.
Loop1Async and Loop2Async in fact are synchronous. Consider using WriteLineAsync method.
You can also use Task.WhenAll in MultiTasks.
Try .WhenAll(...)
static async Task MultiTasks()
{
var Loop1 = Loop1Async();
var Loop2 = Loop2Async();
await Task.WhenAll(Loop1, Loop2);
}
As others have noted your async methods do not currently yield execution.
You need something that will allow threads to yield back to the system so they can actually run in parallel. Something like this...
static async Task Loop1Async()
{
for (int i = 0; i < 500; i++)
{
Console.WriteLine("Loop 1 : " + i);
await Task.Yield();
}
}
static async Task Loop2Async()
{
for (int i = 0; i < 500; i++)
{
Console.WriteLine("Loop 2 : " + i);
await Task.Yield();
}
}
Your Loop1Async and Loop2Async methods does not have an await inside. For a method to be async it needs to have 1 or more await