I was looking for this and I couldn't figure how to do it.
I have a some threats (tasks) in an App.
foreach (string targetMachine in targetMachines)
{
Task<int> task = Task.Run(() => Magic(targetMachine));
}
I created a task for every object in an array.
The task returns a value.
I want to process the values and act based on them. As example, if task return 0 dont do anything, if returns 1 write a log, if returns 2 run a process.
How can I acoomplish this?
If I process the return values inside the foreach:
foreach (string targetMachine in targetMachines)
{
Task<int> task = Task.Run(() => Magic(targetMachine));
Task.Waitforexit()
if (task.result == 2)
{
do something
}
}
I think, task are not going to be useful and the programa will wait each task to be completed to continue.
It can be something like this:
foreach (string targetMachine in targetMachines)
{
Task.Run(() =>
{
var result = Magic(targetMachine);
if (result == 2)
{
DoSomething();
}
});
}
OR (using async/await)
foreach (string targetMachine in targetMachines)
{
var result = await Task.Run(() => Magic(targetMachine));
if (result == 2)
{
DoSomething();
}
}
OR (using ContinueWith)
foreach (string targetMachine in targetMachines)
{
Task<int>.Run(()=>Magic(targetMachine))
.ContinueWith(t =>
{
if (t.Result == 2)
{
DoSomething();
}
});
}
If you want to wait to finish all of your tasks
Parallel.ForEach(targetMachines, targetMachine =>
{
var result = Magic(targetMachine);
if (result == 2)
{
DoSomething();
}
});
Have a look at Task.ContinueWith()
http://msdn.microsoft.com/en-us/library/dd321405.aspx
When each task is complete it passes the result to the ContinueWith which can then either do nothing, log or call a method like you want.
Additionally you could run the foreach .AsParallel() and remove the tasks altogether.
http://msdn.microsoft.com/en-us/library/dd413602.aspx
I'll add a new variant in addition to the variants given in L.B.'s answer:
var tasks = targetMachines.Select(x => Magic(x));
Task.WaitAll(tasks); //fork-join
If you're using async, you can write:
var tasks = targetMachines.Select(x => Magic(x));
await Task.WhenAll(tasks); //fork-join
And instead of
var tasks = targetMachines.Select(x => Magic(x));
you can always write:
var tasks = targetMachines.Select(x =>
{
var result = Magic(x);
if (result == 2) DoSomething();
});
which liberates you from having to use ContinueWith.
Related
Extending: C# async/await await doesn't await
In case the methods executed sequentially are stored in a list as well, how can I add the ExecuteParallelAsync to that list?
private async Task ExecuteSequential()
{
List<Action> sequentialMethods = new List<Action>()
{
SomeMethod1,
SomeMethod2,
await ExecuteParallelAsync, // ???
SomeMethod3,
SomeMethod4
};
for ( int i = 0 ; i < sequentialMethods.Count ; i++ )
{
sequentialMethods.ElementAt( i ).Invoke();
}
}
Clarification:
private async Task ExecuteParallelAsync()
{
List<Action> methods = new List<Action>()
{
MyMethod1,
MyMethod2,
MyMethod3
};
await Task.Run( () => { Parallel.ForEach( methods , ( currentMethod ) => currentMethod.Invoke() ); } );
}
sequentialMethods is a List<Action> but ExecuteParallelAsync is NOT an Action. I've tried to split the list, like suggested. Unfortunately, it doesn't help.
The way suggested by Marc from the original code works fine. But I want to do some extra stuff after each (sequential) method call, which is why I'm trying to use a list and a loop instead of plain method calls.
But when I do so, I'm facing the original problem again, that SomeMethod3 is executed before ExecuteParallelAsync is finished.
Once again, everything in ExecuteParallelAsync can and should be executed simultaneously. Everything in ExecuteSequential has to be executed sequentially.
Solution:
Gabriel is absolutely correct. The crucial statement is this
await Task.Run( () => { Parallel.ForEach( methods , ( currentMethod ) => currentMethod.Invoke() ); } );
When I remove every async and every Task, so that everything is intended to run be executed sequentially this line is the key:
Parallel.ForEach( methods , ( currentMethod ) => currentMethod.Invoke() );
When I use this last statement, everything works correctly.
Thanks to everyone, all your ideas, thoughts and efforts helped and are appreciated.
I'm facing the original problem again, that SomeMethod3 is executed before ExecuteParallelAsync is finished
Then this is not a use case for asynchronous programming. Your requirement is that this is synchronous.
That is especially so since you said that MyMethod1/MyMethod2/MyMethod3 are not asynchronous methods. If they were, that would be a totally different thing. But since they aren't, I don't see any value in trying to use async and await here.
But don't confuse asynchronous with parallel. It seems you want the methods called in ExecuteParallelAsync to be run in parallel, that's fine. You just don't need async and await.
For example:
private void ExecuteSequential()
{
List<Action> sequentialMethods = new List<Action>()
{
SomeMethod1,
SomeMethod2,
ExecuteParallel,
SomeMethod3,
SomeMethod4
};
for ( int i = 0 ; i < sequentialMethods.Count ; i++ )
{
sequentialMethods.ElementAt( i ).Invoke();
}
}
private void ExecuteParallel()
{
List<Action> methods = new List<Action>()
{
MyMethod1,
MyMethod2,
MyMethod3
};
Parallel.ForEach( methods , ( currentMethod ) => currentMethod.Invoke() );
}
You can create an Action that when invoked will start the task and wait for it to complete, like this:
List<Action> sequentialMethods = new List<Action>()
{
SomeMethod1,
SomeMethod2,
() => ExecuteParallelAsync().Wait(),
SomeMethod3,
SomeMethod4
};
Here's a version:
private async Task ExecuteSequential()
{
var sequentialMethods = new List<Func<Task>>()
{
() => Task.Run(SomeMethod1),
() => Task.Run(() => SomeMethod2("Hey!")),
ExecuteParallelAsync,
() => Task.Run(SomeMethod3),
() => Task.Run(SomeMethod4)
};
for ( int i = 0 ; i < sequentialMethods.Count ; i++ )
{
Task t = sequentialMethods[i].Invoke();
await t;
// or just await sequentialMethods[i]();
}
}
Just for the record, ExecuteSequential could just be a standard async method (which are very good in executing things sequentially).
private async Task ExecuteSequential()
{
SomeMethod1();
SomeMethod2();
await ExecuteParallelAsync();
SomeMethod3();
SomeMethod4();
};
Edit: Test
Code
class Program
{
public void MyMethod1() => Console.WriteLine("||My1");
public void MyMethod2() => Console.WriteLine("||My2");
public void MyMethod3() => Console.WriteLine("||My3");
private async Task ExecuteParallelAsync()
{
Console.WriteLine("||Start");
List<Action> methods = new List<Action>() { MyMethod1, MyMethod2, MyMethod3 };
await Task.Run(() => { Parallel.ForEach(methods,
(currentMethod) => currentMethod.Invoke()); }); // This could be just 'currentMethod();' (no Invoke())
Console.WriteLine("||End");
}
public void SomeMethod1() => Console.WriteLine("Some1");
public void SomeMethod2(string s) => Console.WriteLine($"Some2: {s}");
public void SomeMethod3() => Console.WriteLine("Some3");
public void SomeMethod4() => Console.WriteLine("Some4");
private async Task ExecuteSequential()
{
var sequentialMethods = new List<Func<Task>>()
{
() => Task.Run(SomeMethod1),
() => Task.Run(() => SomeMethod2("Hey!")),
ExecuteParallelAsync,
() => Task.Run(SomeMethod3),
() => Task.Run(SomeMethod4)
};
for (int i = 0; i < sequentialMethods.Count; i++)
{
await sequentialMethods[i]();
}
}
static async Task Main(string[] args)
{
await new Program().ExecuteSequential();
Console.WriteLine("All done");
}
}
Output
Some1
Some2: Hey!
||Start
||My3
||My2
||My1
||End
Some3
Some4
All done
My1, My2, and My3 will change order between executions, but always within ||Start and ||End.
I want to translate the following code into a reactive flow (using System.Reactive in C#).
Method1, Method2 and Method3 are long running tasks. The result of Method2 is necessary in order to call Method3, but Method1 and Method2 can run in parallel. If result1 == null the whole operation can be terminated early with null.
Usually, Method2 returns faster than Method1, so Method3 could be started before Method1 finishes.
var result1 = Method1();
if (result1 == null) return null;
var result2 = Method2();
string result3 = null;
if (result2 != null)
{
result3 = Method3(result2);
}
var combinedResult = CreateResult(result1);
if (result2 != null)
{
combinedResult.Attr2 = result2;
}
if (result3 != null)
{
combinedResult.Attr3 = result3;
}
I am lost between nested functions and generics. The following code does not compile, because it has issues with generics and return types (especially the nested Select does not return a value but an Observable).
var observable1 = Observable.Start(() => Method1());
var observable2 = Observable.Start(() => Method2());
Observable.Zip(observable1, observable2, (result1, result2) =>
{
if (result2 != null)
{
var observable3 = Observable.Start(() => Method3(result2));
return observable3.Select(result3 =>
{
return SuperCombiner(result1, result2, result3);
};
}
return SuperCombiner(result1, null, null);
};
Here's what you need to get this to work:
var inner =
from m2 in Observable.Start(() => Method2())
from m3 in Observable.Start(() => Method3(m2))
select new { m2, m3 };
var query =
Observable
.Start(() => Method1())
.Publish(m1s =>
m1s
.Zip(
inner.TakeUntil(m1s.Where(m1 => m1 == null)),
(m1, m23) => new { m1, m23.m2, m23.m3 }))
.Where(x => x.m1 != null);
I have tested this using the following code:
public string Method1()
{
Console.WriteLine("Method1 Start");
Thread.Sleep(TimeSpan.FromSeconds(2.0));
Console.WriteLine("Method1 End");
return null; //"1";
}
public string Method2()
{
Console.WriteLine("Method2 Start");
Thread.Sleep(TimeSpan.FromSeconds(3.0));
Console.WriteLine("Method2 End");
return "2";
}
public string Method3(string x)
{
Console.WriteLine("Method3 Start");
Thread.Sleep(TimeSpan.FromSeconds(2.0));
Console.WriteLine("Method3 End");
return $"3-{x}";
}
The query only produces a value when Method1 returns a non-null value - otherwise it completes without producing a value.
Method3 is executed immediately after Method2 is complete unless Method1 has already returned null in which case Method3 is not executed.
This is computationally the most efficient implementation of what you asked for.
I've added some improvements to your draft and now it works as you described:
var stream1 = Observable.Start(Func1);
var stream2 = Observable.Start(Func2);
Observable.Zip(stream1, stream2, (res1, res2) =>
{
if (res1 == null)
return Observable.Start(() => new string[] { null });
if (res2 == null)
return Observable.Start(() => new string[] { res1, null });
return Observable.Start(() => Func3(res2)).Select(res3 => new[] { res1, res2, res3 });
})
.Merge()
.Subscribe(result =>
{
// 'result' is an array
// result[0] - result of Func1
// result[1] - result of Func2
// result[2] - result of Func3
// result.Length == 1 - means that Func1 returned 'null'
// result.Length == 2 - means that Func2 returned 'null'
});
But it's not a real 'Reactive' way because it contains imperative statements (like if operators for example).
I am having Parallel For loop for following statements but I want to use async task not Parallel. Any idea how can i use async task on same statements? I don't need fully working code but just an idea about how to replace async task with Parallel. Happy coding
Parallel.For(0, allRequests.Count(), i =>
{
var rand = new Random();
var token = allTokens.ElementAt(rand.Next(allTokens.Count()));
var accessKey = token.AccessKey;
var secretKey = token.SecretKey;
using (var ctx = new db_mytestdb())
{
var firstRequest = allRequests[i];
Console.WriteLine("Started scan for: " + firstRequest.SearchedUser.EbayUsername + " and using token: " + allTokens[i % allTokens.Count].TokenName);
var bulkScannedItems = new ConcurrentBag<BulkScannedItems>();
var userPreferences = ctx.UserPreferences.FirstOrDefault(x => x.UserId == firstRequest.UserId);
var userBrekEven = userPreferences.BreakEven;
var intoPast = DateTime.Now.Subtract(TimeSpan.FromDays(firstRequest.Range));
var filteredProducts = ctx.EbayUserTransactions.Where(x => x.SearchedUserID == firstRequest.SearchedUserID && x.TransactionDate >= intoPast && x.TransactionDate <= firstRequest.SearchedUser.LastUpdatedAt)
.ToList()
.GroupBy(x => x.ItemID).Select(x => new ResultItem()
{
ItemID = x.Key,
SaleNumber = x.Sum(y => y.QuantityPurchased)
})
.Where(x => x.SaleNumber >= firstRequest.MinSales)
.ToList();
var itemSpecifics = ctx.SearchedUserItems.Where(x => x.SearchedUserID == firstRequest.SearchedUserID).ToList();
foreach (var item in itemSpecifics)
{
foreach (var filtered in filteredProducts)
{
if (item.ItemID == filtered.ItemID)
{
if (item.UPC != null)
{
filtered.UPC = item.UPC;
}
else
{
filtered.UPC = "does not apply";
}
if (item.EAN != null)
{
filtered.EAN = item.EAN;
}
else
{
filtered.EAN = "does not apply";
}
if (item.MPN != null)
{
filtered.MPN = item.MPN;
}
else
{
filtered.MPN = "does not apply";
}
}
}
}
var bulkScanner = new BulkScannerAlgorithm();
foreach (var dbItem in filteredProducts)
{
var amazonItem = bulkScanner.Found(dbItem.UPC, dbItem.ItemID, accessKey, secretKey);
if (amazonItem.Found)
{
bulkScanner.InsertAmazonData(firstRequest, bulkScannedItems, userBrekEven, amazonItem);
continue;
}
amazonItem = bulkScanner.Found(dbItem.EAN, dbItem.ItemID, accessKey, secretKey);
if (amazonItem.Found)
{
bulkScanner.InsertAmazonData(firstRequest, bulkScannedItems, userBrekEven, amazonItem);
continue;
}
amazonItem = bulkScanner.Found(dbItem.MPN, dbItem.ItemID, accessKey, secretKey);
if (amazonItem.Found)
{
bulkScanner.InsertAmazonData(firstRequest, bulkScannedItems, userBrekEven, amazonItem);
continue;
}
}
List<BulkScannedItems> filteredCompleteBulk;
if (firstRequest.IsPrimeOnly == true)
{
filteredCompleteBulk = bulkScannedItems.Where(x => x.CalculatedProfit >= firstRequest.MinProfit && x.IsPrime == true && x.EbayPrice >= firstRequest.minPrice && x.EbayPrice <= firstRequest.maxPrice).DistinctBy(x => x.ASIN).ToList();
}
else
{
filteredCompleteBulk = bulkScannedItems.Where(x => x.CalculatedProfit >= firstRequest.MinProfit && x.EbayPrice >= firstRequest.minPrice && x.EbayPrice <= firstRequest.maxPrice).DistinctBy(x => x.ASIN).ToList();
}
EFBatchOperation.For(ctx, ctx.BulkScannedItems).InsertAll(filteredCompleteBulk);
ctx.user_scanReq_update(firstRequest.UserSellerScanRequestId);
Console.WriteLine("Scan complete for user: " + firstRequest.SearchedUser.EbayUsername);
}
});
Parallelism and asynchrony are both forms of concurrency, but parallelism works by dividing the problem among multiple threads, and asynchrony works by freeing up threads. So they're kind of opposites in how they work.
That said, to make the code asynchronous, you'd start from your lowest-level I/O calls, e.g., the EF ToList and presumably also whatever APIs are used in the implementation of InsertAll. Replace those with asynchronous equivalents (e.g., ToListAsync) and call them with await.
Next, you'd need to replace the Parallel.For loop with code that creates a collection of asynchronous tasks and then (asynchronously) waits for them all, something like:
var tasks = allRequests.Select(async request => { ... });
await Task.WhenAll(tasks);
That's the basic pattern for asynchronous concurrency.
If you find that you do need true parallelism (multiple threads) in addition to asynchrony, consider using TPL Dataflow.
A little clarification, whether you are using Parallel.For/Foreach, Tasks, or await/async, they are all using the same thing behind the scenes (albeit slightly differently). You should always pick the one that fits your problem the best.
If you want to replace the parallel.for with a method of return type Task, that is straight forward enough but you would end up waiting for this piece to be done before you continued your processing.
Async/Await is generally used when dealing with UIs and web calls, it doesn't appear to be useful here.
What is it that you are trying to accomplish? Why the need to 'replace async task with Parallel'?
The general way you would off load a method to a task would be
Task<T> task = Task<T>.Factory.StartNew(() =>
{
});
or
public Task<T> task()
{
.....
}
I am writing some complex tasks and I can't do success to solve the problem.
I open some tasks for searching some data asynchronously.
At the end I wait for all tasks.
I want to run on all the data and does not stop the tasks until they all finish.
But, when one of the tasks finds some relevant data, I want to continue with the code that is after the wait all tasks, but at the same time, I want to continue searching with my tasks (until all the data will be read).
Here I found a way how to continue the code after the wait all if one of the tasks finds the data, the problem is that it also stops all tasks from running.
Here is my code (this is just code for demonstration of the problem).
private static void RunTasks()
{
TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
Task<bool> t0 = Task.Factory.StartNew<bool>(() => Find(1, 2));
Task<bool> t1 = Task.Factory.StartNew<bool>(() => Find(4, 7));
Task<bool> t2 = Task.Factory.StartNew<bool>(() => Find(13, 14));
t0.ContinueWith(_ =>
{
if (t0.Result)
tcs.TrySetResult(t0.Result);
});
t1.ContinueWith(_ =>
{
if (t1.Result)
tcs.TrySetResult(t1.Result);
});
t2.ContinueWith(_ =>
{
if (t2.Result)
tcs.TrySetResult(t2.Result);
});
tcs.Task.Wait();
Console.WriteLine("Found");
ContinueWork(); //Runs after at least one data found or when all tasks finish.
}
//Just for demonstration...
private static bool Find(int a, int b)
{
Console.WriteLine("a: " + a + " b: " + b);
return a == 4 && b == 7 ? true : false;
}
How can I write it so that when some data is found it will continue to ContinueWork method and also will continue with reading the data with the tasks?
Thanks.
This is not the cleanest way but it will serve your purposes:
TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
Task<bool>[] tasks = new Task<bool>[3];
tasks[0] = Task.Factory.StartNew<bool>(() => Find(1, 2));
tasks[1] = Task.Factory.StartNew<bool>(() => Find(4, 7));
tasks[2] = Task.Factory.StartNew<bool>(() => Find(13, 14));
tasks[0].ContinueWith(_ =>
{
if (tasks[0].Result)
tcs.TrySetResult(tasks[0].Result);
});
tasks[1].ContinueWith(_ =>
{
if (tasks[1].Result)
tcs.TrySetResult(tasks[1].Result);
});
tasks[2].ContinueWith(_ =>
{
if (tasks[2].Result)
tcs.TrySetResult(tasks[2].Result);
});
Task.WaitAny(tasks);
Console.WriteLine("Found");
ContinueWork();
I have the following code:
Task.Factory.ContinueWhenAll(items.Select(p =>
{
return CreateItem(p);
}).ToArray(), completedTasks => { Console.WriteLine("completed"); });
Is it possible to convert ContinueWhenAll to a synchronous method? I want to switch back between async and sync.
Edit: I should metnion that each of the "tasks" in the continuewhenall method should be executing synchronously.
If you want to leave your existing code intact and have a variable option of executing synchronously you should make these changes:
bool isAsync = false; // some flag to check for async operation
var batch = Task.Factory.ContinueWhenAll(items.Select(p =>
{
return CreateItem(p);
}).ToArray(), completedTasks => { Console.WriteLine("completed"); });
if (!isAsync)
batch.Wait();
This way you can toggle it programmatically instead of by editing your source code. And you can keep the continuation code the same for both methods.
Edit:
Here is a simple pattern for having the same method represented as a synchronous and async version:
public Item CreateItem(string name)
{
return new Item(name);
}
public Task<Item> CreateItemAsync(string name)
{
return Task.Factory.StartNew(() => CreateItem(name));
}
Unless am mistaken this is what you're looking for
Task.WaitAll(tasks);
//continuation code here
i think you can try this.
using TaskContinuationOptions for a simple scenario.
var taskFactory = new TaskFactory(TaskScheduler.Defau
var random = new Random();
var tasks = Enumerable.Range(1, 30).Select(p => {
return taskFactory.StartNew(() => {
var timeout = random.Next(5, p * 50);
Thread.Sleep(timeout / 2);
Console.WriteLine(#" 1: ID = " + p);
return p;
}).ContinueWith(t => {
Console.WriteLine(#"* 2: ID = " + t.Result);
}, TaskContinuationOptions.ExecuteSynchronously);
}).ToArray();
Task.WaitAll(tasks);
or using TPL Dataflow for a complex scenario.
var step2 = new ActionBlock<int>(i => {
Thread.Sleep(i);
Console.WriteLine(#"* 2: ID = " + i);
}, new ExecutionDataflowBlockOptions {
MaxDegreeOfParallelism = 1,
//MaxMessagesPerTask = 1
});
var random = new Random();
var tasks = Enumerable.Range(1, 50).Select(p => {
return Task.Factory.StartNew(() => {
var timeout = random.Next(5, p * 50);
Thread.Sleep(timeout / 2);
Console.WriteLine(#" 1: ID = " + p);
return p;
}).ContinueWith(t => {
Thread.Sleep(t.Result);
step2.Post(t.Result);
});
}).ToArray();
await Task.WhenAll(tasks).ContinueWith(t => step2.Complete());
await step2.Completion;