I need your help, with threads I'm full 0 and you only need to create a certain thread and complete it on command, BUT I do not create each thread in advance, as there will be a lot of them, I do it like this:
Thread thread = new Thread(() => Go(..... many many variables that are taken from the listview ......));
thread.Start();
So, as noted above, variables are taken from the listview, which in turn is loaded by me from the file and then I run the threads I need. BUT the process in the stream is infinite and will end only if I completely close the program, and I would like to end the stream in the same way as I started it (right click on the desired line-start/stop). As I said, I have never worked with threads and thought that it was somehow simple, like when you start a thread, you assign it an ID and end it with the same ID, but alas. I have searched all over Google and have not found an EXAMPLE that suits me (I will repeat for the third time - I have never worked with threads and I do not need to say "go read about TPL"), so I ask for help, preferably with an example)
I have a very bad idea: in the sheet there is an invisible column in which an id is generated at the start, then when I send a command to start the thread, a unique variable is created with the name for example int id1=0 and its name is passed to the thread itself and each time the loop starts, id1=0 or 1 is checked in it, respectively, if 0-continue, if 1-empty. Well, it is logical that when you click the stop button, its value changes to 1. But something seems to me that the holy spirit of multithreading will punish me for this when the threads become 100+. I read this idea somewhere, so don't swear)
You do not need hundreds of threads for this. Your worker "threads" are performing HTTP requests, which can be done asynchronously without requiring a new thread. Also, hundreds of threads wouldn't really help you unless you have hundreds of CPU cores (you don't).
For this sort of work, I'd recommend the following:
Write a method that does all the work your thread does, but also checks a CancellationToken with each iteration.
Calls the method in a loop, once for each account, and store the resulting tasks in an array or list. Or use LINQ (as I do in this example) to create the list.
When your program terminates, activate the CancellationToken.
After cancelling, you have to await all the tasks in order to observe any possible exceptions and exit cleanly.
For example
public async Task DoTheWork(Account account, CancellationToken token)
{
while (!token.IsCancellationRequested)
{
var result = await httpClient.GetAsync(account.Url);
await DoSomethingWithResult(result);
await Task.Delay(1000);
}
}
//Main program
var accounts = GetAccountList();
var source = new CancellationTokenSource();
var tasks = accounts.Select( x => DoTheWork(x, source.Token) ).ToList();
//When exiting
source.Cancel();
await Task.WhenAll( tasks );
source.Dispose();
Indivivdual cancellation
Here's another approach that keeps a list of the accounts and a delegate that can be used for cancelling the task for that specific account.
//Declare this somewhere it will persist for the duration of the program
//The key to this dictionary is the account you wish to cancel
//The value is a delegate that you can call to cancel its task
Dictionary<Account, Func<Task>> _tasks = new Dictionary<Account, Func<Task>>();
async Task CreateTasks()
{
var accounts = GetAccounts();
foreach (var account in accounts)
{
var source = new CancellationTokenSource();
var task = DoTheWork(account, source.Token);
_tasks.Add(account, () => { source.Cancel(); return task; });
}
}
//Retrieve the delegate from the dictionary and call it to cancel its task
//Then await the task to observe any exceptions
//Then remove it from the list
async Task CancelTask(Account account)
{
var cancelAction = _tasks[account];
var task = cancelAction();
await task;
_tasks.Remove(account);
}
async Task CancelAllTasks()
{
var tasks = _tasks.Select(x => x.Value()).ToList();
await Task.WhenAll(tasks);
}
Related
I have a parallel foreach for few api calls. Api will return some data and I can process it. Lets say from front end, I call to ProcessInsu method and suddenly the user change his mind and go to other page. (There should be a clear button and user can exit the page after clicking clear button.) After calling ProcessInsu method, it will run APIs in background. It is waste of resources because, the user already change his mind and do other work. I need some methodology to cancel background running jobs.
public async Task ProcessInsu(InsuranceAccounts insuranceCompAccounts,string resourceId)
{
ParallelOptions parallelOptions = new ParallelOptions();
parallelOptions.MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling((Environment.ProcessorCount * 0.75) * 2.0));
Parallel.ForEach(insuranceCompAccounts, parallelOptions, async (insuranceComp) =>
{
await Processor.Process(resourceId,insuranceComp); //When looping, From each call, it will call to different different insurance companies.
});
}
I tried this sample code and I could not do my work with that. Any expert can guide me to do this?
As others have noted, you can't use async with Parallel - it won't work correctly. Instead, you want to do asynchronous concurrency as such:
public async Task ProcessInsu(InsuranceAccounts insuranceCompAccounts, string resourceId)
{
var tasks = insuranceCompAccounts.Select(async (insuranceComp) =>
{
await Processor.Process(resourceId, insuranceComp);
}).ToList();
await Task.WhenAll(tasks);
}
Now that the code is corrected, you can add cancellation support. E.g.:
public async Task ProcessInsu(InsuranceAccounts insuranceCompAccounts, string resourceId)
{
var cts = new CancellationTokenSource();
var tasks = insuranceCompAccounts.Select(async (insuranceComp) =>
{
await Processor.Process(resourceId, insuranceComp, cts.Token);
}).ToList();
await Task.WhenAll(tasks);
}
When you are ready to cancel the operation, call cts.Cancel();.
Note that Process now takes a CancellationToken. It should pass this token on to whatever I/O APIs it's using.
I have a method that runs multiple async methods within it. I have to iterate over a list of devices, and pass the device to this method. I am noticing that this is taking a long time to complete so I am thinking of using Parallel.ForEach so it can run this process against multiple devices at the same time.
Let's say this is my method.
public async Task ProcessDevice(Device device) {
var dev = await _deviceService.LookupDeviceIndbAsNoTracking(device);
var result = await DoSomething(dev);
await DoSomething2(dev);
}
Then DoSomething2 also calls an async method.
public async Task DoSomething2(Device dev) {
foreach(var obj in dev.Objects) {
await DoSomething3(obj);
}
}
The list of devices continuously gets larger over time, so the more this list grows, the longer it takes the program to finish running ProcessDevice() against each device. I would like to process more than one device at a time. So I have been looking into using Parallel.ForEach.
Parallel.ForEach(devices, async device => {
try {
await ProcessDevice(device);
} catch (Exception ex) {
throw ex;
}
})
It appears that the program is finishing before the device is fully processed. I have also tried creating a list of tasks, and then foreach device, add a new task running ProcessDevice to that list and then awaiting Task.WhenAll(listOfTasks);
var listOfTasks = new List<Task>();
foreach(var device in devices) {
var task = Task.Run(async () => await ProcessDevice(device));
listOfTasks.Add(task);
}
await Task.WhenAll(listOfTasks);
But it appears that the task is marked as completed before ProcessDevice() is actually finished running.
Please excuse my ignorance on this issue as I am new to parallel processing and not sure what is going on. What is happening to cause this behavior and is there any documentation that you could provide that could help me better understand what to do?
You can't mix async with Parallel.ForEach. Since your underlying operation is asynchronous, you'd want to use asynchronous concurrency, not parallelism. Asynchronous concurrency is most easily expressed with WhenAll:
var listOfTasks = devices.Select(ProcessDevice).ToList();
await Task.WhenAll(listOfTasks);
In your last example there's a few problems:
var listOfTasks = new List<Task>();
foreach (var device in devices)
{
await Task.Run(async () => await ProcessDevice(device));
}
await Task.WhenAll(listOfTasks);
Doing await Task.Run(async () => await ProcessDevice(device)); means you are not moving to the next iteration of the foreach loop until the previous one is done. Essentially, you're still doing them one at a time.
Additionally, you aren't adding any tasks to listOfTasks so it remains empty and therefore Task.WhenAll(listOfTasks) completes instantly because there's no tasks to await.
Try this:
var listOfTasks = new List<Task>();
foreach (var device in devices)
{
var task = Task.Run(async () => await ProcessDevice(device))
listOfTasks.Add(task);
}
await Task.WhenAll(listOfTasks);
I can explain the problem with Parallel.ForEach. An important thing to understand is that when the await keyword acts on an incomplete Task, it returns. It will return its own incomplete Task if the method signature allows (if it's not void). Then it is up to the caller to use that Task object to wait for the job to finish.
But the second parameter in Parallel.ForEach is an Action<T>, which is a void method, which means no Task can be returned, which means the caller (Parallel.ForEach in this case) has no way to wait until the job has finished.
So in your case, as soon as it hits await ProcessDevice(device), it returns and nothing waits for it to finish so it starts the next iteration. By the time Parallel.ForEach is finished, all it has done is started all the tasks, but not waited for them.
So don't use Parallel.ForEach with asynchronous code.
Stephen's answer is more appropriate. You can also use WSC's answer, but that can be dangerous with larger lists. Creating hundreds or thousands of new threads all at once will not help your performance.
not very sure it this if what you are asking for, but I can give example of how we start async process
private readonly Func<Worker> _worker;
private void StartWorkers(IEnumerable<Props> props){
Parallel.ForEach(props, timestamp => { _worker.Invoke().Consume(timestamp); });
}
Would recommend reading about Parallel.ForEach as it will do some part for you.
Updated to explain things more clearly
I've got an application that runs a number of tasks. Some are created initially and other can be added later. I need need a programming structure that will wait on all the tasks to complete. Once the all the tasks complete some other code should run that cleans things up and does some final processing of data generated by the other tasks.
I've come up with a way to do this, but wouldn't call it elegant. So I'm looking to see if there is a better way.
What I do is keep a list of the tasks in a ConcurrentBag (a thread safe collection). At the start of the process I create and add some tasks to the ConcurrentBag. As the process does its thing if a new task is created that also needs to finish before the final steps I also add it to the ConcurrentBag.
Task.Wait accepts an array of Tasks as its argument. I can convert the ConcurrentBag into an array, but that array won't include any Tasks added to the Bag after Task.Wait was called.
So I have a two step wait process in a do while loop. In the body of the loop I do a simple Task.Wait on the array generated from the Bag. When it completes it means all the original tasks are done. Then in the while test I do a quick 1 millisecond test of a new array generated from the ConcurrentBag. If no new tasks were added, or any new tasks also completed it will return true, so the not condition exits the loop.
If it returns false (because a new task was added that didn't complete) we go back and do a non-timed Task.Wait. Then rinse and repeat until all new and old tasks are done.
// defined on the class, perhaps they should be properties
CancellationTokenSource Source = new CancellationTokenSource();
CancellationToken Token = Source.Token;
ConcurrentBag<Task> ToDoList = new ConcurrentBag<Task>();
public void RunAndWait() {
// start some tasks add them to the list
for (int i = 0; i < 12; i++)
{
Task task = new Task(() => SillyExample(Token), Token);
ToDoList.Add(task);
task.Start();
}
// now wait for those task, and any other tasks added to ToDoList to complete
try
{
do
{
Task.WaitAll(ToDoList.ToArray(), Token);
} while (! Task.WaitAll(ToDoList.ToArray(), 1, Token));
}
catch (OperationCanceledException e)
{
// any special handling of cancel we might want to do
}
// code that should only run after all tasks complete
}
Is there a more elegant way to do this?
I'd recommend using a ConcurrentQueue and removing items as you wait for them. Due to the first-in-first-out nature of queues, if you get to the point where there's nothing left in the queue, you know that you've waited for all the tasks that have been added up to that point.
ConcurrentQueue<Task> ToDoQueue = new ConcurrentQueue<Task>();
...
while(ToDoQueue.Count > 0 && !Token.IsCancellationRequested)
{
Task task;
if(ToDoQueue.TryDequeue(out task))
{
task.Wait(Token);
}
}
Here's a very cool way using Microsoft's Reactive Framework (NuGet "Rx-Main").
var taskSubject = new Subject<Task>();
var query = taskSubject.Select(t => Observable.FromAsync(() => t)).Merge();
var subscription =
query.Subscribe(
u => { /* Each Task Completed */ },
() => Console.WriteLine("All Tasks Completed."));
Now, to add tasks, just do this:
taskSubject.OnNext(Task.Run(() => { }));
taskSubject.OnNext(Task.Run(() => { }));
taskSubject.OnNext(Task.Run(() => { }));
And then to signal completion:
taskSubject.OnCompleted();
It is important to note that signalling completion doesn't complete the query immediately, it will wait for all of the tasks to finish too. Signalling completion just says that you will no longer add any new tasks.
Finally, if you want to cancel, then just do this:
subscription.Dispose();
I have inherited a C#/XAML/Win 8 application. There is some code which is set to run every n seconds.
The code that sets that up is:
if(!_syncThreadStarted)
{
await Task.Run(() => SyncToDatabase());
_syncThreadStarted = true;
}
The above code is ran once.
And then inside SyncToDatabase() we have:
while (true)
{
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate();
await Task.Delay(10); // after elapsed time re-run above code
}
The method DeserializeAndUpdate queries a in-memory collection of objects and pushes those objects to a web service.
Sometimes the send request to the web service takes longer than expected meaning duplicate items are sent.
Question: Is there a way to have a thread or some type of thread pool/background worker which I can stop/abort/destroy inside the method SyncToDatabase() , and then initialize/start it once we are done? This will ensure no subsequent requests are fired while a previous request is still pending.
Edit: I am not very knowledgeable when it comes to Threads, but the logic I want is:
Create thread which runs some method every x seconds, and when it starts that thread stop the "running every x seconds" part, after thread has complete start the "run every x seconds" part again.
E.g. if the thread kicks off at 10:01:30AM and does not complete until 10:01:39AM (9 seconds) the next thread should start at 10:01:44AM (5 seconds after work completed) - does that make sense? I do not want 2 or more threads running at the same time.
Here is my code for the above:
var period = TimeSpan.FromSeconds(5);
var completed = true;
ThreadPoolTimer syncTimer = ThreadPoolTimer.CreatePeriodicTimer(async (source) =>
{
// stop further threads from starting (in case this work takes longer than var period)
syncTimer.Cancel();
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate(); // makes webservices calls
Dispatcher.RunAsync(CoreDispatcerPriority.High, async () =>
{
// Update UI
}
completed = true;
}, period,
(source) =>
{
if(!completed)
{
syncTimer.Cancel(); // not sure if this is correct...
}
}
Thanks,
Andrew)
This is not specific to Windows 8. Usually Task.Run is used for CPU-bound work, to offload it to a pool thread and keep the UI (or the core service loop) responsive. In your case, as far as I can tell, the main payload is dbSyncer.DeserializeAndUpdate, which is already asynchronous and most likely network-IO bound, rather than CPU-bound.
Besides, the author of the original code does _syncThreadStarted = true after await Task.Run(() => SyncToDatabase()). That doesn't make sense, because the work on the pool thread would have been already done by the time _syncThreadStarted = true is executed, thanks to the await.
To cancel the loop inside SyncToDatabase you could use Task Cancellation Pattern. Is SyncToDatabase itself an async method? I presume so, because there's an await in the while loop. Given that, the code which calls it could look something like this:
if(_syncTask != null && !_syncTask.IsCompleted)
{
_ct.Cancel();
// here you may want to make sure that the pending task has been fully shut down,
// keeping possible re-entrancy in mind
// See: https://stackoverflow.com/questions/18999827/a-pattern-for-self-cancelling-and-restarting-task
_syncTask = null;
}
_ct = new CancellationTokenSource();
// _syncTask = SyncToDatabase(ct.Token); // do not await
// edited to run on another thread, as requested by the OP
var _syncTask = Task.Run(async () => await SyncToDatabase(ct.Token), ct.Token);
_syncThreadStarted = true;
And SyncToDatabase could look like:
async Task SyncToDatabase(CancellationToken token)
{
while (true)
{
token.ThrowIfCancellationRequested();
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate();
await Task.Delay(10, token); // after elapsed time re-run above code
}
}
Check this answer for more details on how to cancel and restart a task.
I may have misunderstood the question, but the execution of SynchToDatabase() will wait on the completion of await dbSyncer.DeserializeAndUpdaet() (due to the await keyword, go figure ;)) before executing the continuation, which will then delay for 10 ms (do you want 10ms or did you mean 10 seconds? Parameter for Task.Delay is in milliseconds), then loop back to re-execute the DbSyncer method, so I don't see the problem.
How to modify the following code and make it runs multiple tasks concurrently?
foreach (SyndicationItem item in CurrentFeed.Items)
{
if (m_bDownloadInterrupted)
break;
await Task.Run( async () =>
{
// do some downloading and processing work here
await DoSomethingAsync();
}
}
I also need to make interruption and stop the process possible. Because my DoSomethingAsync method reads the tag (a global boolean value) to stop the process.
Thanks
No, that won't run them concurrently - you're waiting for each one to finish before starting the next one.
You could put the results of each Task.Run call into a collection, then await Task.WhenAll after starting them all though.
(It's a shame that Parallel.ForEach doesn't return a Task you could await. There may be a more async-friendly version around...)
This will process the items concurrently.
Parallel.ForEach(CurrentFeed.Items, DoSomethingAsync)
To be able to cancel you can need a CancellationToken.
CancellationTokenSource cts = new CancellationTokenSource();
ParallelOptions po = new ParallelOptions();
po.CancellationToken = cts.Token;
// Add the ParallelOptions with the token to the ForEach call
Parallel.ForEach(CurrentFeed.Items,po ,DoSomethingAsync)
// set cancel on the token somewhere in the workers to make the loop stop
cts.Cancel();
For detail see (among other sources) http://msdn.microsoft.com/en-us/library/ee256691.aspx