Assume that we have some methods that do some big work.
Initially we don`t know how many methods there are ( can be 1 and can be 10 ).
In code this looks like this:
public interface IWorker
{
void DoWork(DataContainer data);
}
And several classes that implement this interface.
Then we have list of instances.
List<IWorker> workers = new List<IWorker>();
I want to run these methods asynchronously. In addition I want some callback when all of them are executed.
public void Callback()
{
Console.WriteLine("everything done");
}
Is there a way to do this without writing custom wrappers or so? With ThreadPool, Tasks, Parallel?
As I know Parrallel block thread untill tasks are completed so this is not a prefferd behaviour.
When creating Task there should be method without parameters as I`ve seen so this is not good as well.
In ThreadPool there is possibility to use method QueueUserWorkItem, but using this method I won`t get a single "total-completion" callback.
Of course, I can make my own wrapper that will implement the desired functionality using ThreadPool, but the goal is to make this without writing such one.
Can anyone help please?
Thanks.
You're looking for the TPL and the Task class.
Create a Task for each operation, then call Task.WhenAll to get an aggregate task
You're looking for Task.WhenAll. Create a bunch of Tasks that do what you want them to, then wait on all of the tasks and ContinueWith your callback. I split out an async version of the DoWork method - if you're always going to be calling it asynchronously you don't necessarily need to do that.
public interface IWorker
{
Task DoWorkAsync(string data);
void DoWork(string data);
}
public class Worker : IWorker
{
public Task DoWorkAsync(string data)
{
return Task.Run(() => DoWork(data));
}
public void DoWork(string data)
{
Console.WriteLine(data);
Thread.Sleep(100);
}
}
public class Runner
{
public void Callback()
{
Console.WriteLine("Everything done");
}
public void Run()
{
var workers = new List<IWorker> {new Worker(), new Worker(), new Worker()};
var tasks = workers.Select(t => t.DoWorkAsync("some data"));
Task.WhenAll(tasks).ContinueWith(task => Callback());
Console.WriteLine("Waiting");
}
}
Sounds like a prime candidate for the CountdownEvent class:
List<IWorker> workers = new List<IWorker>();
using (CountdownEvent e = new CountdownEvent(workers.Count))
{
foreach (IWorker worker in workers)
{
// Dynamically increment signal count.
e.AddCount();
// run work itself on another thread
ThreadPool.QueueUserWorkItem(delegate(object state)
{
try
{
((IWorker)state[0]).DoWork((DataContainer)state[1]);
}
finally
{
e.Signal();
}
},
// pass required parameters for block of work
new object[] { worker, dataForWorker });
}
// wait for all workers to finish
e.Wait();
// run callback code
}
Related
This question already has answers here:
How to wait for async method to complete?
(7 answers)
Closed 2 years ago.
Let's say I have a MyThread class in my Windows C# app like this:
public class MyThread
{
Thread TheThread;
public MyThread()
{
TheThread = new Thread(MyFunc);
}
public void StartIfNecessary()
{
if (!TheThread.IsAlive)
TheThread.Start();
}
private void MyFunc()
{
for (;;)
{
if (ThereIsStuffToDo)
DoSomeStuff();
}
}
}
That works fine. But now I realize I can make my thread more efficient by using async/await:
public class MyThread
{
Thread TheThread;
public MyThread()
{
TheThread = new Thread(MyFunc);
}
public void StartIfNecessary()
{
if (!TheThread.IsAlive)
TheThread.Start();
}
private async void MyFunc()
{
for (;;)
{
DoSomeStuff();
await MoreStuffIsReady();
}
}
}
What I see now, is that the second time I call StartIfNecessary(), TheThread.IsAlive is false (and ThreadState is Stopped BTW) so it calls TheThread.Start() which then throws the ThreadStateException "Thread is running or terminated; it cannot restart". But I can see that DoMoreStuff() is still getting called, so the function is in fact still executing.
I suspect what is happening, is that when my thread hits the "await", the thread I created is stopped, and when the await on MoreStuffIsReady() completes, a thread from the thread pool is assigned to execute DoSomeStuff(). So it is technically true that the thread I created has been stopped, but the function I created that thread to process is still running.
So how can I tell if "MyFunc" is still active?
I can think of 3 ways to solve this:
1) Add a "bool IsRunning" which is set to true right before calling TheThread.Start(), and MyFunc() sets to false when it completes. This is simple, but requires me to wrap everything in a try/catch/finally which isn't awful but I was hoping there was a way to have the operating system or framework help me out here just in case "MyFunc" dies in some way I wasn't expecting.
2) Find some new function somewhere in System.Threading that will give me the information I need.
3) Rethink the whole thing - since my thread only sticks around for a few milliseconds, is there a way to accomplish this same functionality without creating a thread at all (outside of the thread pool)? Start "MyFunc" as a Task somehow?
Best practices in this case?
Sticking with a Plain Old Thread and using BlockingCollection to avoid a tight loop:
class MyThread
{
private Thread worker = new Thread(MyFunc);
private BlockingCollection<Action> stuff = new BlockingCollection<Action>();
public MyThread()
{
worker.Start();
}
void MyFunc()
{
foreach (var todo in stuff.GetConsumingEnumerable())
{
try
{
todo();
}
catch(Exception ex)
{
// Something went wrong in todo()
}
}
stuff.Dispose(); // should be disposed!
}
public void Shutdown()
{
stuff.CompleteAdding(); // No more adding, but will continue to serve until empty.
}
public void Add( Action stuffTodo )
{
stuff.Add(stuffTodo); // Will throw after Shutdown is called
}
}
BlockingCollection also shows examples with Task if you prefer to go down that road.
Rethink the whole thing
This is definitely the best option. Get rid of the thread completely.
It seems like you have a "consumer" kind of scenario, and you need a consumer with a buffer of data items to work on.
One option is to use ActionBlock<T> from TPL Dataflow:
public class NeedsADifferentName
{
ActionBlock<MyDataType> _block;
public NeedsADifferentName() => _block = new ActionBlock<MyDataType>(MyFunc);
public void QueueData(MyDataType data) => _block.Post(data);
private void MyFunc(MyDataType data)
{
DoSomeStuff(data);
}
}
Alternatively, you can build your own pipeline using something like Channels.
Regarding the right worker method signature I need to understand the following:
is there a point in returning Task instead of void for Worker method (if going sync)?
Should I really wait (call Wait()) on the Worker method (if going sync)?
what should be the return value of Worker method when marked as returning Task object (both if going sync/async)?
what signature and body of Worker method should be, given the work it completes is long-running CPU/IO-bound work? Should I follow this recommendation (if going mixed/async)?
Note
Despite the cpu-bound code, there's a choice to call async versions of io-bound methods (sql queries). So it may be all sync or partially async. As for the nature of code in the Worker method.
public class LoopingService
{
private CancellationTokenSource cts;
// ..
void Worker(CancellationToken cancellationToken)
{
while(!cancellationToken.IsCancellationRequested)
{
// mixed, CPU/IO-bound code
try {
// sql query (can be called either as sync/async)
var lastId = documentService.GetLastDocument().Id;
// get next document from a public resource (third-party code, sync)
// can be moved to a web api
var document = thirdPartyDocumentService.GetNextDocument(lastId);
// apply different processors in parallel
var tasksList = new List<Task>();
foreach(var processor in documentService.Processors) {
// each processor checks if it's applicable
// which may include xml-parsing, additional db calls, regexes
// if it's applicable then document data is inserted into the db
var task = new Task(() => processor.Process(document));
tasksList.Add(task);
task.Start();
}
// or
// var tasksList = documentService.ProcessParallel(document);
Task.WaitAll(tasksList.ToArray(), cancellationToken);
}
catch(Exception ex) {
logger.log(ex);
}
}
}
public void Start()
{
this.cts = new CancellationTokenSource();
Task.Run(() => this.Worker(cts.Token));
}
public void Stop()
{
this.cts.Cancel();
this.cts.Dispose();
}
}
is there a point in returning Task instead of void for Worker method?
If Worker is a truly asynchronous method it should return a Task for you to be able to await it. If it's just a synchronous method runnning on a background thread there is no point of changing the return type from void provided that the method is not supposed to return anything.
what should be the return value of Worker method when marked as returning Task object?
Nothing. Provided that the method is asynchronous and marked as async with a return type of Task, it shouldn't return any value:
async Task Worker(CancellationToken cancellationToken) { ... }
Note that there is no point of defining the method as async unless you actually use the await keyword in it.
what signature and body of Worker method should be given the work it completes is long-running CPU/IO-bound work? Should I follow this recommendation?
Yes, probably. If you for some reason are doing both asynchronous and synchronous (CPU-bound) work in the same method, you should prefer to using an asynchronous signature but not wrap the synchronous stuff in Task.Run. Then your service would look something like this:
public class LoopingService
{
private CancellationTokenSource cts;
async Task Worker(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
await ...
}
}
public async Task Start()
{
this.cts = new CancellationTokenSource();
await this.Worker(cts.Token).ConfigureAwait(false);
}
public void Stop()
{
this.cts.Cancel();
this.cts.Dispose();
}
}
Ideally your method should be either asynchronous or CPU-bound but not both though.
How I can wait for task to complete before repeating it
List<ushort> IdsInProgress = new List<ushort>();
public async Task<string> RunMyTask(ushort Id)
{
if (IdsInProgress.Contains(Id))
{
//Here Wait previous task to finish
//and then re-run it
return await MyTask(Id);
}
return await MyTask(Id);
}
public Task<string> MyTask(ushort Id)
{
IdsInProgress.Add(Id);
String MyString;
//do something
IdsInProgress.Remove(Id);
return Task.FromResult(MyString);
}
After dasblinkenlight comments about thread-safe and sync I ended up with this solution. Not sure if this is the best way.
I remove IdsInProgress because I don't need it anymore.
private readonly object obj = new object();
public Task<string> MyTask(ushort Id)
{
lock(obj)
{
String MyString;
//do something
return Task.FromResult(MyString);
}
}
Now if I run this method 100 times they would run one after the other.
Based on luaan comment it's better to do this like this
private SemaphoreSlim semaphore = new new SemaphoreSlim(1);
public Task<string> MyTask(ushort Id)
{
semaphore.Wait();
String MyString;
//do something
semaphore.Release();
return Task.FromResult(MyString);
}
I would suggest you take a look at the ISynchronizeInvoke interface.
The interface allows methods to be called from any thread and marshals that call to the proper thread.
As the goal is to synchronously executes tasks, the Invoke method can be used to delegate and marshal the call to the thread that created this object.
Using this method ensures that the process or task is finished before returning.
Implementing this method is quite straightforward:
public object Invoke(Delegate method, object[] args)
{
return method.DynamicInvoke(args);
}
Typically you would use this method as followed: invokerObj.Invoke(MyMethod)
Which executes MyMethod on the thread managed by the object (invokerObj) implementing the ISynchronizeInvoke interface.
The property InvokeRequired and the methods BeginInvoke & EndInvokecan be implemented if you also wish the possibility to asynchronously execute a delegate. A nice demonstration can be found over here.
I hope this explains it a bit more.
I'm trying to make code replacement from Thread to Task. The sleep / delay is just representing long running activity.
static void Main(string[] args)
{
ThreadDoWork();
TaskDoWork();
}
public static void ThreadDoWork()
{
using (var dispose = new ThreadDispose())
{
dispose.RunAsync();
}
}
public static async void TaskDoWork()
{
using (var dispose = new TaskDispose())
{
await dispose.RunAsync();
}
}
public class ThreadDispose : IDisposable
{
public void RunAsync ()
{
ThreadPool.QueueUserWorkItem(state =>
{
Thread.Sleep(3000);
});
}
void IDisposable.Dispose()
{
File.AppendAllText("D:\\test.txt", "thread disposing");
}
}
public class TaskDispose : IDisposable
{
public async Task RunAsync()
{
await Task.Delay(3000);
}
void IDisposable.Dispose()
{
File.AppendAllText("D:\\test.txt", "task disposing");
}
}
The result after 3 seconds in test.txt is only
thread disposing
What do I need to change in order TaskDispose::Dispose is always executed just like ThreadDispose?
Let's isolate each piece of code:
public static void ThreadDoWork()
{
using (var dispose = new ThreadDispose())
{
dispose.RunAsync();
}
}
public void RunAsync()
{
ThreadPool.QueueUserWorkItem(state =>
{
Thread.Sleep(3000);
});
}
What you do in this first piece of code is queue work on a threadpool thread. Because you're running this code inside a using scope and it runs asynchronously on a different thread, it disposes immediately. That is why you see the dispose message inside your text file.
public static async void TaskDoWork()
{
using (var dispose = new TaskDispose())
{
await dispose.RunAsync();
}
}
public class TaskDispose : IDisposable
{
public async Task RunAsync()
{
await Task.Delay(3000);
}
}
When you await inside your method, what you actually say is something along the lines of: "Execute this code. Because it's asynchronous by nature, I will return control back to the calling method, please call me back once you complete the asynchronous operation".
Your code hits the await keyword and returns control to your Main method. Inside Main, your async method is the last piece of code to execute, hence finishing your application, and not giving a chance for your Dispose method to execute.
If you want it to dispose, you'll have to change the return type from void to Task and explicitly Wait:
public static async Task TaskDoWork()
{
using (var dispose = new TaskDispose())
{
await dispose.RunAsync();
}
}
And now:
static void Main(string[] args)
{
ThreadDoWork();
TaskDoWork().Wait();
}
Side Note:
There are a couple of guidelines that should be followed:
async void is for compatability with event handlers, there are rarely occasions outside that scope where it should be used. Instead, use async Task.
Methods doing asynchoronous operation using TAP (Task Asynchronous Pattern) should end with the Async postfix. TaskDoWork should be TaskDoWorkAsync.
Using Wait on a Task can cause deadlocks. In this particular case it doesn't because a console application doesn't have a SynchronizationContext and uses the threadpools. The recommended approach is to go "async all the way" and use await.
There is great reading material inside the async-await tag wiki, make sure to check it out.
The both examples has issues.
The first, in the thread case, the Dispose method is calling before the thread has ended.
In the case of the task, the Dispose method will be call when the task has finished.
But the file only has the text written by the thread case, because it is called synchronously after the call of RunAsync method. And the Task case doesn't write anything because the application is ended before the task has finished.
Test to wait the task to finish with a Thread.Sleep(5000) for example at the end of the Main method, and the task should write in the output file.
I have simple method in my C# app, it picks file from FTP server and parses it and stores the data in DB. I want it to be asynchronous, so that user perform other operations on App, once parsing is done he has to get message stating "Parsing is done".
I know it can achieved through asynchronous method call but I dont know how to do that can anybody help me please??
You need to use delegates and the BeginInvoke method that they contain to run another method asynchronously. A the end of the method being run by the delegate, you can notify the user. For example:
class MyClass
{
private delegate void SomeFunctionDelegate(int param1, bool param2);
private SomeFunctionDelegate sfd;
public MyClass()
{
sfd = new SomeFunctionDelegate(this.SomeFunction);
}
private void SomeFunction(int param1, bool param2)
{
// Do stuff
// Notify user
}
public void GetData()
{
// Do stuff
sfd.BeginInvoke(34, true, null, null);
}
}
Read up at http://msdn.microsoft.com/en-us/library/2e08f6yc.aspx
try this method
public static void RunAsynchronously(Action method, Action callback) {
ThreadPool.QueueUserWorkItem(_ =>
{
try {
method();
}
catch (ThreadAbortException) { /* dont report on this */ }
catch (Exception ex) {
}
// note: this will not be called if the thread is aborted
if (callback!= null) callback();
});
}
Usage:
RunAsynchronously( () => { picks file from FTP server and parses it},
() => { Console.WriteLine("Parsing is done"); } );
Any time you're doing something asynchronous, you're using a separate thread, either a new thread, or one taken from the thread pool. This means that anything you do asynchronously has to be very careful about interactions with other threads.
One way to do that is to place the code for the async thread (call it thread "A") along with all of its data into another class (call it class "A"). Make sure that thread "A" only accesses data in class "A". If thread "A" only touches class "A", and no other thread touches class "A"'s data, then there's one less problem:
public class MainClass
{
private sealed class AsyncClass
{
private int _counter;
private readonly int _maxCount;
public AsyncClass(int maxCount) { _maxCount = maxCount; }
public void Run()
{
while (_counter++ < _maxCount) { Thread.Sleep(1); }
CompletionTime = DateTime.Now;
}
public DateTime CompletionTime { get; private set; }
}
private AsyncClass _asyncInstance;
public void StartAsync()
{
var asyncDoneTime = DateTime.MinValue;
_asyncInstance = new AsyncClass(10);
Action asyncAction = _asyncInstance.Run;
asyncAction.BeginInvoke(
ar =>
{
asyncAction.EndInvoke(ar);
asyncDoneTime = _asyncInstance.CompletionTime;
}, null);
Console.WriteLine("Async task ended at {0}", asyncDoneTime);
}
}
Notice that the only part of AsyncClass that's touched from the outside is its public interface, and the only part of that which is data is CompletionTime. Note that this is only touched after the asynchronous task is complete. This means that nothing else can interfere with the tasks inner workings, and it can't interfere with anything else.
Here are two links about threading in C#
Threading in C#
Multi-threading in .NET: Introduction and suggestions
I'd start to read about the BackgroundWorker class
In Asp.Net I use a lot of static methods for jobs to be done. If its simply a job where I need no response or status, I do something simple like below. As you can see I can choose to call either ResizeImages or ResizeImagesAsync depending if I want to wait for it to finish or not
Code explanation: I use http://imageresizing.net/ to resize/crop images and the method SaveBlobPng is to store the images to Azure (cloud) but since that is irrelevant for this demo I didn't include that code. Its a good example of time consuming tasks though
private delegate void ResizeImagesDelegate(string tempuri, Dictionary<string, string> versions);
private static void ResizeImagesAsync(string tempuri, Dictionary<string, string> versions)
{
ResizeImagesDelegate worker = new ResizeImagesDelegate(ResizeImages);
worker.BeginInvoke(tempuri, versions, deletetemp, null, null);
}
private static void ResizeImages(string tempuri, Dictionary<string, string> versions)
{
//the job, whatever it might be
foreach (var item in versions)
{
var image = ImageBuilder.Current.Build(tempuri, new ResizeSettings(item.Value));
SaveBlobPng(image, item.Key);
image.Dispose();
}
}
Or going for threading so you dont have to bother with Delegates
private static void ResizeImagesAsync(string tempuri, Dictionary<string, string> versions)
{
Thread t = new Thread (() => ResizeImages(tempuri, versions, null, null));
t.Start();
}
ThreadPool.QueueUserWorkItem is the quickest way to get a process running on a different thread.
Be aware that UI objects have "thread affinity" and cannot be accessed from any thread other than the one that created them.
So, in addition to checking out the ThreadPool (or using the asynchronous programming model via delegates), you need to check out Dispatchers (wpf) or InvokeRequired (winforms).
In the end you will have to use some sort of threading. The way it basically works is that you start a function with a new thread and it will run until the end of the function.
If you are using Windows Forms then a nice wrapper that they have for this is call the Background Worker. It allows you to work in the background with out locking up the UI form and even provides a way to communicate with the forms and provide progress update events.
Background Worker
.NET got new keyword async for asonchrynous functions. You can start digging at learn.microsoft.com (async). The shortest general howto make function asonchrynous is to change function F:
Object F(Object args)
{
...
return RESULT;
}
to something like this:
async Task<Object> FAsync(Object args)
{
...
await RESULT_FROM_PROMISE;
...
return RESULT;
}
The most important thing in above code is that when your code approach await keyword it return control to function that called FAsync and make other computation until promissed value has been returned and procede with rest of code in function FAsync.