Develop Reference

Refactor a loop to call an async process

I have a process that goes through a loop. During each iteration of the loop, it calls out to an external web service and then adds a object to an EntityFramework repository. The call to the external service is wrapped in a static method. Typically the loop only has one or two iterations but up to 4 is currently possible with the UI. (Each iteration represents an insurance quote).
It seems that this would benefit from being refactored as an asynchronous process. How do I set this up so that each iteration occurs in a seperate thread, and the commit waits until all threads are completed?
public class ProcessRequest
{
private IUnitOfWork = unitOfWork;
public ProcessRequest(IUnitOfWork uow)
{
unitOfWork = uow;
}
public void Execute(MyRequestParams p)
{
foreach (Quote q in p.Quotes)
{
q.Premium = QuoteService.GetQuote(q);
unitOfWork.GetRepository<Quote>().Add(q);
}
unitOfWork.Commit();
}
}
public static class QuoteService
{
public static decimal GetQuote(Quote quote)
{
//I've simplified proprietary code to single line that calls an external service
return ExternalWebService.GetQuote(quote.Deductible);
}
}

You're asking two different things: one is how to execute the loop in parallel, where each iteration occurs (potentially) on a separate thread; this is completely different to executing the entire loop as an asynchronous process which means the thread that initiates it won't wait for it to complete. I assume you meant the first, i.e. that you want to parallelize the iterations in the loop but still block until all of them are done.
Without knowing anything about the context in which this runs, one straightforward way would be to use Parallel Extensions, specifically Parallel Foreach:
public void Execute(MyRequestParams p)
{
Parallel.ForEach(p.Quotes, q => {
q.Premium = QuoteService.GetQuote(q);
unitOfWork.GetRepository<Quote>().Add(q);
});
unitOfWork.Commit();
}
Or maybe something like:
public void Execute(MyRequestParams p)
{
Parallel.ForEach(p.Quotes, q => {
q.Premium = QuoteService.GetQuote(q);
});
unitOfWork.GetRepository<Quote>().AddAll(p.Quotes);
unitOfWork.Commit();
}
This depends heavily on the thread-safety what you're dealing with.

If most of your work is I/O, im not sure i'd go for spinning up a new thread, as you are wasting most of your time idle waiting for your service/DB to reply.
i'd try to go with a async approach:
public async Task Execute(MyRequestParams p)
{
foreach (var quote in p.Quotes)
{
//Of course, you'll need an async endpoint.
var q.Premium = await QuoteService.GetQuoteAsync(q);
}
unitOfWork.GetRepository<Quote>().AddAll(p.Quotes);
await unitOfWork.SaveChangesAsync();
}
With this approach, you save the overhead of spinning up new threads and letting them be idle most of the time.
Hope this makes sense, of course you'd have to have access to an async endpoint of the webservice, and use Entity Framework v6.

How to use await with click handler

I have an application that communicates with a serial port. I am trying to make it so it doesn't freeze the UI when a series of reads are performed. I initially used a thread, but in the long run that isn't going to work well (the code is much more complicated than what is below). I am trying to figure out async/await, and here is what I have tried so far.
I have a click handler and related methods (details, including parameters to some of the methods, removed for simplicity) as follows:
private void cmdOpenModPort_Click(object sender, EventArgs e) {
// stuff
CheckBoardsAsync();
// stuff
}
private async void CheckBoardsAsync() {
// Read from a serial port with different settings.
for (byte iTryAdrs = 125; iTryAdrs <= 144; iTryAdrs++) {
ushort[] registers = await mt.ReadRegistersChkSeeBrdAsync(iTryAdrs);
if (registers != null) {
// Update UI based on registers
} else {
// Update UI
}
}
}
public async Task<ushort[]> ReadRegistersChkSeeBrdAsync(byte b) {
// can't await because the method is not async.
return ReadRegistersChkSeeBrd(b);
}
public ushort[] ReadRegistersChkSeeBrd(byte b) {
try {
// read from serial port. NOT an asynchronous method and it is
// calling a method from a package so I cannot control this.
return master.ReadHoldingRegisters(b);
} catch (Exception) {
return null;
}
}
This doesn't work because the click handler doesn't finish until CheckBoardsAsync finishes. I want the call to CheckBoardsAsync to be asynchronous, but that method makes a series of asynchronous calls and doesn't itself need to return anything, so there isn't a value to wait for. Since it doesn't return anything, I can't await it.
So what is the proper way to deal with this? My understanding is that I should not use threads in this case because I am waiting for I/O, and there is hardly any computation. So this seems like the correct approach. Do I just fake it and have the CheckBoardsAsync return something I don't care about? That seems weak.
I hope I have been clear enough.

You should be getting a warning for having a async method that has no await calls in it. Marking a method as async doesn't automatically make it asynchronous, it just allows the use of the await keyword. Here you have CPU bound work, no IO bound work, so what you want to do is perform that work in another thread (which can be done using Task.Run) and then asynchronously wait on that.
The change is easy enough, just remove ReadRegistersChkSeeBrdAsync entirely and replace:
ushort[] registers = await mt.ReadRegistersChkSeeBrdAsync(iTryAdrs);
with
ushort[] registers = await Task.Run(() =>
mt.ReadRegistersChkSeeBrd(iTryAdrs));
It would also be better practice to remove the CheckBoardsAsync method entirely and have the body of that method simply be the body of cmdOpenModPort_Click. (Or, for that matter, just attach CheckBoardsAsync as the click handler.)

You need to make it return Task (not Task<T>).
You can then await it even though it doesn't return a value.

how to propagate some data to main process from TPL tasks while tasks are running

I have a situation where I create a list of long running tasks which monitors some system/network resources and then sends email, logs into a txt file, and calls a web service when some conditions are met. Then begins monitoring again. These tasks are created in a windows service and hence will be running all the time.
I want them to raise events or something to notify the parent class (which created them) and it will performs the 3 operations i mentioned above instead of each object in tasks doing it by itself.
And how can it be controlled that only a single task uses that parent class's method at a single time. As Email and a web service call is involved, so two concurrent requests may beak the code.
UPDATE
These Watchers are of three types, each implements the following interface.
public interface IWatcher
{
void BeginWatch();
}
Classes that implement are
//this watcher is responsible for watching over a sql query result
public class DBWatcher : IWatcher
{
....
void BeginWatch()
{
//Here a timer is created which contiously checks the SQL query result.
//And would Call SERVICE, send an EMAIL and LOG into a file
Timer watchIterator = new Timer(this._intervalMinutes * 60000);
watchIterator.Elapsed += new ElapsedEventHandler(_watchIterator_Elapsed);
watchIterator.Start();
}
void _watchIterator_Elapsed(object sender, ElapsedEventArgs e)
{
//1. Check Query result
//3. Call SERVICE, send an EMAIL and LOG into a file if result is not as was expected
//I have done the work to this part!
//And I can do the functions as follows .. it should be simple.
//*********************
//SendEmail();
//LogIntoFile();
//CallService();
//But I want the above three methods to be present in one place so i dont have to replicate same functionality in different watcher.
//One approach could be to create a seperate class and wrape the above mentioned functions in it, create an instance of that class here and call them.
//Second option, which I am interested in but dont know how to do, is to have this functionality in the parent class which actually creates the tasks and have each watcher use it from HERE ...
}
....
}
//this watcher is responsible for watching over Folder
public class FolderWatcher : IWatcher
{
....
void BeginWatch()
{
///Same as above
}
....
}
First I create a List from an XML file. This can contain multiple instances of DBWatcher which will continously watch a different query result and FolderWatcher which will continously watch different Folders continously.
After the List is created, I call the following function that I call to create a separate Task. I call this function MANY times to create a different set of watchers.
private void _createWatcherThread(IWatcher wat, CancellationTokenSource cancellationToken)
{
//This represents a watcher that will watch some specific area for any activities
IWatcher watcher = wat.Copy();
bool hasWatchBegin = false;
try
{
//run forever
for (;;)
{
//dispose the watcher and stop this thread if CANCEL token has been issued
if (cancellationToken.IsCancellationRequested)
{
((IDisposable)watcher).Dispose();
break;
}
else if (!hasWatchBegin)
{
//This method of a watcher class creates a timer. which will
//continously check the status after a few minutes... So its the
//timer's elapsed method in Watcher object which will send the mail
//& call service etc to update the admin of current status of the watcher.
//this will be called only once in a watcher!
watcher.BeginWatch();
hasWatchBegin = true;
}
}
}
catch (Exception ex)
{
//Watcher has thrown an exception.
//Again, do the following operations
//*********************
//SendEmail();
//LogIntoFile();
//CallService();
}
}

Provided you make your email, logging & webservice calls threadsafe you can pass references to the code which sends to each of these sinks as a closure (Here's Jon Skeet's excellent explanation of c# closures) into the monitoring tasks. Here's an example where you need to launch multiple tasks:
...
void Email(string message){}
void Log(string message){}
void CallWebService(string message){}
void RunMonitoringTask()
{
var task = Task.TaskFactory.StartNew(() =>
{
string message = Monitor();
if( ShouldNotify(message))
{
Email(mesasge);
Log(message);
CallWebService(message);
}
}
)
}
...
EDIT
vs. an infinite monitor loop triggering tasks when necessary:
...
void Email(string message){}
void Log(string message){}
void CallWebService(string message){}
void Monitor()
{
while(true)
{
string message = Monitor();
if(ShouldNotify(message))
{
var task = Task.TaskFactory.StartNew(() =>
{
Email(mesasge);
Log(message);
CallWebService(message);
}
}
}
)
}
...
As far as how to implement these classes, I'd recomend an approach where each of these sinks accepts the message & then offloads it to it's own processing thread/task to avoid blocking your monitoring tasks & holding up the other notifications.

The Progress class is just perfect for this task. It is a means of allowing a long running process to notify someone (usually the caller) of the current progress of that operation.
Progress<string> progress = new Progress<string>();
progress.ProgressChanged += (s, data) => Console.WriteLine(data);
for (int i = 0; i < 2; i++)
Task.Run(() => DoWork(progress));
public static void DoWork(IProgress<string> progress)
{
int i = 0;
while (true)
{
Thread.Sleep(500);//placeholder for real work
progress.Report(i++.ToString());
}
}
If you have different types of information to report at different times then just pass in multiple IProgress instances to the worker method. (Or, if you are reporting the progress of several types of data at the same time wrap all of the data in a composite object.)
Also note that this is capable of handling the synchronization that you have said that you need. Progress instances, when created, capture the value of SynchronizationContext.Current at the time that it's created, and marshal all of the event handlers for the progress changed event into that sync context. So if your application will already have a context (i.e. a UI context from a desktop application) then you get that for free. If you don't have one (i.e. it's a console application) then you'll need to either manually synchronize the event handler with say a lock, or create your own SynchrnonizationContext to set as the current context.

How can I get my application to continue, only when all async operations are complete?

In my application, I used to create along string of async operations, which passed in functions like this:
public void LotsOfAsync()
{
DoAsync1( ()=> { DoAsync2( ()=> { doAsync3( ()=> { finalAction();}) } ) } );
}
However, now I have moved many of those async operations into separate classes and objects but I want the same results. Mainly I have moved those async operations into classes which become part of a collection.
I'd like my new code to look like this:
public void lotsOfAsync()
{
DoAsync1();
for each ( MyClass in Async1List)
{
MyClass.DoAsyn2();
}
if (allAsyncOperationsAreComplete)
{
FinalAction();
}
}
What things do I need to do, to get this to work? Thanks.
Using the Answer below about Tasks, something still seems to be lacking. The program never continues even after throwing everything into a BackgroundWorker.

You can use Task<T> (using the Task Parallel Library for Silverlight) - something like this maybe:
List<Task> tasks = new List<Task>();
Task.Factory.StartNew(() => DoAsync1()).ContinueWith(_ =>
{
foreach (MyClass myClass in Async1List)
{
tasks.Add(Task.Factory.StartNew(() => myClass.DoSomething()));
}
Task.WaitAll(tasks.ToArray());
FinalAction();
});

Im not familiar with wp7, but you may use counter as static field and check if it's equal to 0 in final action.
Every MyClass.DoAsyn2() should fire maybe a event, or any other code to signal that it is finished.
Another option is to move all async to task and call Task.WaitAll
http://msdn.microsoft.com/en-us/library/dd270695.aspx

Have you had a look at the CountdownEvent in .Net 4? This is a signalling construct where one thread will block and only proceed once other threads have completed and called set on the count down event. You initialise it with the number of signals you need before the thread calling Wait on the construct will proceed. E.g.:
CountdownEvent countdown = new CountdownEvent(3);
will only let the thread calling Wait to proceed once 3 other threads have called Signal.
So your example would perhaps look something like this:
public void lotsOfAsync()
{
Int32 numberOfAsyncProcesses = Async1List.Length + 1;
CountdownEvent countdown = new CountdownEvent (numberOfAsyncProcesses);
DoAsync1(countdown); // call countdown.signal() in the async method once complete.
for each ( MyClass in Async1List)
{
// call countdown.signal() in the async method once complete.
MyClass.DoAsyn2(countdown);
}
if(countDown.Wait(TimeSpan.FromSeconds(3))
{
FinalAction();
}
}
I've also added a timeout where the calling thread will unblock after 3 seconds if failed to get a response from all processing threads. In this case, the final action will not be performed.
You can reproduce this with Monitor/Pulse if you are not targeting .Net 4.
There is a nice example here too. Hope this helps!

After looking through all the previous answers, I was unable to solve my problem.
Instead what I needed to do, was create custom Events within my classes, which triggered upon a successful completion of the asynchronous tasks.
The website that proved the most useful to me to accomplish this was: http://www.marten-online.com/csharp/simple-custom-event-handling.html
My final code looked something like this:
public void lotsOfAsync()
{
DoAsync1();
for each ( MyClass in Async1List)
{
MyClass.PropertyChange += new MyClass.PropertyChangeHandler(Async2Complete);
MyClass.DoAsyn2();
}
}
public void Async2Complete(object sender, PropertyChangeEventArgs data)
{
if (data.PropertyName == "AsyncComplete")
{
totalAsyncCompleted++;
if (totalAsyncCompleted >= Async1List.Count)
{
FinalAction();
}
}
}

Have you heard of the Deferred pattern often used in Javascript?
It is simple to work with and very dynamic and you should be able to implement it on Windows phone aswell.
Have a look at this guide
http://twistedmatrix.com/documents/current/core/howto/defer.html
Regards
Tobias

Design alternatives to thread implementation of coroutines for converting a push method into a pull method

I have a collection class that holds lots of different types of data in a compressed format. In order to enumerate over all of the values in the collection is has an Execute(Query, IDataWriter) method. You pass it a query that defines what data you want, and then for each piece of matching data it calls a method on the IDataWriter object you pass in.
The IDataWriter interface has 15 different methods, one for each different data type in the collection. Now I need to commit this data to the database and I want to be able to implement IEnumerator<SqlDataRecord> to commit stuff to the database. The problem comes in how to convert calling Execute which dumps a ton of data into the IDataWriter object(push), to a pull method so that the IEnumerator's MoveNext and Current can be used.
I have looked at Coroutines and fibers, but none of the examples I have found seem like they would work for an existing method (Execute in my case) that internally knows nothing of the corountine. So my plan in pseudocode is to do the following using Threads and manual synchronization.
The Execute method would be running in a seperate Thread and I would manually wait and signal it inside each IDataWriter method
class EnumeratorAdapterObject : IEnumerator<SqlDataRecord>, IDataWriter
{
public EnumeratorAdapterObject(Store storeObject)
{
workerThread = new Thread(storeObject.Execute(query, this));
}
public bool MoveNext()
{
if (firstTimeCalled)
{
start worker thread
}
else
{
signal resume
}
block for either a call into an Add method or the Execute thread finishes
if (not anything buffered)
return false
else
return true
}
// 14 other methods like this implemented in IDataWriter, each with different types
public void Add_Decimal(IntvlDataHeader header, decimal data)
{
buffer field of current SqlDataRecord = generate record;
signal main thread
wait for resume signal
}
public SqlDataRecord Current
{
get { return buffer field of current SqlDataRecord; }
}
}
Does this look like a good approach? Does anyone know of any examples or questions that already implement this?
Or would there be a way to take advantage of any of the new 4.0 features? I thought about using a blocking concurrent collection with a limit of 1 thing in it, but then how would the consumer(the IEnumerator's MoveNext) know when the other thread is finished adding stuff?

Rather than doing manual Thread creation and synchronization with Signal/Wait I figured out that I can use Blocking collection with a call to CompleteAdding() when done. The following is a quick example, for my problem above I will wrap this in an object that implements IEnumerator<SqlDataRecord> and IDataWriter, so instead of the GenerateStuff call I will be calling Execute and the Add_* methods will be the ones calling col.Add(new SqlDataRecord(....))
static void Main(string[] args)
{
var col = new BlockingCollection<int>(1);
Task.Factory.StartNew(
() =>
{
GenerateStuff(col);
col.CompleteAdding();
});
while (!col.IsCompleted)
{
Thread.Sleep(100);
int result;
if (!col.TryTake(out result, -1))
{
break;
}
Console.WriteLine("Got {0}", result);
}
Console.WriteLine("Done Adding!");
}
static void GenerateStuff(BlockingCollection<int> col)
{
for (int i = 0; i < 10; i++)
{
Thread.Sleep(10);
Console.WriteLine("Adding {0}", i);
col.Add(i);
Console.WriteLine("Added {0}", i);
}
}
This also has the advantage that the worker thread that is running the Execute will be generating the next result concurrently with the IEnumerator returning Current and the sql code doing whatever it does to commit the data. With the manual thread signalling only one thread would ever run at a time.