Develop Reference

How to I design a class so looping is not necessary?

I have to process some records. For reasons that are unnecessary to mention, I can not loop through these records at the UI layer. The client wants to be able to simply call the middle tier using a function call, have the middle tier loop through and process the records. The problem is they want the middle tier to report back a status after each record is processed. How would I design that. For what it's worth, this is c# in .net.

A setup similar to this should work. It's untested/uncompiled so consider it pseudo-code. Also, it should ideally be asynchronous, but this will give you a starting point as an example of how to communicate changes back to the UI through eventing without the UI being aware of any "looping".
Event plumbing:
public class MyEventArgs : EventArgs
{
//add properties you want to send to the UI here.
}
public delegate void ProcessedEventHandler(object sender, MyEventArgs e);
Middle tier raises events.
public class MiddleTier
{
public event ProcessedEventHandler RecordProcessed;
//NOTE it would be best to make a tweak to do this asynchronously
//such that all records can be processed at the same time instead
//of processing them sequentially. if the method were async, you
//could do all of this without the method itself blocking.
public void Process()
{
//this loop/processing should ideally be asynchronous
foreach(var thing in whatever)
{
//process thing
//make event args
var args = new MyEventArgs(); //fill out properties
//raise event
OnProcessed(args);
}
private void OnProcessed(MyEventArgs args)
{
//follow this pattern for thread safety
var p = RecordProcessed;
if(p != null)
p(this, args);
}
}
}
Then in your UI layer:
//in some UI function
var mt = new MiddleTier();
//handle event
mt.RecordProcessed +=
(s, args) =>
{
//update UI
};
//kick things off
mt.Process();

You don't mention what technology your UI will be but assuming it is an application, you want the processing to happen on a separate thread so as to allow your UI to update.
I would look at the backgroundworker component as a starting point. It facilitates a progresschanged event you can use to notify your UI of how it getting on. Similar can be achieved using asynchronous framework.

Call a method only after jobs on other threads have terminated?

in my app I have two methods that load data from csv. These methods are loadData() and loadOtherData(). I want to run them in parallel so I have two threads to run them.
The loadData() method also populate one datagridview, while the data loaded by the loadOtherData() method are stored in a dictionary myDictionary.
After data loading is completed (so the methods running in the two threads have finished) I want to call another method updateGrids() that takes myDictionary as argument.
For updateGrids() to run properly, both loadData() and loadOtherData() must have run successfully cause otherwise the method doesn't have the data to work on.
How can I call method updateGrids() only when the other methods have terminated in other threads?
The structure is as follow:
private void loadData_Click_1(object sender, EventArgs e)
{
ThreadStart thread1Start = new ThreadStart(loadData); // load data and fill the first datagridview
ThreadStart thread2Start = new ThreadStart(loadOtherData); // load data and fill myDictionary
Thread t1 = new Thread(thread1Start);
Thread t2 = new Thread(thread2Start);
t1.Start();
t2.Start();
updateGrids(myDictionary); // call this method to update remaining datagridviews
}
private void updateGrids(Dictionary<string, double> myDictionary)
{
// update the remaining datagridviews by using
// information in the first datagridview and in myDictionary
}
But if I run this, I get the error when updateGrids() since it doesn't have the data to work with. How can I easily change the code to make it work?

You could stop using ThreadStart objects directly and use Tasks instead - have them returned from your loadData and loadOtherData methods.
You have either two options here:
Either Use Task.WaitAll if you're using .NET 4.0
or (more preferable)....
Make loadData and loadOtherData async in .NET 4.5 and above (or use the async BCL package in .NET 4.0). The when you call the methods, you can cache the Task objects they return and await a call to Task.WhenAll
public async Task LoadData(){
}
public async Task LoadOtherData() {
}
private async void loadData_Click_1(object sender, EventArgs e)
{
var loadDataTask = LoadData();
var loadOtherDataTask = LoadOtherData();
await Task.WhenAll(loadDataTask, loadOtherDataTask);
updateGrids(myDictionary);
}

You can use async/await to wait both tasks without blocking the UI
async private void loadData_Click_1(object sender, EventArgs e)
{
await Task.WhenAll(Task.Run(() => loadData()),
Task.Run(() => loadOtherData()));
updateGrids(myDictionary);
}

After you start your threads, you want to wait until they are done. So add these lines of code:
t1.Join();
t2.Join();
Join will cause the current thread to sleep until the thread being joined is complete.
Based on comments, I'm assuming this is WinForms. In this case Join will block the UI thread. You could create a third thread that will join the other two threads and then call Invoke to call your update method.
Since you're using .Net 4.5., you could use Async / Await; add the async modifier to your click method event handler and uses Tasks instead of Threads directly and await them. Other answers cover that, and you can see this blog for other details too.

Best way to run Database access tasks, that return objects, in separate thread keeping UI responsive?

[Windows forms application & .NET 4.0]
I need to execute database access methods that return objects (either list of classes or simple classes).
Also i need to open forms that are responsive while main thread does initialization.
I need to run these on separate threads keeping the User Interface responsive and of course to be able to pass the results back to main thread for UI updates.
I have been reading books regarding the various ways for this.
I understand that my job can be done by:
BackGroundWorker
Thread Class
Task Class
Which one i should dive into ?
Update: using the suggested Task class i am getting errot for cross thread safety using this:
private void BtnCheckClick(object sender, EventArgs e)
{
var itm = Task<JDEItemLotAvailability>.Factory.StartNew(() =>
Dal.GetLotAvailabilityF41021(
txtLot.Text,
cmbMcu.SelectedItem.ToString(),
cmbLocn.SelectedItem.ToString())
);
lblDescriptionValue.Text = itm.Result.Description;
lblItemCodeValue.Text = itm.Result.Code;
lblQuantityValue.Text = itm.Result.AvailableQuantity.ToString();
LotFocus(true);
}
On the above exmaple i am getting the exception in cmbMcu control not the txtLot.

I would use the Task class, it's really easy to synchronize it and it already provides a support for returning objects.
var task = Task.Factory.StartNew(
() => GetDatabaseData(someArguments),
TaskCreationOptions.LongRunning);
// Example method
public DataSet GetDatabaseData(object args) { ... }
this this tells a scheduler to create and begin a new task and gives it a hint that it might be a good idea not to use a thread-pool thread, if the scheduler uses a thread-pool. Anyway you can now decide how do you want to synchronize.
For example to achieve similar behaviour as in Gregor Primar's answer, you can set up a continuation using ContinueWith method as follows,
task.ContinueWith(oldTask => ProcessReturnedData(oldTask.Result));
// Example method
public IEnumerable<SomeEntity> ProcessReturnedData(DataSet data) { ... }
which will schedule calling the ProcessReturnedData method after the task object has done executing. Note that this will be called even if task fails for some reason, so it may not be always a good solution - or you would have to do some checks in the provided delegate.
If you want to do a non-blocking wait on the main thread and use the returned object there, you can simply use the Wait method.
task.Wait(); // Makes current thread wait until the task is comnpleted.
DataSet result = task.Result; // Accessing the result object.

I hade done a lot of projects using Thread, however Task should be more easy to use.
Here is demo how make async operations using Threads.
This is the class that will return data to ui:
public class MyAsyncClass
{
public delegate void NotifyComplete(DataSet data);
public event NotifyComplete NotifyCompleteEvent;
//Starts async thread...
public void Start()
{
System.Threading.Thread t = new System.Threading.Thread(new System.Threading.ThreadStart(DoSomeJob));
t.Start();
}
void DoSomeJob()
{
//just wait 5 sec for nothing special...
System.Threading.Thread.Sleep(5000);
if (NotifyCompleteEvent != null)
{
//TODO: fill your data...
DataSet ds = new System.Data.DataSet();
NotifyCompleteEvent(ds);
}
}
}
And here is ui implementation:
MyAsyncClass myClass = null;
private void button2_Click(object sender, EventArgs e)
{
myClass = new MyAsyncClass();
myClass.NotifyCompleteEvent += new MyAsyncClass.NotifyComplete(myClass_NotifyCompleteEvent);
//here I start the job inside working class...
myClass.Start();
}
//here my class is notified from working class when job is completed...
delegate void myClassDelegate(DataSet data);
void myClass_NotifyCompleteEvent(DataSet data)
{
if (this.InvokeRequired)
{
Delegate d = new myClassDelegate(myClass_NotifyCompleteEvent);
this.Invoke(d, new object[] { data });
}
else
{
//TODO: show your data
MessageBox.Show("Data retrieved!");
}
}

Why Is This ThreadPool.QueueUserWorkItem Code Not Failing My Test?

The code in question
public void StartPlaying()
{
ThreadPool.QueueUserWorkItem(ignoredState =>
{
while (_playlist.Count > 0)
{
var audioFile = _playlist.Dequeue();
if (StartedPlaying != null)
StartedPlaying(this, new TypedAudioFileEventArgs(audioFile));
audioFile.SoundPlayer.PlaySync();
audioFile.SoundPlayer.Dispose();
if (StoppedPlaying != null)
StoppedPlaying(this, new TypedAudioFileEventArgs(audioFile));
}
});
}
and my test:
[TestMethod()]
public void StartPlayIsCalledTwice_OnlyRunningOnce()
{
int timeBetweenPlays = 0;
var target = new TypedAudioFilePlayer(timeBetweenPlays);
target.AddFile(TypedAudioFileCreator.CreateWord(1, "bl"));
target.StartedPlaying += StartedPlaying_Once;
target.StartPlaying();
target.StartPlaying();
}
private bool _once = false;
private void StartedPlaying_Once(object sender, TypedAudioFileEventArgs e)
{
if (!_once)
_once = true;
else
Assert.Fail("Should not be called more than once!");
}
I believe my unit test should fail, judging by the MSDN description of ThreadPool.QueueUserWorkItem:
Queues a method for execution. The method executes when a thread pool thread becomes available.
The default ThreadPool size is 512, so two threads should immediately be available to process the StartPlaying call. I believe my code should fail since I haven't provided any safeguards from race conditions in which both threads can access the same resource.
What's happening here?

Because the StartedPlaying event is only raised if StartPlaying is called when there are items to play.
_playlist.Dequeue(); dequeues the file you enqueue. Therefore the second time you get to while (_playlist.Count > 0) it will immediately fail, passing the second call to StartPlaying straight through without raising the event.
Also, as Bruno Silva points out, the thread spawned by the second call to StartPlaying may not have a chance to execute anything before the test exits.
For what it's worth, there are about a million at least 2 threading mistakes in this code also:
// Where did _playlist come from? Is it shared state among the player threads?
// If so, all access to it should be in locks, since queues are not thread safe
while (_playlist.Count > 0)
// Both of these start threads and then immediately return.
// The test will probably exit before either of those threads do anything much
target.StartPlaying();
target.StartPlaying();
}
If you want to properly unit test, you need to define preconditions, expectations, actions, and postconditions:
Preconditions: you have an initialized TypedAudioFilePlayer with one file queued:
var target = new TypedAudioFilePlayer(timeBetweenPlays);
target.AddFile(TypedAudioFileCreator.CreateWord(1, "bl"));
Expectations: The StartedPlaying event will be raised only once if StartPlaying is called twice
target.StartedPlaying += StartedPlaying_Once;
Actions: The StartPlaying method will be called twice:
target.StartPlaying();
target.StartPlaying();
Postconditions: The StartedPlaying event was only raised once:
private bool _once = false;
private void StartedPlaying_Once(object sender, TypedAudioFileEventArgs e)
{
if (!_once)
_once = true;
else
Assert.Fail("Should not be called more than once!");
}
Now, your test succeeds. That's not good in this case, because of what I explain above. You need to get your test to a failing state by eliminating the queue bug and race condition, then work on making the test pass the right way.

It seems that you are working with a shared resource between two threads so once might not be set to true when the Play is called for the second time. You can use a lock to allow executing part of your code by one thread at a time :
private readonly object lock_object=new object();
private void StartedPlaying_Once(object sender, TypedAudioFileEventArgs e)
{
lock(lock_object)
{
if (!_once)
_once = true;
else
Assert.Fail("Should not be called more than once!");
}
}

Could those be failing outside the text execution? Your test ends right after you queue the item so I'm not sure what is happening to those threads when the test method ends its execution. Have you tried using WaitHandle to wait for the them to finish inside the test?
See http://msdn.microsoft.com/en-us/library/system.threading.waithandle.aspx for an example.

How to respond events while a threaded operation is running in .NET

I have a class to launch background operations in a WinForms application. I need to write this background worker since my requisites are using .NET 1.1, so I cannot use BackgroundWorker, that is only available from .NET 2.0
This class get a delegate and execute it in a thread. I want the main thread to respond to events.
I also want to indicate that the operation is running setting the application cursor to Cursors.WaitCursor.
What do you think about current implementation? I'm interested in the method WaitTillThreadFinishes(), because I'm not sure about Application.DoEvents(), please read the code and share with me opinions about WaitTillThreadFinishes.
The following code executes the operation:
private object ExecuteOperation (Delegate target, params object[] parameters)
{
mTargetDelegate = target;
mTargetParameters = parameters;
mTargetThread = new Thread(new ThreadStart(ThreadProc));
mTargetThread.Name = mTargetDelegate.Method.Name;
mOperationFinished = false;
// start threaded operation
mTargetThread.Start();
// perform active waiting
WaitTillThreadFinishes();
return mTargetResult;
}
The following code is executed in a thread, simply call the delegate, and wrap exceptions:
protected virtual void ThreadProc()
{
try
{
mTargetResult = mTargetDelegate.DynamicInvoke(mTargetParameters);
}
catch (ThreadAbortException) { }
catch (Exception ex)
{
//manage exceptions here ...
}
finally
{
mOperationFinished = true;
}
}
And this is the code performs an active waiting. I'm interested on share with you. Any better option? Any pain calling Application.DoEvents() massively?
private void WaitTillThreadFinishes ()
{
// Active wait to respond to events with a WaitCursor
while (!mOperationFinished)
{
// sleep to avoid CPU usage
System.Threading.Thread.Sleep(100);
Application.DoEvents();
Cursor.Current = Cursors.WaitCursor;
}
Cursor.Current = Cursors.Default;
}
Thanks in advance.

Please let me know if i understood your question correctly.
Why dont you use an event to notify the UI that the worker finished his job?
This way, the UI doen't get blocked by the worker, and you avoid busy waiting.
Sample Implementation
public class MyBackgroundWorker
{
// Fields
private Delegate _target;
private object[] _arguments;
// Events
public event EventHandler RunWorkerStarted;
public event EventHandler<RunWorkerCompletedEventArgs> RunWorkerCompleted;
// Event Invocators
public void InvokeRunWorkerStarted()
{
var handler = RunWorkerStarted;
if (handler != null) handler(this, new EventArgs());
}
public void InvokeRunWorkerCompleted(object result)
{
var handler = RunWorkerCompleted;
if (handler != null) handler(this, new RunWorkerCompletedEventArgs(result));
}
public void RunWorkerAsync(Delegate target, params object[] arguments)
{
_target = target;
_arguments = arguments;
new Thread(DoWork).Start(arguments);
}
// Helper method to run the target delegate
private void DoWork(object obj)
{
_target.DynamicInvoke(_arguments);
// Retrieve the target delegate's result and invoke the RunWorkerCompleted event with it (for simplicity, I'm sending null)
InvokeRunWorkerCompleted(null);
}
}
internal class RunWorkerCompletedEventArgs : EventArgs
{
public RunWorkerCompletedEventArgs(object result)
{
Result = result;
}
public object Result { get; set; }
}
Usage
In the UI you can use it this way:
private void button1_Click(object sender, EventArgs e)
{
var worker = new MyBackgroundWorker();
worker.RunWorkerStarted += worker_RunWorkerStarted;
worker.RunWorkerCompleted += worker_Completed;
worker.RunWorkerAsync(new MethodInvoker(SomeLengthyOperation), null);
}
void worker_RunWorkerStarted(object sender, EventArgs e)
{
}
void worker_Completed(object sender, EventArgs e)
{
MessageBox.Show("Worker completed");
}
private void SomeLengthyOperation()
{
Thread.Sleep(5000);
}
Final Notes
Remember to Invoke() in the event handlers to access the UI thread correctly. You can also modify the worker so this is done in a safe way.

There isn't much support in 1.1 for doing this, but I'll tell you what I'd do (sorry, no code at this time).
As for the asynchronous operation, I'd use the APM to kick off and complete the asynchronous method. This is fully supported in 1.1, so no worries there.
The idea is that in the UI, you store some indication that work is being done (a boolean field, for example) and (optionally) a Timer used to "wake up" the UI on a regular basis to check on the current status of the background work and indicate this to the user.
You would set the boolean to indicate you are working in the background, call BeginInvoke() on your delegate (using the overload that takes a callback search for "Executing a Callback Method When an Asynchronous Call Completes
"), and start the Timer. When the user attempts to use the UI, you would optionally check the boolean and cancel the operation, thus preventing the user from doing something harmful while you are waiting. When the timer Ticks, you can check the status of your asynchronous method by, say, a shared field that the method writes updates to and the UI reads. For example, a double which the UI uses to update a progress bar.
Once the callback fires, you clean up your asynchronous mess (i.e., call EndInvoke, and handle any exceptions thrown, etc), turn off the Timer and reset your boolean running indication field.
By using this method, you can keep the UI completely responsive (and partially usable, depending on your overall design), can set up a mechanism to abort the background worker (through the use of another field, the reverse of the boolean mentioned earlier, and inform the user of the status of the operation.

There is occasionally a case for kicking off a thread and waiting for its return, if you are doing other things in the meantime, but in this case, with the code you have shown, it is meaningless.
If you want the threadProc to allow for events to be processed, then call doevents in that, which will free up the CPU briefly, allowing for processing.
Unless you have a particular reason for needing to thread processes, you should not do it. Getting it right - as Ian Boyd has said - is difficult, and the more you need to interact with it the harder it is. If you can run fire-and-forget threads, that is the easiest.

Ideally you start the asynchronous operation and leave your form alone (aside from maybe using the Cursors.AppStarting cursor).
When your threaded operation completes, it then needs to fire some sort of BackgroundOperationComplete event. This is where your would call from your asynchronous delegate code:
form.Invoke(BackgroundOperationComplete);
The form's BackgroundOperationComplete method is where you can handle the fact that the background operation is complete:
void BackgroundOperationComplete()
{
this.Cursor = Cursors.DefaultCursor;
lblAnswer.Text = "The thread is done";
}
If all else fails, keep the operation synchronous, and use an IProgressDialog. (brief conceptual pseudo-code from memory):
void DoStuff()
{
IProgressDialog pd = new ProgressDialog();
pd.SetTitle = "Calculating Widgets";
pd.StartTimer(PDTIMER_RESET, NULL)
pd.StartProgressDialog(this.Handle, NULL, PROGDLG_MODAL | PROGDLG_NOTIME | PROGDLG_NOPROGRESSBAR | PROGDLG_NOCANCEL, NULL);
try
{
pd.SetLine(1, "Please wait while the widgets are frobbed");
DoTheThingThatDoesTheSynchronousStuff();
}
finally
{
pd.StopProgressDialog();
}
pd = null;
}