Develop Reference

how to mix multithreading with sequential requirement?

i have a program which process price data coming from the broker. the pseudo code are as follow:
Process[] process = new Process[50];
void tickEvent(object sender, EventArgs e)
{
int contractNumber = e.contractNumber;
doPriceProcess(process[contractNumber], e);
}
now i would like to use mutlithreading to speed up my program, if the data are of different contract number, i would like to fire off different threads to speed up the process. However if the data are from the same contract, i would like the program to wait until the current process finishes before i continue with the next data. How do i do it?
can you provide some code please?
thanks in advance~

You have many high level architectural decissions to make here:
How many ticks do you expect to come from that broker?
After all, you should have some kind dispatcher here.
Here is some simple description of what basically is to do:
Encapsulate the incoming ticks in packages, best
single commands that have all the data needed
Have a queue where you can easily (and thread safe) can store those commands
Have a Dispatcher, that takes an item of the queue and assigns some worker
to do the command (or let the command execute itself)
Having a worker, you can have multiple threads, processes or whatsoever
to work multiple commands seemlessly
Maybe you want to do some dispatching already for the input queue, depending
on how many requests you want to be able to complete per time unit.
Here is some more information that can be helpful:
Command pattern in C#
Reactor pattern (with sample code)

Rather than holding onto an array of Processes, I would hold onto an array of BlockingCollections. Each blocking collection can correspond to a particular contract. Then you can have producer threads that add work to do onto the end of a corresponding contract's queue, and you can have producer queues that consume the results from those collections. You can ensure than each thread (I would use threads for this, not processes) is handling 1-n different queues, but that each queue is handled by no more than one thread. That way you can ensure that no bits of work from the same contract are worked on in parallel.
The threading aspect of this can be handled effectiving using C#'s Task class. For your consumers you can create a new task for each BlockingCollection. That task's body will pretty much just be:
foreach(SomeType item in blockingCollections[contractNumber].GetConsumingEnumerable())
processItem(item);
However, by using Tasks you will let the computer schedule them as it sees fit. If it notices most of them sitting around waiting on empty queues it will just have a few (or just one) actual thread rotating between the tasks that it's using. If they are trying to do enough, and your computer can clearly support the load of additional threads, it will add more (possibly adding/removing dynamically as it goes). By letting much smarter people than you or I handle that scheduling it's much more likely to be efficient without under or over parallelizing.

Assigning jobs to BackgroundWorker thread one at a time

My silverlight application fetches file sets from a webservice (async). The webservice method accepts an array of file names and returns the set of files (also as an array). The silverlight client makes several such requests for file sets.
The client issues many requests to the webservice at once. I need a BackgroundWorker thread at the client to process received file sets one after the other.
How can I collect all the file sets as and when they receive and give these sets to the BackgroundWorker thread one at a time.
EDIT:
I could not run multiple BackgorundWorkers as the file set processing module is not thread-safe.

Use a BlockingCollection / ConcurrentQueue to hold the information about file sets to be processed... in the backgroundworker you just have while loop dequeuing the next file set and processing it... the mentioned collections are threadsafe and really fast since most operations are implemented lock-free...

The backgroundworker has no built-in Listen mechanism. It is supposed to perform a long action and terminate.
One solution could consist in firing up one backgroundworker for each file set.
If the processing of those file sets must be synchronized, you could decide to push each request into a queue (basically an array. Make sure you synchronize access to it). Whenever you backgroundworker is done processing a file set, it would report to the main thread (ProgressChanged event IIRC) and loop over further possible requests in the array. Whenever the array is empty, the worker exits.
Pay attention though: If the worker exits while you are sending a request, you'll have a problem. That's why a basic thread may prove stronger than a background thread, especially if you can't know whether there will be further file sets to process. It all depends on your workflow.

Silverlight application not responding when multiple threads launched

I have a silverlight application which kicks off a number of jobs on the server side. Each job is a unit of work which are independent of each other.
I used Parallel.ForEach() and it works fine however I realized that if I had a large number of jobs (say over 300), when the thread count increases by 50 the silverlight application seems to stop responding (it does not freeze the browser but a grid which should have data populated in it is empty and the little doughnut keeps spinning).
Only when the thread counts drop again (i.e. all the jobs have finished processing) does the grid get populated.
I am testing with the Asp.net Development servers (cassini based) and was wondering has something to do with it.
I also switched my code to use the async programming model but I got the same problem as the threads increased.
Any ideas what may be causing this?
JD
I was thinking about doing ThreadPool.SetMaxThread() but I read somewhere that this may not work for web hosted apps.

If you go gobbling up all the threads with a Parallel for each then there aren't any threads left available to service the WCF requests that your Grid is likely depending on to fetch the data it needs to display.
You would probably want to use the ParallelOptions parameter in the ForEach method to specify the maximum number of parallel operations dispatched at one time.

Can I use multiple BackgroundWorker's in a synchronous way in WPF?

In my WPF application - I have a Listview with a Gridview inside. A user is able to add files to this list.
The gridview should allow me to have quite rich "rows", each row showing file name, extra application detail and most importantly a progress bar. Each row in the gridview is databound to a business object in a collection, and specifically the object has a property "Progress" which is bound to the progress bar.
I need to process each file, extract data and send the data to a web service and I would like to display the progress for each file in it's respective progress bar as I work through the collection of files.
Initially I started off with a simple loop containing the processing logic which periodically updated the "Progress" field for each row and noted that the UI didn't update very well.
I then moved the processing logic in the loop to a BackgroundWorker which modified the business object "Progress" field, and because the UI thread was not working so hard - it was a lot more responsive.
However due to the asynchronous nature of BackgroundWorker, it started multiple BackgroundWorkers (one for each row) virtually all at once as the loop spun out. All my progress bars started progressing...
What would be the best approach to limit it to only 2-3 BackgroundWorkers going at once? Or even leave it at one BackgroundWorker - and it doesn't start until the previous one is completed? How can I introduce a dependency so that in essence all the BackgroundWorkers are synchronous?

You can use the Task class for that, like in sample here

If you really want only one to execute at the same time, then there are several approaches. You can use the Task class and a TaskScheduler like #oleksii suggested in his answer or you can post work elements to a Queue and have a background thread work one them one after the other.
But if all you want is to do the work in the most optimal way depending on the current hardware your program is running on then I'd say leave it to the default task scheduler. Just call StartNew on the TaskFactory whenever you want to do work for a grid row and leave scheduling to .NET.
If you do have dependencies between tasks, you can use one of the ContinueWith overloads to maintain order between jobs.

Multithreading help, yet again (winForms)

I have developed an application that pulls X amount of records from my database per X amount of threads. Each thread then iterates the collections that are created and does some data validation for each record.
Once a record is validated / not validated by the application it is updated in the database as being valid / not valid. Records are only pulled by each thread if there are items in the database that have not been run through the application. There is a bit column to indicate if the application retrieved the data.
So, potentially, the system can run out of data depending on the number of threads and records per thread. I want the application to continue to check the database for any records that have not be run, then start the process of creating the threads, and finally validating the data.
Here is an example:
There are 50 records in the database
we are running 5 threads with 10 records per thread.
The application runs, the threads are created, the records are pulled and then processed. Now, the system is out of data. A user imports more data into the DB. The application, still looking to see if there are any records, sees that there are 5 new records in the database. It then starts the process all over to create the threads and process the records.
How can I have the system continue to look for data but allow the user to stop the system if need be. I tried using this:
while(RecordsFound <=0){
…sleepcode
} ;
RunProcessMethod
But the winform locks, obviously, during the waiting period. I tried adding the wait logic to another thread, but was afraid that if I run the process method from that thread, via a delegate, things would get weird since I am creating additional threads inside that method.
Thoughts?

The easiest way to fix this is to use a notification mechanism instead of polling. That is once you've spawned off the threads to read data from the data base, make them responsible for notifying the UI when they are complete instead of having the UI wait for them to be complete.
The easiest way to do this is to pass in a delegate for the threads to call when they are complete with the set of records found. The UI can then update when the records are available
delegate void CallBack(List<Data> found);
void OnDataFound(List<Data> found) {
// Get back on the UI thread
if ( this.InvokeRequired ) {
this.Invoke( new CallBack(OnDataFound), new object[] { found } );
return;
}
// Update display
}

I tried adding the wait logic to another thread, but was afraid that if I run the process method from that thread, via a delegate, things would get weird since I am creating additional threads inside that method. Thoughts?
You don't need to fear this. It's the proper way to handle this type of scenario. There is no problem with a background thread creating additional threads.