I'm trying to build a Windows Forms tool that runs queries asynchronously.
The app has a datagridview with 30 possible queries to run. The user checks the queries he wants to execute, say 10 queries, and hits a button.
The app has a variable called maxthreads = 3 (for the sake of discussion) that indicates how many threads can be used to async run the queries. The queries run on a production environment and we don't want to overload the system with too many threads running in the same time. Each query runs for an average of 30 sec. (some 5 min., others 2 sec.)
In the datagridview there is an image column containing an icon that depicts the status of each query (0- Available to be run, 1-Selected for running, 2- Running, 3- Successfully completed, -1 Error)
I need to be able to communicate with the UI every time a query starts and finishes. Once a query finishes, the results are being displayed in a datagridview contained in a Tabcontrol (one tab per query)
The approach: I was thinking to create a number of maxthread backgroundworkers and let them run the queries. As a backgroundworker finishes it communicates to the UI and is assigned to a new query and so on until all queries have been run.
I tried using an assignmentWorker that would dispatch the work to the background workers but don't know how to wait for all threads to finish. Once a bgw finishes it reports progress on the RunWorkerCompleted event to the assignmentWorker, but that one has already finished.
In the UI thread I call the assignment worker with all the queries that need to be run:
private void btnRunQueries_Click(object sender, EventArgs e)
{
if (AnyQueriesSelected())
{
tcResult.TabPages.Clear();
foreach (DataGridViewRow dgr in dgvQueries.Rows)
{
if (Convert.ToBoolean(dgr.Cells["chk"].Value))
{
Query q = new Query(dgr.Cells["ID"].Value.ToString(),
dgr.Cells["Name"].Value.ToString(),
dgr.Cells["FileName"].Value.ToString(),
dgr.Cells["ShortDescription"].Value.ToString(),
dgr.Cells["LongDescription"].Value.ToString(),
dgr.Cells["Level"].Value.ToString(),
dgr.Cells["Task"].Value.ToString(),
dgr.Cells["Importance"].Value.ToString(),
dgr.Cells["SkillSet"].Value.ToString(),
false,
new Dictionary<string, string>()
{ { "#ClntNb#", txtClntNum.Text }, { "#Staff#", "100300" } });
qryList.Add(q);
}
}
assignmentWorker.RunWorkerAsync(qryList);
}
else
{
MessageBox.Show("Please select at least one query.",
"Warning",
MessageBoxButtons.OK,
MessageBoxIcon.Information);
}
}
Here is the AssignmentWorker:
private void assignmentWorker_DoWork(object sender, DoWorkEventArgs e)
{
foreach (Query q in (List<Query>)e.Argument)
{
while (!q.Processed)
{
for (int threadNum = 0; threadNum < maxThreads; threadNum++)
{
if (!threadArray[threadNum].IsBusy)
{
threadArray[threadNum].RunWorkerAsync(q);
q.Processed = true;
assignmentWorker.ReportProgress(1, q);
break;
}
}
//If all threads are being used, sleep awhile before checking again
if (!q.Processed)
{
Thread.Sleep(500);
}
}
}
}
All bgw run the same event:
private void backgroundWorkerFiles_DoWork(object sender, DoWorkEventArgs e)
{
try
{
Query qry = (Query)e.Argument;
DataTable dtNew = DataAccess.RunQuery(qry).dtResult;
if (dsQryResults.Tables.Contains(dtNew.TableName))
{
dsQryResults.Tables.Remove(dtNew.TableName);
}
dsQryResults.Tables.Add(dtNew);
e.Result = qry;
}
catch (Exception ex)
{
}
}
Once the Query has returned and the DataTable has been added to the dataset:
private void backgroundWorkerFiles_RunWorkerCompleted(object sender,
RunWorkerCompletedEventArgs e)
{
try
{
if (e.Error != null)
{
assignmentWorker.ReportProgress(-1, e.Result);
}
else
{
assignmentWorker.ReportProgress(2, e.Result);
}
}
catch (Exception ex)
{
int o = 0;
}
}
The problem I have is that the assignment worker finishes before the bgw finish and the call to assignmentWorker.ReportProgress go to hell (excuse my French).
How can I wait for all the launched bgw to finish before finishing the assignment worker?
Thank you!
As noted in the comment above, you have overcomplicated your design. If you have a specific maximum number of tasks (queries) that should be executing concurrently, you can and should simply create that number of workers, and have them consume tasks from your queue (or list) of tasks until that queue is empty.
Lacking a good Minimal, Complete, and Verifiable code example that concisely and clearly illustrates your specific scenario, it's not feasible to provide code that would directly address your question. But, here's an example using a List<T> as your original code does, which will work as I describe above:
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace TestSO42101517WaitAsyncTasks
{
class Program
{
static void Main(string[] args)
{
Random random = new Random();
int maxTasks = 30,
maxActive = 3,
maxDelayMs = 1000,
currentDelay = -1;
List<TimeSpan> taskDelays = new List<TimeSpan>(maxTasks);
for (int i = 0; i < maxTasks; i++)
{
taskDelays.Add(TimeSpan.FromMilliseconds(random.Next(maxDelayMs)));
}
Task[] tasks = new Task[maxActive];
object o = new object();
for (int i = 0; i < maxActive; i++)
{
int workerIndex = i;
tasks[i] = Task.Run(() =>
{
DelayConsumer(ref currentDelay, taskDelays, o, workerIndex);
});
}
Console.WriteLine("Waiting for consumer tasks");
Task.WaitAll(tasks);
Console.WriteLine("All consumer tasks completed");
}
private static void DelayConsumer(ref int currentDelay, List<TimeSpan> taskDelays, object o, int workerIndex)
{
Console.WriteLine($"worker #{workerIndex} starting");
while (true)
{
TimeSpan delay;
int delayIndex;
lock (o)
{
delayIndex = ++currentDelay;
if (delayIndex < taskDelays.Count)
{
delay = taskDelays[delayIndex];
}
else
{
Console.WriteLine($"worker #{workerIndex} exiting");
return;
}
}
Console.WriteLine($"worker #{workerIndex} sleeping for {delay.TotalMilliseconds} ms, task #{delayIndex}");
System.Threading.Thread.Sleep(delay);
}
}
}
}
In your case, each worker would report progress to some global state. You don't show the ReportProgress handler for your "assignment" worker, so I can't say specifically what this would look like. But presumably it would involve passing either -1 or 2 to some method that knows what to do with those values (i.e. what would otherwise have been your ReportProgress handler).
Note that the code can simplified somewhat, particularly where the individual tasks are consumed, if you use an actual queue data structure for the tasks. That approach would look something like this:
using System;
using System.Collections.Concurrent;
using System.Threading.Tasks;
namespace TestSO42101517WaitAsyncTasks
{
class Program
{
static void Main(string[] args)
{
Random random = new Random();
int maxTasks = 30,
maxActive = 3,
maxDelayMs = 1000,
currentDelay = -1;
ConcurrentQueue<TimeSpan> taskDelays = new ConcurrentQueue<TimeSpan>();
for (int i = 0; i < maxTasks; i++)
{
taskDelays.Enqueue(TimeSpan.FromMilliseconds(random.Next(maxDelayMs)));
}
Task[] tasks = new Task[maxActive];
for (int i = 0; i < maxActive; i++)
{
int workerIndex = i;
tasks[i] = Task.Run(() =>
{
DelayConsumer(ref currentDelay, taskDelays, workerIndex);
});
}
Console.WriteLine("Waiting for consumer tasks");
Task.WaitAll(tasks);
Console.WriteLine("All consumer tasks completed");
}
private static void DelayConsumer(ref int currentDelayIndex, ConcurrentQueue<TimeSpan> taskDelays, int workerIndex)
{
Console.WriteLine($"worker #{workerIndex} starting");
while (true)
{
TimeSpan delay;
if (!taskDelays.TryDequeue(out delay))
{
Console.WriteLine($"worker #{workerIndex} exiting");
return;
}
int delayIndex = System.Threading.Interlocked.Increment(ref currentDelayIndex);
Console.WriteLine($"worker #{workerIndex} sleeping for {delay.TotalMilliseconds} ms, task #{delayIndex}");
System.Threading.Thread.Sleep(delay);
}
}
}
}
Related
I work on console version of link tester.
I start the function on multiple threads but I can't cancel them by key pressing.
Have an idea how I can do that ?
try
{
Thread[] tr = new Thread[Variables.Threads];
int i = 0;
while (i < Variables.Threads && Variables.running)
{
tr[i] = new Thread(new ThreadStart(Program.Runner));
i++;
}
//Start each thread
foreach (Thread x in tr)
{
x.Start();
}
//Console.ReadKey();
Task.Factory.StartNew(() =>
{
while (Colorful.Console.ReadKey().Key != ConsoleKey.Escape);
Variables.running = false;
foreach (Thread x in tr)
{
x.Abort();
}
Program.Menu();
});
}
catch (Exception)
{
}
EDIT : When my threads are a near of end, all of my app don't move after
The console is a not a good environment to start learning multitasking, much less multithreading. My advise to learning Multitasking - and especially Multithreading - is the BackgroundWorker in a WindowsForms applciation. The event queue does the main thing of keeping your application alive, without blocking I/O. And while the BackgroundWorker is horribly dated and should be removed from production code, it is pretty good training wheels.
Also your current code is swallowing exceptions, including fatal ones. This is a cardinal sin of exception handling. Do not do that.
At the end of the day, you can only do cancelation checking and progress reporting between distinct lines of code. In this BGW example, I had the advantage that I had to write all the loops anyway - so deep cancelation checking and progress reporting was in the cards. But if you use "somebody elses code", chances are high you have to wait for one function call to return before you can report and check.
#region Primenumbers
private void btnPrimStart_Click(object sender, EventArgs e)
{
if (!bgwPrim.IsBusy)
{
//Prepare ProgressBar and Textbox
int temp = (int)nudPrim.Value;
pgbPrim.Maximum = temp;
tbPrim.Text = "";
//Start processing
bgwPrim.RunWorkerAsync(temp);
}
}
private void btnPrimCancel_Click(object sender, EventArgs e)
{
if (bgwPrim.IsBusy)
{
bgwPrim.CancelAsync();
}
}
private void bgwPrim_DoWork(object sender, DoWorkEventArgs e)
{
int highestToCheck = (int)e.Argument;
//Get a reference to the BackgroundWorker running this code
//for Progress Updates and Cancelation checking
BackgroundWorker thisWorker = (BackgroundWorker)sender;
//Create the list that stores the results and is returned by DoWork
List<int> Primes = new List<int>();
//Check all uneven numbers between 1 and whatever the user choose as upper limit
for(int PrimeCandidate=1; PrimeCandidate < highestToCheck; PrimeCandidate+=2)
{
//Report progress
thisWorker.ReportProgress(PrimeCandidate);
bool isNoPrime = false;
//Check if the Cancelation was requested during the last loop
if (thisWorker.CancellationPending)
{
//Tell the Backgroundworker you are canceling and exit the for-loop
e.Cancel = true;
break;
}
//Determin if this is a Prime Number
for (int j = 3; j < PrimeCandidate && !isNoPrime; j += 2)
{
if (PrimeCandidate % j == 0)
isNoPrime = true;
}
if (!isNoPrime)
Primes.Add(PrimeCandidate);
}
//Tell the progress bar you are finished
thisWorker.ReportProgress(highestToCheck);
//Save Return Value
e.Result = Primes.ToArray();
}
private void bgwPrim_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
pgbPrim.Value = e.ProgressPercentage;
}
private void bgwPrim_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
pgbPrim.Value = pgbPrim.Maximum;
this.Refresh();
if (!e.Cancelled && e.Error == null)
{
//Show the Result
int[] Primes = (int[])e.Result;
StringBuilder sbOutput = new StringBuilder();
foreach (int Prim in Primes)
{
sbOutput.Append(Prim.ToString() + Environment.NewLine);
}
tbPrim.Text = sbOutput.ToString();
}
else
{
tbPrim.Text = "Operation canceled by user or Exception";
}
}
#endregion
However in your case, Multithreading seems unesseary. Multitasking without Threads would propably be better. Threads only help if you got a CPU bound task. And "checking links" sounds like a Network bound task. Threads have some extra headaches that you are better off avoiding.
I have a windows forms app that I am checking all the serial ports to see if a particular device is connected.
This is how I spin off each thread. The below code is already spun off the main gui thread.
foreach (cpsComms.cpsSerial ser in availPorts)
{
Thread t = new Thread(new ParameterizedThreadStart(lookForValidDev));
t.Start((object)ser);//start thread and pass it the port
}
I want the next line of code to wait until all the threads have finished.
I've tried using a t.join in there, but that just processes them linearly.
List<Thread> threads = new List<Thread>();
foreach (cpsComms.cpsSerial ser in availPorts)
{
Thread t = new Thread(new ParameterizedThreadStart(lookForValidDev));
t.Start((object)ser);//start thread and pass it the port
threads.Add(t);
}
foreach(var thread in threads)
{
thread.Join();
}
Edit
I was looking back at this, and I like the following better
availPorts.Select(ser =>
{
Thread thread = new Thread(lookForValidDev);
thread.Start(ser);
return thread;
}).ToList().ForEach(t => t.Join());
Use the AutoResetEvent and ManualResetEvent Classes:
private ManualResetEvent manual = new ManualResetEvent(false);
void Main(string[] args)
{
AutoResetEvent[] autos = new AutoResetEvent[availPorts.Count];
manual.Set();
for (int i = 0; i < availPorts.Count - 1; i++)
{
AutoResetEvent Auto = new AutoResetEvent(false);
autos[i] = Auto;
Thread t = new Thread(() => lookForValidDev(Auto, (object)availPorts[i]));
t.Start();//start thread and pass it the port
}
WaitHandle.WaitAll(autos);
manual.Reset();
}
void lookForValidDev(AutoResetEvent auto, object obj)
{
try
{
manual.WaitOne();
// do something with obj
}
catch (Exception)
{
}
finally
{
auto.Set();
}
}
The simplest and safest way to do this is to use a CountdownEvent. See Albahari.
Store the Thread results in a list after they were spawned and iterate the list - during iteration call join then. You still join linearly, but it should do what you want.
You can use a CountDownLatch:
public class CountDownLatch
{
private int m_remain;
private EventWaitHandle m_event;
public CountDownLatch(int count)
{
Reset(count);
}
public void Reset(int count)
{
if (count < 0)
throw new ArgumentOutOfRangeException();
m_remain = count;
m_event = new ManualResetEvent(false);
if (m_remain == 0)
{
m_event.Set();
}
}
public void Signal()
{
// The last thread to signal also sets the event.
if (Interlocked.Decrement(ref m_remain) == 0)
m_event.Set();
}
public void Wait()
{
m_event.WaitOne();
}
}
Example how to use it:
void StartThreads
{
CountDownLatch latch = new CountDownLatch(availPorts.Count);
foreach (cpsComms.cpsSerial ser in availPorts)
{
Thread t = new Thread(new ParameterizedThreadStart(lookForValidDev));
//start thread and pass it the port and the latch
t.Start((object)new Pair(ser, latch));
}
DoSomeWork();
// wait for all the threads to signal
latch.Wait();
DoSomeMoreWork();
}
// In each thread
void NameOfRunMethod
{
while(running)
{
// do work
}
// Signal that the thread is done running
latch.Signal();
}
I try to synchronize three threads (named "1", "2" and "3") using Semaphore class. They have to print a string into console respecting consequence: 1->2->3. Here is my code:
class MyThread
{
public Thread Thrd;
static Semaphore sem = new Semaphore(1, 1);
static int flag = 1;
public MyThread(string name)
{
Thrd = new Thread(this.Run);
Thrd.Name = name;
Thrd.Start();
}
void Run()
{
sem.WaitOne();
if (Convert.ToInt32(Thrd.Name) == flag)
{
Console.WriteLine("Thread " + Thrd.Name);
flag++;
}
if (flag == 4)
flag = 1;
Thread.Sleep(300);
sem.Release();
}
}
class SemaphoreDemo
{
static void Main()
{
for (int i = 0; i < 10; i++)
{
MyThread mt1 = new MyThread("1");
MyThread mt2 = new MyThread("2");
MyThread mt3 = new MyThread("3");
mt1.Thrd.Join();
mt2.Thrd.Join();
mt3.Thrd.Join();
}
}
}
But sometimes strings from threads #2 and #3 are not seen. Where is my mistake and how can I fix this problem?
Thank you very much!
The problem is that sometimes a thread will acquire the semaphore out of order, and you don't have any retry logic. Take a look at your Run method.
void Run()
{
sem.WaitOne();
if (Convert.ToInt32(Thrd.Name) == flag)
{
Console.WriteLine("Thread " + Thrd.Name);
flag++;
}
if (flag == 4)
flag = 1;
Thread.Sleep(300);
sem.Release();
}
Now, what happens if the thread named "3" acquires the semaphore first? flag is equal to 1, so the conditional code won't be executed. The thread will just sleep for 300 milliseconds and then exit. If you want this to work, you have to make the thread retry:
void Run()
{
bool success = false;
while (!success)
{
sem.WaitOne();
if (Convert.ToInt32(Thrd.Name) == flag)
{
Console.WriteLine("Thread " + Thrd.Name);
flag++;
success = true;
}
sem.Release();
if (!success)
{
// let somebody else try
Thread.Sleep(300);
}
}
}
That will make your sample work as expected.
I suspect that this is just an exercise to see how threading and semaphores work. Note, however, that a Mutex is usually more appropriate than a Semaphore with a maximum count of 1.
Note also that there are other ways to make threads execute in sequence, although truthfully if you're going to sequence your threads then you probably don't need more than one thread. Unless those threads are doing other things and they only need to sequence once--or infrequently.
I want to update a progressbar as each task is completed below.
The method var continuation2 = Task.Factory.ContinueWhenAny(..... doesnt work.
What is the correct way to do this?
C# Code
private void radButtonInsertManyErrors_Click(object sender, EventArgs e)
{
try
{
radProgressBarStatus.Maximum = int.Parse(radTextBoxNumberofErrorsInsert.Text);
radProgressBarStatus.Value1 = 0;
Task<int>[] tasks = new Task<int>[int.Parse(radTextBoxNumberofErrorsInsert.Text)];
for (int i = 0; i < int.Parse(radTextBoxNumberofErrorsInsert.Text); i++)
{
int x = i;
tasks[i] = new Task<int>(() =>
{
//insert the error into table FA_Errors
Accessor.Insert_FAErrors(BLLErrorCodes.BLL_Error_Codes.Error_Log_Event_Login.ToString(),
(int)BLLErrorCodes.BLL_Error_Codes.Error_Log_Event_Login,
"Some Error", "",
MethodBase.GetCurrentMethod().DeclaringType.Namespace.ToString(),
MethodBase.GetCurrentMethod().Name.ToString(),
BLLErrorCategory.BLL_Error_Category.WEB_APP.ToString(),
"pc source", "damo",
sConn.ToString());
return 1;
});
}
var continuation = Task.Factory.ContinueWhenAll(
tasks,
(antecedents) =>
{
RadMessageBox.Show("Finished inserting errors ");
});
var continuation2 = Task.Factory.ContinueWhenAny(
tasks,
(antecedents) =>
{
radProgressBarStatus.Value1++;
});
for (int i = 0; i < int.Parse(radTextBoxNumberofErrorsInsert.Text); i++)
tasks[i].Start();
// Use next line if you want to block the main thread until all the tasks are complete
//continuation.Wait();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}
You can use this function:
public static void TaskProgress(IEnumerable<Task> tasks, Action<int> callback)
{
int count = 0;
foreach (var task in tasks)
task.ContinueWith(t => callback(Interlocked.Increment(ref count)));
}
It will call the callback each time a task completes with the number of currently completed tasks. Note that the callbacks are not synchronized, so it can be called while the previous callback is still running.
Set up a continuation with each of the tasks. Keep a (thread-safe) counter on how many completed and update the UI on completion of each task.
Actually, Task.WhenAll does keep such a counter under the hood. It is just not accessible.
i'm using the BlockingCollection for a Producer Consumer pattern and i got an excecption i think to write a patent on it- only two results in google!
the expection is "CompleteAdding may not be used concurrently with additions to the collection" and it happens when i TryAdd on th BlockingCollection as Follows:
public void EnqueueTask(T item)
{
if (!_cancellationTokenSource.IsCancellationRequested)
{
_workerQueue.Add(item);
}
}
the CompleteAdding is called on the dispose of the Consumer-Producer wrapper class:
public void Dispose()
{
if (!_IsActive)
return;
_IsActive = false;
_cancellationTokenSource.Cancel();
_workerQueue.CompleteAdding();
// Wait for the consumer's thread to finish.
for (int i = 0; i < _workers.Length; ++i)
{
Task t1 = Task.Factory.StartNew(() =>
{
try
{
if (!_workers[i].Join(4000))
LogWriter.Trace("Failed to join thread", "ThreadFailureOnDispose");
}
catch (Exception ex)
{
OnLogged(ex.Message + ex.StackTrace);
}
});
}
// Release any OS resources.
}
Anyone from microsoft got an idea? should i sleep after the cancelation and before calling the CompleteAdding?
Look at this piece of the code:
for (int i = 0; i < _workers.Length; ++i)
{
Task t1 = Task.Factory.StartNew(() =>
{
try
{
if (!_workers[i].Join(4000)) << == Here
LogWriter.Trace("Failed to join thread", "ThreadFailureOnDispose");
}
In _workers[i].Join(4000), the value of i is not what you think it is. Try again with:
for (int i = 0; i < _workers.Length; ++i)
{
int j = i; // copy
Task t1 = Task.Factory.StartNew(() =>
{
try
{
if (!_workers[j].Join(4000)) // j
LogWriter.Trace("Failed to join thread", "ThreadFailureOnDispose");
}
In your version, the variable 'i' is captured and all Tasks use the same var. All but the first few will see i == _workers.Length because they are executed after the for-loop is completed.
It is a classic lambda + captured var problem.