I am stress testing my application and have written a simple test to spawn hundreds of threads that call a method.
The code below works fine for 1000 threads and 100 ms delay.
In the code below when the number of threads is 2000 and the delay is 100 i get the error
Cannot load the "shell32.dll" DLL into memory in the catch statement for radButtonEmptyThread_Click
How do i fix this?
The value written "Debug.Print(count.ToString());" is always 1000 - why?
C# Code
private void radButtonEmptyThread_Click(object sender, EventArgs e)
{
try
{
for (int i = 0; i < int.Parse(radTextBoxWaitThreads.Text); i++)
{
Thread Trd = new Thread(() => EmptyThreadRequest(int.Parse(radTextBoxFloodDelay.Text), i));
Trd.IsBackground = true;
Trd.Start();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}
private void EmptyThreadRequest(int delay, int count)
{
try
{
System.Threading.Thread.Sleep(delay);
Debug.Print(count.ToString());
}
catch (Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}
}
Stop creating so many threads. That's very resource-intensive. Instead, use Tasks.
i is a captured variable, which means the threads all access the original variable, not a copy. If you create a copy of the variable inside the loop, it will work as expected.
To deal with the captured variable issue, inside the loop do this:
int x = i;
Thread Trd = new Thread(() => EmptyThreadRequest(int.Parse(radTextBoxFloodDelay.Text), x));
And of course, consider using Tasks.
2000 is a functional limit enforced by Windows. I think it might have something to do with minimum stack allocated to each thread, but I would not bet my life on it. Tasks are very light weight threads, prefer them over threads when possible.
C# Code
private void radButtonCallEmptyTasks_Click(object sender, EventArgs e)
{
try
{
for (int i = 0; i < int.Parse(radTextBoxWaitThreads.Text); i++)
{
// Create a task and supply a user delegate by using a lambda expression.
var taskA = new Task(() => EmptyTaskRequest(int.Parse(radTextBoxFloodDelay.Text), i));
// Start the task.
taskA.Start();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}
private void EmptyTaskRequest(int delay, int count)
{
try
{
System.Threading.Thread.Sleep(delay);
Debug.Print(count.ToString());
}
catch (Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}
}
Related
Im looking for a simple way, to repeat a method, when an element(selector) for example a button on a webpage cant be found.
My idea or my plan is, that:
If the selector can be found, the method is done
If the selector cant be found, it should repeat the "FindElement" method up to maximal 3 times
If the selector cant be found after the third try, it should give me an output message of the "NoSuchElementException e"
I tried different loops and i always end up having endless loops like in the code below.
public static void FindElement(IWebDriver webDriver, string selector)
{
int maxTries = 0;
try
{
webDriver.FindElement(By.XPath(selector));
Console.WriteLine("Element found.");
}
catch (NoSuchElementException e)
{
if (maxTries !> 3 )
{
Console.WriteLine("Element, not found. Retrying.");
maxTries++;
FindElement(webDriver, selector);
}
else
{
Console.WriteLine(e);
}
}
}
What a coincidence: Just a few days ago I wrote a short helper method to separate "retry logic" from "business logic".
private T Retry<T>(Func<T> action, int maxRetryCount, int waitMilliseconds,
Func<Exception, bool> retryCondition)
{
var retryCount = 0;
while (true)
{
try
{
// If the action was successful (no exception thrown),
// we can leave the method.
return action();
}
catch (Exception ex) when (retryCount < maxRetryCount && retryCondition(ex))
{
retryCount += 1;
Thread.Sleep(waitMilliseconds);
}
}
}
Which would be called as follows in your case:
var element = Retry(() => webDriver.FindElement(By.XPath(selector)),
2, 0,
ex => ex is NoSuchElementException);
Whenever the maximum number of retries is reached, the exception is not caught any more (the when condition fails) and can be caught by your regular exception handling logic instead.
Since you are trying to automate a web browser, you might consider passing something else than 0 as waitMilliseconds, to give the browser time to render the elements that are still missing.
Why don't you put the maxTry outside of your function?
int maxTry=3;
string errorMessage="";
for(int i=0;i<maxTry;i++)
{
try
{
FindElement(webDriver,selector)
errorMessage="";
break;
}
catch (NoSuchElementException e)
{
errorMessage=e;
}
}
if(errorMessage!="")
{
Console.WriteLine(e);
}
else
{
Console.WriteLine("Element found.");
}
For recursion, you should pass the amount of remaining iterations to the function, and decrement it for each iteration.
public static void FindElement(IWebDriver webDriver, string selector, int iterations = 3)
{
iterations--;
if (iterations < 0)
{
Console.WriteLine("Max iterations passed, exiting")
return;
}
try
{
webDriver.FindElement(By.XPath(selector));
Console.WriteLine("Element found.");
}
catch (NoSuchElementException e)
{
Console.WriteLine("Element not found. Retrying.");
FindElement(webDriver, selector, iterations);
}
}
I'm trying to build a Windows Forms tool that runs queries asynchronously.
The app has a datagridview with 30 possible queries to run. The user checks the queries he wants to execute, say 10 queries, and hits a button.
The app has a variable called maxthreads = 3 (for the sake of discussion) that indicates how many threads can be used to async run the queries. The queries run on a production environment and we don't want to overload the system with too many threads running in the same time. Each query runs for an average of 30 sec. (some 5 min., others 2 sec.)
In the datagridview there is an image column containing an icon that depicts the status of each query (0- Available to be run, 1-Selected for running, 2- Running, 3- Successfully completed, -1 Error)
I need to be able to communicate with the UI every time a query starts and finishes. Once a query finishes, the results are being displayed in a datagridview contained in a Tabcontrol (one tab per query)
The approach: I was thinking to create a number of maxthread backgroundworkers and let them run the queries. As a backgroundworker finishes it communicates to the UI and is assigned to a new query and so on until all queries have been run.
I tried using an assignmentWorker that would dispatch the work to the background workers but don't know how to wait for all threads to finish. Once a bgw finishes it reports progress on the RunWorkerCompleted event to the assignmentWorker, but that one has already finished.
In the UI thread I call the assignment worker with all the queries that need to be run:
private void btnRunQueries_Click(object sender, EventArgs e)
{
if (AnyQueriesSelected())
{
tcResult.TabPages.Clear();
foreach (DataGridViewRow dgr in dgvQueries.Rows)
{
if (Convert.ToBoolean(dgr.Cells["chk"].Value))
{
Query q = new Query(dgr.Cells["ID"].Value.ToString(),
dgr.Cells["Name"].Value.ToString(),
dgr.Cells["FileName"].Value.ToString(),
dgr.Cells["ShortDescription"].Value.ToString(),
dgr.Cells["LongDescription"].Value.ToString(),
dgr.Cells["Level"].Value.ToString(),
dgr.Cells["Task"].Value.ToString(),
dgr.Cells["Importance"].Value.ToString(),
dgr.Cells["SkillSet"].Value.ToString(),
false,
new Dictionary<string, string>()
{ { "#ClntNb#", txtClntNum.Text }, { "#Staff#", "100300" } });
qryList.Add(q);
}
}
assignmentWorker.RunWorkerAsync(qryList);
}
else
{
MessageBox.Show("Please select at least one query.",
"Warning",
MessageBoxButtons.OK,
MessageBoxIcon.Information);
}
}
Here is the AssignmentWorker:
private void assignmentWorker_DoWork(object sender, DoWorkEventArgs e)
{
foreach (Query q in (List<Query>)e.Argument)
{
while (!q.Processed)
{
for (int threadNum = 0; threadNum < maxThreads; threadNum++)
{
if (!threadArray[threadNum].IsBusy)
{
threadArray[threadNum].RunWorkerAsync(q);
q.Processed = true;
assignmentWorker.ReportProgress(1, q);
break;
}
}
//If all threads are being used, sleep awhile before checking again
if (!q.Processed)
{
Thread.Sleep(500);
}
}
}
}
All bgw run the same event:
private void backgroundWorkerFiles_DoWork(object sender, DoWorkEventArgs e)
{
try
{
Query qry = (Query)e.Argument;
DataTable dtNew = DataAccess.RunQuery(qry).dtResult;
if (dsQryResults.Tables.Contains(dtNew.TableName))
{
dsQryResults.Tables.Remove(dtNew.TableName);
}
dsQryResults.Tables.Add(dtNew);
e.Result = qry;
}
catch (Exception ex)
{
}
}
Once the Query has returned and the DataTable has been added to the dataset:
private void backgroundWorkerFiles_RunWorkerCompleted(object sender,
RunWorkerCompletedEventArgs e)
{
try
{
if (e.Error != null)
{
assignmentWorker.ReportProgress(-1, e.Result);
}
else
{
assignmentWorker.ReportProgress(2, e.Result);
}
}
catch (Exception ex)
{
int o = 0;
}
}
The problem I have is that the assignment worker finishes before the bgw finish and the call to assignmentWorker.ReportProgress go to hell (excuse my French).
How can I wait for all the launched bgw to finish before finishing the assignment worker?
Thank you!
As noted in the comment above, you have overcomplicated your design. If you have a specific maximum number of tasks (queries) that should be executing concurrently, you can and should simply create that number of workers, and have them consume tasks from your queue (or list) of tasks until that queue is empty.
Lacking a good Minimal, Complete, and Verifiable code example that concisely and clearly illustrates your specific scenario, it's not feasible to provide code that would directly address your question. But, here's an example using a List<T> as your original code does, which will work as I describe above:
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace TestSO42101517WaitAsyncTasks
{
class Program
{
static void Main(string[] args)
{
Random random = new Random();
int maxTasks = 30,
maxActive = 3,
maxDelayMs = 1000,
currentDelay = -1;
List<TimeSpan> taskDelays = new List<TimeSpan>(maxTasks);
for (int i = 0; i < maxTasks; i++)
{
taskDelays.Add(TimeSpan.FromMilliseconds(random.Next(maxDelayMs)));
}
Task[] tasks = new Task[maxActive];
object o = new object();
for (int i = 0; i < maxActive; i++)
{
int workerIndex = i;
tasks[i] = Task.Run(() =>
{
DelayConsumer(ref currentDelay, taskDelays, o, workerIndex);
});
}
Console.WriteLine("Waiting for consumer tasks");
Task.WaitAll(tasks);
Console.WriteLine("All consumer tasks completed");
}
private static void DelayConsumer(ref int currentDelay, List<TimeSpan> taskDelays, object o, int workerIndex)
{
Console.WriteLine($"worker #{workerIndex} starting");
while (true)
{
TimeSpan delay;
int delayIndex;
lock (o)
{
delayIndex = ++currentDelay;
if (delayIndex < taskDelays.Count)
{
delay = taskDelays[delayIndex];
}
else
{
Console.WriteLine($"worker #{workerIndex} exiting");
return;
}
}
Console.WriteLine($"worker #{workerIndex} sleeping for {delay.TotalMilliseconds} ms, task #{delayIndex}");
System.Threading.Thread.Sleep(delay);
}
}
}
}
In your case, each worker would report progress to some global state. You don't show the ReportProgress handler for your "assignment" worker, so I can't say specifically what this would look like. But presumably it would involve passing either -1 or 2 to some method that knows what to do with those values (i.e. what would otherwise have been your ReportProgress handler).
Note that the code can simplified somewhat, particularly where the individual tasks are consumed, if you use an actual queue data structure for the tasks. That approach would look something like this:
using System;
using System.Collections.Concurrent;
using System.Threading.Tasks;
namespace TestSO42101517WaitAsyncTasks
{
class Program
{
static void Main(string[] args)
{
Random random = new Random();
int maxTasks = 30,
maxActive = 3,
maxDelayMs = 1000,
currentDelay = -1;
ConcurrentQueue<TimeSpan> taskDelays = new ConcurrentQueue<TimeSpan>();
for (int i = 0; i < maxTasks; i++)
{
taskDelays.Enqueue(TimeSpan.FromMilliseconds(random.Next(maxDelayMs)));
}
Task[] tasks = new Task[maxActive];
for (int i = 0; i < maxActive; i++)
{
int workerIndex = i;
tasks[i] = Task.Run(() =>
{
DelayConsumer(ref currentDelay, taskDelays, workerIndex);
});
}
Console.WriteLine("Waiting for consumer tasks");
Task.WaitAll(tasks);
Console.WriteLine("All consumer tasks completed");
}
private static void DelayConsumer(ref int currentDelayIndex, ConcurrentQueue<TimeSpan> taskDelays, int workerIndex)
{
Console.WriteLine($"worker #{workerIndex} starting");
while (true)
{
TimeSpan delay;
if (!taskDelays.TryDequeue(out delay))
{
Console.WriteLine($"worker #{workerIndex} exiting");
return;
}
int delayIndex = System.Threading.Interlocked.Increment(ref currentDelayIndex);
Console.WriteLine($"worker #{workerIndex} sleeping for {delay.TotalMilliseconds} ms, task #{delayIndex}");
System.Threading.Thread.Sleep(delay);
}
}
}
}
I want to update a progressbar as each task is completed below.
The method var continuation2 = Task.Factory.ContinueWhenAny(..... doesnt work.
What is the correct way to do this?
C# Code
private void radButtonInsertManyErrors_Click(object sender, EventArgs e)
{
try
{
radProgressBarStatus.Maximum = int.Parse(radTextBoxNumberofErrorsInsert.Text);
radProgressBarStatus.Value1 = 0;
Task<int>[] tasks = new Task<int>[int.Parse(radTextBoxNumberofErrorsInsert.Text)];
for (int i = 0; i < int.Parse(radTextBoxNumberofErrorsInsert.Text); i++)
{
int x = i;
tasks[i] = new Task<int>(() =>
{
//insert the error into table FA_Errors
Accessor.Insert_FAErrors(BLLErrorCodes.BLL_Error_Codes.Error_Log_Event_Login.ToString(),
(int)BLLErrorCodes.BLL_Error_Codes.Error_Log_Event_Login,
"Some Error", "",
MethodBase.GetCurrentMethod().DeclaringType.Namespace.ToString(),
MethodBase.GetCurrentMethod().Name.ToString(),
BLLErrorCategory.BLL_Error_Category.WEB_APP.ToString(),
"pc source", "damo",
sConn.ToString());
return 1;
});
}
var continuation = Task.Factory.ContinueWhenAll(
tasks,
(antecedents) =>
{
RadMessageBox.Show("Finished inserting errors ");
});
var continuation2 = Task.Factory.ContinueWhenAny(
tasks,
(antecedents) =>
{
radProgressBarStatus.Value1++;
});
for (int i = 0; i < int.Parse(radTextBoxNumberofErrorsInsert.Text); i++)
tasks[i].Start();
// Use next line if you want to block the main thread until all the tasks are complete
//continuation.Wait();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}
You can use this function:
public static void TaskProgress(IEnumerable<Task> tasks, Action<int> callback)
{
int count = 0;
foreach (var task in tasks)
task.ContinueWith(t => callback(Interlocked.Increment(ref count)));
}
It will call the callback each time a task completes with the number of currently completed tasks. Note that the callbacks are not synchronized, so it can be called while the previous callback is still running.
Set up a continuation with each of the tasks. Keep a (thread-safe) counter on how many completed and update the UI on completion of each task.
Actually, Task.WhenAll does keep such a counter under the hood. It is just not accessible.
In my multi threaded web app I invoke in the ThreadPool SomeMethod which can throw an exception. Suppose I want to make a few attempts if it causes an exception at first call. I decide to use System.Timers.Timer inside my action for attempts. Can I use the code below? Is it safely?
static void Caller()
{
ThreadPool.QueueUserWorkItem(action =>
{
try
{
SomeMethod();
Console.WriteLine("Done.");
}
catch
{
var t = new System.Timers.Timer(1000);
t.Start();
var count = 0;
t.Elapsed += new System.Timers.ElapsedEventHandler((o, a) =>
{
var timer = o as System.Timers.Timer;
count++;
var done = false;
Exception exception = null;
try
{
Console.WriteLine(count);
SomeMethod();
done = true;
}
catch (Exception ex)
{
exception = ex;
}
if (done || count == 10)
{
Console.WriteLine(String.Format("Stopped. done: {0}, count: {1}", done, count));
t.Stop();
if (!done) throw exception;
}
});
}
});
Thread.Sleep(100000);
}
static void SomeMethod()
{
var x = 1 / new Random().Next(0, 2);
}
You should Dispose each Timer after use, that's for sure. But, probably you could do something even simpler:
static void Main()
{
ThreadPool.QueueUserWorkItem(action =>
{
while (TrySomeMethod() == false)
Thread.Sleep(1000);
});
// wait here
Console.Read();
}
static bool TrySomeMethod()
{
try
{
SomeMethod();
return true;
}
catch
{
return false;
}
}
I do not think that using a timer in a thread pool thread is a safe approach. I may be wrong, but the timer will raise its elapsed event when the thread method has already been finished to execute. In this case, the exception will be thrown. Also, I do not see that you are not disposing the timer which leads to resource leaks. If you explain why you need the timer, I will try to find a safe solution...
I don't see the point of using a Timer within a ThreadPool queue, because the ThreadPool would spawn a new thread, and the Timer would spawn a new thread as well.
I would just have a loop within that delegate, because it would not block the main thread either way. Groo showed a good example of that.
The following code was a proof of concept for a message batching routine. Do I avoid goto like the plague and rewrite this code? Or do you think the goto is an expressive way to get this done?
If you'd rewrite please post some code...
var queue = new Queue<TraceItem>(this.batch);
while (this.connected)
{
byte[] buffer = null;
try
{
socket.Recv(out buffer);
}
catch
{
// ignore the exception we get when the socket is shut down from another thread
// the connected flag will be set to false and we'll break the loop
}
HaveAnotherMessage:
if (buffer != null)
{
try
{
var item = TraceItemSerializer.FromBytes(buffer);
if (item != null)
{
queue.Enqueue(item);
buffer = null;
if (queue.Count < this.batch && socket.Recv(out buffer, ZMQ.NOBLOCK))
{
goto HaveAnotherMessage;
}
}
}
catch (Exception ex)
{
this.ReceiverPerformanceCounter.IncrementDiagnosticExceptions();
this.tracer.TraceException(TraceEventType.Error, 0, ex);
}
}
// queue processing code
}
Pretty much sums up my thoughts on "goto."
Goto is bad programming practice for many reasons. Chief among them is that there is almost never a reason for it. Someone posted a do..while loop, use that. Use a boolean to check if you should continue. Use a while loop. Goto's are for interpreted languages and a call back to assembler days (JMP anyone?). You're using a high level language for a reason. So that you and everyone else doesn't look at your code and get lost.
To keep this answer somewhat current I'd like to point out that a combination of goto and bracing errors caused a major SSL bug in iOS and OS X.
Replace the goto with a do-while, or simply a while loop if you don't want the "always run once" functionality you have right now.
var queue = new Queue<TraceItem>(this.batch);
while (this.connected)
{
byte[] buffer = null;
try
{
socket.Recv(out buffer);
}
catch
{
// ignore the exception we get when the socket is shut down from another thread
// the connected flag will be set to false and we'll break the loop
}
do {
if (buffer != null)
{
try
{
var item = TraceItemSerializer.FromBytes(buffer);
if (item != null)
{
queue.Enqueue(item);
buffer = null;
}
}
catch (Exception ex)
{
this.ReceiverPerformanceCounter.IncrementDiagnosticExceptions();
this.tracer.TraceException(TraceEventType.Error, 0, ex);
}
}
} while(queue.Count < this.batch && socket.Recv(out buffer, ZMQ.NOBLOCK))
// queue processing code
}
It's so amazingly easy to rid yourself of GOTO in this situation it makes me cry:
var queue = new Queue<TraceItem>(this.batch);
while (this.connected)
{
byte[] buffer = null;
try
{
socket.Recv(out buffer);
}
catch
{
// ignore the exception we get when the socket is shut down from another thread
// the connected flag will be set to false and we'll break the loop
}
bool hasAnotherMessage = true
while(hasAnotherMessage)
{
hasAnotherMessage = false;
if (buffer != null)
{
try
{
var item = TraceItemSerializer.FromBytes(buffer);
if (item != null)
{
queue.Enqueue(item);
buffer = null;
if (queue.Count < this.batch && socket.Recv(out buffer, ZMQ.NOBLOCK))
{
hasAnotherMessage = true;
}
}
}
catch (Exception ex)
{
this.ReceiverPerformanceCounter.IncrementDiagnosticExceptions();
this.tracer.TraceException(TraceEventType.Error, 0, ex);
}
}
}
// queue processing code
}
I guess the goto is SLIGHTLY more readable intuitively... But if you WANTED to avoid it I think all you'd have to do is throw the code in a while(true) loop, and then have a break statement at the end of the loop for a normal iteration. And the goto could be replaced with a continue statement.
Eventually you just learn to read and write loops and other control flow structures instead of using goto statements, at least in my experience.
Kind of related to Josh K post but I'm writing it here since comments doesn't allow code.
I can think of a good reason: While traversing some n-dimensional construct to find something. Example for n=3 //...
for (int i = 0; i < X; i++)
for (int j = 0; j < Y; j++)
for (int k = 0; k < Z; k++)
if ( array[i][j][k] == someValue )
{
//DO STUFF
goto ENDFOR; //Already found my value, let's get out
}
ENDFOR: ;
//MORE CODE HERE...
I know you can use "n" whiles and booleans to see if you should continue.. or you can create a function that maps that n-dimensional array to just one dimension and just use one while but i believe that the nested for its far more readable.
By the way I'm not saying we should all use gotos but in this specific situation i would do it the way i just mentioned.
You could refactor is to something like this.
while (queue.Count < this.batch && buffer != null)
{
try
{
var item = TraceItemSerializer.FromBytes(buffer);
buffer = null;
if (item != null)
{
queue.Enqueue(item);
socket.Recv(out buffer, ZMQ.NOBLOCK)
}
}
catch (Exception ex)
{
this.ReceiverPerformanceCounter.IncrementDiagnosticExceptions();
this.tracer.TraceException(TraceEventType.Error, 0, ex);
}
}
Umm, I'm not really sure you want to goto out of a try block. I'm pretty sure that is not a safe thing to do, though I'm not 100% sure on that. That just doesn't look very safe...
Wrap the "HaveAnotherMessage" into a method that takes in the buffer and may call itself recursively. That would seem to be the easiest way to fix this.
I would avoid goto in this case, and refactor it. The method reads too long in my opinion.
I think your method is too big. It mixes different levels of abstraction, like error processing, message retrieval and message processing.
If you refactor it in different methods, the goto naturally goes away (note: I assume your main method is called Process):
...
private byte[] buffer;
private Queue<TraceItem> queue;
public void Process() {
queue = new Queue<TraceItem>(batch);
while (connected) {
ReceiveMessage();
TryProcessMessage();
}
}
private void ReceiveMessage() {
try {
socket.Recv(out buffer);
}
catch {
// ignore the exception we get when the socket is shut down from another thread
// the connected flag will be set to false and we'll break the processing
}
}
private void TryProcessMessage() {
try {
ProcessMessage();
}
catch (Exception ex) {
ProcessError(ex);
}
}
private void ProcessMessage() {
if (buffer == null) return;
var item = TraceItemSerializer.FromBytes(buffer);
if (item == null) return;
queue.Enqueue(item);
if (HasMoreData()) {
TryProcessMessage();
}
}
private bool HasMoreData() {
return queue.Count < batch && socket.Recv(out buffer, ZMQ.NOBLOCK);
}
private void ProcessError(Exception ex) {
ReceiverPerformanceCounter.IncrementDiagnosticExceptions();
tracer.TraceException(TraceEventType.Error, 0, ex);
}
...