This question already has answers here:
Question about terminating a thread cleanly in .NET
(8 answers)
Closed 6 years ago.
I have an application which will spin up a List of threads then kick them off one after another until a predefined thread limit has been hit, once this happens it will wait until a thread has finished before starting another. I want to implement a way to exit all threads that are currently running if a threshold of consecutive errors is reached. I have got as far as clearing the list of threads to process, so no more will be started but I'm not sure how to go about closing the remaining threads that are running once the thread limit has been reached, I recognise one thread can't talk to another and that's where Im getting hung up..I've provided a cut down version of main method below.
public static void Run(string distinguishedName, string hrExtractFile, string sanctionsFileLocation, string exemptionsFileLocation, string url, string soapAction, int threadLimit)
{
UsersList Employees = new UsersList(distinguishedName, hrExtractFile); //imports users from HR file
int errorcount = 0;
ManualResetEvent resetEventThreadComplete = new ManualResetEvent(false);
ManualResetEvent resetEventNoMoreThreads = new ManualResetEvent(false);
List<Thread> threads = new List<Thread>();
int toProcess = Employees.Count;
for (int i = 0; i < Employees.Count; i++)
{
int current = i;
threads.Add(new Thread(delegate ()
{
User u = Employees[current];
User drUser = new User();
try
{
drUser = fetchUser(u, url, soapAction);
bool userExists = false;
if (drUser != null)
{
userExists = true;
}
//removes a user if they're in the deleted users OU as well as being in system
if (u.IsDeleted)
{
if (userExists == true)
{
Console.WriteLine("Removing " + u.AccountName);
Log.writeToLogs("activitylog.txt", "Removing " + u.AccountName + ":", u, drUser);
DeleteUser(u, url, soapAction);
}
}
errorcount = 0;
}
}
catch (Exception e)
{
if (errorcount <= 5)
{
Log.writeToLogs("syslog.txt", e.ToString());
Log.writeToLogs("activitylog.txt", u.AccountName + " - Failed to true up!");
Console.WriteLine("Failed on " + u.AccountName + ": An error occured, check logs for details");
errorcount++;
}
else
{
lock (_syncObject)
{
//removes threads from list of threads that are pending
if (threads.Count > 0)
{
threads.Clear();
}
}
}
}
resetEventThreadComplete.Set();
if (Interlocked.Decrement(ref toProcess) == 0)
resetEventNoMoreThreads.Set();
}));
}
/*
* Kicks off the first x number of threads (where x = threadLimit) and removes them from list of pending threads
*/
for (int i = 0; i < threadLimit; i++)
{
if (threads.Count < 1)
break;
//runningThreads++;
threads[0].Start();
threads.RemoveAt(0);
}
/*
*Controls the initiation of thread exection; When one thread finishes, another will be started and removed from the list of pending threads
*/
while (threads.Count > 0)
{
resetEventThreadComplete.WaitOne();
resetEventThreadComplete.Reset();
threads[0].Start();
threads.RemoveAt(0);
}
if (toProcess > 0) resetEventNoMoreThreads.WaitOne();
//Log.sendLogs();
}
Assuming you don't mind killing your threads (not usually a great idea), you could use some code like this, just before clearing the list:
if (threads.Count > 0)
{
threads.ForEach(t => t.Abort()); //Kill all threads
threads.Clear();
}
That being said, you should probably not use Thread.Abort(), if you believe this poster:
Did you read the "remark" section on the MSDN? NEVER USE ABORT. IT IS INTENDED AS LAST RESORT. IF YOU USE THREAD.ABORT, SKY MAY FALL DOWN, AND KITTEN WILL BE KILLED
Whatever method you choose, you can put it into that ForEach expression, e.g. if you prefer to use Interrupt:
threads.ForEach(t => t.Interrupt());
Another (far simpler) approach is to modify your threads to check a global flag variable and exit cleanly when it is set. In your main program, when you wish to kill the threads, just set the flag. Be sure to use a volatile variable, or better yet, a WaitHandle.
Related
This question already has an answer here:
Thread alters passed Int, if start() is called separately
(1 answer)
Closed 7 years ago.
I have no idea what is going on in this. I'm trying to test thread safety of a class by spawning 100 threads to access it constantly, but it seems my anonymous method parameters are changing themselves to values they should never be and I'm confused as to why. As I have no idea what's going on, I'll just post all the functions involved in testing. Somehow I'm ending up with "Thread 98" getting the parameter "num = 100"... That shouldn't be possible and I have no idea what kind of hokey pokey is going on that is changing the integer. (in method "ThreadWriting(int num)" you'll see the point where I check for "num" to equal 100, where I put a break point to catch the offending thread in the act, and it breaks every time. Otherwise it throws an "IndexOutofRangeException" on the array "counts". I'm just trying to see if my threads are generally getting equal access to the class they're all trying to use at once.
public delegate void TempDel();
public TempDel InvokeTest;
public void TRTest3(Form1 sender)
{
InvokeTest = new TempDel(UpdateInvoke);
Thread t = new Thread(() => ConsoleUpdateTest(sender));
t.IsBackground = true;
t.Start();
POConsole.Instance.MaxLines = 20;
for(int i = 0; i < 100; i++)
{
Thread t2 = new Thread(() => ThreadWriting(i));
t2.IsBackground = true;
t2.Name = String.Format("Thread {0}", i);
t2.Start();
}
}
public ulong[] counts = new ulong[100];
public void ThreadWriting(int num)
{
if(num == 100)
{
bool stop = true;
}
while (true)
{
POConsole.Instance.WriteLine("Hello from Thread " + num);
counts[num]++;
}
}
public void ConsoleUpdateTest(Form1 sender)
{
while(true)
{
sender.Invoke(InvokeTest);
Thread.Sleep(5);
}
}
public void UpdateInvoke()
{
QuickTestBox.Text = POConsole.Instance.FullFeed;
}
All my threads are named, as you can see, and none of them receives the name "Thread 100" so I have no idea how one of the other threads could get passed a parameter of 100 or the parameter could be corrupted in some way.
Apparently my thread-safety checking isn't thread safe in some way?
This is a simple closure issue, you should not be using the for loop counter as a threading parameter issue, issue happens out here, for loop and thread execution do not run at same speed, so value of i can change for multiple threads:
for(int i = 0; i < 100; i++)
{
Thread t2 = new Thread(() => ThreadWriting(i));
t2.IsBackground = true;
t2.Name = String.Format("Thread {0}", i);
t2.Start();
}
Use following modification, create a local variable from loop counter
for(int i = 0; i < 100; i++)
{
int j = i;
Thread t2 = new Thread(() => ThreadWriting(j));
t2.IsBackground = true;
t2.Name = String.Format("Thread {0}", j);
t2.Start();
}
I'm making service for watch on some controller data and if it changing then I write it to DB. Seems simple. previosly I realized the same with Delphi, but now I am on C# (.Net 4.5). Now service works good with 100 tasks, but eats about 7-8% of CPU time. My Delphi service eats about 0%.
How can I reduce time which service eat from CPU?
P.S.: each task has own nstance of class to connect and insert into DB and work with local copy of data.
int TagCnt = DataCtrl.TagList.Count;
stopExec = false;
if (TagCnt != 0)
{
tasks = new Task[TagCnt];
for (int i = 0; i <= TagCnt - 1; i++)
{
int TempID = i;
tasks[TempID] = Task.Run(async () => // make threads for parallel read-write tasks // async
{
Random rand = new Random();
TimeSpan delay = TimeSpan.FromMilliseconds(rand.Next(1000, 1500))
try
{
while (!stopExec)
{
cToken.ThrowIfCancellationRequested();
//do basic job here
await Task.Delay(delay, cToken);
}//while end
}
catch (...)
{
...
}
}, cToken);
}
Recently I've been facing a similar conundrum and managed to solve the erratic CPU usage by using a set of dedicated long-running tasks to carry out the asynchronous work in my app like so:
Dim NumThreads As Integer = 10
Dim CanTokSrc As New CancellationTokenSource
Dim LongRunningTasks As Task() = New Task(NumThreads) {}
Dim i As Integer
Do Until i = LongRunningTasks.Count
LongRunningTasks(i) = Task.Factory.StartNew(Sub()
Do Until CanTokSrc.IsCancellationRequested
'DO WORK HERE
Loop
End Sub, CanTokSrc.Token, TaskCreationOptions.LongRunning)
i = i + 1
Loop
This image shows the difference it made in CPU usage for the same workload (shown after 9am).
So I think bypassing the thread pool by using dedicated/ long running tasks like above could improve CPU utilization in some cases. It certainly did in mine :-)
I moved to timer instructions because it's a windows service. Every event on timer load is about 7-10% and between is 0%. I tried to apply tasks, ThreadSchedule - they seems more heavy.
private void OnReadTimer(object source, ElapsedEventArgs e) //check states on timer
{
int TagCnt = DataCtrl.TagList.Count;
po.MaxDegreeOfParallelism = DataCtrl.TagList.Count;
// string ss = "tags=" + TagCnt;
//int TempID;
Random rand = new Random();
try
{
if (TagCnt != 0)
{
ParallelLoopResult loopResult = Parallel.For(0, TagCnt - 1, po, (i, loopState) =>
{
po.CancellationToken.ThrowIfCancellationRequested();
int TempID = i;
Thread.Sleep(rand.Next(100, 200));
int ID = 0;
bool State = false;
long WT = 0;
int ParID = 0;
bool Save = false;
ReadStates(TempID, out ID, out State, out WT, out ParID, out Save);
lock (locker)
{
if (Save) WriteState(ID, State, WT, ParID);
}
});
}
}
catch (TaskCanceledException)
{
}
catch (System.NullReferenceException eNullRef)
{
AddLog("Error:" + eNullRef);
}
catch (System.ArgumentOutOfRangeException e0)
{
AddLog("Error:" + e0);
}
catch (Exception e1)
{
//AddLog("Error while processing data: " + e1);
}
}
I moved to basic threads with infinite loops inside. It gets endless threads for my needs. No heavy recreating/restarting and so on. Now it works nice like Delphi service, but more comfortable job with data and DB. I starts threads with this procedure from lambda new thread()=>:
void RWDeviceState(int i)
{
try
{
int TempID = i;
long StartTime;
long NextTime;
long Period = 3000;
int ID = 0;
bool State = false;
long WT = 0;
int ParID = 0;
bool Save = false;
while (ExecutionAllowed)
{
Save = false;
ReadStates(TempID, out ID, out State, out WT, out ParID, out Save);
lock (locker)
{
if (Save) WriteState(ID, State, WT, ParID);
}
StartTime = DateTime.Now.Ticks / TimeSpan.TicksPerMillisecond;
NextTime = StartTime + Period;
while (DateTime.Now.Ticks / TimeSpan.TicksPerMillisecond < NextTime && ExecutionAllowed)
{
Thread.Sleep(40);
}
}
There are two particular techniques that will help reduce CPU usage in long loop waits. One, is to use the threading sleep method. This is good for example in standalone applications, less in windows services.
In a service, for the second, you should be using timers. These fire at regular intervals, so in between the intervals the CPU is not solicited.
I'm working on my university project. One of main requirement is to use multithreading (user can choose threads numbers).
I'm new in C# and based on internet research. I choose ThreadPool.
I spent a lot of time observing how the threads act using parallel watch in VS and i have no idea how this thing works. For example threadNumber = 10 but parallel watch shows only 4 activated threads.
Here is my code:
public void calculateBeta()
{
var finished = new CountdownEvent(1);
for (int i = 0; i < threadNumber; i++)
{
finished.AddCount();
ThreadPool.QueueUserWorkItem(
(state) =>
{
try
{
doSth();
}
finally
{
finished.Signal();
}
});
}
finished.Signal();
finished.Wait();
}
What am I doing wrong? I tried to test this code with many different values of threads number and it didn't work as i looked for.
EDIT:
private void myTask(object index)
{
int z = (int)index;
double[] result = countBeta(createTableB(z), createTableDiagonalA(z));
int counter = 0;
if ((rest != 0) && (z == threadNumber - 1))
{
for (int j = z * numbersInRow; j < (z + 1) * numbersInRow + rest; j++)
{
N[j] = result[counter];
counter++;
}
}
else
{
for (int j = z * numbersInRow; j < (z + 1) * numbersInRow; j++)
{
N[j] = result[counter];
counter++;
}
}
threads[z] = true;
}
public void calculateBeta()
{
N = new double[num];
setThreadNumber(2);
checkThreadNumber();
setNumberInRow();
setRest();
threads = new bool[threadNumber];
for (int i = 0; i < threadNumber; i++)
{
Thread thread = new Thread(this.myTask);
thread.IsBackground = true;
thread.Start(i);
}
while (!checkThreads())
{
}
}
private bool checkThread()
{
bool result = true;
for (int i = 0; i < threads.Length; i++)
{
if (!threads[i])
result = false;
}
return result;
}
static void Main(string[] args)
{
Jacobi jacobi = new Jacobi();
Console.WriteLine("Metoda Jacobiego");
Console.WriteLine("Rozwiazywanie ukladu n-rownan z n-niewiadomymi Ax=b");
jacobi.getNum();
jacobi.getA();
jacobi.getB();
jacobi.calculateBeta();
jacobi.calculateM();
jacobi.calculateX();
jacobi.countNorms();
Console.ReadLine();
}
I need results from calculateBeta to further calculations. Sometimes threads are not finished yet but the program moves forward without data that need to be provided by threads. I'm using bool variable now but this solution is not an elegant way to deal with it(Creating bool table, checking if all thread are fnished) How can i manage with that in a different way?
This is because you're using ThreadPool to manage your threads. It will create a certain number of threads based on many factors. You can tweak some of the settings but by and large when you commit to using ThreadPool to managing your threads you commit to a black box. Check out GetMaxThreads and GetMinThreads and their setter counterparts for some of your options.
Check out this ThreadPool Architecture article on MSDN. It gives good background to the hows and whys of the class. But in the introductory paragraph you will see this sentence, which is key to your conundrum:
The thread pool is primarily used to reduce the number of application
threads and provide management of the worker threads.
If you want to have the kind of control where you launch 10 threads in quick succession you should avoid ThreadPool and just manage the threads yourself. Here is a simple, absolutely minimal example of launching ten threads and also passing different data to each, in this case an index:
void ButtonClickHandlerOrSomeOtherMethod()
{
for (int i=1; i<=10; i++) // using a 1-based index
{
new Thread(ThreadTask).Start(i);
}
}
void ThreadTask(object i)
{
Console.WriteLine("Thread " + i + " ID: " + Thread.CurrentThread.ManagedThreadId);
}
And some sample output:
Thread 1 ID: 19
Thread 2 ID: 34
Thread 3 ID: 26
Thread 4 ID: 5
Thread 5 ID: 36
Thread 6 ID: 18
Thread 7 ID: 9
Thread 8 ID: 38
Thread 9 ID: 39
Thread 10 ID: 40
Follow-up code demonstrating synching with threads and "waiting" until they are all finished:
void ButtonClickHandlerOrSomeOtherMethod()
{
// need a collection of threads to call Join after Start(s)
var threads = new List<Thread>();
// create threads, add to List and start them
for (int i=1; i<=10; i++) {
var thread = new Thread(ThreadTask);
threads.Add(thread);
// a background thread will allow main app to exit even
// if the thread is still running
thread.IsBackground = true;
thread.Start(i);
}
// call Join on each thread which makes this thread wait on
// all 10 other threads
foreach (var thread in threads)
thread.Join();
// this message will not show until all threads are finished
Console.WriteLine("All threads finished.");
}
void ThreadTask(object i)
{
Console.WriteLine("Thread " + i + " ID: " + Thread.CurrentThread.ManagedThreadId);
// introducing some randomness to how long a task "works on something"
Thread.Sleep(100 * new Random().Next(0, 10));
Console.WriteLine("Thread " + i + " finished.");
}
The whole design of the thread pool is that it doesn't have to create a new actual thread every time a new item is queued up. If the pool notices that it has items pending in the queue for an extended period of time it will eventually start spinning up new threads, over time. If you're continually saturating the thread pool with operations, you'll see the number of actual threads rise. It will also only add new threads up to a limit; based on what it feels is going to have the best throughput. For example, it will avoid creating a lot more threads than cores assuming all of the threads are actively running CPU bound work.
The idea of using the thread pool is if you don't care how many actual threads there are, but rather just want to have efficient throughput of the operations that you have, allowing the framework lots of freedom on how to best optimize that work. If you have very specific requirements as to how many threads you have, you'll need to create threads manually rather than using a pool.
// Array of threads launched.
// This array is useful to trace threads status.
Thread[] threads;
private void myTask(object index)
{
Console.Write("myTask {0} started\n", index);
Console.Write("myTask {0} finisced\n", index);
}
public void calculateBeta(UInt16 threadNumber)
{
// Allocate a new array with size of requested number of threads
threads = new Thread[threadNumber];
// For each thread
for (int i = 0; i < threadNumber; i++)
{
// Thread creation
threads[i] = new Thread(this.myTask);
// IsBackground set to true grants that the allication can be "killed" without wait for all threads termination
// This is useful in debug to be sure that an error in task doesn't freeze the app.
// Leave it to false in release
#if DEBUG
threads[i].IsBackground = true;
#endif
// Start the thread
threads[i].Start(i);
}
// Waits until all threads complete.
while (!checkThreads());
}
private bool checkThreads()
{
bool result = true;
for (int i = 0; i < threads.Length; i++)
{
// If the thread wasn't disposed
if (threads[i] != null)
{
// Check if the thead is alive (means is working)
if (threads[i].IsAlive == true)
{
result = false;
}
else // The thread is not working
{
// Dispose the thread
threads[i].Join();
// Set pointer to null to signal that the task was
threads[i] = null;
}
}
}
return result;
}
private void Button_Click(object sender, RoutedEventArgs e)
{
Console.Write("Starting tasks!!\n");
calculateBeta(10);
Console.Write("All tasks finished!!\n");
}
I have a Web Service, to make the load of the database server for a local database, making 100 requests for records.
Since the process is slow, I want to create ten threads, not to use too much memory, making Web Service calls, and when one of the threads, finished, over 100 call records. How do part of the thread?
Example:
Create thread 1
Create thread 2
Create thread 3
Create thread 4
thread 1 complete change Web Service again
Edit
My code not working. Variable sendalways gets the value 10 and not 0,1,2,3,4 and etc.
Int32 page = 0;
do
{
for (int iterator=0; iterator < 10; iterator++)
{
listTask[iterator] = Task.Factory.StartNew(() =>
{
Int32 send = iterator + page * 10;
DoStatus("Page: " + send.ToString());
Processamento(parametros, filial, send);
});
}
Task.WaitAll(listTask);
page++;
}
while (true); // Test only
You're closing over the loop variable. You need to remember that lambdas close over variables not over values. Your tasks will each read the value of iterator at the time that the lambda executes iterator + page * 10. By the time that that happens the main thread has already incremented it to 10.
This is simple enough to resolve. Make a copy of the loop variable inside of your for loop so that the closure closes over that variable, which never changes.
for (int iterator=0; iterator < 10; iterator++)
{
int i = iterator;
listTask[iterator] = Task.Factory.StartNew(() =>
{
Int32 send = i + page * 10;
DoStatus("Page: " + send.ToString());
Processamento(parametros, filial, send);
});
}
If I understand your question, you want to create 10 threads, wait for all, then recreate 10 threads, etc. Each thread load 100 results.
In this answer, results are String but that can be changed.
private void Load()
{
Boolean loading = true;
List<String> listResult = new List<String>();
Int32 boucle = 0;
Task[] listTask = new Task[10];
do
{
// create 10 threads (=1000 results)
for (int iterator=0; iterator < 10; iterator++)
{
// [0-99] [100-199] [200-299] ...
Int32 start = 100 * iterator + 1000 * boucle;
Int32 end = start + 99;
listTask[iterator] = Task<List<String>>.Factory.StartNew(() =>
{
List<String> data = LoadData(start, end);
return data;
});
}
// wait for 10 threads to finish
Task.WaitAll(listTask);
// collapse results
for (int i=0; i < 10; i++)
{
listResult.AddRange((listTask[i] as Task<List<String>>).Result);
}
// check if there is 100 results in last thread
loading = (listTask[9] as Task<List<String>>).Result.Count == 100;
// ready for another iteration (next 1000 results)
boucle++;
}
while (loading);
}
private List<string> LoadData(int p1, int p2)
{
// TODO : load data from p1 to p2
throw new NotImplementedException();
}
I want to achieve the below requirement; please suggest some solution.
string[] filenames = Directory.GetFiles("C:\Temp"); //10 files
for (int i = 0; i < filenames.count; i++)
{
ProcessFile(filenames[i]); //it takes time to execute
}
I wanted to implement multi-threading. e.g There are 10 files. I wanted to process 3 files at a time (configurable, say maxthreadcount). So 3 files will be processed in 3 threads from the for loop and if any thread completes the execution, it should pick the next item from the for loop. Also wanted to ensure all the files are processed before it exits the for loop.
Please suggest best approach.
Try
Parallel.For(0, filenames.Length, i => {
ProcessFile(filenames[i]);
});
MSDN
It's only available since .Net 4. Hope that acceptable.
This will do the job in .net 2.0:
class Program
{
static int workingCounter = 0;
static int workingLimit = 10;
static int processedCounter = 0;
static void Main(string[] args)
{
string[] files = Directory.GetFiles("C:\\Temp");
int checkCount = files.Length;
foreach (string file in files)
{
//wait for free limit...
while (workingCounter >= workingLimit)
{
Thread.Sleep(100);
}
workingCounter += 1;
ParameterizedThreadStart pts = new ParameterizedThreadStart(ProcessFile);
Thread th = new Thread(pts);
th.Start(file);
}
//wait for all threads to complete...
while (processedCounter< checkCount)
{
Thread.Sleep(100);
}
Console.WriteLine("Work completed!");
}
static void ProcessFile(object file)
{
try
{
Console.WriteLine(DateTime.Now.ToString() + " recieved: " + file + " thread count is: " + workingCounter.ToString());
//make some sleep for demo...
Thread.Sleep(2000);
}
catch (Exception ex)
{
//handle your exception...
string exMsg = ex.Message;
}
finally
{
Interlocked.Decrement(ref workingCounter);
Interlocked.Increment(ref processedCounter);
}
}
}
Take a look at the Producer/Consumer Queue example by Joe Albahari. It should provide a good starting point for what you're trying to accomplish.
You could use the ThreadPool.
Example:
ThreadPool.SetMaxThreads(3, 3);
for (int i = 0; i < filenames.count; i++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessFile), filenames[i]);
}
static void ProcessFile(object fileNameObj)
{
var fileName = (string)fileNameObj;
// do your processing here.
}
If you are using the ThreadPool elsewhere in your application then this would not be a good solution since it is shared across your app.
You could also grab a different thread pool implementation, for example SmartThreadPool
Rather than starting a thread for each file name, put the file names into a queue and then start up three threads to process them. Or, since the main thread is now free, start up two threads and let the main thread work on it, too:
Queue<string> MyQueue;
void MyProc()
{
string[] filenames = Directory.GetFiles(...);
MyQueue = new Queue(filenames);
// start two threads
Thread t1 = new Thread((ThreadStart)ProcessQueue);
Thread t2 = new Thread((ThreadStart)ProcessQueue);
t1.Start();
t2.Start();
// main thread processes the queue, too!
ProcessQueue();
// wait for threads to complete
t1.Join();
t2.Join();
}
private object queueLock = new object();
void ProcessQueue()
{
while (true)
{
string s;
lock (queueLock)
{
if (MyQueue.Count == 0)
{
// queue is empty
return;
}
s = MyQueue.Dequeue();
}
ProcessFile(s);
}
}
Another option is to use a semaphore to control how many threads are working:
Semaphore MySem = new Semaphore(3, 3);
void MyProc()
{
string[] filenames = Directory.GetFiles(...);
foreach (string s in filenames)
{
mySem.WaitOne();
ThreadPool.QueueUserWorkItem(ProcessFile, s);
}
// wait for all threads to finish
int count = 0;
while (count < 3)
{
mySem.WaitOne();
++count;
}
}
void ProcessFile(object state)
{
string fname = (string)state;
// do whatever
mySem.Release(); // release so another thread can start
}
The first will perform somewhat better because you don't have the overhead of starting and stopping a thread for each file name processed. The second is much shorter and cleaner, though, and takes full advantage of the thread pool. Likely you won't notice the performance difference.
Can set max threads unsing ParallelOptions
Parallel.For Method (Int32, Int32, ParallelOptions, Action)
ParallelOptions.MaxDegreeOfParallelism
var results = filenames.ToArray().AsParallel().Select(filename=>ProcessFile(filename)).ToArray();
bool ProcessFile(object fileNameObj)
{
var fileName = (string)fileNameObj;
// do your processing here.
return true;
}