I have array of 2863 objects. I want in two "runs" per 1000 objects read array data by 4 threads (running PC # of CPUs).
Currenly my source code is partitioning data to correct number of threads and runs:
Single run size (default) = 1000 elements
Number of runs = 2
Extra thread run size = 866 elements
Starting run [1 / 2]
Thread as readDCMTags(i=0,firstIndex=0, lastIndex=249
Thread as readDCMTags(i=1,firstIndex=250, lastIndex=499
Thread as readDCMTags(i=2,firstIndex=500, lastIndex=749
Thread as readDCMTags(i=3,firstIndex=750, lastIndex=999
Starting run [2 / 2]
Thread as readDCMTags(i=0,firstIndex=1000, lastIndex=1249
Thread as readDCMTags(i=1,firstIndex=1250, lastIndex=1499
Thread as readDCMTags(i=2,firstIndex=1500, lastIndex=1749
Thread as readDCMTags(i=3,firstIndex=1750, lastIndex=1999
Extra Thread as readDCMTags(i=1,firstIndex=2000, lastIndex=2865
However current source code is starting all threads at once, it is not waiting for RUN TO END. When I join threads from current run, the GUI is hanging out. How to solve the issue?
Source Code is:
nrOfChunks = 2866 / 1000;
int leftOverChunk = 2866 % 1000;
for(int z = 0; z < nrOfChunks; z++)
{
addToStatusPanel("\nStarting run [" + (z+1).ToString() + " / " + nrOfChunks.ToString() + "]");
int indexesPerThread = 1000 / 5; #nrOfThreads
int leftOverIndexes = 1000 % 5; #nrOfThreads
threads = new Thread[nrOfThreads];
threadProgress = new int[nrOfThreads];
threadDCMRead = new int[nrOfThreads];
for(int i = 0; i < nrOfThreads; i++)
{
int firstIndex = (i * indexesPerThread+z*Convert.ToInt32(chunkSizeTextBox.Text));
int lastIndex = firstIndex + indexesPerThread - 1;
if(i == (nrOfThreads- 1))
{
if(i == (nrOfThreads - 1))
{
lastIndex += leftOverIndexes;
}
}
addToStatusPanel("readDCMTags(i=" + i.ToString() + ",firstIndex=" + firstIndex.ToString() + ", lastIndex=" + lastIndex.ToString());
threads[i] = new Thread(() => readDCMTags(i.ToString(), firstIndex, lastIndex));
threads[i].Name = i.ToString();
threads[i].Start();
}
if(z == (nrOfChunks - 1))
{
int firstIndex = (nrOfChunks * Convert.ToInt32(chunkSizeTextBox.Text));
int lastIndex = firstIndex + leftOverChunk - 1;
addToStatusPanel("readDCMTags(i=" + z.ToString() + ",firstIndex=" + firstIndex.ToString() + ", lastIndex=" + lastIndex.ToString());
}
}
Adding after a loop for(int i = 0; i < nrOfThreads; i++) a join command for the threads array, before going for next next run loop for(int z = 0; z < nrOfChunks; z++) is hanging the GUI.
By definition, if you wait, you block (the current executing threads blocks waiting for something else).
What you want, instead, is for something to happen when all threads are finished. That, "all threads have finished" is an event. So your best option will be to wait in a background thread and fire the event when all threads complete.
If the GUI is interested on that, then the GUI thread will need to subscribe to that particular event.
Edit: Pseudocode (not tested, just the idea).
waitBg = new Thread(() =>
{
foreach (thread in threads)
thread.WaitFor();
// All threads have finished
if (allThreadFinishedEvent != null)
allThreadFinishedEvent();
}
);
Then, on the handler for the allThreadFinishedEvent you do whatever you want to do (remember to dispatch it to the main thread if you want to change something in the UI, as that'll be executed in the bg thread context).
How about placing this threading logic in a backgroundworker and sending back the result of your backgroundworker to your interface. This way, your interface will not be locked while the program is processing the threads.
you can find the msdn example of initializing and using the backgroundworker here.
I think that should be the correct way forward.
Related
I have a windows form program. In the main form I have a label where I update with progress. I use the following code to perform my operations:
Thread workThread = new Thread(delegate ()
{
for (int i = 1; i < bytes.Length; i++)
{
SliderOverBytesAndDoProcessing();
}
});
workThread.Start();
The function SliderOverBytesAndDoProcessing() performs operations where I read 4 bytes and perform some operations on more bytes. I do this for each 4 bytes of the 200MB file (read 4 bytes process something and continue moving). Everything works fine until I decide to report my progress. Here is how I do the update:
Thread workThread = new Thread(delegate ()
{
for (int i = 1; i < bytes.Length; i++)
{
Lbl.Invoke((Action)delegate { Lbl.Text = "Processing " + (ii / 1048576) + "/" + (bytes.Length / 1048576); }
SliderOverBytesAndDoProcessing();
}
});
workThread.Start();
If I add that one line the program becomes too slow. I mean the program runs in around a few minutes without that line but with that line it takes around 1.5 hours. Also, on some other operations if I do that, I end up in infinite loops. How do I report my progress from the other thread to form thread without facing this problem.
Thanks.
It is slow because you’re dispatching to the UI thread on every byte, even though the progress message only seems to care about megabytes. So you change the label over a million times to the same value.
Consider
var totalMb = bytes.Length / 1024 / 1024;
var lastMb = -1;
for (int i = 1; i < bytes.Length; i++) {
var currentMb = i / 1024 / 1024;
if (currentMb > lastMb) {
Lbl.Invoke(() => { Lbl.Text = $"Processing {currentMb}/{totalMb} MiB" });
lastMb = currentMb;
}
SliderOverBytesAndDoProcessing();
}
I want to create 10 threads with different names. Name should be a combination of name and id.ToString()
If I make a thread using Thread t = new Thread() then I know that I can do t.Name = name + id.ToString();, but if I there is some way to do it with the code example below I would love to know it.
string name = "Thread #";
int id = 0;
for (int i = 0; i < 10; i++)
{
new Thread(() =>
{
for (int j = 0; j <= 100; j++)
{
//Do something
}
}).Start();
}
you could do
var t = new Thread(...);
t.Name = "Thread " + i;
t.Start();
But you should probably use the thread pool instead of creating explicit threads. In this specific case where you have a loop, you should probably use a Parallel.For. In general when you want to run something on a background thread you should use Task.Run, or in some cases Dataflow. When using the task pool you should not name threads, since the threads may be reused for other things. See How can I assign a name to a task in TPL for similar concept for tasks.
Using the thread pool reduces the overhead for running things in the background, and usually also provides interfaces that are easier to use.
I just added Thread t = Thread.CurrentThread; and then I could do t.Name = $"Thread #{id}"; after.
This is what it looks like in its' entirety:
int tId = 0;
for (int i = 0; i < 10; i++)
{
new Thread(() =>
{
Thread t = Thread.CurrentThread;
tId++;
t.Name = $"Thread #{tId}";
Console.WriteLine($"{t.Name}");
}).Start();
So it takes too long, if I try to do this code in serialization. So I want to do it with threading. But I'm running into safety-problems. Here is what starts the threads:
protected void Page_LoadComplete(object sender, EventArgs e)
{
if (!IsPostBack)
{
using (var finished = new CountdownEvent(1))
{
for (int i = 1; i <= Convert.ToInt32(ViewState["Count"]); i++)
{
string k = i.ToString();
ThreadInfo threadInfo = new ThreadInfo();
threadInfo.f = k;
finished.AddCount(); // Indicate that there is another work item.
ThreadPool.QueueUserWorkItem(
(state) =>
{
try
{
Debug.WriteLine("Thread-Start Part " + k.ToString());
CalcThread(threadInfo);
}
finally
{
finished.Signal(); // Signal that the work item is complete.
}
}, null);
Thread.Sleep(300);
}
Debug.WriteLine("Waiting till threads are done! ");
finished.Signal(); // Signal that queueing is complete.
finished.Wait(); // Wait for all work items to complete.
Debug.WriteLine("Threads have completed! ");
}
}
Here is one place that I believe I'm getting some unsafe conditions, and I really don't know if there is a way to solve the problems. I cannot Lock ( ... ) all the code. Because that would defeat the purpose. So a lot of the calculations happen in a sub-class. The problem, I believe is when this sub-class is called, in multiple threads... the answers I'm getting back are often the same answers.
m_calculation.StartQuotePart(runnerweight, waste, machinesize, injectioncycle, volumefactor, MOQ);
m_calculation.partCostNoShiping = m_calculation.partCostNoShiping / partquantity * (1.0 + partcommission);
m_calculation.FedExShippingCost = m_calculation.FedExShippingCost / partquantity * (1.0 + partcommission);
m_calculation.DHLShippingCost = m_calculation.DHLShippingCost / partquantity * (1.0 + partcommission);
m_calculation.UPSShippingCost = m_calculation.UPSShippingCost / partquantity * (1.0 + partcommission);
m_calculation.OceanShippingCost = m_calculation.OceanShippingCost / partquantity * (1.0 + partcommission);
m_calculation.materialcost_out = m_calculation.materialcost_out / partquantity;
m_calculation.ProcessCost_out = m_calculation.ProcessCost_out / partquantity;
m_calculation.DHLshippingcost_out = m_calculation.DHLshippingcost_out / partquantity;
I have an instance for m_calculation... in the same class that is kicking off the threads. Is there a better way to access it, that wouldn't cause the issues. Is there a better way to create the threads that would cause the variable-mis-mash? This is supposed to run during the 'Page-Load-Complete' and then wait for the threads to complete with 'Finish-Wait'
Edit: I'm updating the Load-Complete with this based on jjxtra's post...
int count = Convert.ToInt32(ViewState["Count"]);
var tasks = new Task[count];
for (int i = 1; i <= count; i++)
{
string k = i.ToString();
ThreadInfo threadInfo = new ThreadInfo();
threadInfo.f = k;
tasks[i - 1] = Task.Factory.StartNew(() =>
{
CalcThread(threadInfo);
});
}
Task.WaitAll(tasks);
That's what the state object is for, you can pass an object to the thread, let it modify that object in isolation and then execute some sort of callback at the end of the thread to another method, or you can use the Join method to wait for the work to complete.
Instead of pasing null to QueueUserWorkItem, pass an instance of a class with everything the thread needs to do it's work and report results.
Having said all this, if you switch to using Task everything might be much simpler, but not sure what version of .NET / .NET core you are using...
I'm working on my university project. One of main requirement is to use multithreading (user can choose threads numbers).
I'm new in C# and based on internet research. I choose ThreadPool.
I spent a lot of time observing how the threads act using parallel watch in VS and i have no idea how this thing works. For example threadNumber = 10 but parallel watch shows only 4 activated threads.
Here is my code:
public void calculateBeta()
{
var finished = new CountdownEvent(1);
for (int i = 0; i < threadNumber; i++)
{
finished.AddCount();
ThreadPool.QueueUserWorkItem(
(state) =>
{
try
{
doSth();
}
finally
{
finished.Signal();
}
});
}
finished.Signal();
finished.Wait();
}
What am I doing wrong? I tried to test this code with many different values of threads number and it didn't work as i looked for.
EDIT:
private void myTask(object index)
{
int z = (int)index;
double[] result = countBeta(createTableB(z), createTableDiagonalA(z));
int counter = 0;
if ((rest != 0) && (z == threadNumber - 1))
{
for (int j = z * numbersInRow; j < (z + 1) * numbersInRow + rest; j++)
{
N[j] = result[counter];
counter++;
}
}
else
{
for (int j = z * numbersInRow; j < (z + 1) * numbersInRow; j++)
{
N[j] = result[counter];
counter++;
}
}
threads[z] = true;
}
public void calculateBeta()
{
N = new double[num];
setThreadNumber(2);
checkThreadNumber();
setNumberInRow();
setRest();
threads = new bool[threadNumber];
for (int i = 0; i < threadNumber; i++)
{
Thread thread = new Thread(this.myTask);
thread.IsBackground = true;
thread.Start(i);
}
while (!checkThreads())
{
}
}
private bool checkThread()
{
bool result = true;
for (int i = 0; i < threads.Length; i++)
{
if (!threads[i])
result = false;
}
return result;
}
static void Main(string[] args)
{
Jacobi jacobi = new Jacobi();
Console.WriteLine("Metoda Jacobiego");
Console.WriteLine("Rozwiazywanie ukladu n-rownan z n-niewiadomymi Ax=b");
jacobi.getNum();
jacobi.getA();
jacobi.getB();
jacobi.calculateBeta();
jacobi.calculateM();
jacobi.calculateX();
jacobi.countNorms();
Console.ReadLine();
}
I need results from calculateBeta to further calculations. Sometimes threads are not finished yet but the program moves forward without data that need to be provided by threads. I'm using bool variable now but this solution is not an elegant way to deal with it(Creating bool table, checking if all thread are fnished) How can i manage with that in a different way?
This is because you're using ThreadPool to manage your threads. It will create a certain number of threads based on many factors. You can tweak some of the settings but by and large when you commit to using ThreadPool to managing your threads you commit to a black box. Check out GetMaxThreads and GetMinThreads and their setter counterparts for some of your options.
Check out this ThreadPool Architecture article on MSDN. It gives good background to the hows and whys of the class. But in the introductory paragraph you will see this sentence, which is key to your conundrum:
The thread pool is primarily used to reduce the number of application
threads and provide management of the worker threads.
If you want to have the kind of control where you launch 10 threads in quick succession you should avoid ThreadPool and just manage the threads yourself. Here is a simple, absolutely minimal example of launching ten threads and also passing different data to each, in this case an index:
void ButtonClickHandlerOrSomeOtherMethod()
{
for (int i=1; i<=10; i++) // using a 1-based index
{
new Thread(ThreadTask).Start(i);
}
}
void ThreadTask(object i)
{
Console.WriteLine("Thread " + i + " ID: " + Thread.CurrentThread.ManagedThreadId);
}
And some sample output:
Thread 1 ID: 19
Thread 2 ID: 34
Thread 3 ID: 26
Thread 4 ID: 5
Thread 5 ID: 36
Thread 6 ID: 18
Thread 7 ID: 9
Thread 8 ID: 38
Thread 9 ID: 39
Thread 10 ID: 40
Follow-up code demonstrating synching with threads and "waiting" until they are all finished:
void ButtonClickHandlerOrSomeOtherMethod()
{
// need a collection of threads to call Join after Start(s)
var threads = new List<Thread>();
// create threads, add to List and start them
for (int i=1; i<=10; i++) {
var thread = new Thread(ThreadTask);
threads.Add(thread);
// a background thread will allow main app to exit even
// if the thread is still running
thread.IsBackground = true;
thread.Start(i);
}
// call Join on each thread which makes this thread wait on
// all 10 other threads
foreach (var thread in threads)
thread.Join();
// this message will not show until all threads are finished
Console.WriteLine("All threads finished.");
}
void ThreadTask(object i)
{
Console.WriteLine("Thread " + i + " ID: " + Thread.CurrentThread.ManagedThreadId);
// introducing some randomness to how long a task "works on something"
Thread.Sleep(100 * new Random().Next(0, 10));
Console.WriteLine("Thread " + i + " finished.");
}
The whole design of the thread pool is that it doesn't have to create a new actual thread every time a new item is queued up. If the pool notices that it has items pending in the queue for an extended period of time it will eventually start spinning up new threads, over time. If you're continually saturating the thread pool with operations, you'll see the number of actual threads rise. It will also only add new threads up to a limit; based on what it feels is going to have the best throughput. For example, it will avoid creating a lot more threads than cores assuming all of the threads are actively running CPU bound work.
The idea of using the thread pool is if you don't care how many actual threads there are, but rather just want to have efficient throughput of the operations that you have, allowing the framework lots of freedom on how to best optimize that work. If you have very specific requirements as to how many threads you have, you'll need to create threads manually rather than using a pool.
// Array of threads launched.
// This array is useful to trace threads status.
Thread[] threads;
private void myTask(object index)
{
Console.Write("myTask {0} started\n", index);
Console.Write("myTask {0} finisced\n", index);
}
public void calculateBeta(UInt16 threadNumber)
{
// Allocate a new array with size of requested number of threads
threads = new Thread[threadNumber];
// For each thread
for (int i = 0; i < threadNumber; i++)
{
// Thread creation
threads[i] = new Thread(this.myTask);
// IsBackground set to true grants that the allication can be "killed" without wait for all threads termination
// This is useful in debug to be sure that an error in task doesn't freeze the app.
// Leave it to false in release
#if DEBUG
threads[i].IsBackground = true;
#endif
// Start the thread
threads[i].Start(i);
}
// Waits until all threads complete.
while (!checkThreads());
}
private bool checkThreads()
{
bool result = true;
for (int i = 0; i < threads.Length; i++)
{
// If the thread wasn't disposed
if (threads[i] != null)
{
// Check if the thead is alive (means is working)
if (threads[i].IsAlive == true)
{
result = false;
}
else // The thread is not working
{
// Dispose the thread
threads[i].Join();
// Set pointer to null to signal that the task was
threads[i] = null;
}
}
}
return result;
}
private void Button_Click(object sender, RoutedEventArgs e)
{
Console.Write("Starting tasks!!\n");
calculateBeta(10);
Console.Write("All tasks finished!!\n");
}
I have a Web Service, to make the load of the database server for a local database, making 100 requests for records.
Since the process is slow, I want to create ten threads, not to use too much memory, making Web Service calls, and when one of the threads, finished, over 100 call records. How do part of the thread?
Example:
Create thread 1
Create thread 2
Create thread 3
Create thread 4
thread 1 complete change Web Service again
Edit
My code not working. Variable sendalways gets the value 10 and not 0,1,2,3,4 and etc.
Int32 page = 0;
do
{
for (int iterator=0; iterator < 10; iterator++)
{
listTask[iterator] = Task.Factory.StartNew(() =>
{
Int32 send = iterator + page * 10;
DoStatus("Page: " + send.ToString());
Processamento(parametros, filial, send);
});
}
Task.WaitAll(listTask);
page++;
}
while (true); // Test only
You're closing over the loop variable. You need to remember that lambdas close over variables not over values. Your tasks will each read the value of iterator at the time that the lambda executes iterator + page * 10. By the time that that happens the main thread has already incremented it to 10.
This is simple enough to resolve. Make a copy of the loop variable inside of your for loop so that the closure closes over that variable, which never changes.
for (int iterator=0; iterator < 10; iterator++)
{
int i = iterator;
listTask[iterator] = Task.Factory.StartNew(() =>
{
Int32 send = i + page * 10;
DoStatus("Page: " + send.ToString());
Processamento(parametros, filial, send);
});
}
If I understand your question, you want to create 10 threads, wait for all, then recreate 10 threads, etc. Each thread load 100 results.
In this answer, results are String but that can be changed.
private void Load()
{
Boolean loading = true;
List<String> listResult = new List<String>();
Int32 boucle = 0;
Task[] listTask = new Task[10];
do
{
// create 10 threads (=1000 results)
for (int iterator=0; iterator < 10; iterator++)
{
// [0-99] [100-199] [200-299] ...
Int32 start = 100 * iterator + 1000 * boucle;
Int32 end = start + 99;
listTask[iterator] = Task<List<String>>.Factory.StartNew(() =>
{
List<String> data = LoadData(start, end);
return data;
});
}
// wait for 10 threads to finish
Task.WaitAll(listTask);
// collapse results
for (int i=0; i < 10; i++)
{
listResult.AddRange((listTask[i] as Task<List<String>>).Result);
}
// check if there is 100 results in last thread
loading = (listTask[9] as Task<List<String>>).Result.Count == 100;
// ready for another iteration (next 1000 results)
boucle++;
}
while (loading);
}
private List<string> LoadData(int p1, int p2)
{
// TODO : load data from p1 to p2
throw new NotImplementedException();
}