I'm trying to test the C# parallel methods and this is my test program:
class Program
{
static int counter;
static void Main(string[] args)
{
counter = 0;
Parallel.Invoke(
() => func(1),
() => func(2),
() => func(3)
);
Console.Read();
}
static void func(int num)
{
for (int i = 0; i < 5;i++ )
{
Console.WriteLine(string.Format("This is function #{0} loop. counter - {1}", num, counter));
counter++;
}
}
}
What I tried to do is to have 1 static shared variable and each function instance will increase it by 1.
I expected that counter will be printed in order (1,2,3,...)
But the output is surprising:
This is function #1 loop. counter - 0
This is function #1 loop. counter - 1
This is function #1 loop. counter - 2
This is function #1 loop. counter - 3
This is function #1 loop. counter - 4
This is function #3 loop. counter - 5
This is function #2 loop. counter - 1
This is function #3 loop. counter - 6
This is function #3 loop. counter - 8
This is function #3 loop. counter - 9
This is function #3 loop. counter - 10
This is function #2 loop. counter - 7
This is function #2 loop. counter - 12
This is function #2 loop. counter - 13
This is function #2 loop. counter - 14
Can anyone explain to me why this is happening?
The problem is that your code is not thread-safe. For example, what can happen is this:
function #2 gets the value of counter to use it in Console.WriteLine()
function #1 gets the value of counter, calls Console.WriteLine(), increments counter
function #1 gets the value of counter, calls Console.WriteLine(), increments counter
function #2 finally calls Console.WriteLine() with the old value
Also, ++ by itself is not thread-safe, so the final value may not be 15.
To fix both of these issues, you can use Interlocked.Increment():
for (int i = 0; i < 5; i++)
{
int incrementedCounter = Interlocked.Increment(ref counter);
Console.WriteLine("This is function #{0} loop. counter - {1}", num, incrementedCounter);
}
This way, you will get the number after increment, not before, as in your original code. Also, this code still won't print the numbers in the correct order, but you can be sure that each number will be printed exactly once.
If you do want to have the numbers in the correct order, you will need to use lock:
private static readonly object lockObject = new object();
…
for (int i = 0; i < 5; i++)
{
lock (lockObject)
{
Console.WriteLine("This is function #{0} loop. counter - {1}", num, counter);
counter++;
}
}
Of course, if you do this, you won't actually get any parallelism, but I assume this is not your real code.
Actually what happens - Invoke just queues up these tasks, and runtime assigns threads for these tasks, what gives alot of random element to it (which one is going to get picked up first etc).
Even msdn article states this:
This method can be used to execute a set of operations, potentially in parallel.
No guarantees are made about the order in which the operations execute or whether they execute in parallel. This method does not return until each of the provided operations has completed, regardless of whether completion occurs due to normal or exceptional termination.
This problem looks like that many threads access the same variable. This is a problem of concurrency.
You can try it:
static object syncObj = new object();
static void func(int num)
{
for (int i = 0; i < 5; i++)
{
lock (syncObj)
{
Console.WriteLine(string.Format("This is function #{0} loop. counter - {1}", num, counter));
counter++;
}
}
}
Related
The code successfully build no compilation error however nothing iteration on runtime. I Stopped the loop iteration at 200 so loop will not proceed further but loop does not execute iteration lower than < 200.
I am not sure. Is there anything alternative of Stop I can use to fix this code?
Why Iterations with lower index is not performed?
How to fix this issue. I googled stuff but all vain.
Please consider the following code.
static void Main(string[] args)
{
var values= Enumerable.Range(0, 500).ToArray();
ParallelLoopResult result = Parallel.For(0, values.Count(),
(int i, ParallelLoopState loopState) => {
if (i == 200)
loopState.Stop();
WorkOnItem(values[i]);
});
Console.WriteLine(result);
}
static void WorkOnItem(object value) {
System.Console.WriteLine("Started working on: " + value);
Thread.Sleep(100);
System.Console.WriteLine("Finished working on: " + value); }
Any help to solve this issue would be appreciated. Thanks
You should call loopState.Break() instead of loopState.Stop().
From the documentation of ParallelLoopState.Break method:
Break indicates that no iterations after the current iteration should be run. It effectively cancels any additional iterations of the loop. However, it does not stop any iterations that have already begun execution. For example, if Break is called from the 100th iteration of a parallel loop iterating from 0 to 1,000, all iterations less than 100 should still be run, but the iterations from 101 through to 1000 that have not yet started are not executed.
MSDN
My question is: The third parameter in the parallel.for, what does it do?
When I change it to ()=> 1d, it doubles my result, set to two it triples, but it ignores the decimals.
Why does it ignore the decimals, if it was some sort of doubling? What is really happening there?
I've now tried adding locks. And it does not just initialize the interimresult to the value specified.
Here is the code Im using:
static void RunParallelForCorrectedAdam()
{
object _lock = new object();
double result = 0d;
// Here we call same method several times.
// for (int i = 0; i < 32; i++)
Parallel.For(0, 32,
// Func<TLocal> localInit,
() => 3d,
// Func<int, ParallelLoopState, TLocal, TLocal> body,
(i, state, interimResult) =>
{
lock (_lock)
{
return interimResult + 1;
}
},
//Final step after the calculations
//we add the result to the final result
// Action<TLocal> localFinally
(lastInterimResult) =>
{
lock (_lock)
{
result += lastInterimResult;
}
}
);
// Print the result
Console.WriteLine("The result is {0}", result);
}
With () => 3d, result will be 32 + 3 * t, where t is the number of threads that were used. 3d is passed as interimResult to the first call to body within each thread.
The whole purpose of Parallel.For is to distribute the work on several threads. So interimResult + 1 is executed exactly 32 times (possibly on different threads). But each thread has to have some initial value for interimResult. That's the value that is returned by localInit.
So if the work is distributed on e.g. two threds, each one does + 1 16 times and thus calculates 3 + 16. At the end, the partial results are summed yielding 6 + 32.
In short, in this example, it doesn't make much sense for localInit to return somthing different than 0d.
My question is: The third parameter in the parallel.for, what does it do?
It's a Func that gets executed once per thread. If your loop requires thread-local variable, this is where you initialize it.
EDIT:
Step by step:
(i, state, interimResult) => interimResult + 1,
Do you understand that interimResult is your local variable, the same one you initialized as 0d?
I have some code that loops through a list of records, starts an export task for each one, and increases a progress counter by 1 each time a task finishes so the user knows how far along the process is.
But depending on the timing of my loops, I often see the output showing a higher number before a lower number.
For example, I would expect to see output like this:
Exporting A
Exporting B
Exporting C
Exporting D
Exporting E
Finished 1 / 5
Finished 2 / 5
Finished 3 / 5
Finished 4 / 5
Finished 5 / 5
But instead I get output like this
Exporting A
Exporting B
Exporting C
Exporting D
Exporting E
Finished 1 / 5
Finished 2 / 5
Finished 5 / 5
Finished 4 / 5
Finished 3 / 5
I don't expect the output to be exact since I'm not locking the value when I update/use it (sometimes it outputs the same number twice, or skips a number), however I wouldn't expect it to go backwards.
My test data set is 72 values, and the relevant code looks like this:
var tasks = new List<Task>();
int counter = 0;
StatusMessage = string.Format("Exporting 0 / {0}", count);
foreach (var value in myValues)
{
var valueParam = value;
// Create async task, start it, and store the task in a list
// so we can wait for all tasks to finish at the end
tasks.Add(
Task.Factory.StartNew(() =>
{
Debug.WriteLine("Exporting " + valueParam );
System.Threading.Thread.Sleep(500);
counter++;
StatusMessage = string.Format("Exporting {0} / {1}", counter, count);
Debug.WriteLine("Finished " + counter.ToString());
})
);
}
// Begin async task to wait for all tasks to finish and update output
Task.Factory.StartNew(() =>
{
Task.WaitAll(tasks.ToArray());
StatusMessage = "Finished";
});
The output can appear backwards in both the debug statements and the StatusMessage output.
What's the correct way to keep count of how many async tasks in a loop are completed so that this problem doesn't occur?
You get mixed output, because counter is not incremented in the same order as Debug.WriteLine(...) method is executed.
To get a consistent progress report, you can introduce a reporting lock into the task
tasks.Add(
Task.Factory.StartNew(() =>
{
Debug.WriteLine("Exporting " + valueParam );
System.Threading.Thread.Sleep(500);
lock(progressReportLock)
{
counter++;
StatusMessage = string.Format("Exporting {0} / {1}", counter, count);
Debug.WriteLine("Finished " + counter.ToString());
}
})
);
In this sample the counter variable represents shared state among several threads. Using the ++ operator on shared state is simply unsafe and will give you incorrect results. It essentially boils down to the following instructions
push counter to stack
push 1 to stack
add values on the stack
store into counter
Because multiple threads are executing this statement it's possible for one to interrupt the other partway through completing the above sequence. This would cause the incorrect value to end up in counter.
Instead of ++ use the following statement
Interlocked.Increment(ref counter);
This operation is specifically designed to update state which may be shared among several threads. The interlocked will happen atomically and won't suffer from the race conditions I outlined
The actual out of order display of values suffers from a similar problem even after my suggested fix. The increment and display operation aren't atomic and hence one thread can interrupt the other in between the increment and display. If you want the operations to be un-interruptable by other threads then you will need to use a lock.
object lockTarget = new object();
int counter = 0;
...
lock (lockTarget) {
counter++;
StatusMessage = string.Format("Exporting {0} / {1}", counter, count);
Debug.WriteLine("Finished " + counter.ToString());
}
Note that because the increment of counter now occurs inside the lock there is no longer a need to use Interlocked.Increment
In My Parallel.ForEach Loop the localFinally delegate does get called on all the threads.
I have found this to happen as my Parallel Loop stalls.
In my Parallel Loop I have about three condition check stages that return before completion of the Loop. And it seems that it is when the Threads are returned from these stages and not the execution of the entire body that it does not execute the localFinally delegate.
The Loop structure is as follows:
var startingThread = Thread.CurrentThread;
Parallel.ForEach(fullList, opt,
()=> new MultipleValues(),
(item, loopState, index, loop) =>
{
if (cond 1)
return loop;
if (cond 2)
{
process(item);
return loop;
}
if (cond 3)
return loop;
Do Work(item);
return loop;
},
partial =>
{
Log State of startingThread and threads
} );
I have run the loop on a small data set and logged in detail and found that while the Parallel.ForEach completes all the iterations and the Log at the last thread of localFinally is --
Calling Thread State is WaitSleepJoin for Thread 6 Loop Indx 16
the Loop still does not complete gracefully and remains stalled... any clues why the stalls ?
Cheers!
Just did a quick test run after seeing the definition of localFinally (executed after each thread finished), which had me suspecting that that could mean there would be far less threads created by parallelism than loops executed. e.g.
var test = new List<List<string>> ();
for (int i = 0; i < 1000; i++)
{
test.Add(null);
}
int finalcount = 0;
int itemcount = 0;
int loopcount = 0;
Parallel.ForEach(test, () => new List<string>(),
(item, loopState, index, loop) =>
{
Interlocked.Increment(ref loopcount);
loop.Add("a");
//Thread.Sleep(100);
return loop;
},
l =>
{
Interlocked.Add(ref itemcount, l.Count);
Interlocked.Increment(ref finalcount);
});
at the end of this loop, itemcount and loopcount were 1000 as expected, and (on my machine) finalcount 1 or 2 depending on the speed of execution. In the situation with the conditions: when returned directly the execution is probably much faster and no extra threads are needed. only when the dowork is executed more threads are needed. However the parameter (l in my case) contains the combined list of all executions.
Could this be the cause of the logging difference?
I think you just misunderstood what localFinally means. It's not called for each item, it's called for each thread that is used by Parallel.ForEach(). And many items can share the same thread.
The reason why it exists is that you can perform some aggregation independently on each thread, and join them together only in the end. This way, you have to deal with synchronization (and have it impact your performance) only in a very small piece of code.
For example, if you want to compute the sum of score for a collection of items, you could do it like this:
int totalSum = 0;
Parallel.ForEach(
collection, item => Interlocked.Add(ref totalSum, ComputeScore(item)));
But here, you call Interlocked.Add() for every item, which can be slow. Using localInit and localFinally, you can rewrite the code like this:
int totalSum = 0;
Parallel.ForEach(
collection,
() => 0,
(item, state, localSum) => localSum + ComputeScore(item),
localSum => Interlocked.Add(ref totalSum, localSum));
Notice that the code uses Interlocked.Add() only in the localFinally and does access the global state in body. This way, the cost of synchronization is paid only a few times, once for each thread used.
Note: I used Interlocked in this example, because it is very simple and quite obviously correct. If the code was more complicated, I would use lock first, and try to use Interlocked only when it was necessary for good performance.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace ThreadExample
{
public class Info
{
public int Counter;
private static object _lock = new object();
private List<Thread> ThreadList;
public Info(int counter)
{
Counter = counter;
ThreadList = new List<Thread>();
ThreadList.Add(new Thread(ThreadBody));
ThreadList.Add(new Thread(ThreadBody));
ThreadList[0].Name = "t1";
ThreadList[1].Name = "t2";
}
public void Start()
{
ThreadList.ForEach(t => t.Start(t.Name));
}
public void ThreadBody(object name)
{
while (Counter != 20)
{
lock (_lock)
{
Counter++;
Console.WriteLine("Thread {0} : the value of the counter is {1}", name.ToString(), Counter);
}
}
}
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ThreadExample
{
class Program
{
static void Main(string[] args)
{
Info info = new Info(0);
info.Start();
}
}
}
if the lock is just lock counter ++
lock (_lock)
{
Counter++;
}
I don't have an infinite loop, but if the lock is like in example, it run infinite loop
It could be that when Counter gets to 19 both threads enter the loop and it ends up getting incremented to 21 before they test the value again.
You'll need to hold the lock while reading the value of Counter. A double check of Counter might be adequate (re-read it again inside the while loop while holding the lock). However, I'm not sure about this because my head just can't keep track of all the details of various threading memory models between native, .NET, Java, and whatever. Even on .NET the ECMA model is apparently different than what MS guarantees for their CLR (see http://msdn.microsoft.com/en-us/magazine/cc163715.aspx and http://www.bluebytesoftware.com/blog/PermaLink,guid,543d89ad-8d57-4a51-b7c9-a821e3992bf6.aspx). For more details on why double-checking might or might not work, search for "double checked locking" - there's an awful lot of complexity behind something that apparently should be simple.
For example, here's snippet of a run on my machine:
Thread t1 : the value of the counter is 1
Thread t2 : the value of the counter is 2
Thread t2 : the value of the counter is 3
Thread t2 : the value of the counter is 4
Thread t2 : the value of the counter is 5
Thread t2 : the value of the counter is 6
Thread t2 : the value of the counter is 7
Thread t2 : the value of the counter is 8
Thread t2 : the value of the counter is 9
Thread t2 : the value of the counter is 10
Thread t2 : the value of the counter is 11
Thread t2 : the value of the counter is 12
Thread t2 : the value of the counter is 13
Thread t2 : the value of the counter is 14
Thread t2 : the value of the counter is 15
Thread t2 : the value of the counter is 16
Thread t2 : the value of the counter is 17
Thread t2 : the value of the counter is 18
Thread t2 : the value of the counter is 19
Thread t2 : the value of the counter is 20
Thread t1 : the value of the counter is 21
Thread t1 : the value of the counter is 22
... Thread t1 never stops ...
You'll notice that t2 stops once it gets Counter to 20, but the t1 doesn't notice that. It's already entered the loop (or decided to enter the loop) thinking that Counter is 1 (or maybe 2 or something else - just not 20).
The problem is your line here:
while (Counter != 20)
Since you're locking around the increment to counter, at some point, Counter can equal 19. Both threads can do the check, then increment counter internally, making it 21 before the threads check again.
That being said, even if the two threads don't hit that at the same time, one thread may see the 20 and stop, while the other thread has a value of 21 when that is hit, and the loop will continue forever.
Your "fix" (locking just the increment) doesn't really fix it, btw - it just makes the error case less likely. The reason for this is the Console.WriteLine call is much, much slower, so more of the processing time is happening in your lock, making it more likely that the threads increment past your condition check before they see it again. However, this could still occur with just locking the counter increment (though it'd be more rare.)
You could easily correct this by having a more flexible condition, such as:
while (Counter < 20)
This will cause the threads to exit as soon as it hits 20 or higher.
The way your code is written, it's possible for both threads to increment Counter before their respective while clause gets evaluated. In that case, Counter can go from 19 to 21 before the next while is hit.
Try refactoring your loop into something like:
while (true) {
lock (_lock) {
Counter++;
Console.WriteLine("Thread {0} : the value of the counter is {1}",
name.ToString(), Counter);
if (Counter >= 20) {
break;
}
}
}