Develop Reference

Producer/consumer doesn't generate expected results

I've written such producer/consumer code, which should generate big file filled with random data
class Program
{
static void Main(string[] args)
{
Random random = new Random();
String filename = #"d:\test_out";
long numlines = 1000000;
var buffer = new BlockingCollection<string[]>(10); //limit to not get OOM.
int arrSize = 100; //size of each string chunk in buffer;
String[] block = new string[arrSize];
Task producer = Task.Factory.StartNew(() =>
{
long blockNum = 0;
long lineStopped = 0;
for (long i = 0; i < numlines; i++)
{
if (blockNum == arrSize)
{
buffer.Add(block);
blockNum = 0;
lineStopped = i;
}
block[blockNum] = random.Next().ToString();
//null is sign to stop if last block is not fully filled
if (blockNum < arrSize - 1)
{
block[blockNum + 1] = null;
}
blockNum++;
};
if (lineStopped < numlines)
{
buffer.Add(block);
}
buffer.CompleteAdding();
}, TaskCreationOptions.LongRunning);
Task consumer = Task.Factory.StartNew(() =>
{
using (var outputFile = new StreamWriter(filename))
{
foreach (string[] chunk in buffer.GetConsumingEnumerable())
{
foreach (string value in chunk)
{
if (value == null) break;
outputFile.WriteLine(value);
}
}
}
}, TaskCreationOptions.LongRunning);
Task.WaitAll(producer, consumer);
}
}
And it does what is intended to do. But for some unknown reason it produces only ~550000 strings, not 1000000 and I can not understand why this is happening.
Can someone point on my mistake? I really don't get what's wrong with this code.

The buffer
String[] block = new string[arrSize];
is declared outside the Lambda. That means it is captured and re-used.
That would normally go unnoticed (you would just write out the wrong random data) but because your if (blockNum < arrSize - 1) is placed inside the for loop you regularly write a null into the shared buffer.
Exercise, instead of:
block[blockNum] = random.Next().ToString();
use
block[blockNum] = i.ToString();
and predict and verify the results.

Is System.Threading.Semaphore weak?

There are two types of Semaphore
Strong Semaphore: maintains an order internally.
Weak semaphore: which does not provide any ordered access to a critical section which can cause starvation.
“There is no guaranteed order, such as FIFO or LIFO, in which blocked threads enter the semaphore.” from MSDN remarks in System.Threading.Semaphore.
I would like to confirm what type of semaphore implementation is provided by .Net Framework ?

static int count = 0;
static Semaphore writerSem = new Semaphore(0, 10);
static void Main(string[] args)
{
Thread[] readers = new Thread[10];
for (int i = 0; i < readers.Length; i++)
{
readers[i] = new Thread(new ThreadStart(Reader));
readers[i].Name = "Reader: " + i;
readers[i].Start();
}
Thread writer = new Thread(new ThreadStart(Writer));
writer.Start();
for (int i = 0; i < readers.Length; i++)
{
readers[i].Join();
}
writer.Join();
}
static void Reader()
{
while (true)
{
writerSem.WaitOne();
Console.WriteLine(count + " " + Thread.CurrentThread.Name);
}
}
static void Writer()
{
while (true)
{
count++;
writerSem.Release(10);
Thread.Sleep(1000);
}
}
I have tested it by writing this program. It is just confirmed that System.Threading.Semaphore is implemented as Weak Semaphore.

Am i using semaphore wrong?

I need to do some task in parallel using semaphore. I try this:
Semaphore sema = new Semaphore(2,2);
Thread[] Threads = new Thread[5];
for (int k = 0; k < 5; k++) {
sema.WaitOne();
Console.WriteLine((k + 1) + " started");
Threads[k] = new Thread(ThreadMethod1);
Threads[k].Start(k + 1);
sema.Release();
}
static void ThreadMethod1(object id) {
Thread.Sleep(50);
Console.WriteLine(id + " completed");
}
Output looks like:
1 started
2 started
3 started
4 started
5 started
1 completed
2 completed
4 completed
3 completed
5 completed
Isn't semaphore supposed to let only 2 threads to run? I don't get it or doing something wrong?

You are entering/exiting the semaphore in the "main" thread. It's useless, because in each "cycle" you'll both enter and exit it. In this modified example, you enter the semaphore in the main thread and upon finishing the worker thread you exit it.
Note that I had to pass the semaphore to the worker thread (I used a Tuple, but other methods are ok)
static void Main(string[] args) {
Semaphore sema = new Semaphore(2, 2);
Thread[] Threads = new Thread[5];
for (int k = 0; k < 5; k++) {
sema.WaitOne();
Console.WriteLine((k + 1) + " started");
Threads[k] = new Thread(ThreadMethod1);
Threads[k].Start(Tuple.Create(k + 1, sema));
}
}
static void ThreadMethod1(object tuple) {
Tuple<int, Semaphore> tuple2 = (Tuple<int, Semaphore>)tuple;
Thread.Sleep(50);
Console.WriteLine(tuple2.Item1 + " completed");
tuple2.Item2.Release();
}
You could move the sema.WaitOne "inside" the ThreadMethod1, but it would be different: all the threads would be created but would "wait" and only 2 at a time would do the "real work". As written instead up to two threads are created (and do the work)

All you have to do is to move operations on the semaphore from main thread. Small correction to your code will solve it.
public static Semaphore sema = new Semaphore(2, 2);
static void Main(string[] args)
{
Thread[] Threads = new Thread[5];
for (int k = 0; k < 5; k++)
{
Console.WriteLine((k + 1) + " started");
Threads[k] = new Thread(ThreadMethod1);
Threads[k].Start(k + 1);
}
}
static void ThreadMethod1(object id)
{
sema.WaitOne();
Thread.Sleep(1000);
Console.WriteLine(id + " completed");
sema.Release();
}

Am I doing multi threading the right way in c#? [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I have a program that runs multiple threads to convert pdf files to images in a folder and its subdirectories. It iterates through that folder, putting all pdf file names in a list, and then use that list to divide up the work between 4 threads that i created. Now it all works perfectly. Multiple threads are running and converting pdf files at the same time to different locations i specify.
I would just like to know if im doing it in the right way. Almost every website i visit do multi threading in a different way, and i don't know which one would be the most effective and ultimately the right one.
No good doing some thing that works, if its done wrong i guess... Just going to get me in the future.
I would love if you could just look at the code here and see if there is anything Drastically wrong that i need to change regarding multiple threads running
static object LockInteger = new object();
static object LockIfCheck = new object();
static object LockInteger = new object();
static object LockIfCheck = new object();
private void button1_Click(object sender, EventArgs e)
{
IterateThrough(txtboxdirectory.Text);
}
public void IterateThrough(string sourceDir)
{
MulThread = new Thread(delegate()
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
int lowerbound = 0;
int upperbound = (fileList.Count / 4);
for (int i = lowerbound; i < upperbound; i++)
{
forFunction(exceptionFileList, fileList[i], compltetedFileList,sourceDir, dir1);
stopWatch.Stop();
lblFileExecutionTime.BeginInvoke(((Action)(() => lblFileExecutionTime.Text = "Execution Per File " + stopWatch.Elapsed.ToString())));
stopWatch.Reset();
stopWatch.Start();
}
});
MulThread.Start();
MulThread1 = new Thread(delegate()
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
int lowerbound = fileList.Count / 4;
int upperbound = (fileList.Count / 4) * 2;
for (int i = lowerbound; i < upperbound; i++)
{
forFunction(exceptionFileList, fileList[i], compltetedFileList, sourceDir, dir2);
stopWatch.Stop();
lblFileExecutionTime.BeginInvoke(((Action)(() => lblFileExecutionTime.Text = "Execution Per File " + stopWatch.Elapsed.ToString())));
stopWatch.Reset();
stopWatch.Start();
}
});
MulThread1.Start();
MulThread2 = new Thread(delegate()
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
int lowerbound = (fileList.Count / 4) * 2;
int upperbound = (fileList.Count / 4) * 3;
for (int i = lowerbound; i < upperbound; i++)
{
forFunction(exceptionFileList, fileList[i], compltetedFileList, sourceDir, dir3);
stopWatch.Stop();
lblFileExecutionTime.BeginInvoke(((Action)(() => lblFileExecutionTime.Text = "Execution Per File " + stopWatch.Elapsed.ToString())));
stopWatch.Reset();
stopWatch.Start();
}
});
MulThread2.Start();
MulThread3 = new Thread(delegate()
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
int lowerbound = (fileList.Count / 4) * 3;
int upperbound;
if (fileList.Count % 4 != 0)
{
upperbound = ((fileList.Count / 4) * 4) + (fileList.Count % 4) + 1;
}
else
{
upperbound = ((fileList.Count / 4) * 4) + (fileList.Count % 4);
}
for (int i = lowerbound; i < upperbound; i++)
{
forFunction(exceptionFileList, fileList[i], compltetedFileList, sourceDir, dir4);
stopWatch.Stop();
lblFileExecutionTime.BeginInvoke(((Action)(() => lblFileExecutionTime.Text = "Execution Per File " + stopWatch.Elapsed.ToString())));
stopWatch.Reset();
stopWatch.Start();
}
});
MulThread3.Start();
}
Then i lock some of the methods in "forFunction".
private int forFunction(String exceptionFileList, FileInfo z, String compltetedFileList, String sourceDir, String imagedirectory)
{
//heres where it locked up because of this global variable
lock (LockInteger)
{
atPDFNumber++;
}
int blankImage = 1;
int pagesMissing = 0;
//delete the images currently in the folder
deleteCreatedImages(imagedirectory);
//Get the amount of pages in the pdf
int numberPDFPage = numberOfPagesPDF(z.FullName);
//Convert the pdf to images on the users pc
convertToImage(z.FullName, imagedirectory);
//Check the images for blank pages
blankImage = testPixels(imagedirectory, z.FullName);
//Check if the conversion couldnt convert a page because of an error
pagesMissing = numberPDFPage - numberOfFiles;
//int pagesMissing = 0;
//Cancel button is pressed
if (toContinue == 0)
{
return 0;
}
lock (LockIfCheck)
{
//If there is a blank page, or if there is a missing page
if (blankImage == 0 || pagesMissing > 0)
{
myholder = 1;
exceptionFileList += "File Name: " + z.Name + "\r\n"
+ "File Path: " + z.FullName + "\r\n \r\n";
String currentValue = exceptionFileList;
txtboxProblemFiles.BeginInvoke(((Action)(() => txtboxProblemFiles.Text += currentValue.ToString())));
String currentValue3 = z.FullName;
txtboxProblemFiles.BeginInvoke(((Action)(() => listboxProblemFiles.Items.Add(currentValue3))));
}
else
{
compltetedFileList += "Scanning Completed of file: " + "\r\n"
+ "File Name: " + z.Name + "\r\n"
+ "File Path: " + sourceDir + "\r\n \r\n";
}
String currentValue1 = "File Name: " + z.Name + "\r\n";
txtboxCheckedFiles.BeginInvoke(((Action)(() => txtboxCheckedFiles.Text += currentValue1)));
}
myWorkerClass();
return 1;
}
I'll accept any up building criticism from this code.

The suggested way for multitrheading is by using ThreadPool.
The benefit of ThreadPool is that it manages thread creation, job assignment ,... and gives you better performance alongside less resource usage.
A sample of ThreadPool from MSDN:
using System;
using System.Threading;
public class Example {
public static void Main() {
// Queue the task.
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
Thread.Sleep(1000);
Console.WriteLine("Main thread exits.");
}
// This thread procedure performs the task.
static void ThreadProc(Object stateInfo) {
// No state object was passed to QueueUserWorkItem, so
// stateInfo is null.
Console.WriteLine("Hello from the thread pool.");
}
}

I guess the answer is that depends. Are you starting threads, yes. Does it look like you are protecting shared data yes. It does look like you have copy-pasted code everywhere. That could be abstracted into a function and the upper and lower bounds made into arguments. I think you might be able to reduce this and use the Parallel.For or Parallel.ForEach. Lastly, I can't quite remember the multithreading issue that relates to GUI. I see that you are using some GUI here. But you need to make sure that the change to the labels are made by the GUI thread and not some other thread.

C# using loop to start threads and pass parameters

In below sample code, I use lambda function to make 3 threads doing different things. My goal is make the thread count configurable, so I was thinking using a loop to start threads. But I always got in static function can't call non-static members error. Can the community help me or direct me to a tutorial? Thanks a lot!
My Code:
internal class FeedClient
{
private static void Main(string[] args)
{
int iteration = 10;
int ranSleepTime = 1000;
var obj = new MyClass();
var threads = new Thread[3];
(threads[0] = new Thread(() =>
{
Random random = new System.Random();
for (int i = 0; i < iteration; i++)
{
obj.MyMethod("my string 1");
Thread.Sleep(random.Next(ranSleepTime));
}
})).Start();
(threads[1] = new Thread(() =>
{
Random random = new System.Random();
for (int i = 0; i < iteration; i++)
{
obj.MyMethod("my string 2");
Thread.Sleep(random.Next(ranSleepTime));
}
})).Start();
(threads[2] = new Thread(() =>
{
Random random = new System.Random();
for (int i = 0; i < iteration; i++)
{
obj.MyMethod("my string 3");
Thread.Sleep(random.Next(ranSleepTime));
}
})).Start();
foreach (Thread thread in threads)
{
thread.Join();
}
obj.Close(false);
Console.WriteLine("Press any key to exit.");
Console.ReadKey();
}
}
Desired look:
for(int i=0;i<3;i++){
threads[i] = new Thread(func); // func is the lambda function
threads[i].Start(myData[i]); // myData[] may be a string array
}

The error message seems to indicate that you are attempting to use an instance member from a static method somewhere. Naturally that is not allowed since a static method does not have a this reference. Here is how I would refactor your code.
public static void Main()
{
string[] myData = GetStringArray();
int iteration = 10;
int ranSleepTime = 1000;
var obj = new MyClass();
var threads = new Thread[myData.Length];
for (int i = 0; i < threads.Length; i++)
{
int captured = i; // This is required to avoid capturing the loop variable.
threads[i] = new Thread(
() =>
{
var random = new Random();
for (int i = 0; i < iteration; i++)
{
obj.MyMethod(myData[captured]);
Thread.Sleep(random.Next(ranSleepTime));
}
});
threads[i].Start();
}
foreach (Thread thread in threads)
{
thread.Join();
}
obj.Close(false);
}
I must mention, however, that creating new threads in an unbounded loop is generally undesirable. If the loop has a tight bound then maybe, but I would have to get a better understanding of the problem before making any further comments regarding this point.