Develop Reference

Producer/consumer doesn't generate expected results

I've written such producer/consumer code, which should generate big file filled with random data
class Program
{
static void Main(string[] args)
{
Random random = new Random();
String filename = #"d:\test_out";
long numlines = 1000000;
var buffer = new BlockingCollection<string[]>(10); //limit to not get OOM.
int arrSize = 100; //size of each string chunk in buffer;
String[] block = new string[arrSize];
Task producer = Task.Factory.StartNew(() =>
{
long blockNum = 0;
long lineStopped = 0;
for (long i = 0; i < numlines; i++)
{
if (blockNum == arrSize)
{
buffer.Add(block);
blockNum = 0;
lineStopped = i;
}
block[blockNum] = random.Next().ToString();
//null is sign to stop if last block is not fully filled
if (blockNum < arrSize - 1)
{
block[blockNum + 1] = null;
}
blockNum++;
};
if (lineStopped < numlines)
{
buffer.Add(block);
}
buffer.CompleteAdding();
}, TaskCreationOptions.LongRunning);
Task consumer = Task.Factory.StartNew(() =>
{
using (var outputFile = new StreamWriter(filename))
{
foreach (string[] chunk in buffer.GetConsumingEnumerable())
{
foreach (string value in chunk)
{
if (value == null) break;
outputFile.WriteLine(value);
}
}
}
}, TaskCreationOptions.LongRunning);
Task.WaitAll(producer, consumer);
}
}
And it does what is intended to do. But for some unknown reason it produces only ~550000 strings, not 1000000 and I can not understand why this is happening.
Can someone point on my mistake? I really don't get what's wrong with this code.

The buffer
String[] block = new string[arrSize];
is declared outside the Lambda. That means it is captured and re-used.
That would normally go unnoticed (you would just write out the wrong random data) but because your if (blockNum < arrSize - 1) is placed inside the for loop you regularly write a null into the shared buffer.
Exercise, instead of:
block[blockNum] = random.Next().ToString();
use
block[blockNum] = i.ToString();
and predict and verify the results.

Simultaneous printing on the console in c#, is it possible?

I'm new to programming and currently I'm trying to learn c#. Here is my question:
I have this following piece of code:
static void Main()
{
string loading = "LOADING...";
for (int i = 0; i < 5; i++)
{
foreach (var letter in loading)
{
Console.Write("{0}", letter);
Thread.Sleep(250);
}
Console.Clear();
Console.Write("\r");
}
for (int k = 0; k <= 100; k++)
{
Console.Write("\r{0}%", k);
Thread.Sleep(150);
}
}
I am trying to find a way to execute these two for loops simultaneously. I want to keep re-writing LOADING on the console and under it to print the percentage from 1 to 100. Right now the second for loop runs after the first quits. Is it possible to make them run at the same time? I have been trying to find an answer for a few hours now, but with no luck so far.
Thanks!

As krillgar noted you can use Tasks to run each loop independently. Something like this:
string loading = "LOADING...";
var loadingTask = Task.Run(() =>
{
for (int i = 0; i < 5; i++)
{
foreach (var letter in loading)
{
Console.Write("{0}", letter);
Thread.Sleep(250);
}
Console.Clear();
Console.Write("\r");
}
});
var pocTask = Task.Run(() =>
{
for (int k = 0; k <= 100; k++)
{
Console.Write("\r{0}%", k);
Thread.Sleep(150);
}
});
Task.WaitAll(loadingTask, pocTask);
Please note however it'll not work as you would expect( first line with LOADING... and second with percentage). That would require some synchronization of displayed messages between those loops and I think that's whole another story and not in the scope of the answer to your question.

There is more to this challenge than just running the code in two threads. When multiple threads are attempting to write to the console simultaneously, you are likely to end up with race conditions, where one thread starts writing in the middle of another thread's output.
To protect against this, you need to enforce critical sections, which are regions of your code where only one thread can execute at any time. These would generally consist of:
// Critical section
lock (syncLock)
{
Console.SetCursorPosition(x, y);
Console.Write(yourText);
}
Thread.Sleep(yourDelay);
Here is the full code:
// Dummy object to serve as mutual-exclusion lock when synchronizing threads.
private static readonly object syncLock = new object();
public static void Main(string[] args)
{
// Run two anonymous functions in parallel,
// then wait for both to complete.
Parallel.Invoke(
// Anonymous function for printing "LOADING..."
() =>
{
const string loading = "LOADING...";
for (int i = 0; i < 5; i++)
{
for (int j = 0; j < loading.Length; j++)
{
// Critical section
lock (syncLock)
{
Console.SetCursorPosition(j, 0);
Console.Write("{0}", loading[j]);
}
Thread.Sleep(250);
}
// Critical section
lock (syncLock)
{
Console.SetCursorPosition(0, 0);
Console.Write("\r ");
}
Thread.Sleep(250);
}
},
// Anonymous function for printing "x%"
() =>
{
for (int k = 0; k <= 100; k++)
{
// Critical section
lock (syncLock)
{
Console.SetCursorPosition(0, 1);
Console.Write("\r{0}%", k);
}
Thread.Sleep(150);
}
});
}
}

Multithreaded code to do work using configured number of thread

I want to create a multithreaded application code. I want to execute configured no of threads and each thread do the work. I want to know is this the write approach or do we have better approach. All the threads needs to be executed asynchronously.
public static bool keepThreadsAlive = false;
static void Main(string[] args)
{
Program pgm = new Program();
int noOfThreads = 4;
keepThreadsAlive = true;
for (int i = 1; i <= noOfThreads; i++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(DoWork), (object)i);
}
System.Console.ReadLine();
StopAllThreads();
System.Console.ReadLine();
}
private static void DoWork(object threadNumber)
{
int threadNum = (int)threadNumber;
int counter = 1;
while (keepThreadsAlive)
{
counter = ProcessACK(threadNum, counter);
}
}
private static int ProcessACK(int threadNum, int counter)
{
System.Console.WriteLine("Thread {0} count {1}", threadNum, counter++);
Random ran = new Random();
int randomNumber = ran.Next(5000, 100000);
for (int i = 0; i < randomNumber; i++) ;
Thread.Sleep(2000);
return counter;
}

As others have pointed out, the methods you are using are dated and not as elegant as the more modern C# approach to accomplishing the same tasks.
Have a look at System.Threading.Tasks for an overview of what is available to you these days. There is even a way to set the maximum threads used in a parallel operation. Here is a simple (pseudocode) example:
Parallel.ForEach(someListOfItems, new ParallelOptions { MaxDegreeOfParallelism = 8 }, item =>
{
//do stuff for each item in "someListOfItems" using a maximum of 8 threads.
});
Hope this helps.

Main thread not proceeding when child threads have finished

I am trying to use multithreading in my application. The method test5 tries to fetch some content from Internet, while the main thread waits for all threads to finish before continuing with other work.
But my main thread doesn't come back after calling test5, and my console lines Done Inside!! and thread all got back!! are never reached.
How can I resolve this issue?
class Program
{
static void Main(string[] args)
{
string[] url =
{
"http://...", "http://...", "http://...", "http://...", "http://..."
};
test5(url);
Console.WriteLine("thread all got back!!");
// Do some other work after all threads come back
Console.ReadLine();
}
private static void test5(string[] _url)
{
int numThreads = _url.Length;
ManualResetEvent resetEvent = new ManualResetEvent(false);
int toProcess = numThreads;
for (int i = 0; i < numThreads - 1; i++)
{
new Thread( delegate() {
testWebWorking(_url[i]);
if (Interlocked.Decrement(ref toProcess) == 0)
resetEvent.Set();
}).Start();
}
resetEvent.WaitOne();
Console.WriteLine("Done inside!!");
}
private static void test6(string[] _url)
{
int numThreads = _url.Length;
var countdownEvent = new CountdownEvent(numThreads);
for (int i = 0; i < numThreads - 1; i++)
{
new Thread(delegate() {
testWebWorking(_url[i]);
countdownEvent.Signal();
}).Start();
}
countdownEvent.Wait();
Console.WriteLine("Done inside!!");
}
private static void testWebWorking(object url)
{
Console.WriteLine("start {0}", Thread.CurrentThread.ManagedThreadId);
string uri = (string)url;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.KeepAlive = true;
request.Timeout = 5000;
request.ReadWriteTimeout = 5000;
request.Proxy = null;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
//Console.WriteLine(response.ContentType + "; uri = " + uri);
Stream receiveStream = response.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, encode);
//Console.WriteLine("\r\nResponse stream received.");
Char[] read = new Char[256];
// Reads 256 characters at a time.
int count = readStream.Read(read, 0, 256);
//Console.WriteLine("HTML...\r\n");
String str = "";
while (count > 0)
{
// Dumps the 256 characters on a string and displays the string to the console.
str = new String(read, 0, count);
//Console.Write(str);
count = readStream.Read(read, 0, 256);
}
//Console.WriteLine(str);
// Releases the resources of the response.
response.Close();
// Releases the resources of the Stream.
readStream.Close();
Console.WriteLine("end {0}", Thread.CurrentThread.ManagedThreadId);
}
}
catch (WebException ex)
{
//Console.WriteLine(ex.GetBaseException().ToString() );
//Console.WriteLine(url);
Console.WriteLine("time out !!");
}
finally
{
request.Abort();
request = null;
GC.Collect();
}
}
}

Look at this:
for (int i = 0; i < numThreads - 1; i++)
You're only starting numThreads - 1 threads. Your counter starts at numThreads and counts down, so it'll only ever reach 1, not 0.
As an aside, this is also broken::
for (int i = 0; i < numThreads - 1; i++)
{
new Thread( delegate()
{
testWebWorking(_url[i]);
...
}
...
}
Here, you're capturing the variable i within the delegate, so it will have whatever value i has when you execute it. You may very well test the same URL more than once, and skip other URLs. Instead, you should copy the value of i into a variable within the loop, so that you capture a different variable each time:
// Fixed the loop boundary as well
for (int i = 0; i < numThreads; i++)
{
int copy = i;
new Thread(() => // Use a lambda expression for brevity
{
// Fix naming convention for method name, too...
TestWebWorking(_url[copy]);
if (Interlocked.Decrement(ref toProcess) == 0))
{
resetEvent.Set();
}
}).Start()
}
Both of your methods have the same set of problems.
Personally I wouldn't use a for loop here anyway - I'd use a foreach loop:
private static void TestWithResetEvent(string[] urls)
{
ManualResetEvent resetEvent = new ManualResetEvent(false);
int counter = urls.Length;
foreach (string url in urls)
{
string copy = url;
Thread t = new Thread(() =>
{
TestWebWorking(copy);
if (Interlocked.Decrement(ref toProcess) == 0))
{
resetEvent.Set();
}
});
t.Start();
}
resetEvent.WaitOne();
Console.WriteLine("Done inside!!");
}
Alternatively, it would be simpler to use Parallel.ForEach, which is designed for exactly this sort of thing.

ThreadQueue problems in "Accelerated C# 2008"

Example for threading queue book "Accelerated C# 2008" (CrudeThreadPool class) not work correctly. If I insert long job in WorkFunction() on 2-processor machine executing for next task don't run before first is over. How to solve this problem? I want to load the processor to 100 percent
public class CrudeThreadPool
{
static readonly int MAX_WORK_THREADS = 4;
static readonly int WAIT_TIMEOUT = 2000;
public delegate void WorkDelegate();
public CrudeThreadPool()
{
stop = 0;
workLock = new Object();
workQueue = new Queue();
threads = new Thread[MAX_WORK_THREADS];
for (int i = 0; i < MAX_WORK_THREADS; ++i)
{
threads[i] = new Thread(new ThreadStart(this.ThreadFunc));
threads[i].Start();
}
}
private void ThreadFunc()
{
lock (workLock)
{
int shouldStop = 0;
do
{
shouldStop = Interlocked.Exchange(ref stop, stop);
if (shouldStop == 0)
{
WorkDelegate workItem = null;
if (Monitor.Wait(workLock, WAIT_TIMEOUT))
{
// Process the item on the front of the queue
lock (workQueue)
{
workItem = (WorkDelegate)workQueue.Dequeue();
}
workItem();
}
}
} while (shouldStop == 0);
}
}
public void SubmitWorkItem(WorkDelegate item)
{
lock (workLock)
{
lock (workQueue)
{
workQueue.Enqueue(item);
}
Monitor.Pulse(workLock);
}
}
public void Shutdown()
{
Interlocked.Exchange(ref stop, 1);
}
private Queue workQueue;
private Object workLock;
private Thread[] threads;
private int stop;
}
public class EntryPoint
{
static void WorkFunction()
{
Console.WriteLine("WorkFunction() called on Thread 0}", Thread.CurrentThread.GetHashCode());
//some long job
double s = 0;
for (int i = 0; i < 100000000; i++)
s += Math.Sin(i);
}
static void Main()
{
CrudeThreadPool pool = new CrudeThreadPool();
for (int i = 0; i < 10; ++i)
{
pool.SubmitWorkItem(
new CrudeThreadPool.WorkDelegate(EntryPoint.WorkFunction));
}
pool.Shutdown();
}
}

I can see 2 problems:
Inside ThreadFunc() you take a lock(workLock) for the duration of the method, meaning your threadpool is no longer async.
in the Main() method, you close down the threadpool w/o waiting for it to finish. Oddly enough that is why it is working now, stopping each ThreadFunc after 1 loop.

It's hard to tell because there's no indentation, but it looks to me like it's executing the work item while still holding workLock - which is basically going to serialize all the work.
If at all possible, I suggest you start using the Parallel Extensions framework in .NET 4, which has obviously had rather more time spent on it. Otherwise, there's the existing thread pool in the framework, and there are other implementations around if you're willing to have a look. I have one in MiscUtil although I haven't looked at the code for quite a while - it's pretty primitive.