Develop Reference

Deadlock using async Task and SemaphoreSlim

we are running an ASP.NET 6 webapplication and are having strange issues with deadlocks.
The app suddenly freezes after some weeks of operations and it seems that it might be caused by our locking mechanism with the SemaphoreSlim class.
I tried to reproduce the issue with a simple test-project and found something strange.
The following code is simply starting 1000 tasks where each is doing some work (requesting semaphore-handle, waiting for 10 ms and releasing the semaphore).
I expected this code to simply execute one task after another. But it freezes because of a deadlock in the first call of the DoWork method (at await Task.Delay(10)).
Does anyone know why this causes a deadlock? I tried exactly the same code with ThreadPool.QueueUserWorkItem instead of Task.Run and Thread.Sleep instead of Task.Delay and this worked as expected. But as soon as I use the tasks it stops working.
Here is the complete code-snippet:
internal class Program
{
static int timeoutSec = 60;
static SemaphoreSlim semaphore = new SemaphoreSlim(1);
static int numPerIteration = 1000;
static int iteration = 0;
static int doneCounter = numPerIteration;
static int successCount = 0;
static int failedCount = 0;
static Stopwatch sw = new Stopwatch();
static Random rnd = new Random();
static void Main(string[] args)
{
Task.WaitAll(TestUsingTasks());
}
static async Task TestUsingTasks()
{
while (true)
{
var tasks = new List<Task>();
if (doneCounter >= numPerIteration)
{
doneCounter = 0;
if (iteration >= 1)
{
Log($"+++++ FINISHED TASK ITERATION {iteration} - SUCCESS: {successCount} - FAILURES: {failedCount} - Seconds: {sw.Elapsed.TotalSeconds:F1}", ConsoleColor.Magenta);
}
iteration++;
sw.Restart();
for (int i = 0; i < numPerIteration; i++)
{
// Start indepdent tasks to do some work
Task.Run(async () =>
{
if (await DoWork())
{
successCount++;
}
else
{
failedCount++;
}
doneCounter++;
});
}
}
await Task.Delay(10);
}
}
static async Task<bool> DoWork()
{
if (semaphore.Wait(timeoutSec * 1000)) // Request the semaphore to ensure that one 1 task at a time can enter
{
Log($"Got handle for {iteration} within {sw.Elapsed.TotalSeconds:F1}", ConsoleColor.Green);
var totalSec = sw.Elapsed.TotalSeconds;
await Task.Delay(10); // Wait for 10ms to simulate some work => Deadlock seems to happen here
Log($"RELEASING LOCK handle for {iteration} within {sw.Elapsed.TotalSeconds:F1}. WAIT took " + (sw.Elapsed.TotalSeconds - totalSec) + " seconds", ConsoleColor.Gray);
semaphore.Release();
return true;
}
else
{
Log($"ERROR: TASK handle failed for {iteration} within {sw.Elapsed.TotalSeconds:F1} sec", ConsoleColor.Red);
return false;
}
}
static void Log(string message, ConsoleColor color)
{
Console.ForegroundColor = color;
Console.WriteLine(message);
Console.ForegroundColor = ConsoleColor.White;
}
}
Thanks in advance!

But it freezes because of a deadlock in the first call of the DoWork method (at await Task.Delay(10)).
I would argue that it is not deadlock but a thread starvation issue. If you wait long enough you will see that threads will be able to finish the simulation wait from time to time.
The quick fix here is using non-blocking WaitAsync call with await:
static async Task<bool> DoWork()
{
if (await semaphore.WaitAsync(timeoutSec * 1000))
{
...
}
}
Also note:
It is recommended to wrap the code after Wait.. into try-finally block and release the semaphore in the finally.
Incrementing counters in parallel environments better should be done in atomic fashion, for example with Interlocked.Increment.

I am working on implementing job queue. Using TPL for the same. Trying to restrict number of jobs at a time using ThreadPool.SetMaxThreads. no luck

****To Restrict the thread :****
int workerThreads, completionPortThreads;
ThreadPool.GetMaxThreads(out workerThreads, out completionPortThreads);
workerThreads = 2;
ThreadPool.SetMaxThreads(workerThreads, completionPortThreads);
To run the job I tried 2 options
Option 1.
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc),task);
Option 2:
Task runner = new Task(() => taskProcessor.ImportIntoArt(task),TaskCreationOptions.LongRunning|TaskCreationOptions.PreferFairness);
runner.Start();
I expect this code has to pick up first two jobs for processing and 3rd one should go in to the queue. As expected first two jobs will start, however 3rd one will also be picked up for processing.
Any help is highly appreciated.

Use the QueuedTaskScheduler from this package in conjunction with Task.Factory.StartNew method:
var scheduler = new QueuedTaskScheduler(TaskScheduler.Default, 2);
var jobAction = new Action<string>(
jobName =>
{
Console.WriteLine("I am job " + jobName + " and I start at " + DateTime.Now.ToLongTimeString());
Thread.Sleep(10000);
Console.WriteLine("I am job " + jobName + " and I finish at " + DateTime.Now.ToLongTimeString());
});
var jobs = Enumerable
.Range(1, 6)
.Select(num => Task.Factory.StartNew(
() => jobAction("Job" + num),
CancellationToken.None,
TaskCreationOptions.LongRunning,
scheduler))
.ToList();
Task.WhenAll(jobs).Wait();

I know you want to achieve this task using TPL, but as #stuartd has made a comment that we can't do that with threadpool, then you can achieve this task traditional way by creating required number of thread and run them infinitely and observe the collection of a task which of type query.
Please refer below code if you want to achieve the task without using other libraries.
//Declare queue of task.
static Queue<int> taskQueue = new Queue<int>();
static readonly object lockObj = new object();
//Get task to perform.
static int? GetNextTask()
{
lock (lockObj)
{
if (taskQueue.Count > 0)
return taskQueue.Dequeue();
else return null;
}
}
//Add task to queue from different thread.
static void AddTask(int task)
{
lock (lockObj)
{
taskQueue.Enqueue(task);
}
}
static void PerformThreadOperation()
{
//Run infinite for current thread.
while (true)
{
var task = GetNextTask();
//If there is task then perform some action else make thread sleep for some time you can set event to resume thread.
if (task.HasValue)
{
Console.WriteLine("Task Initiate => {0}", task.Value);
Thread.Sleep(4000);
Console.WriteLine("Task Complete => {0}", task.Value);
}
else
{
Console.WriteLine("Task not found, thread is going to be sleep for some moment.");
Console.WriteLine("Thread {0} enter in sleep mode.", Thread.CurrentThread.Name);
Thread.Sleep(5000);
}
}
}
//Create required thread to process task parallely.
static void TestThreadApplication()
{
Thread thread = new Thread(new ThreadStart(PerformThreadOperation));
Thread thread1 = new Thread(PerformThreadOperation);
thread.Start();
thread1.Start();
}
static void Main(string[] args)
{
for (int i = 0; i < 6; i++)
{
taskQueue.Enqueue(i);
}
TestThreadApplication();
Thread.Sleep(20000);
for (int i = 6; i < 10; i++)
{
taskQueue.Enqueue(i);
}
Console.ReadKey();
}

Repeat a task (TPL) in windows service, using ContinueWith

I have a windows service (written in C#) that use the task parallel library dll to perform some parallel tasks (5 tasks a time)
After the tasks are executed once I would like to repeat the same tasks on an on going basis (hourly). Call the QueuePeek method
Do I use a timer or a counter like I have setup in the code snippet below?
I am using a counter to set up the tasks, once I reach five I exit the loop, but I also use a .ContinueWith to decrement the counter, so my thought is that the counter value would be below 5 hence the loop would continue. But my ContinueWith seems to be executing on the main thread and the loop then exits.
The call to DecrementCounter using the ContinueWith does not seem to work
FYI : The Importer class is to load some libraries using MEF and do the work
This is my code sample:
private void QueuePeek()
{
var list = SetUpJobs();
while (taskCounter < 5)
{
int j = taskCounter;
Task task = null;
task = new Task(() =>
{
DoLoad(j);
});
taskCounter += 1;
tasks[j] = task;
task.ContinueWith((t) => DecrementTaskCounter());
task.Start();
ds.SetJobStatus(1);
}
if (taskCounter == 0)
Console.WriteLine("Completed all tasks.");
}
private void DoLoad(int i)
{
ILoader loader;
DataService.DataService ds = new DataService.DataService();
Dictionary<int, dynamic> results = ds.AssignRequest(i);
var data = results.Where(x => x.Key == 2).First();
int loaderId = (int)data.Value;
Importer imp = new Importer();
loader = imp.Run(GetLoaderType(loaderId));
LoaderProcessor lp = new LoaderProcessor(loader);
lp.ExecuteLoader();
}
private void DecrementTaskCounter()
{
Console.WriteLine(string.Format("Decrementing task counter with threadId: {0}",Thread.CurrentThread.ManagedThreadId) );
taskCounter--;
}

I see a few issues with your code that can potentially lead to some hard to track-down bugs. First, if using a counter that all of the tasks can potentially be reading and writing to at the same time, try using Interlocked. For example:
Interlocked.Increment(ref _taskCounter); // or Interlocked.Decrement(ref _taskCounter);
If I understand what you're trying to accomplish, I think what you want to do is to use a timer that you re-schedule after each group of tasks is finished.
public class Worker
{
private System.Threading.Timer _timer;
private int _timeUntilNextCall = 3600000;
public void Start()
{
_timer = new Timer(new TimerCallback(QueuePeek), null, 0, Timeout.Infinite);
}
private void QueuePeek(object state)
{
int numberOfTasks = 5;
Task[] tasks = new Task[numberOfTasks];
for(int i = 0; i < numberOfTasks; i++)
{
tasks[i] = new Task(() =>
{
DoLoad();
});
tasks[i].Start();
}
// When all tasks are complete, set to run this method again in x milliseconds
Task.Factory.ContinueWhenAll(tasks, (t) => { _timer.Change(_timeUntilNextCall, Timeout.Infinite); });
}
private void DoLoad() { }
}

How to use multi threading in a For loop

I want to achieve the below requirement; please suggest some solution.
string[] filenames = Directory.GetFiles("C:\Temp"); //10 files
for (int i = 0; i < filenames.count; i++)
{
ProcessFile(filenames[i]); //it takes time to execute
}
I wanted to implement multi-threading. e.g There are 10 files. I wanted to process 3 files at a time (configurable, say maxthreadcount). So 3 files will be processed in 3 threads from the for loop and if any thread completes the execution, it should pick the next item from the for loop. Also wanted to ensure all the files are processed before it exits the for loop.
Please suggest best approach.

Try
Parallel.For(0, filenames.Length, i => {
ProcessFile(filenames[i]);
});
MSDN
It's only available since .Net 4. Hope that acceptable.

This will do the job in .net 2.0:
class Program
{
static int workingCounter = 0;
static int workingLimit = 10;
static int processedCounter = 0;
static void Main(string[] args)
{
string[] files = Directory.GetFiles("C:\\Temp");
int checkCount = files.Length;
foreach (string file in files)
{
//wait for free limit...
while (workingCounter >= workingLimit)
{
Thread.Sleep(100);
}
workingCounter += 1;
ParameterizedThreadStart pts = new ParameterizedThreadStart(ProcessFile);
Thread th = new Thread(pts);
th.Start(file);
}
//wait for all threads to complete...
while (processedCounter< checkCount)
{
Thread.Sleep(100);
}
Console.WriteLine("Work completed!");
}
static void ProcessFile(object file)
{
try
{
Console.WriteLine(DateTime.Now.ToString() + " recieved: " + file + " thread count is: " + workingCounter.ToString());
//make some sleep for demo...
Thread.Sleep(2000);
}
catch (Exception ex)
{
//handle your exception...
string exMsg = ex.Message;
}
finally
{
Interlocked.Decrement(ref workingCounter);
Interlocked.Increment(ref processedCounter);
}
}
}

Take a look at the Producer/Consumer Queue example by Joe Albahari. It should provide a good starting point for what you're trying to accomplish.

You could use the ThreadPool.
Example:
ThreadPool.SetMaxThreads(3, 3);
for (int i = 0; i < filenames.count; i++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessFile), filenames[i]);
}
static void ProcessFile(object fileNameObj)
{
var fileName = (string)fileNameObj;
// do your processing here.
}
If you are using the ThreadPool elsewhere in your application then this would not be a good solution since it is shared across your app.
You could also grab a different thread pool implementation, for example SmartThreadPool

Rather than starting a thread for each file name, put the file names into a queue and then start up three threads to process them. Or, since the main thread is now free, start up two threads and let the main thread work on it, too:
Queue<string> MyQueue;
void MyProc()
{
string[] filenames = Directory.GetFiles(...);
MyQueue = new Queue(filenames);
// start two threads
Thread t1 = new Thread((ThreadStart)ProcessQueue);
Thread t2 = new Thread((ThreadStart)ProcessQueue);
t1.Start();
t2.Start();
// main thread processes the queue, too!
ProcessQueue();
// wait for threads to complete
t1.Join();
t2.Join();
}
private object queueLock = new object();
void ProcessQueue()
{
while (true)
{
string s;
lock (queueLock)
{
if (MyQueue.Count == 0)
{
// queue is empty
return;
}
s = MyQueue.Dequeue();
}
ProcessFile(s);
}
}
Another option is to use a semaphore to control how many threads are working:
Semaphore MySem = new Semaphore(3, 3);
void MyProc()
{
string[] filenames = Directory.GetFiles(...);
foreach (string s in filenames)
{
mySem.WaitOne();
ThreadPool.QueueUserWorkItem(ProcessFile, s);
}
// wait for all threads to finish
int count = 0;
while (count < 3)
{
mySem.WaitOne();
++count;
}
}
void ProcessFile(object state)
{
string fname = (string)state;
// do whatever
mySem.Release(); // release so another thread can start
}
The first will perform somewhat better because you don't have the overhead of starting and stopping a thread for each file name processed. The second is much shorter and cleaner, though, and takes full advantage of the thread pool. Likely you won't notice the performance difference.

Can set max threads unsing ParallelOptions
Parallel.For Method (Int32, Int32, ParallelOptions, Action)
ParallelOptions.MaxDegreeOfParallelism

var results = filenames.ToArray().AsParallel().Select(filename=>ProcessFile(filename)).ToArray();
bool ProcessFile(object fileNameObj)
{
var fileName = (string)fileNameObj;
// do your processing here.
return true;
}

Multi thread worker thread status

I create my threads as
for (int i = 0; i < threadCount; i++)
{
Searcher src = new Searcher(i, this);
threads[i] = new Thread(new ThreadStart(src.getIpRange));
threads[i].Name = string.Format(i.ToString());
}
foreach (Thread t in threads)
{
t.Start();
}
with threadCount(= 100, 150, 255 etc...) but I can't learn how many threads working. on execute time.
and I want to control when all threads finishes their job. and give me a message like "All threads are dead, jobs completed..."
like backgroundWorker's RunWorkerCompleted event

Determining when all the threads are finished is simple.
for (int i = 0; i < threadCount; i++)
{
threads[i].Join();
}
Console.WriteLine("All threads are done!");
Can you elaborate on your other requirements?

You can check the ThreadState property of the Thread.
Might be better to use async methods. This gives you a WaitHandle object, and you can use WaitHandle.WaitAll to wait for all of your async methods to finish.
Here's an intro to asynchronous programming:
http://msdn.microsoft.com/en-us/library/aa719598%28v=VS.71%29.aspx

You definitely want to use the Task class for this or a higher-level concept like Parallel.ForEach. Using the Thread class directly is quite painful.
I recently wrote a blog post comparing various asynchronous approaches, listed in order from best (Task) to worst (Thread).
Here's an example using Task, demonstrating what you wanted to do:
// Start all tasks
var threads = new Task[threadCount];
for (int i = 0; i < threadCount; i++)
{
Searcher src = new Searcher(i, this);
threads[i] = Task.Factory.StartNew(src.getIpRange);
}
// How many are running right now?
var runningCount = threads.Count(x => x.Status == TaskStatus.Running);
// Register a callback when they have all completed (this does not block)
Task.Factory.ContinueWhenAll(threads, MyCallback);

Add a delegate to Searcher and pass it a callback method from your main thread that each thread will call when it finishes. As you launch each thread, add it to a Dictionary keyed by the thread's ManagedThreadId. When each thread finishes, the callback removes the thread from the Dictionary and checks to see if the count is zero.
Dictionary<int, Thread> activeThreads = new Dictionary<int, Thread>();
for (int i = 0; i < threadCount; i++)
{
Searcher src = new Searcher(i, this);
src.Done = new SearcherDoneDelegate(ThreadDone);
threads[i] = new Thread(new ThreadStart(src.getIpRange));
threads[i].Name = string.Format(i.ToString());
}
foreach (Thread t in threads)
{
lock (activeThreads)
{
activeThreads.Add(t.ManagedThreadId, t);
}
t.Start();
}
}
public void ThreadDone(int threadIdArg)
{
lock (activeThreads)
{
activeThreads.Remove(threadIdArg);
if (activeThreads.Count == 0)
{
// all done
}
}
}
public delegate void SearcherDoneDelegate(int threadIdArg);
public static object locker = new object();
public class Searcher
{
public SearcherDoneDelegate Done { get; set; }
public void getIpRange()
{
Done(Thread.CurrentThread.ManagedThreadId);
}
}
If you have more threads than you want to run at one time, put them into a Queue and peel them off as older threads finish (use the callback).

First, I have to point out that creating 100, 150, 255, etc. threads is probably not a good idea. You might be better off using the ThreadPool or Task class (if using .NET 4.0). Aside from that there are two well established methods for waiting until all threads complete.
Join the thread.
Thread.Join blocks until the target thread finishes.
for (int i = 0; i < threadCount; i++)
{
Searcher src = new Searcher(i, this);
threads[i] = new Thread(new ThreadStart(src.getIpRange));
threads[i].Name = string.Format(i.ToString());
}
foreach (Thread t in threads)
{
t.Start();
}
foreach (Thread t in threads)
{
t.Join();
}
Use a CountdownEvent.
A CountdownEvent waits until its internal count reaches zero. This method is better suited if you want to use the ThreadPool. If you are not using .NET 4.0 you can get a really simple implementation over at Joe Albahari's website.
var finished = new CountdownEvent(1);
for (int i = 0; i < threadCount; i++)
{
finished.AddCount();
Searcher src = new Searcher(i, this);
threads[i] = new Thread(
() =>
{
try
{
src.getIpRange();
}
finally
{
finished.Signal();
}
}
threads[i].Name = string.Format(i.ToString());
}
foreach (Thread t in threads)
{
t.Start();
}
finished.Signal();
finished.WaitOne();

Why can't you use critical section protected single variable to control a number of active threads? Thread function can modify this variable (having entered critical section, of course).