Develop Reference

Deadlock using async Task and SemaphoreSlim

we are running an ASP.NET 6 webapplication and are having strange issues with deadlocks.
The app suddenly freezes after some weeks of operations and it seems that it might be caused by our locking mechanism with the SemaphoreSlim class.
I tried to reproduce the issue with a simple test-project and found something strange.
The following code is simply starting 1000 tasks where each is doing some work (requesting semaphore-handle, waiting for 10 ms and releasing the semaphore).
I expected this code to simply execute one task after another. But it freezes because of a deadlock in the first call of the DoWork method (at await Task.Delay(10)).
Does anyone know why this causes a deadlock? I tried exactly the same code with ThreadPool.QueueUserWorkItem instead of Task.Run and Thread.Sleep instead of Task.Delay and this worked as expected. But as soon as I use the tasks it stops working.
Here is the complete code-snippet:
internal class Program
{
static int timeoutSec = 60;
static SemaphoreSlim semaphore = new SemaphoreSlim(1);
static int numPerIteration = 1000;
static int iteration = 0;
static int doneCounter = numPerIteration;
static int successCount = 0;
static int failedCount = 0;
static Stopwatch sw = new Stopwatch();
static Random rnd = new Random();
static void Main(string[] args)
{
Task.WaitAll(TestUsingTasks());
}
static async Task TestUsingTasks()
{
while (true)
{
var tasks = new List<Task>();
if (doneCounter >= numPerIteration)
{
doneCounter = 0;
if (iteration >= 1)
{
Log($"+++++ FINISHED TASK ITERATION {iteration} - SUCCESS: {successCount} - FAILURES: {failedCount} - Seconds: {sw.Elapsed.TotalSeconds:F1}", ConsoleColor.Magenta);
}
iteration++;
sw.Restart();
for (int i = 0; i < numPerIteration; i++)
{
// Start indepdent tasks to do some work
Task.Run(async () =>
{
if (await DoWork())
{
successCount++;
}
else
{
failedCount++;
}
doneCounter++;
});
}
}
await Task.Delay(10);
}
}
static async Task<bool> DoWork()
{
if (semaphore.Wait(timeoutSec * 1000)) // Request the semaphore to ensure that one 1 task at a time can enter
{
Log($"Got handle for {iteration} within {sw.Elapsed.TotalSeconds:F1}", ConsoleColor.Green);
var totalSec = sw.Elapsed.TotalSeconds;
await Task.Delay(10); // Wait for 10ms to simulate some work => Deadlock seems to happen here
Log($"RELEASING LOCK handle for {iteration} within {sw.Elapsed.TotalSeconds:F1}. WAIT took " + (sw.Elapsed.TotalSeconds - totalSec) + " seconds", ConsoleColor.Gray);
semaphore.Release();
return true;
}
else
{
Log($"ERROR: TASK handle failed for {iteration} within {sw.Elapsed.TotalSeconds:F1} sec", ConsoleColor.Red);
return false;
}
}
static void Log(string message, ConsoleColor color)
{
Console.ForegroundColor = color;
Console.WriteLine(message);
Console.ForegroundColor = ConsoleColor.White;
}
}
Thanks in advance!

But it freezes because of a deadlock in the first call of the DoWork method (at await Task.Delay(10)).
I would argue that it is not deadlock but a thread starvation issue. If you wait long enough you will see that threads will be able to finish the simulation wait from time to time.
The quick fix here is using non-blocking WaitAsync call with await:
static async Task<bool> DoWork()
{
if (await semaphore.WaitAsync(timeoutSec * 1000))
{
...
}
}
Also note:
It is recommended to wrap the code after Wait.. into try-finally block and release the semaphore in the finally.
Incrementing counters in parallel environments better should be done in atomic fashion, for example with Interlocked.Increment.

Await Task.Delay() spend too much time

In C# I have an example:
public async static Task TaskTest(int i)
{
await Task.Delay(1);
Console.WriteLine($"{i}. {DateTime.Now.ToString("HH:mm:ss fff")} " +
$"ThreadId:{Thread.CurrentThread.ManagedThreadId} Start");
int count = 1;
while (true)
{
DoSomeThing(count);
var stopWatch = new Stopwatch();
stopWatch.Start();
await Task.Delay(100);
stopWatch.Stop();
if (stopWatch.Elapsed.TotalMilliseconds > 200)
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine($"Id:{count} Time:{DateTime.Now.ToString("HH:mm:ss fff")} " +
$"ThreadID:{Thread.CurrentThread.ManagedThreadId} Time Delay:{stopWatch.Elapsed.TotalMilliseconds }");
Console.ForegroundColor = ConsoleColor.White;
count++;
}
}
public async static Task DoSomeThing(int index)
{
await Task.Delay(1);
Task.Delay(1000).Wait();
}
private static void Main(string[] args)
{
int i = 1;
while (i < 2)
{
TaskTest(i);
Task.Delay(1).Wait();
i++;
}
Console.ReadKey();
}
Here is my result
Result
Id:8 Time:23:03:59 972 ThreadID:12 Time Delay:582.6348
Id:22 Time:23:04:01 974 ThreadID:14 Time Delay:552.7234000000001
Id:42 Time:23:04:04 967 ThreadID:8 Time Delay:907.3214
I don't know why Task sometimes delay more than 200 milliseconds.
Update:
Thank for all answer.
I update my code to use Thread and Thread.Sleep() and Task.Run(). I increase number of Threads run forever to 500. I tested in 30 minutes and 500 threads never sleep more than 200ms.
Do you think that is bad code?
Please leave a comment!
Thank you so much!
public static void TaskTest(object i)
{
Console.WriteLine($"{i} Start");
int count = 1;
while (true)
{
// Open Task to do work
Task.Run(() => { DoSomeThing(count); });
var stopWatch = new Stopwatch();
stopWatch.Start();
Thread.Sleep(100);
stopWatch.Stop();
if (stopWatch.Elapsed.TotalMilliseconds > 200)
{
Console.WriteLine($"Id:{count} Time:{DateTime.Now.ToString("HH:mm:ss fff")} " +
$"ThreadID:{Thread.CurrentThread.ManagedThreadId} Time Delay:{stopWatch.Elapsed.TotalMilliseconds }");
}
count++;
}
}
public static void DoSomeThing(int index)
{
Thread.Sleep(1000); // Time spent complete work
}
private static void Main(string[] args)
{
int i = 0;
while (i < 500)
{
// Open Thread for TaskTest
Thread tesThread = new Thread(TaskTest);
tesThread.IsBackground = true;
tesThread.Start(i);
i++;
}
Console.WriteLine("Finish init");
Console.ReadKey();
}

Task.Delay, like any other multi-threaded sleep function, yields the thread it's running on back to the system (or in the case of the thread pool, back to the thread pool scheduler), asking to be re-scheduled some time after the amount of time specified.
That is the only guarantee you have, that it will wait at least the amount specified. How long it will actually wait heavily depends on your thread pool load (you're delaying an awful lot of tasks there), general system load (there's thousands of threads at any given point in time to be scheduled on an average computer OS) and on your CPU&OS's capability to schedule threads quickly (in Windows, look at timeBeginPeriod).
Long story short, if precise timing matters to you, don't relinquish your thread.

I am working on implementing job queue. Using TPL for the same. Trying to restrict number of jobs at a time using ThreadPool.SetMaxThreads. no luck

****To Restrict the thread :****
int workerThreads, completionPortThreads;
ThreadPool.GetMaxThreads(out workerThreads, out completionPortThreads);
workerThreads = 2;
ThreadPool.SetMaxThreads(workerThreads, completionPortThreads);
To run the job I tried 2 options
Option 1.
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc),task);
Option 2:
Task runner = new Task(() => taskProcessor.ImportIntoArt(task),TaskCreationOptions.LongRunning|TaskCreationOptions.PreferFairness);
runner.Start();
I expect this code has to pick up first two jobs for processing and 3rd one should go in to the queue. As expected first two jobs will start, however 3rd one will also be picked up for processing.
Any help is highly appreciated.

Use the QueuedTaskScheduler from this package in conjunction with Task.Factory.StartNew method:
var scheduler = new QueuedTaskScheduler(TaskScheduler.Default, 2);
var jobAction = new Action<string>(
jobName =>
{
Console.WriteLine("I am job " + jobName + " and I start at " + DateTime.Now.ToLongTimeString());
Thread.Sleep(10000);
Console.WriteLine("I am job " + jobName + " and I finish at " + DateTime.Now.ToLongTimeString());
});
var jobs = Enumerable
.Range(1, 6)
.Select(num => Task.Factory.StartNew(
() => jobAction("Job" + num),
CancellationToken.None,
TaskCreationOptions.LongRunning,
scheduler))
.ToList();
Task.WhenAll(jobs).Wait();

I know you want to achieve this task using TPL, but as #stuartd has made a comment that we can't do that with threadpool, then you can achieve this task traditional way by creating required number of thread and run them infinitely and observe the collection of a task which of type query.
Please refer below code if you want to achieve the task without using other libraries.
//Declare queue of task.
static Queue<int> taskQueue = new Queue<int>();
static readonly object lockObj = new object();
//Get task to perform.
static int? GetNextTask()
{
lock (lockObj)
{
if (taskQueue.Count > 0)
return taskQueue.Dequeue();
else return null;
}
}
//Add task to queue from different thread.
static void AddTask(int task)
{
lock (lockObj)
{
taskQueue.Enqueue(task);
}
}
static void PerformThreadOperation()
{
//Run infinite for current thread.
while (true)
{
var task = GetNextTask();
//If there is task then perform some action else make thread sleep for some time you can set event to resume thread.
if (task.HasValue)
{
Console.WriteLine("Task Initiate => {0}", task.Value);
Thread.Sleep(4000);
Console.WriteLine("Task Complete => {0}", task.Value);
}
else
{
Console.WriteLine("Task not found, thread is going to be sleep for some moment.");
Console.WriteLine("Thread {0} enter in sleep mode.", Thread.CurrentThread.Name);
Thread.Sleep(5000);
}
}
}
//Create required thread to process task parallely.
static void TestThreadApplication()
{
Thread thread = new Thread(new ThreadStart(PerformThreadOperation));
Thread thread1 = new Thread(PerformThreadOperation);
thread.Start();
thread1.Start();
}
static void Main(string[] args)
{
for (int i = 0; i < 6; i++)
{
taskQueue.Enqueue(i);
}
TestThreadApplication();
Thread.Sleep(20000);
for (int i = 6; i < 10; i++)
{
taskQueue.Enqueue(i);
}
Console.ReadKey();
}

Do I need two queues for producer-consumer pattern?

I have a difficult time to understand the BlockingCollection.
The following code is from an old answer but it used two queues.
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;
namespace Demo
{
class Program
{
static void Main(string[] args)
{
new Program().run();
}
void run()
{
int threadCount = 4;
Task[] workers = new Task[threadCount];
Task.Factory.StartNew(consumer);
for (int i = 0; i < threadCount; ++i)
{
int workerId = i;
Task task = new Task(() => worker(workerId));
workers[i] = task;
task.Start();
}
for (int i = 0; i < 100; ++i)
{
Console.WriteLine("Queueing work item {0}", i);
inputQueue.Add(i);
Thread.Sleep(50);
}
Console.WriteLine("Stopping adding.");
inputQueue.CompleteAdding();
Task.WaitAll(workers);
outputQueue.CompleteAdding();
Console.WriteLine("Done.");
Console.ReadLine();
}
void worker(int workerId)
{
Console.WriteLine("Worker {0} is starting.", workerId);
foreach (var workItem in inputQueue.GetConsumingEnumerable())
{
Console.WriteLine("Worker {0} is processing item {1}", workerId, workItem);
Thread.Sleep(100); // Simulate work.
outputQueue.Add(workItem); // Output completed item.
}
Console.WriteLine("Worker {0} is stopping.", workerId);
}
void consumer()
{
Console.WriteLine("Consumer is starting.");
foreach (var workItem in outputQueue.GetConsumingEnumerable())
{
Console.WriteLine("Consumer is using item {0}", workItem);
Thread.Sleep(25);
}
Console.WriteLine("Consumer is finished.");
}
BlockingCollection<int> inputQueue = new BlockingCollection<int>();
BlockingCollection<int> outputQueue = new BlockingCollection<int>();
}
}
Can I just use one queue? If using .net 4.5(TPL) to replace it, what can I do?

Producer-consumer is a model that defines two actors:
One who creates items - producer
One who processes those items - consumer
There are different ways to organize their interoperation, but the most simple one is Queue - producer enqueues new items to it, consumer dequeues them.
The code example shown above actually contains three actors and two queues:
Producer 1 (in Main)
for (int i = 0; i < 100; ++i)
{
Console.WriteLine("Queueing work item {0}", i);
inputQueue.Add(i);
Thread.Sleep(50);
}
Consumer 1 and Producer 2 at the same time ( worker method):
foreach (var workItem in inputQueue.GetConsumingEnumerable())
{
Console.WriteLine("Worker {0} is processing item {1}", workerId, workItem);
Thread.Sleep(100); // Simulate work.
outputQueue.Add(workItem); // Output completed item.
}
Consumer 2 (consumer method):
foreach (var workItem in outputQueue.GetConsumingEnumerable())
{
Console.WriteLine("Consumer is using item {0}", workItem);
Thread.Sleep(25);
}
This is not producer-consumer - it is Conveyor "pattern" organized upon two sequentially connected producer-consumer systems.
So, the ANSWER:
No, there is no need to use two queues. You can reduce the example to proper one-level producer-consumer just removing the worker method and operating in the consumer method upon inputQueue.
ABOUT THE old answer:
The old question does not deal with producer-consumer - it deals with conveyor model - data flow. Simple producer-consumer does not need any complex multilevel data transitions.

Waiting for all threads to complete, with a timeout

I'm running into a common pattern in the code that I'm writing, where I need to wait for all threads in a group to complete, with a timeout. The timeout is supposed to be the time required for all threads to complete, so simply doing Thread.Join(timeout) for each thread won't work, since the possible timeout is then timeout * numThreads.
Right now I do something like the following:
var threadFinishEvents = new List<EventWaitHandle>();
foreach (DataObject data in dataList)
{
// Create local variables for the thread delegate
var threadFinish = new EventWaitHandle(false, EventResetMode.ManualReset);
threadFinishEvents.Add(threadFinish);
var localData = (DataObject) data.Clone();
var thread = new Thread(
delegate()
{
DoThreadStuff(localData);
threadFinish.Set();
}
);
thread.Start();
}
Mutex.WaitAll(threadFinishEvents.ToArray(), timeout);
However, it seems like there should be a simpler idiom for this sort of thing.

I still think using Join is simpler. Record the expected completion time (as Now+timeout), then, in a loop, do
if(!thread.Join(End-now))
throw new NotFinishedInTime();

With .NET 4.0 I find System.Threading.Tasks a lot easier to work with. Here's spin-wait loop which works reliably for me. It blocks the main thread until all the tasks complete. There's also Task.WaitAll, but that hasn't always worked for me.
for (int i = 0; i < N; i++)
{
tasks[i] = Task.Factory.StartNew(() =>
{
DoThreadStuff(localData);
});
}
while (tasks.Any(t => !t.IsCompleted)) { } //spin wait

This doesn't answer the question (no timeout), but I've made a very simple extension method to wait all threads of a collection:
using System.Collections.Generic;
using System.Threading;
namespace Extensions
{
public static class ThreadExtension
{
public static void WaitAll(this IEnumerable<Thread> threads)
{
if(threads!=null)
{
foreach(Thread thread in threads)
{ thread.Join(); }
}
}
}
}
Then you simply call:
List<Thread> threads=new List<Thread>();
//Add your threads to this collection
threads.WaitAll();

Since the question got bumped I will go ahead and post my solution.
using (var finished = new CountdownEvent(1))
{
for (DataObject data in dataList)
{
finished.AddCount();
var localData = (DataObject)data.Clone();
var thread = new Thread(
delegate()
{
try
{
DoThreadStuff(localData);
threadFinish.Set();
}
finally
{
finished.Signal();
}
}
);
thread.Start();
}
finished.Signal();
finished.Wait(YOUR_TIMEOUT);
}

Off the top of my head, why don't you just Thread.Join(timeout) and remove the time it took to join from the total timeout?
// pseudo-c#:
TimeSpan timeout = timeoutPerThread * threads.Count();
foreach (Thread thread in threads)
{
DateTime start = DateTime.Now;
if (!thread.Join(timeout))
throw new TimeoutException();
timeout -= (DateTime.Now - start);
}
Edit: code is now less pseudo. don't understand why you would mod an answer -2 when the answer you modded +4 is exactly the same, only less detailed.

This may not be an option for you, but if you can use the Parallel Extension for .NET then you could use Tasks instead of raw threads and then use Task.WaitAll() to wait for them to complete.

I read the book C# 4.0: The Complete Reference of Herbert Schildt. The author use join to give a solution :
class MyThread
{
public int Count;
public Thread Thrd;
public MyThread(string name)
{
Count = 0;
Thrd = new Thread(this.Run);
Thrd.Name = name;
Thrd.Start();
}
// Entry point of thread.
void Run()
{
Console.WriteLine(Thrd.Name + " starting.");
do
{
Thread.Sleep(500);
Console.WriteLine("In " + Thrd.Name +
", Count is " + Count);
Count++;
} while (Count < 10);
Console.WriteLine(Thrd.Name + " terminating.");
}
}
// Use Join() to wait for threads to end.
class JoinThreads
{
static void Main()
{
Console.WriteLine("Main thread starting.");
// Construct three threads.
MyThread mt1 = new MyThread("Child #1");
MyThread mt2 = new MyThread("Child #2");
MyThread mt3 = new MyThread("Child #3");
mt1.Thrd.Join();
Console.WriteLine("Child #1 joined.");
mt2.Thrd.Join();
Console.WriteLine("Child #2 joined.");
mt3.Thrd.Join();
Console.WriteLine("Child #3 joined.");
Console.WriteLine("Main thread ending.");
Console.ReadKey();
}
}

I was tying to figure out how to do this but i could not get any answers from google.
I know this is an old thread but here was my solution:
Use the following class:
class ThreadWaiter
{
private int _numThreads = 0;
private int _spinTime;
public ThreadWaiter(int SpinTime)
{
this._spinTime = SpinTime;
}
public void AddThreads(int numThreads)
{
_numThreads += numThreads;
}
public void RemoveThread()
{
if (_numThreads > 0)
{
_numThreads--;
}
}
public void Wait()
{
while (_numThreads != 0)
{
System.Threading.Thread.Sleep(_spinTime);
}
}
}
Call Addthreads(int numThreads) before executing a thread(s).
Call RemoveThread() after each one has completed.
Use Wait() at the point that you want to wait for all the threads to complete
before continuing

Possible solution:
var tasks = dataList
.Select(data => Task.Factory.StartNew(arg => DoThreadStuff(data), TaskContinuationOptions.LongRunning | TaskContinuationOptions.PreferFairness))
.ToArray();
var timeout = TimeSpan.FromMinutes(1);
Task.WaitAll(tasks, timeout);
Assuming dataList is the list of items and each item needs to be processed in a separate thread.

Here is an implementation inspired by Martin v. Löwis's answer:
/// <summary>
/// Blocks the calling thread until all threads terminate, or the specified
/// time elapses. Returns true if all threads terminated in time, or false if
/// at least one thread has not terminated after the specified amount of time
/// elapsed.
/// </summary>
public static bool JoinAll(IEnumerable<Thread> threads, TimeSpan timeout)
{
ArgumentNullException.ThrowIfNull(threads);
if (timeout < TimeSpan.Zero)
throw new ArgumentOutOfRangeException(nameof(timeout));
Stopwatch stopwatch = Stopwatch.StartNew();
foreach (Thread thread in threads)
{
if (!thread.IsAlive) continue;
TimeSpan remaining = timeout - stopwatch.Elapsed;
if (remaining < TimeSpan.Zero) return false;
if (!thread.Join(remaining)) return false;
}
return true;
}
For measuring the remaining time, instead of the DateTime.Now it uses a Stopwatch. The Stopwatch component is not sensitive to system-wide clock adjustments.
Usage example:
bool allTerminated = JoinAll(new[] { thread1, thread2 }, TimeSpan.FromSeconds(10));
The timeout must be a positive or zero TimeSpan. The Timeout.InfiniteTimeSpan constant is not supported.