I am adding messages to a threading channel and reading from this channel constantly. I noticed that if I do not add a Task.Delay then all of the messages are not processed. The program will exit with maybe 10 messages processed when it should be 1000.
Adding a Task.Delay after each write seems hacky. Is there a better way to read all messages in the channel without adding a Task.Delay after each write?
Is there something wrong with my StartListener() method?
internal class Program
{
static List<Task> taskList = new List<Task>();
private static Channel<string> messageList = Channel.CreateUnbounded<string>();
static int midpointCount = 0;
static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
sw.Start();
Task.WhenAll(Task.Run(() => StartListener()));
for (int i = 0; i < 10; i++)
{
int task = i;
taskList.Add(Task.Run(() => StartJob(task)));
}
Task.WaitAll(taskList.ToArray());
sw.Stop();
Console.WriteLine("Total Messages Processed: {0} in time {1} MessageListCount {2}", midpointCount, sw.Elapsed, messageList.Reader.Count);
}
private static async Task<string> StartListener()
{
var cancellationtoken = new CancellationTokenSource(TimeSpan.FromMinutes(60)).Token;
await foreach (var msg in messageList.Reader.ReadAllAsync(cancellationtoken))
{
if (!string.IsNullOrEmpty(msg))
{
Console.WriteLine(msg);
Interlocked.Increment(ref midpointCount);
}
}
return "Finished";
}
private static async Task<string> StartJob(int TaskNum)
{
Random rnd = new Random();
for (int i = 0; i < 100; i++)
{
var cancellationtoken = new CancellationTokenSource(TimeSpan.FromMinutes(60)).Token;
try
{
var message = string.Format("TaskNum {0} Message added #{1}", TaskNum, rnd.Next(1, 3000));
await messageList.Writer.WriteAsync(message);
await Task.Delay(50); //<--- Here it seems it will only read all messages with a delay involved.
}
catch (OperationCanceledException)
{
// ignored
}
}
return "Finished";
}
}
Task.WhenAll(Task.Run(() => StartListener()));
StartListener returns a Task. You wrap that in Task.Run, starting another thread to run that task. You then pass than task to the Task.WhenAll method, which returns a Task that you promptly throw away.
The only tasks you add to the taskList variable are the StartJob tasks. Your Main method will finish as soon as all of the StartJob tasks have finished. It will not wait for the StartListener task to finish.
Change your code to wait for the listener to finish.
static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
sw.Start();
taskList.Add(Task.Run(() => StartListener()));
for (int i = 0; i < 10; i++)
{
int task = i;
taskList.Add(Task.Run(() => StartJob(task)));
}
Task.WaitAll(taskList.ToArray());
sw.Stop();
Console.WriteLine("Total Messages Processed: {0} in time {1} MessageListCount {2}",
midpointCount, sw.Elapsed, messageList.Reader.Count);
}
Related
we are running an ASP.NET 6 webapplication and are having strange issues with deadlocks.
The app suddenly freezes after some weeks of operations and it seems that it might be caused by our locking mechanism with the SemaphoreSlim class.
I tried to reproduce the issue with a simple test-project and found something strange.
The following code is simply starting 1000 tasks where each is doing some work (requesting semaphore-handle, waiting for 10 ms and releasing the semaphore).
I expected this code to simply execute one task after another. But it freezes because of a deadlock in the first call of the DoWork method (at await Task.Delay(10)).
Does anyone know why this causes a deadlock? I tried exactly the same code with ThreadPool.QueueUserWorkItem instead of Task.Run and Thread.Sleep instead of Task.Delay and this worked as expected. But as soon as I use the tasks it stops working.
Here is the complete code-snippet:
internal class Program
{
static int timeoutSec = 60;
static SemaphoreSlim semaphore = new SemaphoreSlim(1);
static int numPerIteration = 1000;
static int iteration = 0;
static int doneCounter = numPerIteration;
static int successCount = 0;
static int failedCount = 0;
static Stopwatch sw = new Stopwatch();
static Random rnd = new Random();
static void Main(string[] args)
{
Task.WaitAll(TestUsingTasks());
}
static async Task TestUsingTasks()
{
while (true)
{
var tasks = new List<Task>();
if (doneCounter >= numPerIteration)
{
doneCounter = 0;
if (iteration >= 1)
{
Log($"+++++ FINISHED TASK ITERATION {iteration} - SUCCESS: {successCount} - FAILURES: {failedCount} - Seconds: {sw.Elapsed.TotalSeconds:F1}", ConsoleColor.Magenta);
}
iteration++;
sw.Restart();
for (int i = 0; i < numPerIteration; i++)
{
// Start indepdent tasks to do some work
Task.Run(async () =>
{
if (await DoWork())
{
successCount++;
}
else
{
failedCount++;
}
doneCounter++;
});
}
}
await Task.Delay(10);
}
}
static async Task<bool> DoWork()
{
if (semaphore.Wait(timeoutSec * 1000)) // Request the semaphore to ensure that one 1 task at a time can enter
{
Log($"Got handle for {iteration} within {sw.Elapsed.TotalSeconds:F1}", ConsoleColor.Green);
var totalSec = sw.Elapsed.TotalSeconds;
await Task.Delay(10); // Wait for 10ms to simulate some work => Deadlock seems to happen here
Log($"RELEASING LOCK handle for {iteration} within {sw.Elapsed.TotalSeconds:F1}. WAIT took " + (sw.Elapsed.TotalSeconds - totalSec) + " seconds", ConsoleColor.Gray);
semaphore.Release();
return true;
}
else
{
Log($"ERROR: TASK handle failed for {iteration} within {sw.Elapsed.TotalSeconds:F1} sec", ConsoleColor.Red);
return false;
}
}
static void Log(string message, ConsoleColor color)
{
Console.ForegroundColor = color;
Console.WriteLine(message);
Console.ForegroundColor = ConsoleColor.White;
}
}
Thanks in advance!
But it freezes because of a deadlock in the first call of the DoWork method (at await Task.Delay(10)).
I would argue that it is not deadlock but a thread starvation issue. If you wait long enough you will see that threads will be able to finish the simulation wait from time to time.
The quick fix here is using non-blocking WaitAsync call with await:
static async Task<bool> DoWork()
{
if (await semaphore.WaitAsync(timeoutSec * 1000))
{
...
}
}
Also note:
It is recommended to wrap the code after Wait.. into try-finally block and release the semaphore in the finally.
Incrementing counters in parallel environments better should be done in atomic fashion, for example with Interlocked.Increment.
So, I have a super simple application, but as I am testing this out it is only writing to the console from the method DoWork(). I am unsure why that is, but I am fairly sure it has to do with the fact that it is async code. Any ideas, why it only writes from method DoWork()?
class Program
{
static void Main(string[] args)
{
MainAsync().Wait();
System.Threading.Thread.Sleep(50000);
}
static async Task MainAsync()
{
Console.WriteLine("Hello World!");
for (int i = 0; i < 300; i++)
{
List<Task> myWork = new List<Task>();
myWork.Add(DoWork(i));
if (myWork.Count == 50)
{
await Task.WhenAll(myWork);
Console.WriteLine("before delay");
await Task.Delay(1000);
Console.WriteLine("after delay");
myWork.Clear();
Console.WriteLine("List cleared.");
}
}
}
public static async Task DoWork(int i)
{
await Task.Delay(0);
Console.WriteLine("Run: " + i);
}
}
You're creating a new List for each iteration of the loop...it will only ever have one thing in it.
Declare the List outside of the loop.
Change:
for (int i = 0; i < 300; i++)
{
List<Task> myWork = new List<Task>();
To:
List<Task> myWork = new List<Task>();
for (int i = 0; i < 300; i++)
{
I have some code that runs thousands of URLs through a third party library. Occasionally the method in the library hangs which takes up a thread. After a while all threads are taken up by processes doing nothing and it grinds to a halt.
I am using a SemaphoreSlim to control adding new threads so I can have an optimal number of tasks running. I need a way to identify tasks that have been running too long and then to kill them but also release a thread from the SemaphoreSlim so a new task can be created.
I am struggling with the approach here so I made some test code that immitates what I am doing. It create tasks that have a 10% chance of hanging so very quickly all threads have hung.
How should I be checking for these and killing them off?
Here is the code:
class Program
{
public static SemaphoreSlim semaphore;
public static List<Task> taskList;
static void Main(string[] args)
{
List<string> urlList = new List<string>();
Console.WriteLine("Generating list");
for (int i = 0; i < 1000; i++)
{
//adding random strings to simulate a large list of URLs to process
urlList.Add(Path.GetRandomFileName());
}
Console.WriteLine("Queueing tasks");
semaphore = new SemaphoreSlim(10, 10);
Task.Run(() => QueueTasks(urlList));
Console.ReadLine();
}
static void QueueTasks(List<string> urlList)
{
taskList = new List<Task>();
foreach (var url in urlList)
{
Console.WriteLine("{0} tasks can enter the semaphore.",
semaphore.CurrentCount);
semaphore.Wait();
taskList.Add(DoTheThing(url));
}
}
static async Task DoTheThing(string url)
{
Random rand = new Random();
// simulate the IO process
await Task.Delay(rand.Next(2000, 10000));
// add a 10% chance that the thread will hang simulating what happens occasionally with http request
int chance = rand.Next(1, 100);
if (chance <= 10)
{
while (true)
{
await Task.Delay(1000000);
}
}
semaphore.Release();
Console.WriteLine(url);
}
}
As people have already pointed out, Aborting threads in general is bad and there is no guaranteed way of doing it in C#. Using a separate process to do the work and then kill it is a slightly better idea than attempting Thread.Abort; but still not the best way to go. Ideally, you want co-operative threads/processes, which use IPC to decide when to bail out themselves. This way the cleanup is done properly.
With all that said, you can use code like below to do what you intend to do. I have written it assuming your task will be done in a thread. With slight changes, you can use the same logic to do your task in a process
The code is by no means bullet-proof and is meant to be illustrative. The concurrent code is not really tested well. Locks are held for longer than needed and some places I am not locking (like the Log function)
class TaskInfo {
public Thread Task;
public DateTime StartTime;
public TaskInfo(ParameterizedThreadStart startInfo, object startArg) {
Task = new Thread(startInfo);
Task.Start(startArg);
StartTime = DateTime.Now;
}
}
class Program {
const int MAX_THREADS = 1;
const int TASK_TIMEOUT = 6; // in seconds
const int CLEANUP_INTERVAL = TASK_TIMEOUT; // in seconds
public static SemaphoreSlim semaphore;
public static List<TaskInfo> TaskList;
public static object TaskListLock = new object();
public static Timer CleanupTimer;
static void Main(string[] args) {
List<string> urlList = new List<string>();
Log("Generating list");
for (int i = 0; i < 2; i++) {
//adding random strings to simulate a large list of URLs to process
urlList.Add(Path.GetRandomFileName());
}
Log("Queueing tasks");
semaphore = new SemaphoreSlim(MAX_THREADS, MAX_THREADS);
Task.Run(() => QueueTasks(urlList));
CleanupTimer = new Timer(CleanupTasks, null, CLEANUP_INTERVAL * 1000, CLEANUP_INTERVAL * 1000);
Console.ReadLine();
}
// TODO: Guard against re-entrancy
static void CleanupTasks(object state) {
Log("CleanupTasks started");
lock (TaskListLock) {
var now = DateTime.Now;
int n = TaskList.Count;
for (int i = n - 1; i >= 0; --i) {
var task = TaskList[i];
Log($"Checking task with ID {task.Task.ManagedThreadId}");
// kill processes running for longer than anticipated
if (task.Task.IsAlive && now.Subtract(task.StartTime).TotalSeconds >= TASK_TIMEOUT) {
Log("Cleaning up hung task");
task.Task.Abort();
}
// remove task if it is not alive
if (!task.Task.IsAlive) {
Log("Removing dead task from list");
TaskList.RemoveAt(i);
continue;
}
}
if (TaskList.Count == 0) {
Log("Disposing cleanup thread");
CleanupTimer.Dispose();
}
}
Log("CleanupTasks done");
}
static void QueueTasks(List<string> urlList) {
TaskList = new List<TaskInfo>();
foreach (var url in urlList) {
Log($"Trying to schedule url = {url}");
semaphore.Wait();
Log("Semaphore acquired");
ParameterizedThreadStart taskRoutine = obj => {
try {
DoTheThing((string)obj);
} finally {
Log("Releasing semaphore");
semaphore.Release();
}
};
var task = new TaskInfo(taskRoutine, url);
lock (TaskListLock)
TaskList.Add(task);
}
Log("All tasks queued");
}
// simulate all processes get hung
static void DoTheThing(string url) {
while (true)
Thread.Sleep(5000);
}
static void Log(string msg) {
Console.WriteLine("{0:HH:mm:ss.fff} Thread {1,2} {2}", DateTime.Now, Thread.CurrentThread.ManagedThreadId.ToString(), msg);
}
}
Im new with c# (and with english language too), i try do some work in background parallel. I have a list with "MyClass" objects. Each has a "DoWork()" function, i put this functions in a List then i'll run all. I have 2 questions: 1, Is it all wrong? 2, when the first answere is "no", then is it possible to set a max running time to functions (when a "DoWork()" take more than 600ms, then i want to stop it, but the others no.)? Sorry for my bad english!
Here is my code:
public class MyClass
{
static int nextId;
public int id;
public int x;
public MyClass()
{
id = Interlocked.Increment(ref nextId);
x = (id % 4 + 1) * 250;
}
public void DoWork()
{
Console.Write("start: {0}", this.id);
Thread.Sleep(x); //this simulate the work
Console.WriteLine("end: {0}", this.id);
}
}
in main:
for (int i = 0; i < db; i++)
{
xy_list.Add(new MyClass());
}
List<Task> tasks3 = new List<Task>();
foreach (var item in xy_list)
{
Task work = new Task(() => item.DoWork());
tasks3.Add(work);
}
Parallel.ForEach(tasks3, task =>
{
task.Start();
});
You will need to do some work here yourself. Cancellation of tasks is a cooperative process (see Cancellation in Managed Threads for more information).
To be able to cancel your background tasks and have them time-out, you will need to create a CancellationTokenSource that you provide with a timeout. This token source can be used to provide cancellation tokens to the tasks that you are starting, and you also need to pass them on to the DoWork method, that should check if cancellation is requested.
If you constructed the cancelation token source with a timeout, it will signal all connected cancellation tokens to cancel when the timeout has expired.
Using this on your code example, would make it look something like this:
public class MyClass
{
static int nextId;
public int id;
public int x;
public MyClass()
{
id = Interlocked.Increment(ref nextId);
x = (id % 4 + 1) * 250;
}
public void DoWork(CancellationToken cancelToken)
{
bool is_canceled = false;
while (!cancelToken.IsCancellationRequested && cycle < 5)
{
try
{
Console.WriteLine("Task {0} waiting, tid = {1}", id, Thread.CurrentThread.ManagedThreadId);
Task.Delay(x / 5, cancelToken).Wait(); // don't do Thread.Sleep()
Console.WriteLine("Task {0} waking up, tid = {1}", id, Thread.CurrentThread.ManagedThreadId);
}
catch (AggregateException ex)
{
if (ex.InnerExceptions.Any(x => typeof(OperationCanceledException).IsAssignableFrom(x.GetType())))
{
Console.WriteLine("Task {0} canceled, tid = {1}", id, Thread.CurrentThread.ManagedThreadId);
is_canceled = true;
break;
}
throw;
}
catch (OperationCanceledException)
{
Console.WriteLine("Task {0} canceled, tid = {1}", id, Thread.CurrentThread.ManagedThreadId);
is_canceled = true;
}
cycle++;
}
is_canceled |= cycle < 5;
Console.WriteLine("{0} {1}, tid = {2}", this.id, is_canceled ? "canceled" : "completed", Thread.CurrentThread.ManagedThreadId);
}
}
And in your "main":
for (int i = 0; i < db; i++)
{
xy_list.Add(new MyClass());
}
// set cancellation timeout for 600 ms.
var cancelSource = new CancellationTokenSource(TimeSpan.FromMilliseconds(600));
// start all tasks
List<Task> tasks3 = new List<Task>();
foreach (var item in xy_list)
tasks3.Add(Task.Run(() => item.DoWork(cancelSource.Token), cancelSource.Token));
// Wait for all tasks to be finished.
Task.WaitAll(tasks3.ToArray());
I got confused why when I used task the time of stopwatch will display but when I used thread it will not. What's wrong with my code? Am i missing something?
static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
sw.Start();
//if I used this sw.Elapsed will display
//Task t1 = Task.Factory.StartNew(runTask1);
//Task t2 = Task.Factory.StartNew(runTask2);
//Task.WaitAll(t1, t2);
//if I used this sw.Elapsed will not display
//Thread t1 = new Thread(runTask1);
//Thread t2 = new Thread(runTask2);
//t1.Start();
//t2.Start();
sw.Stop();
Console.WriteLine(sw.Elapsed);
Console.ReadLine();
}
public static void runTask1()
{
for (int x = 1; x <= 5000; x++)
{
Console.WriteLine("Run task tester 1");
}
}
public static void runTask2()
{
for (int x = 1; x <= 5000; x++)
{
Console.WriteLine("Run task tester 2");
}
}
When you're using tasks, you wait for them to complete their work before stopping the stopwatch and displaying the time. When you're using threads you don't wait for them to finish before displaying the result therefore it is printed at the top above the text from the threads.
You want to wait for the threads to finish as well:
static void Main(string[] args)
{
Stopwatch sw = new Stopwatch();
sw.Start();
// add Thread.Join at the end
Thread t1 = new Thread(runTask1);
Thread t2 = new Thread(runTask2);
t1.Start();
t2.Start();
t1.Join();
t2.Join();
sw.Stop();
Console.WriteLine(sw.Elapsed);
Console.ReadLine();
}