In my code in the main thread I call a 3rd party API. For each result from the API I call 2 async tasks. Sometimes all works perfectly, sometimes not all async tasks run. I suppose that when the main thread finishes, the garbage collector kills all my other tasks that run in the background. Is there any way to tell garbage collector not to kill the background services when the main thread finishes?
The code is like this:
for (int i = 0; i < 1000; i++)
{
var demo = new AsyncAwaitTest();
demo.DoStuff1(guid);
demo.DoStuff2(guid);
}
public class AsyncAwaitTest
{
public async Task DoStuff1(string guid)
{
await Task.Run(() =>
{
DoSomething1(guid);
});
}
public async Task DoStuff2(string guid)
{
await Task.Run(() =>
{
DoSomething2(guid);
});
}
private static async Task<int> DoSomething1(string guid)
{
// insert in db or something else
return 1;
}
private static async Task<int> DoSomething2(string guid)
{
// insert in db or something else
return 1;
}
Thanks
If you want to wait for all your tasks to finish, you have to collect them and wait for them. Because when your process ends, that normally is the end of everything.
var tasks = List<Task>();
for (int i = 0; i < 1000; i++)
{
var demo = new AsyncAwaitTest();
tasks.Add(demo.DoStuff1(guid));
tasks.Add(demo.DoStuff2(guid));
}
// before your process ends, you need to make sure all tasks ave finished.
Task.WaitAll(tasks.ToArray());
You also have 2 levels of carelessness (meaning you start a task and don't care whether it's done). You need to remove the second one, too:
public async Task DoStuff1(string guid)
{
await DoSomething1(guid);
}
Related
I'm puzzled with this situation, where a class has a method that launches two periodic Tasks and then a property is used to check if both Tasks are still running or not, but the result is unexpected. Here is the code (simplified):
public partial class UdpClientConnector
{
Task localListener;
Task periodicSubscriber;
bool keepWorking = false;
public bool IsRunning
{
get
{
if ((localListener != null) && (periodicSubscriber != null))
{
return (localListener.Status == TaskStatus.Running) &&
(periodicSubscriber.Status == TaskStatus.Running);
}
else
return false;
}
}
public void Start()
{
keepWorking = true;
localListener = new Task(() => LocalListenerWorker());
localListener.Start();
periodicSubscriber = new Task(() => PeriodicSubscriberWorker());
periodicSubscriber.Start();
}
public void Stop()
{
keepWorking = false;
localListener.Wait();
periodicSubscriber.Wait();
}
async void LocalListenerWorker()
{
while (keepWorking)
{
// Do some work and then wait a bit
await Task.Delay(1000);
}
}
async void PeriodicSubscriberWorker()
{
while (keepWorking)
{
// Do some (other) work and then wait a bit
await Task.Delay(1000);
}
}
}
To test this boilerplate I used the following:
UdpClientConnector connector = new UdpClientConnector();
// This assert is successful because the two Tasks are not yet started
Assert.IsTrue(!connector.IsRunning);
// Starts the tasks and wait a bit
Connector.Start();
Task.Delay(2000).Wait();
// This fails
Assert.IsTrue(connector.IsRunning);
When I've tried to debug the test case, I've found that two Tasks are in the RanToCompletion state, which is unexpected due the fact that both tasks are just loops and should not terminate until keepWorking becomes false.
I've tried also to start the Tasks using Task.Factory.StartNew(..) with same results.
What I'm missing? Thank you!
The problem is with how you start the tasks, and indeed the task methods.
localListener = new Task(() => LocalListenerWorker());
That task will complete when LocalListenerWorker returns - which it will do pretty much immediately (when it hits the first await expression). It doesn't wait for the asynchronous operation to actually complete (i.e. the loop to finish).
async void methods should almost never be used - they're basically only there to support event handlers.
I suggest you rewrite your methods to return Task, and then use Task.Run to start them, passing in a method group:
Task.Run(LocalListenerWorker);
...
private async Task LocalListenerWorker()
{
// Body as before
}
The task by Task.Run will only complete when the task returned by LocalListenerWorker completes, which is when the loop body finishes.
Here's a complete demo:
using System;
using System.Threading.Tasks;
class Program
{
static void Main(string[] args)
{
Task task1 = Task.Run(Loop);
// Don't do this normally! It's just as a simple demo
// in a console app...
task1.Wait();
Console.WriteLine("First task done");
Task task2 = new Task(() => Broken());
task2.Start();
// Don't do this normally! It's just as a simple demo
// in a console app...
task2.Wait();
Console.WriteLine("Second task done");
}
static async Task Loop()
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(1000);
Console.WriteLine(i);
}
}
static async void Broken()
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(1000);
Console.WriteLine(i);
}
}
}
The output shows:
0
1
2
3
4
First task done
Second task done
The first task behaves as expected, and only completes when the first async method has really completed. The second task behaves like your current code: it completes as soon as the second async method has returned - which happens almost immediately due to the await.
I have two methods I want to call within a loop. Step1() has to complete before Step2() is called. But in a loop, Step1() can start while Step2() is asynchronously executing. Should I simply wait for the Step2 task, before allowing any other 'Step2' tasks from being executed, as I do in the code below?
public MainViewModel()
{
StartCommand = new RelayCommand(Start);
}
public ICommand StartCommand { get; set; }
private async void Start()
{
await Task.Factory.StartNew(() =>
{
Console.WriteLine($"{DateTime.Now:hh:mm:ss.fff} - Started processing.");
for (int i = 0; i < 10; i++)
{
_counter++;
string result = Step1(i);
_step2Task?.Wait(); //Is this OK to do???
Step2(result).ConfigureAwait(false);
}
_step2Task?.Wait();
Console.WriteLine($"{DateTime.Now:hh:mm:ss.fff} - Finished processing.");
});
}
private string Step1(int i)
{
Thread.Sleep(5000); //simulates time-consuming task
Console.WriteLine($"{DateTime.Now:hh:mm:ss.fff} - Step 1 completed - Iteration {i}.");
return $"Step1Result{i}";
}
private async Task Step2(string result)
{
_step2Task = Task.Run(() =>
{
Thread.Sleep(4000); //simulates time-consuming task
Console.WriteLine($"{DateTime.Now:hh:mm:ss.fff} - Step 2 completed. - {result}");
});
await _step2Task;
}
Don't do any of this stuff; you will risk getting deadlocks all over the place. Also, don't move stuff onto threads unless it is CPU bound.
Start over:
Find every long-running synchronous method that is CPU intensive and write an async wrapper around it. The async wrapper should grab a worker thread, execute the CPU intensive task, and complete when the execution is done. Now you consistently have an abstraction in terms of tasks, not threads.
Move all of your control flow logic onto the UI thread.
Put an await everywhere that you mean "the code that comes after this must not execute until the awaited task is complete".
If we do that, your code gets a lot simpler:
// Return Task, not void
// Name async methods accordingly
private async Task StartAsync()
{
Console.WriteLine($"{DateTime.Now:hh:mm:ss.fff} - Started processing.");
Task task2 = null;
for (int i = 0; i < 10; i++)
{
// We cannot do Step2Async until Step1Async's task
// completes, so await it.
string result = await Step1Async(i);
// We can't run a new Step2Async until the old one is done:
if (task2 != null) {
await task2;
task2 = null;
}
// Now run a new Step2Async:
task2 = Step2Async(result);
// But *do not await it*. We don't care if a new Step1Async
// starts up before Step2Async is done.
}
// Finally, don't complete StartAsync until any pending Step2 is done.
if (task2 != null) {
await task2;
task2 = null;
}
Console.WriteLine($"{DateTime.Now:hh:mm:ss.fff} - Finished processing.");
}
private string Step1(int i)
{
// TODO: CPU intensive work here
}
private async Task<string> Step1Async(int i) {
// TODO: Run CPU-intensive Step1(i) on a worker thread
// return a Task<string> representing that work, that is
// completed when the work is done.
}
private void Step2(string result)
{
// TODO: CPU-intensive work here
}
private async Task Step2Async(string result)
{
// TODO: Again, make a worker thread that runs Step2
// and signals the task when it is complete.
}
Remember, await is the sequencing operation on workflows. It means don't proceed with this workflow until this task is complete; go find some other workflow.
Exercise: How would you write the code to represent the workflow:
Step1 must complete before Step2
Any number of Step2 may be running at the same time
All the Step2 must complete before Start completes
?
Originally I wrote my C# program using Threads and ThreadPooling, but most of the users on here were telling me Async was a better approach for more efficiency. My program sends JSON objects to a server until status code of 200 is returned and then moves on to the next task.
The problem is once one of my Tasks retrieves a status code of 200, it waits for the other Tasks to get 200 code and then move onto the next Task. I want each task to continue it's next task without waiting for other Tasks to finish (or catch up) by receiving a 200 response.
Below is my main class to run my tasks in parallel.
public static void Main (string[] args)
{
var tasks = new List<Task> ();
for (int i = 0; i < 10; i++) {
tasks.Add (getItemAsync (i));
}
Task.WhenAny (tasks);
}
Here is the getItemAsync() method that does the actual sending of information to another server. Before this method works, it needs a key. The problem also lies here, let's say I run 100 tasks, all 100 tasks will wait until every single one has a key.
public static async Task getItemAsync (int i)
{
if (key == "") {
await getProductKey (i).ConfigureAwait(false);
}
Uri url = new Uri ("http://www.website.com/");
var content = new FormUrlEncodedContent (new[] {
...
});
while (!success) {
using (HttpResponseMessage result = await client.PostAsync (url, content)) {
if (result.IsSuccessStatusCode) {
string resultContent = await result.Content.ReadAsStringAsync ().ConfigureAwait(false);
Console.WriteLine(resultContent);
success=true;
}
}
}
await doMoreAsync (i);
}
Here's the function that retrieves the keys, it uses HttpClient and gets parsed.
public static async Task getProductKey (int i)
{
string url = "http://www.website.com/key";
var stream = await client.GetStringAsync (url).ConfigureAwait(false);
var doc = new HtmlDocument ();
doc.LoadHtml (stream);
HtmlNode node = doc.DocumentNode.SelectNodes ("//input[#name='key']") [0];
try {
key = node.Attributes ["value"].Value;
} catch (Exception e) {
Console.WriteLine ("Exception: " + e);
}
}
After every task receives a key and has a 200 status code, it runs doMoreAsync(). I want any individual Task that retrieved the 200 code to run doMoreAsync() without waiting for other tasks to catch up. How do I go about this?
Your Main method is not waiting for a Task to complete, it justs fires off a bunch of async tasks and returns.
If you want to await an async task you can only do that from an async method. Workaround is to kick off an async task from the Main method and wait for its completion using the blocking Task.Wait():
public static void Main(string[] args) {
Task.Run(async () =>
{
var tasks = new List<Task> ();
for (int i = 0; i < 10; i++) {
tasks.Add (getItemAsync (i));
}
var finishedTask = await Task.WhenAny(tasks); // This awaits one task
}).Wait();
}
When you are invoking getItemAsync() from an async method, you can remove the ConfigureAwait(false) as well. ConfigureAwait(false) just makes sure some code does not execute on the UI thread.
If you want to append another task to a task, you can also append it to the previous Task directly by using ContinueWith():
getItemAsync(i).ContinueWith(anotherTask);
Your main problem seems to be that you're sharing the key field across all your concurrently-running asynchronous operations. This will inevitably lead to race hazards. Instead, you should alter your getProductKey method to return each retrieved key as the result of its asynchronous operation:
// ↓ result type of asynchronous operation
public static async Task<string> getProductKey(int i)
{
// retrieval logic here
// return key as result of asynchronous operation
return node.Attributes["value"].Value;
}
Then, you consume it like so:
public static async Task getItemAsync(int i)
{
string key;
try
{
key = await getProductKey(i).ConfigureAwait(false);
}
catch
{
// handle exceptions
}
// use local variable 'key' in further asynchronous operations
}
Finally, in your main logic, use a WaitAll to prevent the console application from terminating before all your tasks complete. Although WaitAll is a blocking call, it is acceptable in this case since you want the main thread to block.
public static void Main (string[] args)
{
var tasks = new List<Task> ();
for (int i = 0; i < 10; i++) {
tasks.Add(getItemAsync(i));
}
Task.WaitAll(tasks.ToArray());
}
I am using Async await with Task.Factory method.
public async Task<JobDto> ProcessJob(JobDto jobTask)
{
try
{
var T = Task.Factory.StartNew(() =>
{
JobWorker jobWorker = new JobWorker();
jobWorker.Execute(jobTask);
});
await T;
}
This method I am calling inside a loop like this
for(int i=0; i < jobList.Count(); i++)
{
tasks[i] = ProcessJob(jobList[i]);
}
What I notice is that new tasks opens up inside Process explorer and they also start working (based on log file). however out of 10 sometimes 8 or sometimes 7 finishes. Rest of them just never come back.
why would that be happening ?
Are they timing out ? Where can I set timeout for my tasks ?
UPDATE
Basically above, I would like each Task to start running as soon as they are called and wait for the response on AWAIT T keyword. I am assuming here that once they finish each of them will come back at Await T and do the next action. I am alraedy seeing this result for 7 out of 10 tasks but 3 of them are not coming back.
Thanks
It is hard to say what the issues is without the rest if the code, but you code can be simplified by making ProcessJob synchronous and then calling Task.Run with it.
public JobDto ProcessJob(JobDto jobTask)
{
JobWorker jobWorker = new JobWorker();
return jobWorker.Execute(jobTask);
}
Start tasks and wait for all tasks to finish. Prefer using Task.Run rather than Task.Factory.StartNew as it provides more favourable defaults for pushing work to the background. See here.
for(int i=0; i < jobList.Count(); i++)
{
tasks[i] = Task.Run(() => ProcessJob(jobList[i]));
}
try
{
await Task.WhenAll(tasks);
}
catch(Exception ex)
{
// handle exception
}
First, let's make a reproducible version of your code. This is NOT the best way to achieve what you are doing, but to show you what is happening in your code!
I'll keep the code almost same as your code, except I'll use simple int rather than your JobDto and on completion of the job Execute() I'll write in a file that we can verify later. Here's the code
public class SomeMainClass
{
public void StartProcessing()
{
var jobList = Enumerable.Range(1, 10).ToArray();
var tasks = new Task[10];
//[1] start 10 jobs, one-by-one
for (int i = 0; i < jobList.Count(); i++)
{
tasks[i] = ProcessJob(jobList[i]);
}
//[4] here we have 10 awaitable Task in tasks
//[5] do all other unrelated operations
Thread.Sleep(1500); //assume it works for 1.5 sec
// Task.WaitAll(tasks); //[6] wait for tasks to complete
// The PROCESS IS COMPLETE here
}
public async Task ProcessJob(int jobTask)
{
try
{
//[2] start job in a ThreadPool, Background thread
var T = Task.Factory.StartNew(() =>
{
JobWorker jobWorker = new JobWorker();
jobWorker.Execute(jobTask);
});
//[3] await here will keep context of calling thread
await T; //... and release the calling thread
}
catch (Exception) { /*handle*/ }
}
}
public class JobWorker
{
static object locker = new object();
const string _file = #"C:\YourDirectory\out.txt";
public void Execute(int jobTask) //on complete, writes in file
{
Thread.Sleep(500); //let's assume does something for 0.5 sec
lock(locker)
{
File.AppendAllText(_file,
Environment.NewLine + "Writing the value-" + jobTask);
}
}
}
After running just the StartProcessing(), this is what I get in the file
Writing the value-4
Writing the value-2
Writing the value-3
Writing the value-1
Writing the value-6
Writing the value-7
Writing the value-8
Writing the value-5
So, 8/10 jobs has completed. Obviously, every time you run this, the number and order might change. But, the point is, all the jobs did not complete!
Now, if I un-comment the step [6] Task.WaitAll(tasks);, this is what I get in my file
Writing the value-2
Writing the value-3
Writing the value-4
Writing the value-1
Writing the value-5
Writing the value-7
Writing the value-8
Writing the value-6
Writing the value-9
Writing the value-10
So, all my jobs completed here!
Why the code is behaving like this, is already explained in the code-comments. The main thing to note is, your tasks run in ThreadPool based Background threads. So, if you do not wait for them, they will be killed when the MAIN process ends and the main thread exits!!
If you still don't want to await the tasks there, you can return the list of tasks from this first method and await the tasks at the very end of the process, something like this
public Task[] StartProcessing()
{
...
for (int i = 0; i < jobList.Count(); i++)
{
tasks[i] = ProcessJob(jobList[i]);
}
...
return tasks;
}
//in the MAIN METHOD of your application/process
var tasks = new SomeMainClass().StartProcessing();
// do all other stuffs here, and just at the end of process
Task.WaitAll(tasks);
Hope this clears all confusion.
It's possible your code is swallowing exceptions. I would add a ContineWith call to the end of the part of the code that starts the new task. Something like this untested code:
var T = Task.Factory.StartNew(() =>
{
JobWorker jobWorker = new JobWorker();
jobWorker.Execute(jobTask);
}).ContinueWith(tsk =>
{
var flattenedException = tsk.Exception.Flatten();
Console.Log("Exception! " + flattenedException);
return true;
});
},TaskContinuationOptions.OnlyOnFaulted); //Only call if task is faulted
Another possibility is that something in one of the tasks is timing out (like you mentioned) or deadlocking. To track down whether a timeout (or maybe deadlock) is the root cause, you could add some timeout logic (as described in this SO answer):
int timeout = 1000; //set to something much greater than the time it should take your task to complete (at least for testing)
var task = TheMethodWhichWrapsYourAsyncLogic(cancellationToken);
if (await Task.WhenAny(task, Task.Delay(timeout, cancellationToken)) == task)
{
// Task completed within timeout.
// Consider that the task may have faulted or been canceled.
// We re-await the task so that any exceptions/cancellation is rethrown.
await task;
}
else
{
// timeout/cancellation logic
}
Check out the documentation on exception handling in the TPL on MSDN.
In my Asp.Net MVC 5 project I have a ~3 minute task that I pass to Task.Factory.StartNew().
I would like to pause the task from within the task if there is a validation issue in one of the steps of my code running in the task. I don't want to delay it async because the rest of the task will continue to run, which can't happen.
Could I use thread.sleep() without any repercussions since I'm within a task? I read that I may have to use TaskScheduler.Default to have the Task.Factory create a new thread for each task.
I'm using a PauseToken similar to a CancellationToken so I'll be able to resume the task or cancel this task based on user input.
Multithreading really scares me, and I don't want to overlook something.
Here is an example of the Thread.Sleep implementation:
public void WaitIfPaused(PauseToken pauseToken, CancellationToken cancellationToken, IProgressBar progressBar)
{
//TODO: If paused for an hour notify user via noty and abort the task to keep it from completing by cancellation via cancellationToken.
//wait for 1 hour
for (int i = 0; i < 3600; i++)
{
ThrowExceptionIfCancelled(cancellationToken, progressBar);
if (pauseToken.IsPaused)
{
Thread.Sleep(1000);
}
else
{
break;
}
}
}
PauseToken: http://blogs.msdn.com/b/pfxteam/archive/2013/01/13/cooperatively-pausing-async-methods.aspx
Requested: Implementation of task structure in shared code library.
public void StartTask(params object[] data)
{
//throw an exception if no ITask was found
if (_taskToRun == null)
throw new Exception("Task cannot be null");
//set up task cancellation
CancellationTokenSource = new CancellationTokenSource();
var cancellationToken = CancellationTokenSource.Token;
//set up task pausing
PauseTokenSource = new PauseTokenSource();
var pauseToken = PauseTokenSource.Token;
//start a new task using the Task that was set
_task = Task.Factory.StartNew(() => _taskToRun.Execute(cancellationToken, pauseToken, data), cancellationToken);
}
My Execute method that is invoked by _taskToRun.Execute:
Public override void Execute(CancellationToken cancellationToken, PauseToken pauseToken, params object[] data)
{
var managerList = (List<IFileManager>) data[0];
var instr = (List<InstructionSet>) data[1];
ProcessInstructions(managerList, instr, cancellationToken, pauseToken);
}
Update due to comments:
Code example: 3 instructions
For(var instruction in instructions)
{
instruction.Execute();
}
In my execute method I run into a scenario for pause and call WaitWhilePausedAsync from within the execute. It will continue to execute the other two instructions, but pause the only the current instructions execute method.
Edit: By awaiting instruction.Execute() it will wait until instruction.Execute() completes or is unpaused.
Final Edit:
I was able to resolve the issue by awaiting the Execute method and making it async like Servy and I3arnon suggested.
Final Code Sample:
foreach(var instruction in instructions)
{
try
{
await instruction.Execute(pauseToken);
}
catch(InvalidOperationException)
{
pauseTokenSource.IsPaused = true;
//ask if user wants to cancel or resume.
}
}
//Simplified
public async Task<bool> Execute(PauseToken pauseToken)
{
await pauseToken.WaitWhilePausedAsync();
//do work
}
You can safely use Thread.Sleep. The only drawback is that the thread would be wasted blocking synchronously.
You should be using await Task.Delay(1000) instead. The code after that line would not execute until the wait is complete, but you won't be wasting a thread in the meantime:
public async Task WaitIfPausedAsync(PauseToken pauseToken, CancellationToken cancellationToken, IProgressBar progressBar)
{
for (int i = 0; i < 3600; i++)
{
ThrowExceptionIfCancelled(cancellationToken, progressBar);
if (pauseToken.IsPaused)
{
await Task.Delay(1000)
}
else
{
break;
}
}
}
Edit: I was unaware of PauseToken.WaitWhilePausedAsync. You should definitly use that instead of replicating that yourself with polling over PauseToken.IsPaused