Develop Reference

Return data from long running Task on demand

I want to create a Task, which may run for many minutes, collecting data via an API call to another system. At some point in the future I need to stop the task and return the collected data. This future point is unknown at the time of starting the task.
I have read many question about returning data from tasks, but I can't find any that answer this scenario. I may be missing a trick, but all of the examples actually seem to wait in the man thread for the task to finish before continuing. This seems counter-intuitive, surely the purpose of a task is to hand off an activity whilst continuing with other activities in your main thread?
Here is one of those many examples, taken from DotNetPearls..
namespace TaskBasedAsynchronousProgramming
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine($"Main Thread Started");
Task<double> task1 = Task.Run(() =>
{
return CalculateSum(10);
});
Console.WriteLine($"Sum is: {task1.Result}");
Console.WriteLine($"Main Thread Completed");
Console.ReadKey();
}
static double CalculateSum(int num)
{
double sum = 0;
for (int count = 1; count <= num; count++)
{
sum += count;
}
return sum;
}
}
}
Is it possible to do what I need, and have a long-running task running in parallel, stop it and return the data at an arbitrary future point?

Here is a sample application how you can do that:
static double partialResult = -1;
static void Main()
{
CancellationTokenSource calculationEndSignal = new(TimeSpan.FromSeconds(3));
Task meaningOfLife = Task.Run(() =>
GetTheMeaningOfLife(calculationEndSignal.Token),
calculationEndSignal.Token);
calculationEndSignal.Token.Register(() => Console.WriteLine(partialResult));
Console.ReadLine();
}
static async Task GetTheMeaningOfLife(CancellationToken cancellationToken)
{
foreach (var semiResult in Enumerable.Range(1, 42))
{
partialResult = semiResult;
cancellationToken.ThrowIfCancellationRequested();
await Task.Delay(1000);
}
}
partialResult is a shared variable between the two threads
The worker thread (GetTheMeaningOfLife) only writes it
The main thread (Main) only reads it
The read operation is performed only after the Task has been cancelled
calculationEndSignal is used to cancel the long-running operation
I've have specified a timeout, but you can call the Cancel method if you want
meaningOfLife is the Task which represents the long-running operation call
I have passed the CancellationToken to the GetTheMeaningOfLife and to the Task.Run as well
For this very simple example the Task.Run should not need to receive the token but it is generally a good practice to pass there as well
Register is receiving a callback which should be called after the token is cancelled
ReadLine can be any other computation
I've used ReadLine to keep the application running
GetTheMeaningOfLife simply increments the partialResult shared variable
either until it reaches the meaning of life
or until it is cancelled

Here is one approach. It features a CancellationTokenSource that is used as a stopping mechanism, instead of its normal usage as a cancellation mechanism. That's because you want to get the partial results, and a canceled Task does not propagate results:
CancellationTokenSource stoppingTokenSource = new();
Task<List<int>> longRunningTask = Task.Run(() =>
{
List<int> list = new();
for (int i = 1; i <= 60; i++)
{
if (stoppingTokenSource.IsCancellationRequested) break;
// Simulate a synchronous operation that has 1 second duration.
Thread.Sleep(1000);
list.Add(i);
}
return list;
});
Then, somewhere else in your program, you can send a stopping signal to the task, and then await asynchronously until the task acknowledges the signal and completes successfully. The await will also propagate the partial results:
stoppingTokenSource.Cancel();
List<int> partialResults = await longRunningTask;
Or, if you are not in an asynchronous workflow, you can wait synchronously until the partial results are available:
stoppingTokenSource.Cancel();
List<int> partialResults = longRunningTask.Result;

Recursively call a method with the same thread

I have the following method:
public async Task ScrapeObjects(int page = 1)
{
try
{
while (!isObjectSearchCompleted)
{
..do calls..
}
}
catch (HttpRequestException ex)
{
Thread.Sleep(TimeSpan.FromSeconds(60));
ScrapeObjects(page);
Log.Fatal(ex, ex.Message);
}
}
I call this long running method async and I don't wait for it to finish. Thing is that an exception my occur and in that case I want to handle it. But then I want to start from where I left and with the same thread. At the current state a new thread gets used when I recursively call the method after handling the exception. I would like to keep using the same thread. Is there a way to do so? Thank you!

You probably need to move the try/catch block inside the while loop, and add a counter with the errors occurred, to bail out in case of continuous faulted attempts.
public async Task ScrapeObjects()
{
int failedCount = 0;
int page = 1;
while (!isObjectSearchCompleted)
{
try
{
//..do calls..
}
catch (HttpRequestException ex)
{
failedCount++;
if (failedCount < 3)
{
Log.Info(ex, ex.Message);
await Task.Delay(TimeSpan.FromSeconds(60));
}
else
{
Log.Fatal(ex, ex.Message);
throw; // or return;
}
}
}
}
As a side note it is generally better to await Task.Delay instead of Thread.Sleep inside asynchronous methods, to avoid blocking a thread without a reason.

One simple question before you read the long answer below:
Why you need the same thread? Are you accessing thread static / contextual data?
If yes, there will be ways to solve that easily than limiting your tasks to run on the same thread.
How to limit tasks to run on a single thread
As long as you use async calls on the default synchronization context, and as soon as the code is resumed from an await, it is possible that the thread can change after an await. This is because the default context schedules tasks to the next available thread in the thread pool. Like in the below case, before can be different from after:
public async Task ScrapeObjects(int page = 1)
{
var before = Thread.CurrentThread.ManagedThreadId;
await Task.Delay(1000);
var after = Thread.CurrentThread.ManagedThreadId;
}
The only reliable way to guarantee that your code could come back on the same thread is to schedule your async code onto a single threaded synchronization context:
class SingleThreadSynchronizationContext : SynchronizationContext
{
private readonly BlockingCollection<Action> _actions = new BlockingCollection<Action>();
private readonly Thread _theThread;
public SingleThreadSynchronizationContext()
{
_theThread = new Thread(DoWork);
_theThread.IsBackground = true;
_theThread.Start();
}
public override void Send(SendOrPostCallback d, object state)
{
// Send requires run the delegate immediately.
d(state);
}
public override void Post(SendOrPostCallback d, object state)
{
// Schedule the action by adding to blocking collection.
_actions.Add(() => d(state));
}
private void DoWork()
{
// Keep picking up actions to run from the collection.
while (!_actions.IsAddingCompleted)
{
try
{
var action = _actions.Take();
action();
}
catch (InvalidOperationException)
{
break;
}
}
}
}
And you need to schedule ScrapeObjects to the custom context:
SynchronizationContext.SetSynchronizationContext(new SingleThreadSynchronizationContext());
await Task.Factory.StartNew(
() => ScrapeObjects(),
CancellationToken.None,
TaskCreationOptions.DenyChildAttach | TaskCreationOptions.LongRunning,
TaskScheduler.FromCurrentSynchronizationContext()
).Unwrap();
By doing that, all your async code shall be scheduled to the same context, and run by the thread on that context.
However
This is typically dangerous, as you suddenly lose the ability to use the thread pool. If you block the thread, the entire async operation is blocked, meaning you will have deadlocks.

Why does PauseToken break the task?

I'm trying to understand Microsoft.VisualStudio.Workspace.PauseToken.
When I have called PauseTokenSource.Pause(), the method is paused and the task is completed, and I can see "Completed" message.
After that I have called PauseTokenSource.Resume(), and the task is continued.
But I have no message when the process has finished.
Two questions:
Why does this happen?
How can I fix it?
I want PauseToken to not break the task during the work. I don't want to get the message after pause. I want to get message at true end of process.
public PauseToken PauseToken;
private async Task DoDoneAsync()
{
FlagCommandDone = true;
try
{
task = Task.Factory.StartNew(async () =>
{
double i = 0;
while (i < 3)
{
i++;
await PauseToken.WaitWhilePausedAsync();
Thread.Sleep(5000);
}
}
);
await task;
}
catch (Exception e)
{
isError = true;
message = e.Message;
}
if (!isError) message = task.IsCompleted ? "Completed" : "Canceled";
MessageBox.Show(message);
FlagCommandDone = false;
}
EDIT:
task = Task.Factory.StartNew(async () =>
{
double i = 0;
while (i < 3)
{
i++;
CurrentStatus = i.ToString();
var a = (int)(double)((i / 3.0) * 100.0);
MessageBox.Show(PauseToken.IsPaused.ToString());
await PauseToken.WaitWhilePausedAsync();
CancellationToken.ThrowIfCancellationRequested();
Thread.Sleep(5000);
CurrentProgress = a;
}
}
, CancellationToken);

The problem with your code is that the task created by StartNew() is deemed complete when the invoked method returns. And of course, it being an async method with an await, that method returns as soon as the first await is executed on an incomplete task.
So, your DoDoneAsync() method is in fact completing before the whole you thought you were starting completes. It essentially is waiting only until your StartNew() method reaches its first await.
Without a good Minimal, Complete, and Verifiable code example that reliably reproduces your problem, it's impossible to know for sure what will fix your program. But, I think the main thing is to stop trying to mix blocking code with async code. Instead, just factor the anonymous method you have invoked by StartNew() out into its own named method, make it properly async, and just call it directly.
For example:
private async Task DoDoneAsync()
{
FlagCommandDone = true;
try
{
await DoDoneLoopAsync();
}
catch (Exception e)
{
isError = true;
message = e.Message;
}
if (!isError) message = task.IsCompleted ? "Completed" : "Canceled";
MessageBox.Show(message);
FlagCommandDone = false;
}
private async Task DoDoneLoopAsync()
{
double i = 0;
while (i < 3)
{
i++;
await PauseToken.WaitWhilePausedAsync();
await Task.Delay(5000);
}
}

I needed to use Task.Run() instead Task.Factory.StartNew().
Task.Run vs Task.Factory.StartNew is here.
Thanks to VladD.

How to make a task more independent?

Originally I wrote my C# program using Threads and ThreadPooling, but most of the users on here were telling me Async was a better approach for more efficiency. My program sends JSON objects to a server until status code of 200 is returned and then moves on to the next task.
The problem is once one of my Tasks retrieves a status code of 200, it waits for the other Tasks to get 200 code and then move onto the next Task. I want each task to continue it's next task without waiting for other Tasks to finish (or catch up) by receiving a 200 response.
Below is my main class to run my tasks in parallel.
public static void Main (string[] args)
{
var tasks = new List<Task> ();
for (int i = 0; i < 10; i++) {
tasks.Add (getItemAsync (i));
}
Task.WhenAny (tasks);
}
Here is the getItemAsync() method that does the actual sending of information to another server. Before this method works, it needs a key. The problem also lies here, let's say I run 100 tasks, all 100 tasks will wait until every single one has a key.
public static async Task getItemAsync (int i)
{
if (key == "") {
await getProductKey (i).ConfigureAwait(false);
}
Uri url = new Uri ("http://www.website.com/");
var content = new FormUrlEncodedContent (new[] {
...
});
while (!success) {
using (HttpResponseMessage result = await client.PostAsync (url, content)) {
if (result.IsSuccessStatusCode) {
string resultContent = await result.Content.ReadAsStringAsync ().ConfigureAwait(false);
Console.WriteLine(resultContent);
success=true;
}
}
}
await doMoreAsync (i);
}
Here's the function that retrieves the keys, it uses HttpClient and gets parsed.
public static async Task getProductKey (int i)
{
string url = "http://www.website.com/key";
var stream = await client.GetStringAsync (url).ConfigureAwait(false);
var doc = new HtmlDocument ();
doc.LoadHtml (stream);
HtmlNode node = doc.DocumentNode.SelectNodes ("//input[#name='key']") [0];
try {
key = node.Attributes ["value"].Value;
} catch (Exception e) {
Console.WriteLine ("Exception: " + e);
}
}
After every task receives a key and has a 200 status code, it runs doMoreAsync(). I want any individual Task that retrieved the 200 code to run doMoreAsync() without waiting for other tasks to catch up. How do I go about this?

Your Main method is not waiting for a Task to complete, it justs fires off a bunch of async tasks and returns.
If you want to await an async task you can only do that from an async method. Workaround is to kick off an async task from the Main method and wait for its completion using the blocking Task.Wait():
public static void Main(string[] args) {
Task.Run(async () =>
{
var tasks = new List<Task> ();
for (int i = 0; i < 10; i++) {
tasks.Add (getItemAsync (i));
}
var finishedTask = await Task.WhenAny(tasks); // This awaits one task
}).Wait();
}
When you are invoking getItemAsync() from an async method, you can remove the ConfigureAwait(false) as well. ConfigureAwait(false) just makes sure some code does not execute on the UI thread.
If you want to append another task to a task, you can also append it to the previous Task directly by using ContinueWith():
getItemAsync(i).ContinueWith(anotherTask);

Your main problem seems to be that you're sharing the key field across all your concurrently-running asynchronous operations. This will inevitably lead to race hazards. Instead, you should alter your getProductKey method to return each retrieved key as the result of its asynchronous operation:
// ↓ result type of asynchronous operation
public static async Task<string> getProductKey(int i)
{
// retrieval logic here
// return key as result of asynchronous operation
return node.Attributes["value"].Value;
}
Then, you consume it like so:
public static async Task getItemAsync(int i)
{
string key;
try
{
key = await getProductKey(i).ConfigureAwait(false);
}
catch
{
// handle exceptions
}
// use local variable 'key' in further asynchronous operations
}
Finally, in your main logic, use a WaitAll to prevent the console application from terminating before all your tasks complete. Although WaitAll is a blocking call, it is acceptable in this case since you want the main thread to block.
public static void Main (string[] args)
{
var tasks = new List<Task> ();
for (int i = 0; i < 10; i++) {
tasks.Add(getItemAsync(i));
}
Task.WaitAll(tasks.ToArray());
}

Random tasks from Task.Factory.StartNew never finishes

I am using Async await with Task.Factory method.
public async Task<JobDto> ProcessJob(JobDto jobTask)
{
try
{
var T = Task.Factory.StartNew(() =>
{
JobWorker jobWorker = new JobWorker();
jobWorker.Execute(jobTask);
});
await T;
}
This method I am calling inside a loop like this
for(int i=0; i < jobList.Count(); i++)
{
tasks[i] = ProcessJob(jobList[i]);
}
What I notice is that new tasks opens up inside Process explorer and they also start working (based on log file). however out of 10 sometimes 8 or sometimes 7 finishes. Rest of them just never come back.
why would that be happening ?
Are they timing out ? Where can I set timeout for my tasks ?
UPDATE
Basically above, I would like each Task to start running as soon as they are called and wait for the response on AWAIT T keyword. I am assuming here that once they finish each of them will come back at Await T and do the next action. I am alraedy seeing this result for 7 out of 10 tasks but 3 of them are not coming back.
Thanks

It is hard to say what the issues is without the rest if the code, but you code can be simplified by making ProcessJob synchronous and then calling Task.Run with it.
public JobDto ProcessJob(JobDto jobTask)
{
JobWorker jobWorker = new JobWorker();
return jobWorker.Execute(jobTask);
}
Start tasks and wait for all tasks to finish. Prefer using Task.Run rather than Task.Factory.StartNew as it provides more favourable defaults for pushing work to the background. See here.
for(int i=0; i < jobList.Count(); i++)
{
tasks[i] = Task.Run(() => ProcessJob(jobList[i]));
}
try
{
await Task.WhenAll(tasks);
}
catch(Exception ex)
{
// handle exception
}

First, let's make a reproducible version of your code. This is NOT the best way to achieve what you are doing, but to show you what is happening in your code!
I'll keep the code almost same as your code, except I'll use simple int rather than your JobDto and on completion of the job Execute() I'll write in a file that we can verify later. Here's the code
public class SomeMainClass
{
public void StartProcessing()
{
var jobList = Enumerable.Range(1, 10).ToArray();
var tasks = new Task[10];
//[1] start 10 jobs, one-by-one
for (int i = 0; i < jobList.Count(); i++)
{
tasks[i] = ProcessJob(jobList[i]);
}
//[4] here we have 10 awaitable Task in tasks
//[5] do all other unrelated operations
Thread.Sleep(1500); //assume it works for 1.5 sec
// Task.WaitAll(tasks); //[6] wait for tasks to complete
// The PROCESS IS COMPLETE here
}
public async Task ProcessJob(int jobTask)
{
try
{
//[2] start job in a ThreadPool, Background thread
var T = Task.Factory.StartNew(() =>
{
JobWorker jobWorker = new JobWorker();
jobWorker.Execute(jobTask);
});
//[3] await here will keep context of calling thread
await T; //... and release the calling thread
}
catch (Exception) { /*handle*/ }
}
}
public class JobWorker
{
static object locker = new object();
const string _file = #"C:\YourDirectory\out.txt";
public void Execute(int jobTask) //on complete, writes in file
{
Thread.Sleep(500); //let's assume does something for 0.5 sec
lock(locker)
{
File.AppendAllText(_file,
Environment.NewLine + "Writing the value-" + jobTask);
}
}
}
After running just the StartProcessing(), this is what I get in the file
Writing the value-4
Writing the value-2
Writing the value-3
Writing the value-1
Writing the value-6
Writing the value-7
Writing the value-8
Writing the value-5
So, 8/10 jobs has completed. Obviously, every time you run this, the number and order might change. But, the point is, all the jobs did not complete!
Now, if I un-comment the step [6] Task.WaitAll(tasks);, this is what I get in my file
Writing the value-2
Writing the value-3
Writing the value-4
Writing the value-1
Writing the value-5
Writing the value-7
Writing the value-8
Writing the value-6
Writing the value-9
Writing the value-10
So, all my jobs completed here!
Why the code is behaving like this, is already explained in the code-comments. The main thing to note is, your tasks run in ThreadPool based Background threads. So, if you do not wait for them, they will be killed when the MAIN process ends and the main thread exits!!
If you still don't want to await the tasks there, you can return the list of tasks from this first method and await the tasks at the very end of the process, something like this
public Task[] StartProcessing()
{
...
for (int i = 0; i < jobList.Count(); i++)
{
tasks[i] = ProcessJob(jobList[i]);
}
...
return tasks;
}
//in the MAIN METHOD of your application/process
var tasks = new SomeMainClass().StartProcessing();
// do all other stuffs here, and just at the end of process
Task.WaitAll(tasks);
Hope this clears all confusion.

It's possible your code is swallowing exceptions. I would add a ContineWith call to the end of the part of the code that starts the new task. Something like this untested code:
var T = Task.Factory.StartNew(() =>
{
JobWorker jobWorker = new JobWorker();
jobWorker.Execute(jobTask);
}).ContinueWith(tsk =>
{
var flattenedException = tsk.Exception.Flatten();
Console.Log("Exception! " + flattenedException);
return true;
});
},TaskContinuationOptions.OnlyOnFaulted); //Only call if task is faulted
Another possibility is that something in one of the tasks is timing out (like you mentioned) or deadlocking. To track down whether a timeout (or maybe deadlock) is the root cause, you could add some timeout logic (as described in this SO answer):
int timeout = 1000; //set to something much greater than the time it should take your task to complete (at least for testing)
var task = TheMethodWhichWrapsYourAsyncLogic(cancellationToken);
if (await Task.WhenAny(task, Task.Delay(timeout, cancellationToken)) == task)
{
// Task completed within timeout.
// Consider that the task may have faulted or been canceled.
// We re-await the task so that any exceptions/cancellation is rethrown.
await task;
}
else
{
// timeout/cancellation logic
}
Check out the documentation on exception handling in the TPL on MSDN.