I have inherited a C#/XAML/Win 8 application. There is some code which is set to run every n seconds.
The code that sets that up is:
if(!_syncThreadStarted)
{
await Task.Run(() => SyncToDatabase());
_syncThreadStarted = true;
}
The above code is ran once.
And then inside SyncToDatabase() we have:
while (true)
{
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate();
await Task.Delay(10); // after elapsed time re-run above code
}
The method DeserializeAndUpdate queries a in-memory collection of objects and pushes those objects to a web service.
Sometimes the send request to the web service takes longer than expected meaning duplicate items are sent.
Question: Is there a way to have a thread or some type of thread pool/background worker which I can stop/abort/destroy inside the method SyncToDatabase() , and then initialize/start it once we are done? This will ensure no subsequent requests are fired while a previous request is still pending.
Edit: I am not very knowledgeable when it comes to Threads, but the logic I want is:
Create thread which runs some method every x seconds, and when it starts that thread stop the "running every x seconds" part, after thread has complete start the "run every x seconds" part again.
E.g. if the thread kicks off at 10:01:30AM and does not complete until 10:01:39AM (9 seconds) the next thread should start at 10:01:44AM (5 seconds after work completed) - does that make sense? I do not want 2 or more threads running at the same time.
Here is my code for the above:
var period = TimeSpan.FromSeconds(5);
var completed = true;
ThreadPoolTimer syncTimer = ThreadPoolTimer.CreatePeriodicTimer(async (source) =>
{
// stop further threads from starting (in case this work takes longer than var period)
syncTimer.Cancel();
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate(); // makes webservices calls
Dispatcher.RunAsync(CoreDispatcerPriority.High, async () =>
{
// Update UI
}
completed = true;
}, period,
(source) =>
{
if(!completed)
{
syncTimer.Cancel(); // not sure if this is correct...
}
}
Thanks,
Andrew)
This is not specific to Windows 8. Usually Task.Run is used for CPU-bound work, to offload it to a pool thread and keep the UI (or the core service loop) responsive. In your case, as far as I can tell, the main payload is dbSyncer.DeserializeAndUpdate, which is already asynchronous and most likely network-IO bound, rather than CPU-bound.
Besides, the author of the original code does _syncThreadStarted = true after await Task.Run(() => SyncToDatabase()). That doesn't make sense, because the work on the pool thread would have been already done by the time _syncThreadStarted = true is executed, thanks to the await.
To cancel the loop inside SyncToDatabase you could use Task Cancellation Pattern. Is SyncToDatabase itself an async method? I presume so, because there's an await in the while loop. Given that, the code which calls it could look something like this:
if(_syncTask != null && !_syncTask.IsCompleted)
{
_ct.Cancel();
// here you may want to make sure that the pending task has been fully shut down,
// keeping possible re-entrancy in mind
// See: https://stackoverflow.com/questions/18999827/a-pattern-for-self-cancelling-and-restarting-task
_syncTask = null;
}
_ct = new CancellationTokenSource();
// _syncTask = SyncToDatabase(ct.Token); // do not await
// edited to run on another thread, as requested by the OP
var _syncTask = Task.Run(async () => await SyncToDatabase(ct.Token), ct.Token);
_syncThreadStarted = true;
And SyncToDatabase could look like:
async Task SyncToDatabase(CancellationToken token)
{
while (true)
{
token.ThrowIfCancellationRequested();
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate();
await Task.Delay(10, token); // after elapsed time re-run above code
}
}
Check this answer for more details on how to cancel and restart a task.
I may have misunderstood the question, but the execution of SynchToDatabase() will wait on the completion of await dbSyncer.DeserializeAndUpdaet() (due to the await keyword, go figure ;)) before executing the continuation, which will then delay for 10 ms (do you want 10ms or did you mean 10 seconds? Parameter for Task.Delay is in milliseconds), then loop back to re-execute the DbSyncer method, so I don't see the problem.
Related
I need your help, with threads I'm full 0 and you only need to create a certain thread and complete it on command, BUT I do not create each thread in advance, as there will be a lot of them, I do it like this:
Thread thread = new Thread(() => Go(..... many many variables that are taken from the listview ......));
thread.Start();
So, as noted above, variables are taken from the listview, which in turn is loaded by me from the file and then I run the threads I need. BUT the process in the stream is infinite and will end only if I completely close the program, and I would like to end the stream in the same way as I started it (right click on the desired line-start/stop). As I said, I have never worked with threads and thought that it was somehow simple, like when you start a thread, you assign it an ID and end it with the same ID, but alas. I have searched all over Google and have not found an EXAMPLE that suits me (I will repeat for the third time - I have never worked with threads and I do not need to say "go read about TPL"), so I ask for help, preferably with an example)
I have a very bad idea: in the sheet there is an invisible column in which an id is generated at the start, then when I send a command to start the thread, a unique variable is created with the name for example int id1=0 and its name is passed to the thread itself and each time the loop starts, id1=0 or 1 is checked in it, respectively, if 0-continue, if 1-empty. Well, it is logical that when you click the stop button, its value changes to 1. But something seems to me that the holy spirit of multithreading will punish me for this when the threads become 100+. I read this idea somewhere, so don't swear)
You do not need hundreds of threads for this. Your worker "threads" are performing HTTP requests, which can be done asynchronously without requiring a new thread. Also, hundreds of threads wouldn't really help you unless you have hundreds of CPU cores (you don't).
For this sort of work, I'd recommend the following:
Write a method that does all the work your thread does, but also checks a CancellationToken with each iteration.
Calls the method in a loop, once for each account, and store the resulting tasks in an array or list. Or use LINQ (as I do in this example) to create the list.
When your program terminates, activate the CancellationToken.
After cancelling, you have to await all the tasks in order to observe any possible exceptions and exit cleanly.
For example
public async Task DoTheWork(Account account, CancellationToken token)
{
while (!token.IsCancellationRequested)
{
var result = await httpClient.GetAsync(account.Url);
await DoSomethingWithResult(result);
await Task.Delay(1000);
}
}
//Main program
var accounts = GetAccountList();
var source = new CancellationTokenSource();
var tasks = accounts.Select( x => DoTheWork(x, source.Token) ).ToList();
//When exiting
source.Cancel();
await Task.WhenAll( tasks );
source.Dispose();
Indivivdual cancellation
Here's another approach that keeps a list of the accounts and a delegate that can be used for cancelling the task for that specific account.
//Declare this somewhere it will persist for the duration of the program
//The key to this dictionary is the account you wish to cancel
//The value is a delegate that you can call to cancel its task
Dictionary<Account, Func<Task>> _tasks = new Dictionary<Account, Func<Task>>();
async Task CreateTasks()
{
var accounts = GetAccounts();
foreach (var account in accounts)
{
var source = new CancellationTokenSource();
var task = DoTheWork(account, source.Token);
_tasks.Add(account, () => { source.Cancel(); return task; });
}
}
//Retrieve the delegate from the dictionary and call it to cancel its task
//Then await the task to observe any exceptions
//Then remove it from the list
async Task CancelTask(Account account)
{
var cancelAction = _tasks[account];
var task = cancelAction();
await task;
_tasks.Remove(account);
}
async Task CancelAllTasks()
{
var tasks = _tasks.Select(x => x.Value()).ToList();
await Task.WhenAll(tasks);
}
In my application I have the need to continually process some piece(s) of Work on some set interval(s). I had originally written a Task to continually check a given Task.Delay to see if it was completed, if so the Work would be processed that corresponded to that Task.Delay. The draw back to this method is the Task that checks these Task.Delays would be in a psuedo-infinite loop when no Task.Delay is completed.
To solve this problem I found that I could create a "recursive Task" (I am not sure what the jargon for this would be) that processes the work at the given interval as needed.
// New Recurring Work can be added by simply creating
// the Task below and adding an entry into this Dictionary.
// Recurring Work can be removed/stopped by looking
// it up in this Dictionary and calling its CTS.Cancel method.
private readonly object _LockRecurWork = new object();
private Dictionary<Work, Tuple<Task, CancellationTokenSource> RecurringWork { get; set; }
...
private Task CreateRecurringWorkTask(Work workToDo, CancellationTokenSource taskTokenSource)
{
return Task.Run(async () =>
{
// Do the Work, then wait the prescribed amount of time before doing it again
DoWork(workToDo);
await Task.Delay(workToDo.RecurRate, taskTokenSource.Token);
// If this Work's CancellationTokenSource is not
// cancelled then "schedule" the next Work execution
if (!taskTokenSource.IsCancellationRequested)
{
lock(_LockRecurWork)
{
RecurringWork[workToDo] = new Tuple<Task, CancellationTokenSource>
(CreateRecurringWorkTask(workToDo, taskTokenSource), taskTokenSource);
}
}
}, taskTokenSource.Token);
}
Should/Could this be represented with a chain of Task.ContinueWith? Would there be any benefit to such an implementation? Is there anything majorly wrong with the current implementation?
Yes!
Calling ContinueWith tells the Task to call your code as soon as it finishes. This is far faster than manually polling it.
I'm having trouble assimilating the c# Task, async and await patterns.
Windows service, .NET v4.5.2 server-side.
I have a Windows service accepting a variety of sources of incoming records, arriving externally ad-hoc via a self-hosted web api. I would like to batch up these records and then forward them on to another service. If the number of batched records exceeds a threshold, the batch should be dispatched immediately. Furthermore, the batch as it stands should also be dispatched if a time interval has elapsed. This means that a record is never held for more than N seconds.
I'm struggling to fit this into a Task based async pattern.
In days gone by, I would have created a Thread, a ManualResetEvent and a System.Threading.Timer. The Thread would loop around a Wait on the reset event. The Timer would set the event when fired, as would the code doing the aggregation when the batch size exceeded the threshold. Following the Wait, the Thread would stop the Timer, do the dispatch (an HTTP Post), reset the Timer and clear the ManualResetEvent, the loop back and Wait.
However, I am seeing folk say that this is 'bad' as the Wait just blocks a valuable thread resource, and that async/await is my panacea.
First off, are they right? Is my way out-of-date and inefficient or can I JFDI?
I've found examples here for batching and here for tasks at intervals, but not a combination of the two.
Is this requirement actually compatible with async/await?
Actually, you're almost doing the right thing, and they are also partially right.
What you should know is that you should avoid idle threads, with long waiting on events or waiting for I/O to complete (waiting on locks with few contention and fast statement blocks or spinning loops with compare-and-swap are usually OK).
What most of them don't know is that tasks are not magic, for instance, Task.Delay uses a Timer (more exactly, a System.Threading.Timer) and waiting on a non-complete task ends up using a ManualResetEventSlim (an improvement over ManualResetEvent, as it doesn't create a Win32 event unless explicitly asked for, e.g. ((IAsyncResult)task).AsyncWaitHandle).
So yes, your requirements are achievable with async/await, or tasks in general.
Runnable example at .NET Fiddle:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
public class Record
{
private int n;
public Record(int n)
{
this.n = n;
}
public int N { get { return n; } }
}
public class RecordReceiver
{
// Arbitrary constants
// You should fetch value from configuration and define sensible defaults
private static readonly int threshold = 5;
// I chose a low value so the example wouldn't timeout in .NET Fiddle
private static readonly TimeSpan timeout = TimeSpan.FromMilliseconds(100);
// I'll use a Stopwatch to trace execution times
private readonly Stopwatch sw = Stopwatch.StartNew();
// Using a separate private object for locking
private readonly object lockObj = new object();
// The list of accumulated records to execute in a batch
private List<Record> records = new List<Record>();
// The most recent TCS to signal completion when:
// - the list count reached the threshold
// - enough time has passed
private TaskCompletionSource<IEnumerable<Record>> batchTcs;
// A CTS to cancel the timer-based task when the threshold is reached
// Not strictly necessary, but it reduces resource usage
private CancellationTokenSource delayCts;
// The task that will be completed when a batch of records has been dispatched
private Task dispatchTask;
// This method doesn't use async/await,
// because we're not doing an async flow here.
public Task ReceiveAsync(Record record)
{
Console.WriteLine("Received record {0} ({1})", record.N, sw.ElapsedMilliseconds);
lock (lockObj)
{
// When the list of records is empty, set up the next task
//
// TaskCompletionSource is just what we need, we'll complete a task
// not when we've finished some computation, but when we reach some criteria
//
// This is the main reason this method doesn't use async/await
if (records.Count == 0)
{
// I want the dispatch task to run on the thread pool
// In .NET 4.6, there's TaskCreationOptions.RunContinuationsAsynchronously
// .NET 4.6
//batchTcs = new TaskCompletionSource<IEnumerable<Record>>(TaskCreationOptions.RunContinuationsAsynchronously);
//dispatchTask = DispatchRecordsAsync(batchTcs.Task);
// Previously, we have to set up a continuation task using the default task scheduler
// .NET 4.5.2
batchTcs = new TaskCompletionSource<IEnumerable<Record>>();
var asyncContinuationsTask = batchTcs.Task
.ContinueWith(bt => bt.Result, TaskScheduler.Default);
dispatchTask = DispatchRecordsAsync(asyncContinuationsTask);
// Create a cancellation token source to be able to cancel the timer
//
// To be used when we reach the threshold, to release timer resources
delayCts = new CancellationTokenSource();
Task.Delay(timeout, delayCts.Token)
.ContinueWith(
dt =>
{
// When we hit the timer, take the lock and set the batch
// task as complete, moving the current records to its result
lock (lockObj)
{
// Avoid dispatching an empty list of records
//
// Also avoid a race condition by checking the cancellation token
//
// The race would be for the actual timer function to start before
// we had a chance to cancel it
if ((records.Count > 0) && !delayCts.IsCancellationRequested)
{
batchTcs.TrySetResult(new List<Record>(records));
records.Clear();
}
}
},
// Since our continuation function is fast, we want it to run
// ASAP on the same thread where the actual timer function runs
//
// Note: this is just a hint, but I trust it'll be favored most of the time
TaskContinuationOptions.ExecuteSynchronously);
// Remember that we want our batch task to have continuations
// running outside the timer thread, since dispatching records
// is probably too much work for a timer thread.
}
// Actually store the new record somewhere
records.Add(record);
// When we reach the threshold, set the batch task as complete,
// moving the current records to its result
//
// Also, cancel the timer task
if (records.Count >= threshold)
{
batchTcs.TrySetResult(new List<Record>(records));
delayCts.Cancel();
records.Clear();
}
// Return the last saved dispatch continuation task
//
// It'll start after either the timer or the threshold,
// but more importantly, it'll complete after it dispatches all records
return dispatchTask;
}
}
// This method uses async/await, since we want to use the async flow
internal async Task DispatchRecordsAsync(Task<IEnumerable<Record>> batchTask)
{
// We expect it to return a task right here, since the batch task hasn't had
// a chance to complete when the first record arrives
//
// Task.ConfigureAwait(false) allows us to run synchronously and on the same thread
// as the completer, but again, this is just a hint
//
// Remember we've set our task to run completions on the thread pool?
//
// With .NET 4.6, completing a TaskCompletionSource created with
// TaskCreationOptions.RunContinuationsAsynchronously will start scheduling
// continuations either on their captured SynchronizationContext or TaskScheduler,
// or forced to use TaskScheduler.Default
//
// Before .NET 4.6, completing a TaskCompletionSource could mean
// that continuations ran withing the completer, especially when
// Task.ConfigureAwait(false) was used on an async awaiter, or when
// Task.ContinueWith(..., TaskContinuationOptions.ExecuteSynchronously) was used
// to set up a continuation
//
// That's why, before .NET 4.6, we need to actually run a task for that effect,
// and we used Task.ContinueWith without TaskContinuationOptions.ExecuteSynchronously
// and with TaskScheduler.Default, to ensure it gets scheduled
//
// So, why am I using Task.ConfigureAwait(false) here anyway?
// Because it'll make a difference if this method is run from within
// a Windows Forms or WPF thread, or any thread with a SynchronizationContext
// or TaskScheduler that schedules tasks on a dedicated thread
var batchedRecords = await batchTask.ConfigureAwait(false);
// Async methods are transformed into state machines,
// much like iterator methods, but with async specifics
//
// What await actually does is:
// - check if the awaitable is complete
// - if so, continue executing
// Note: if every awaited awaitable is complete along an async method,
// the method will complete synchronously
// This is only expectable with tasks that have already completed
// or I/O that is always ready, e.g. MemoryStream
// - if not, return a task and schedule a continuation for just after the await expression
// Note: the continuation will resume the state machine on the next state
// Note: the returned task will complete on return or on exception,
// but that is something the compiled state machine will handle
foreach (var record in batchedRecords)
{
Console.WriteLine("Dispatched record {0} ({1})", record.N, sw.ElapsedMilliseconds);
// I used Task.Yield as a replacement for actual work
//
// It'll force the async state machine to always return here
// and shedule a continuation that reenters the async state machine right afterwards
//
// This is not something you usually want on production code,
// so please replace this with the actual dispatch
await Task.Yield();
}
}
}
public class Program
{
public static void Main()
{
// Our main entry point is synchronous, so we run an async entry point and wait on it
//
// The difference between MainAsync().Result and MainAsync().GetAwaiter().GetResult()
// is in the way exceptions are thrown:
// - the former aggregates exceptions, throwing an AggregateException
// - the latter doesn't aggregate exceptions if it doesn't have to, throwing the actual exception
//
// Since I'm not combining tasks (e.g. Task.WhenAll), I'm not expecting multiple exceptions
//
// If my main method returned int, I could return the task's result
// and I'd make MainAsync return Task<int> instead of just Task
MainAsync().GetAwaiter().GetResult();
}
// Async entry point
public static async Task MainAsync()
{
var receiver = new RecordReceiver();
// I'll provide a few records:
// - a delay big enough between the 1st and the 2nd such that the 1st will be dispatched
// - 8 records in a row, such that 5 of them will be dispatched, and 3 of them will wait
// - again, a delay big enough that will provoke the last 3 records to be dispatched
// - and a final record, which will wait to be dispatched
//
// We await for Task.Delay between providing records,
// but we'll await for the records in the end only
//
// That is, we'll not await each record before the next,
// as that would mean each record would only be dispatched after at least the timeout
var t1 = receiver.ReceiveAsync(new Record(1));
await Task.Delay(TimeSpan.FromMilliseconds(300));
var t2 = receiver.ReceiveAsync(new Record(2));
var t3 = receiver.ReceiveAsync(new Record(3));
var t4 = receiver.ReceiveAsync(new Record(4));
var t5 = receiver.ReceiveAsync(new Record(5));
var t6 = receiver.ReceiveAsync(new Record(6));
var t7 = receiver.ReceiveAsync(new Record(7));
var t8 = receiver.ReceiveAsync(new Record(8));
var t9 = receiver.ReceiveAsync(new Record(9));
await Task.Delay(TimeSpan.FromMilliseconds(300));
var t10 = receiver.ReceiveAsync(new Record(10));
// I probably should have used a list of records, but this is just an example
await Task.WhenAll(t1, t2, t3, t4, t5, t6, t7, t8, t9, t10);
}
}
You can make this more interesting, like returning a distinct task, such as Task<RecordDispatchReport>, from ReceiveAsync which is completed by the processing part of DispatchRecords, using a TaskCompletionSource for each record.
I'm writing a Windows Service that will kick off multiple worker threads that will listen to Amazon SQS queues and process messages. There will be about 20 threads listening to 10 queues.
The threads will have to be always running and that's why I'm leaning towards to actually using actual threads for the worker loops rather than threadpool threads.
Here is a top level implementation. Windows service will kick off multiple worker threads and each will listen to it's queue and process messages.
protected override void OnStart(string[] args)
{
for (int i = 0; i < _workers; i++)
{
new Thread(RunWorker).Start();
}
}
Here is the implementation of the work
public async void RunWorker()
{
while(true)
{
// .. get message from amazon sqs sync.. about 20ms
var message = sqsClient.ReceiveMessage();
try
{
await PerformWebRequestAsync(message);
await InsertIntoDbAsync(message);
}
catch(SomeExeception)
{
// ... log
//continue to retry
continue;
}
sqsClient.DeleteMessage();
}
}
I know I can perform the same operation with Task.Run and execute it on the threadpool thread rather than starting individual thread, but I don't see a reason for that since each thread will always be running.
Do you see any problems with this implementation? How reliable would it be to leave threads always running in this fashion and what can I do to make sure that each thread is always running?
One problem with your existing solution is that you call your RunWorker in a fire-and-forget manner, albeit on a new thread (i.e., new Thread(RunWorker).Start()).
RunWorker is an async method, it will return to the caller when the execution point hits the first await (i.e. await PerformWebRequestAsync(message)). If PerformWebRequestAsync returns a pending task, RunWorker returns and the new thread you just started terminates.
I don't think you need a new thread here at all, just use AmazonSQSClient.ReceiveMessageAsync and await its result. Another thing is that you shouldn't be using async void methods unless you really don't care about tracking the state of the asynchronous task. Use async Task instead.
Your code might look like this:
List<Task> _workers = new List<Task>();
CancellationTokenSource _cts = new CancellationTokenSource();
protected override void OnStart(string[] args)
{
for (int i = 0; i < _MAX_WORKERS; i++)
{
_workers.Add(RunWorkerAsync(_cts.Token));
}
}
public async Task RunWorkerAsync(CancellationToken token)
{
while(true)
{
token.ThrowIfCancellationRequested();
// .. get message from amazon sqs sync.. about 20ms
var message = await sqsClient.ReceiveMessageAsync().ConfigureAwait(false);
try
{
await PerformWebRequestAsync(message);
await InsertIntoDbAsync(message);
}
catch(SomeExeception)
{
// ... log
//continue to retry
continue;
}
sqsClient.DeleteMessage();
}
}
Now, to stop all pending workers, you could simple do this (from the main "request dispatcher" thread):
_cts.Cancel();
try
{
Task.WaitAll(_workers.ToArray());
}
catch (AggregateException ex)
{
ex.Handle(inner => inner is OperationCanceledException);
}
Note, ConfigureAwait(false) is optional for Windows Service, because there's no synchronization context on the initial thread, by default. However, I'd keep it that way to make the code independent of the execution environment (for cases where there is synchronization context).
Finally, if for some reason you cannot use ReceiveMessageAsync, or you need to call another blocking API, or simply do a piece of CPU intensive work at the beginning of RunWorkerAsync, just wrap it with Task.Run (as opposed to wrapping the whole RunWorkerAsync):
var message = await Task.Run(
() => sqsClient.ReceiveMessage()).ConfigureAwait(false);
Well, for one I'd use a CancellationTokenSource instantiated in the service and passed down to the workers. Your while statement would become:
while(!cancellationTokenSource.IsCancellationRequested)
{
//rest of the code
}
This way you can cancel all your workers from the OnStop service method.
Additionally, you should watch for:
If you're playing with thread states from outside of the thread, then a ThreadStateException, or ThreadInterruptedException or one of the others might be thrown. So, you want to handle a proper thread restart.
Do the workers need to run without pause in-between iterations? I would throw in a sleep in there (even a few ms's) just so they don't keep the CPU up for nothing.
You need to handle ThreadStartException and restart the worker, if it occurs.
Other than that there's no reason why those 10 treads can't run for as long as the service runs (days, weeks, months at a time).
I want to run a block of code (or method) in C#. In this block I use Web Service method. I want to run it asynchronously (to avoid freeze application) with time out. My code is:
SmsSender s = new SmsSender();
dataGrid.ItemsSource =
s.GetAllInboxMessagesDataSet().Tables[0].DefaultView;
before this I use thread.Abort. finally I find out that thread.Abrot is evil
please help me
If you are using C# 4.5 you can do it like that:
var cts = new CancellationTokenSource(3000); // Set timeout
var task = Task.Run(() =>
{
while (!cts.Token.IsCancellationRequested)
{
// Working...
}
}, cts.Token);
There are different solutions to the problem (not to freeze the main thread). My solution is to create one Task and within to create a second task for which i wait. The wrapper task is not blocked by wait or join and so the main thread is not blocked. With events i can notify the caller, that the worker task has timed out or not. The code looks like this:
// create asynchronous task. in order not to block the calling thread,
// create and start another task in this one and wait for its completion
var synchronize = new System.Threading.Tasks.Task(() =>
{
var worker = new System.Threading.Tasks.TaskFactory().StartNew(() =>
{
// do something work intensive
});
var workCompleted = worker.Wait(10000 /* timeout */);
if (!workCompleted)
{
// worker task has timed-out
}
});