Partly as an exercise in exploring async, I though I'd try creating a ServiceBrokerWatcher class. The idea is much the same as a FileSystemWatcher - watch a resource and raise an event when something happens. I was hoping to do this with async rather than actually creating a thread, because the nature of the beast means that most of the time it is just waiting on a SQL waitfor (receive ...) statement. This seemed like an ideal use of async.
I have written code which "works", in that when I send a message through broker, the class notices it and fires off the appropriate event. I thought this was super neat.
But I suspect I have gotten something fundamentally wrong somewhere in my understanding of what is going on, because when I try to stop the watcher it doesn't behave as I expect.
First a brief overview of the components, and then the actual code:
I have a stored procedure which issues a waitfor (receive...) and returns a result set to the client when a message is received.
There is a Dictionary<string, EventHandler> which maps message type names (in the result set) to the appropriate event handler. For simplicity I only have the one message type in the example.
The watcher class has an async method which loops "forever" (until cancellation is requested), which contains the execution of the procedure and the raising of the events.
So, what's the problem? Well, I tried hosting my class in a simple winforms application, and when I hit a button to call the StopListening() method (see below), execution isn't cancelled right away as I thought it would be. The line listener?.Wait(10000) will in fact wait for 10 seconds (or however long I set the timeout). If I watch what happens with SQL profiler I can see that the attention event is being sent "straight away", but still the function does not exit.
I have added comments to the code starting with "!" where I suspect I have misunderstood something.
So, main question: Why isn't my ListenAsync method "honoring" my cancellation request?
Additionally, am I right in thinking that this program is (most of the time) consuming only one thread? Have I done anything dangerous?
Code follows, I tried to cut it down as much as I could:
// class members //////////////////////
private readonly SqlConnection sqlConnection;
private CancellationTokenSource cts;
private readonly CancellationToken ct;
private Task listener;
private readonly Dictionary<string, EventHandler> map;
public void StartListening()
{
if (listener == null)
{
cts = new CancellationTokenSource();
ct = cts.Token;
// !I suspect assigning the result of the method to a Task is wrong somehow...
listener = ListenAsync(ct);
}
}
public void StopListening()
{
try
{
cts.Cancel();
listener?.Wait(10000); // !waits the whole 10 seconds for some reason
} catch (Exception) {
// trap the exception sql will raise when execution is cancelled
} finally
{
listener = null;
}
}
private async Task ListenAsync(CancellationToken ct)
{
using (SqlCommand cmd = new SqlCommand("events.dequeue_target", sqlConnection))
using (CancellationTokenRegistration ctr = ct.Register(cmd.Cancel)) // !necessary?
{
cmd.CommandTimeout = 0;
while (!ct.IsCancellationRequested)
{
var events = new List<string>();
using (var rdr = await cmd.ExecuteReaderAsync(ct))
{
while (rdr.Read())
{
events.Add(rdr.GetString(rdr.GetOrdinal("message_type_name")));
}
}
foreach (var handler in events.Join(map, e => e, m => m.Key, (e, m) => m.Value))
{
if (handler != null && !ct.IsCancellationRequested)
{
handler(this, null);
}
}
}
}
}
You don't show how you've bound it to the WinForms app, but if you are using regular void button1click methods, you may be running into this issue.
So your code will run fine in a console app (it does when I try it) but deadlock when called via the UI thread.
I'd suggest changing your controller class to expose async start and stop methods, and call them via e.g.:
private async void btStart_Click(object sender, EventArgs e)
{
await controller.StartListeningAsync();
}
private async void btStop_Click(object sender, EventArgs e)
{
await controller.StopListeningAsync();
}
Peter had the right answer. I was confused for several minutes about what was deadlocking, but then I had my forehead slapping moment. It is the continuation of ListenAsync after the ExecuteReaderAsync is cancelled, because it's just a task, not a thread of its own. That was, after all, the whole point!
Then I wondered... OK, what if I tell the async part of ListenAsync() that it doesn't need the UI thread. I will call ExecuteReaderAsync(ct) with .ConfigureAwait(false)! Aha! Now the class methods don't have to be async anymore, because in StopListening() I can just listener.Wait(10000), the wait will continue the task internally on a different thread, and the consumer is none the wiser. Oh boy, so clever.
But no, I can't do that. Not in a webforms application at least. If I do that then the textbox is not updated. And the reason for that seems clear enough: the guts of ListenAsync invoke an event handler, and that event handler is a function which wants to update text in a textbox - which no doubt has to happen on the UI thread. So it doesn't deadlock, but it also can't update the UI. If I set a breakpoint in the handler which wants to update the UI the line of code is hit, but the UI can't be changed.
So in the end it seems the only solution in this case is indeed to "go async all the way down". Or in this case, up!
I was hoping that I didn't have to do that. The fact that the internals of my Watcher are using async methodologies rather than just spawning a thread is, in my mind, an "implementation detail" that the caller shouldn't have to care about. But a FileSystemWatcher has exactly the same issue (the need to control.Invoke if you want to update a GUI based on a watcher event), so that's not so bad. If I was a consumer that had to choose between using async or using Invoke, I'd choose async!
Related
Yes, I know, there are tons of threads on this topic. I read a lot of them and used them often (more or less) successfully. Now I got an old DLL (programmed in .net 4.0) and that is using BackgroundWorkers to fire result events. Whatever I try to stop and wait for such a result seems to miss its mark. But maybe some of you have ideas that I haven't tried yet.
I register the answer event in thread 1 (according to Thread.CurrentThread.ManagedThreadId), call the method to connect and wait with a SemaphoreSlim for the answer. But in each and every constellation I get the answering event AFTER the timeout occurred. The event is on thread 3 and when I tried to raise the AwaitAsync() (also with Await) of the SemaphoreSlim on a special Thread.Run(() => ...); it was on thread id 8. But still thread number 3 always just comes AFTER the timeout.
private void ConnectDevice()
{
MobileDevice.DeviceConnected += new DeviceConnectedHandler(MobileDevice_Connected);
...
mSignal = new SemaphoreSlim(0, 1);
MobileDevice.Connect();
// Task.Run(() => MobileDevice.Connect());
int i = Thread.CurrentThread.ManagedThreadId;
var task = Task.Run(() => mSignal.WaitAsync(new TimeSpan(0, 0, cTimeout)).GetAwaiter().GetResult());
// Task<bool> task = Task.Run(async () => await WaitForSemaphore());
// var result = task.Wait();
IsConnected = task.Result;
// WaitForSemaphore();
...
}
private async Task<bool> WaitForSemaphore()
{
int j = Thread.CurrentThread.ManagedThreadId;
if (!await mSignal.WaitAsync(new TimeSpan(0, 0, 5)))
{
throw new MobileDeviceException("Device timed out");
}
return true;
}
//private void WaitForSemaphore()
//{
// if (!mSignal.Wait(new TimeSpan(0, 0, cTimeout)))
// {
// throw new MobileDeviceException("Device timed out");
// }
//}
private void MobileDevice_Connected(object sender, DeviceConnectedEventArgs e)
{
int k = Thread.CurrentThread.ManagedThreadId;
mSignal?.Release();
...
}
And yes, I know this is chaotic. But I wanted to show you, that I tried a lot already. I tried a lot more, but deleted also a mass of mistakes.
I begin to think, that, even if the answer comes on thread id 3, the listener to these events is still the main thread (id 1). And as long as that is blocked, the event doesn't gets fired.
Am I right? And how do I get around this? Register the event on a different way?
Oh, I nearly forgot: I am serving an Interface here and this will become a plugin for a complex application, so I cannot make the Connect-Method async and use the async/await-Pattern. I have to call the Connect of the device, block the main thread till the answer arrives and then release it, so the main part of the application can continue.
Anyone an idea of solving this?
Edit (the 1st): Ok, to sort some confusions out. This is a plugin that is called from a non-async method. I cannot change the calling method to an async one or else I would have to reprogram a few hundred thousand lines of code.
The call comes from the main program and looks like this:
firstDevice.Connect();
I COULD change that to something within reason, but I cannot use something like: await firstDevice.Connect(); or else I would have to change the main programs calls all to async. And this is simply out of question.
The connect method inside the plug-in I could change. At this moment it does nothing more than to call the ConnectDevice(), so I could test some things with async, SemaphoreSlims, and so on.
And as soon as I use an await inside an async method, the calling thread moves on. There would also have to be an await, but you cannot use await outside of async methods.
What seems strange to me is, that Thread.CurrentThread.ManagedThreadId says that both threads are thread 1. But when I step through they are clearly moving asynchronously.
Edit (the 2nd): I heard a clue. Maybe the problem here is the BackgroundWorker of the API. A colleague of mine once heard that the BackgroundWorker blocks the GUI-thread, when it is started on the GUI-thread. So the events of the API cannot get to me on thread 3 until the GUI-thread is released. So the solution would be to call the MobileDevice.Connect(); on a different thread. But it seems that the API will have to change. So we will discuss this internally. As soon as I have a solution I will update this a last time for anyone interested.
Edit (the 3rd): Ok, it seems nearly all of this solutions are working my problem was really with this goddamn BackgroundWorker. The API communicated with the mobile device on thread 1. And as soon as you block thread 1, there is also a block in the communication between API and device, so the answer of the API never comes...
But thanks anyway for you help. ;)
You may wrap the Connect call in a TaskCompletionSource:
public static class DeviceExtension
{
public static Task ConnectAsync(this Device device)
{
var tcs = new TaskCompletionSource<object>();
device.DeviceConnected += (s, e) => tcs.SetResult(null);
device.Connect();
return tcs.Task;
}
}
which you would call like
await MobileDevice.ConnectAsync();
or in a synchronous context like
MobileDevice.ConnectAsync().Wait();
I have a piece of code that has two event handlers. I want these two event handlers to notify another method that there is some work to be done.
I have implemented this using a ManualResetEvent, but I am unsure if this is the best way to achieve what I want, or if there is some better way.
static ManualResetEvent autoEvent = new ManualResetEvent(false);
void begin() {
ThreadPool.QueueUserWorkItem(new WaitCallback(genericHandler));
}
void OnEvent1(object sender) {
autoEvent.Set();
}
void OnEvent2(object sender) {
autoEvent.Set();
}
void genericHandler(object info) {
while (true) {
autoEvent.WaitOne();
// do some work
}
}
One of the most important questions I have is: After autoEvent.WaitOne(), I do some work and that work consumes time. In the meanwhile, another event is triggered, and Set() is called before the genericHandler gets to WaitOne() again. When WaitOne is reached again, will it wait for another Set(), or proceed if a Set() has been called before reaching the WaitOne()?
Is this the best way to implement multiple publishers and one subscriber pattern in C#? Or should I use another thing instead of the ManualResetEvent?
Note: The genericHandler is in a different thread because Event1 and Event2 have different priorities, so in the handler I check whether Event1 has pending work, before checking Event2.
Your code does indeed do what you think it does, and the race condition you describe is not a problem at all. As per the documentation of a MRE, when it is set it will remain in the "signaled" state until it is reset by a call to WaitOne.
The question and code is presented too vaguely to offer good, specific advice. That said...
No, the use of ManualResetEvent is not appropriate here. Not only does it needlessly complicate the code, your code relies on a long-running thread that you have taken from the thread pool (where only short-lived tasks should be executed).
If you have a need for events to trigger the execution of some asynchronous work, then you should be using the async/await pattern, where each new unit of work is invoked via the Task class.
For example:
async void OnEvent1(object sender) {
var workUnit = ... ; // something here that represents your unit of work
await Task.Run(() => genericHandler(workUnit));
}
void OnEvent2(object sender) {
var workUnit = ... ; // something here that represents your unit of work
await Task.Run(() => genericHandler(workUnit));
}
void genericHandler(object info) {
// do some work using info
}
Note that the event object and the begin() method are eliminated entirely.
It's not clear from your question whether each work unit is entirely independent of each other. If not, then you may also require some synchronization to protect shared data. Again, without a more specific question it's not possible to say what this would be, but most likely you'd either use the lock statement, or one of the Concurrent... collections.
I'm performing an async operation for an upload using Starksoft.Net.Ftp.
Looks like that:
public void UploadFile(string filePath, string packageVersion)
{
_uploadFtpClient= new FtpClient(Host, Port, FtpSecurityProtocol.None)
{
DataTransferMode = UsePassiveMode ? TransferMode.Passive : TransferMode.Active,
FileTransferType = TransferType.Binary,
};
_uploadFtpClient.TransferProgress += TransferProgressChangedEventHandler;
_uploadFtpClient.PutFileAsyncCompleted += UploadFinished;
_uploadFtpClient.Open(Username, Password);
_uploadFtpClient.ChangeDirectoryMultiPath(Directory);
_uploadFtpClient.MakeDirectory(newDirectory);
_uploadFtpClient.ChangeDirectory(newDirectory);
_uploadFtpClient.PutFileAsync(filePath, FileAction.Create);
_uploadResetEvent.WaitOne();
_uploadFtpClient.Close();
}
private void UploadFinished(object sender, PutFileAsyncCompletedEventArgs e)
{
if (e.Error != null)
{
if (e.Error.InnerException != null)
UploadException = e.Error.InnerException;
}
_uploadResetEvent.Set();
}
As you can see, there is a ManualResetEvent in there, which is declared as private variable on top of the class:
private ManualResetEvent _uploadResetEvent = new ManualResetEvent(false);
Well, the sense is just that it should wait for the upload to complete, but it must be async for reporting progress, that's all.
Now, this just works fine.
I have a second method that should cancel the upload, if wished.
public void Cancel()
{
_uploadFtpClient.CancelAsync();
}
When the upload is cancelled a directory on the server also must be deleted.
I have a method for this, too:
public void DeleteDirectory(string directoryName)
{
_uploadResetEvent.Set(); // As the finished event of the upload is not called when cancelling, I need to set the ResetEvent manually here.
if (!_hasAlreadyFixedStrings)
FixProperties();
var directoryEmptyingClient = new FtpClient(Host, Port, FtpSecurityProtocol.None)
{
DataTransferMode = UsePassiveMode ? TransferMode.Passive : TransferMode.Active,
FileTransferType = TransferType.Binary
};
directoryEmptyingClient.Open(Username, Password);
directoryEmptyingClient.ChangeDirectoryMultiPath(String.Format("/{0}/{1}", Directory, directoryName));
directoryEmptyingClient.GetDirListAsyncCompleted += DirectoryListingFinished;
directoryEmptyingClient.GetDirListAsync();
_directoryFilesListingResetEvent.WaitOne(); // Deadlock appears here
if (_directoryCollection != null)
{
foreach (FtpItem directoryItem in _directoryCollection)
{
directoryEmptyingClient.DeleteFile(directoryItem.Name);
}
}
directoryEmptyingClient.Close();
var directoryDeletingClient = new FtpClient(Host, Port, FtpSecurityProtocol.None)
{
DataTransferMode = UsePassiveMode ? TransferMode.Passive : TransferMode.Active,
FileTransferType = TransferType.Binary
};
directoryDeletingClient.Open(Username, Password);
directoryDeletingClient.ChangeDirectoryMultiPath(Directory);
directoryDeletingClient.DeleteDirectory(directoryName);
directoryDeletingClient.Close();
}
private void DirectoryListingFinished(object sender, GetDirListAsyncCompletedEventArgs e)
{
_directoryCollection = e.DirectoryListingResult;
_directoryFilesListingResetEvent.Set();
}
As the finished event of the upload is not called when cancelling, I need to set the ResetEvent manually in the DeleteDirectory-method.
Now, what am I doing here: I first list all files in the directory in order to delete them, as a filled folder can't be deleted.
This method GetDirListAsync is also async which means I need another ManualResetEvent as I don't want the form to freeze.
This ResetEvent is _directoryFilesListingResetEvent. It is declared like the _uploadResetEvent above.
Now, the problem is, it goes to the WaitOne-call of the _directoryFilesListingResetEvent and then it stucks. A deadlock appears and the form freezes. (I've also marked it in the code)
Why is that?
I tried to move the call of _uploadResetEvent.Set(), but it doesn't change.
Does anyone see the problem?
When I try to call the DeleteDirectory-method alone without any upload, it works as well.
I think the problem is that both ResetEvents use the same resource or something and overlap themselves, I don't know.
Thanks for your help.
You are not using this library correctly. The MREs you added cause deadlock. That started with _uploadResetEvent.WaitOne(), blocking the UI thread. This is normally illegal, the CLR ensures that your UI does not go completely dead by pumping a message loop itself. That makes it look like it is still alive, it still repaints for example. A rough equivalent of DoEvents(), although not nearly as dangerous.
But the biggest problem with it is that it will not allow your PutFileAsyncCompleted event handler to run, the underlying async worker is a plain BackgroundWorker. It fires its events on the same thread that started it, which is very nice. But it cannot call its RunWorkerCompleted event handler until the UI thread goes idle. Which is not nice, the thread is stuck in the WaitOne() call. Exact same story for what you are debugging now, your GetDirListAsyncCompleted event handler cannot run for the same reason. So it just freezes there without being able to make progress.
So eliminate _uploadResetEvent completely, rely on your UploadFinished() method instead. You can find out if it was canceled from the e.Cancelled property. Only then do you start the code to delete the directory. Follow the same pattern, using the corresponding XxxAsyncCompleted event to decide what to do next. No need for MREs at all.
Looking at the source, it appears FtpClient uses a BackgroundWorker to perform asynchronous operations. That means its completion event will be posted to whatever SynchronizationContext was set at the time the worker was created. I'll bet the completion of CancelAsync pushes you back onto the UI thread, which blocks when you call WaitOne on the directory list reset event. The GetDirListAsyncCompleted event gets posted to UI message loop, but since the UI thread is blocked, it will never run, and the reset event will never be set.
BOOM! Deadlock.
So I'm working on a windows 8 application with some asynchronous methods.
In one particular place I need the aplication to wait for the async method to finish, but it doesn't seem it sends the EventHandle it's state.
Here are the methods that need to work together:
public class Film : Page
private User loggedinUser = new User();
private EventWaitHandle handle = new AutoResetEvent(false);
private dynamic parameters;
protected override void OnNavigatedTo(NavigationEventArgs e)
{
this.parameters = e.Parameter;
LoadFacebookData(parameters);
handle.WaitOne();
LoadUserMovies(loggedinUser.UserName);
}
private async void LoadFacebookData(dynamic parameter)
{
//async code that gets info from facebook whichs determines what user is logged in
handle.Set();
}
private void LoadUserMovies(string username)
{
// irrelevant code
}
the moment the code hits the handle.WaitOne() bit, it stops working completely
In short you are not supposed to use async and wait on something. The standard mistake is to wait on the task returned (deadlock). You did a variation on this: You created an event and waited on it (also deadlock).
Solution: Either embrace async-await or don't do async at all. Can't do both in a mixed style (generally).
If await is available to you, this is a good start:
await LoadFacebookData(parameters);
Get rid of the event.
I suggest you to use the
await LoadFacebookData(parameters);
code line to get the execution continue on the same thread it was started once your async operation finishes.
As far as I can see, there is nothing wrong with the code you have posted. This leads me to believe that the problem is in the code you have commented away. The problem is probably that your code-execution never reaches the line
handle.Set();
This can be because of an exception, or your async code simply never completes. To "deal" with the possibility of an exception, you should probably use the builtin Wait-method of the Task-object, rather than your own EventWaitHandle.
You can remove the EventWaitHandle as a whole, and simply do:
LoadFacebookData(parameters).Wait();
This will propagate any exceptions that occur in the thread within.
Lets say I have a component called Tasking (that I cannot modify) which exposes a method “DoTask” that does some possibly lengthy calculations and returns the result in via an event TaskCompleted. Normally this is called in a windows form that the user closes after she gets the results.
In my particular scenario I need to associate some data (a database record) with the data returned in TaskCompleted and use that to update the database record.
I’ve investigated the use of AutoResetEvent to notify when the event is handled. The problem with that is AutoResetEvent.WaitOne() will block and the event handler will never get called. Normally AutoResetEvents is called be a separate thread, so I guess that means that the event handler is on the same thread as the method that calls.
Essentially I want to turn an asynchronous call, where the results are returned via an event, into a synchronous call (ie call DoSyncTask from another class) by blocking until the event is handled and the results placed in a location accessible to both the event handler and the method that called the method that started the async call.
public class SyncTask
{
TaskCompletedEventArgs data;
AutoResetEvent taskDone;
public SyncTask()
{
taskDone = new AutoResetEvent(false);
}
public string DoSyncTask(int latitude, int longitude)
{
Task t = new Task();
t.Completed = new TaskCompletedEventHandler(TaskCompleted);
t.DoTask(latitude, longitude);
taskDone.WaitOne(); // but something more like Application.DoEvents(); in WinForms.
taskDone.Reset();
return data.Street;
}
private void TaskCompleted(object sender, TaskCompletedEventArgs e)
{
data = e;
taskDone.Set(); //or some other mechanism to signal to DoSyncTask that the work is complete.
}
}
In a Windows App the following works correctly.
public class SyncTask
{
TaskCompletedEventArgs data;
public SyncTask()
{
taskDone = new AutoResetEvent(false);
}
public string DoSyncTask(int latitude, int longitude)
{
Task t = new Task();
t.Completed = new TaskCompletedEventHandler(TaskCompleted);
t.DoTask(latitude, longitude);
while (data == null) Application.DoEvents();
return data.Street;
}
private void TaskCompleted(object sender, TaskCompletedEventArgs e)
{
data = e;
}
}
I just need to replicate that behaviour in a window service, where Application.Run isn’t called and the ApplicationContext object isn’t available.
I've had some trouble lately with making asynchronous calls and events at threads and returning them to the main thread.
I used SynchronizationContext to keep track of things. The (pseudo)code below shows what is working for me at the moment.
SynchronizationContext context;
void start()
{
//First store the current context
//to call back to it later
context = SynchronizationContext.Current;
//Start a thread and make it call
//the async method, for example:
Proxy.BeginCodeLookup(aVariable,
new AsyncCallback(LookupResult),
AsyncState);
//Now continue with what you were doing
//and let the lookup finish
}
void LookupResult(IAsyncResult result)
{
//when the async function is finished
//this method is called. It's on
//the same thread as the the caller,
//BeginCodeLookup in this case.
result.AsyncWaitHandle.WaitOne();
var LookupResult= Proxy.EndCodeLookup(result);
//The SynchronizationContext.Send method
//performs a callback to the thread of the
//context, in this case the main thread
context.Send(new SendOrPostCallback(OnLookupCompleted),
result.AsyncState);
}
void OnLookupCompleted(object state)
{
//now this code will be executed on the
//main thread.
}
I hope this helps, as it fixed the problem for me.
Maybe you could get DoSyncTask to start a timer object that checks for the value of your data variable at some appropriate interval. Once data has a value, you could then have another event fire to tell you that data now has a value (and shut the timer off of course).
Pretty ugly hack, but it could work... in theory.
Sorry, that's the best I can come up with half asleep. Time for bed...
I worked out a solution to the async to sync problem, at least using all .NET classes.
Link
It still doesn't work with COM. I suspect because of STA threading. The Event raised by the .NET component that hosts the COM OCX is never handled by my worker thread, so I get a deadlock on WaitOne().
someone else may appreciate the solution though :)
If Task is a WinForms component, it might be very aware of threading issues and Invoke the event handler on the main thread -- which seems to be what you're seeing.
So, it might be that it relies on a message pump happening or something. Application.Run has overloads that are for non-GUI apps. You might consider getting a thread to startup and pump to see if that fixes the issue.
I'd also recommend using Reflector to get a look at the source code of the component to figure out what it's doing.
You've almost got it. You need the DoTask method to run on a different thread so the WaitOne call won't prevent work from being done. Something like this:
Action<int, int> doTaskAction = t.DoTask;
doTaskAction.BeginInvoke(latitude, longitude, cb => doTaskAction.EndInvoke(cb), null);
taskDone.WaitOne();
My comment on Scott W's answer seems a little cryptic after I re-read it. So let me be more explicit:
while( !done )
{
taskDone.WaitOne( 200 );
Application.DoEvents();
}
The WaitOne( 200 ) will cause it to return control to your UI thread 5 times per second (you can adjust this as you wish). The DoEvents() call will flush the windows event queue (the one that handles all windows event handling like painting, etc.). Add two members to your class (one bool flag "done" in this example, and one return data "street" in your example).
That is the simplest way to get what you want done. (I have very similar code in an app of my own, so I know it works)
Your code is almost right... I just changed
t.DoTask(latitude, longitude);
for
new Thread(() => t.DoTask(latitude, longitude)).Start();
TaskCompleted will be executed in the same thread as DoTask does. This should work.