Develop Reference

Can this "Synchronous" Code Cause Deadlocks?

My company has a Nuget Package they wrote that can do various common tasks easily for you. One of which is making HTTP requests. Normally I always make my HTTP requests asynchronous, however in this Nuget package is the following code:
protected T GetRequest<T>(string requestUri)
{
// Call the async method within a task to run it synchronously
return Task.Run(() => GetRequestAsync<T>(requestUri)).Result;
}
Which calls this function:
protected async Task<T> GetRequestAsync<T>(string requestUri)
{
// Set up the uri and the client
string uri = ParseUri(requestUri);
var client = ConfigureClient();
// Call the web api
var response = await client.GetAsync(uri);
// Process the response
return await ProcessResponse<T>(response);
}
My question is, is this code really running synchronously by just wrapping the GetRequestAsync(requestUri) inside a Task.Run and calling .Result on the returned task? This seems like a deadlock waiting to happen, and we are seeing issues in areas of our app that utilize this function when running at higher loads.

Accessing Task.Result will block the current thread until the Task is complete, so it is not asynchronous.
As for deadlocks, this shouldn't happen, as Task.Run uses another thread for GetRequestAsync, which is not being blocked by the call to Result.

The reason that will not cause a deadlock is because Task.Run will push the delegate to be executed in a threadpool thread. Threadpool threads has no SynchronizationContext therefore no deadlock happens as there is no sync context to lock on between the async method GetRequestAsync and the caller. Same as you could have called .Result directly on the actual async method as well within Task.Run() block and that would not have caused a deadlock either.
Very inefficient though as you freeze 1 thread ie. 1 Core in the CPU do nothing but wait for the async method and I/O calls within it to complete. That s probably why you see a freeze in high load scenarios.
If you have a sync/async deadlock issue due to capturing sync context and blocking on async call, the deadlock will happen irrespective of load on a single call..

This won't cause a deadlock. But it's surely a resources wasting as one of the threads may be blocked.
The deadlock though may be possible if GetRequest looked like this:
protected T GetRequest<T>(string requestUri)
{
var task = GetRequestAsync<T>(requestUri);
return task.Result;
// or
// return GetRequestAsync<T>(requestUri).Result;
}
In example above you can see that I call GetRequestAsync within the current thread. Let's give the thread a number 0. Consider this line from the GetRequestAsync - await client.GetAsync(uri). .GetAsync is executed by a thread 1. After .GetAsync is done, default task scheduler is trying to return execution flow to the thread that executed the line - to the thread 0. But the thread that executed the line (0) is blocked right now as after we executed GetRequestAsync(), we are blocking it (thread 0) with task.Result. Hence our thread 0 remains blocked as it cannot proceed with execution of GetRequestAsync after await client.GetAsync(uri) is done nor it can give us the Result.
It's a pretty common mistake and I suppose you meant this one when asked about the deadlock. Your code is not causing it because you are executing GetRequestAsync from within another thread.

How to prevent HttpListener from aborting pending requests on stoppage? HttpListener.Stop is not working

I have a problem here. In below code the async/await pattern is used with HttpListener. When the request is sent via HTTP "delay" query string argument is expected and its value causes the server to delay the mentioned request processing for the given period. I need the server to process the pending requests even after the server stopped receiving new requests.
static void Main(string[] args)
{
HttpListener httpListener = new HttpListener();
CountdownEvent sessions = new CountdownEvent(1);
bool stopRequested = false;
httpListener.Prefixes.Add("http://+:9000/GetData/");
httpListener.Start();
Task listenerTask = Task.Run(async () =>
{
while (true)
{
try
{
var context = await httpListener.GetContextAsync();
sessions.AddCount();
Task childTask = Task.Run(async () =>
{
try
{
Console.WriteLine($"Request accepted: {context.Request.RawUrl}");
int delay = int.Parse(context.Request.QueryString["delay"]);
await Task.Delay(delay);
using (StreamWriter sw = new StreamWriter(context.Response.OutputStream, Encoding.Default, 4096, true))
{
await sw.WriteAsync("<html><body><h1>Hello world</h1></body></html>");
}
context.Response.Close();
}
finally
{
sessions.Signal();
}
});
}
catch (HttpListenerException ex)
{
if (stopRequested && ex.ErrorCode == 995)
{
break;
}
throw;
}
}
});
Console.WriteLine("Server is running. ENTER to stop...");
Console.ReadLine();
sessions.Signal();
stopRequested = true;
httpListener.Stop();
Console.WriteLine("Stopped accepting requests. Waiting for the pendings...");
listenerTask.Wait();
sessions.Wait();
Console.WriteLine("Finished");
Console.ReadLine();
httpListener.Close();
}
The exact problem here, is that when the server is stopped the HttpListener.Stop is called, but all the pending requests are aborted immediately, i.e. the code is unable to send the responses back.
In non-async/await pattern (i.e. simple Thread based implementation) I have a choice to abort the thread (which I suppose is very bad) and this will allow me to process pending requests, because this simply Aborts HttpListener.GetContext call.
Can you please point me out, what am I doing wrong and how to can I prevent HttpListener to abort pending requests in async/await pattern?

It seems that when HttpListener closes the request queue handle, the requests in progress are aborted. As far as I can tell, there is no way to avoid having HttpListener do that - apparently, it's a compatibility thing. In any case, that's how its GetContext-ending system works - when the handle is closed, the native method GetContext call to actually get the request context returns an error immediately.
Thread.Abort doesn't help - really, I've yet to see a place where Thread.Abort is used correctly outside of the "application domain unloading" scenario. Thread.Abort can only ever abort managed code. Since your code is currently running native, it will only be aborted when it returns back to managed code - which is almost exactly equivalent to just doing this:
var context = await httpListener.GetContextAsync();
if (stopRequested) return;
... and since there's no better cancellation API for HttpListener, this is really your only option if you want to stick with HttpListener.
The shutdown will look like this:
stopRequested = true;
sessions.Wait();
httpListener.Dispose();
listenerTask.Wait();
I'd also suggest using CancellationToken instead of a bool flag - it handles all the synchronization woes for you. If that's not desirable for some reason, make sure you synchronize access to the flag - contractually, the compiler is allowed to omit the check, since it's impossible for the flag to change in single-threaded code.
If you want to, you can make listenerTask complete sooner by sending a dummy HTTP request to yourself right after setting stopRequested - this will cause GetContext to return immediately with the new request, and you can return. This is an approach that's commonly used when dealing with APIs that don't support "nice" cancellation, e.g. UdpClient.Receive.

Is it safe to use ContinueWith as a "finally" operation?

Consider a piece of code like:
private Task<string> Download()
{
var wc = new WebClient();
Task<string> backgroundDownload = wc.DownloadStringTaskAsync(this.Uri);
// Make sure the WebClient is disposed no matter what.
backgroundDownload.ContinueWith((downloadTask) => { wc.Dispose(); });
return backgroundDownload;
}
Can I be certain that the WebClient.Dispose() call occurs and that any exception produced is rethrown to the caller as if there was no call to ContinueWith?
Can clients observe this ContinueWith? (e.g. will later calls to ContinueWith remove the Dispose call?)

With the code that you have you can be certain that the continuation will be fired regardless of whether or not the code completed successfully, was cancelled, or throws an exception.
The one potential problem with the solution that you have is that other continuations can potentially run, before, during, or after the web client is disposed. If you don't have a problem with other continuations running before this cleanup runs then what you have is fine. If that's a problem then you'll need to return the continuation, not the original task, but you'll also need to propagate the result (and exceptions/cancellation) correctly. The use of async makes all of this way easier:
private async Task<string> Download()
{
using(var wc = new WebClient())
return await wc.DownloadStringTaskAsync(this.Uri);
}

First of all, the continuation will be performed even if a regular exception occurred. However it is less likely to run than a regular finally block in the eventuality of exceptional conditions such as an OutOfMemoryException.
Now I would not try to dispose the webclient. Remember that disposing is an optimization, because native resources will be disposed by the finalizer anyway. The only reason we are disposing is that a finalizer is expensive because it triggers a second GC pass.
But in the order to perform your optimization the system may have to create new threads. Besides you may be prolongating the lifetime of your webclient a lot if the threadpool is filled with long running tasks.
Basically, you have to choose the lesser of two evils and I am not convinced that one less GC run is worth what you're doing. You should consider this decision in the context of your application.

Why does an async method not return the awaitable immediately?

I had thought this was a reasonable pattern to use to call WebClient DownloadData asynchronously in an event handler:
private async void button1_Click(object sender, EventArgs e)
{
WebClient wc = new WebClient();
//wc.Proxy = null;
byte[] bytes = await wc.DownloadDataTaskAsync("http://www.google.com");
label1.Text = Encoding.ASCII.GetString(bytes);
}
But I find DownloadDataTaskAsync blocks for about 5 seconds before returning (unless the wc.Proxy = null statement is uncommented). What's the point of a method being awaitable if it can at a whim perform non-trivial work before even returning the task?
Presumably this means to be safe I should never call xAsync methods as above but instead should always wrap them in a Task.Run() myself to be sure. Or not?

This is a known issue with WebClient/HttpWebRequest: the proxies and DNS lookups are always done synchronously. This is a bug, but Microsoft has pretty much decided not to fix it for backwards compatibility reasons.
The first thing I'd recommend is to use HttpClient. If that doesn't work and you need asynchrony, then you can wrap the call in Task.Run.

It turns out that WebClient.DownloadDataTaskAsync is calling HttpWebRequest.BeginGetResponse
MSDN points out:
The BeginGetResponse method requires some synchronous setup tasks to
complete (DNS resolution, proxy detection, and TCP socket connection,
for example) before this method becomes asynchronous. As a result,
this method should never be called on a user interface (UI) thread
because it might take considerable time (up to several minutes
depending on network settings) to complete the initial synchronous
setup tasks before an exception for an error is thrown or the method
succeeds.
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse(v=vs.110).aspx
Unfortunately the MSDN documentation for WebClient.DownloadDataTaskAsync says:
This operation will not block.
which seems to not be strictly true.

Using RegisterWaitForSingleObject if operation completes first

I'm doing some asynchronous network I/O using Begin/End style methods. (It's actually a query against Azure Table Storage, but I don't think that matters.) I've implemented a client side timeout using the ThreadPool.RegisterWaitForSingleObject(). This is working fine as far as I can tell.
Because ThreadPool.RegisterWaitForSingleObject() takes a WaitHandle as an argument, I have to begin the I/O operation, then execute ThreadPool.RegisterWaitForSingleObject(). It seems like this introduces the possibility that the I/O completes before I even register the wait.
A simplified code sample:
private void RunQuery(QueryState queryState)
{
//Start I/O operation
IAsyncResult asyncResult = queryState.Query.BeginExecuteSegmented(NoopAsyncCallback, queryState);
//What if the I/O operation completes here?
queryState.TimeoutWaitHandle = ThreadPool.RegisterWaitForSingleObject(asyncResult.AsyncWaitHandle, QuerySegmentCompleted, asyncResult, queryTimeout, true);
}
private void QuerySegmentCompleted(object opState, bool timedOut){
IAsyncResult asyncResult = opState as IAsyncResult;
QueryState state = asyncResult.AsyncState as QueryState;
//If the I/O completed quickly, could TimeoutWaitHandle could be null here?
//If so, what do I do about that?
state.TimeoutWaitHandle.Unregister(asyncResult.AsyncWaitHandle);
}
What's the proper way to handle this? Do I still need to worry about Unregister()'ing the AsyncWaitHandle? If so, is there a fairly easy way to wait for it to be set?

Yep, you and everyone else has this problem. And it does not matter if the IO completed synchronously or not. There is still a race between the callback and the assignment. Microsoft should have provided the RegisteredWaitHandle to that callback function automatically. That would have solved everything. Oh well, hindsight is always 20-20 as they say.
What you need to do is keep reading the RegisteredWaitHandle variable until it is no longer null. It is okay to do this in a tight loop because the race is subtle enough that the loop will not be spinning around very many times.
private void RunQuery(QueryState queryState)
{
// Start the operation.
var asyncResult = queryState.Query.BeginExecuteSegmented(NoopAsyncCallback, queryState);
// Register a callback.
RegisteredWaitHandle shared = null;
RegisteredWaitHandle produced = ThreadPool.RegisterWaitForSingleObject(asyncResult.AsyncWaitHandle,
(state, timedout) =>
{
var asyncResult = opState as IAsyncResult;
var state = asyncResult.AsyncState as QueryState;
while (true)
{
// Keep reading until the value is no longer null.
RegisteredWaitHandle consumed = Interlocked.CompareExchange(ref shared, null, null);
if (consumed != null)
{
consumed.Unregister(asyncResult.AsyncWaitHandle);
break;
}
}
}, asyncResult, queryTimeout, true);
// Publish the RegisteredWaitHandle so that the callback can see it.
Interlocked.CompareExchange(ref shared, produced, null);
}

You do not need to Unregister if the I/O completed before the timeout as it was the completion that signalled your callback. In fact upon reading the docs of the Unregister method it seems totally unnecessary to call it as you are executing only once and you are not Unregistering in an unrelated method.
http://msdn.microsoft.com/en-us/library/system.threading.registeredwaithandle.unregister.aspx
If a callback method is in progress when Unregister executes, waitObject is not signaled until the callback method completes. In particular, if a callback method executes Unregister, waitObject is not signaled until that callback method completes.