Is there a way, using HttpClient, to differentiate a time-out "when we get no response from the server" from a "time exceeded" operation?
Let me explain our issue:
Case1: If we don't get any response from the server in 10 seconds then this is an issue.
Case2: If we get a response from the server, but the server continues to transfer data and it takes a while, maybe 30 seconds or more. Then this is not an issue.
Is there a way using .NET HttpClient class to handle this scenario? From what I tested specifying a TimeOut on HttpClient will put the same time-out for case1 and case2.
Here is the solution I managed to do:
// Create the cancelation token, when we don't get any feedback from server within 20 seconds
var cancelHeadersToken = new CancellationTokenSource();
cancelHeadersToken.CancelAfter(TimeSpan.FromSeconds(20)); // if we don't receive server headers after 20 seconds then something went wrong
// We have another cancelation token, that allows the user to cancel the request, so here we create a linked token source which uses both tokens
var linkedToken = CancellationTokenSource.CreateLinkedTokenSource(userCancelToken, cancelHeadersToken.Token);
// The linked token is then used in GetAsync and we use the overload which allows to specify the HttpCompletionOption
// We only want to receive headers and not all content
var httpMessage = await customClient.CustomizedHttpClient.GetAsync(address, HttpCompletionOption.ResponseHeadersRead, linkedToken.Token).ConfigureAwait(false);
// We can then download the content, and we still allow to cancel anything by the user
using (var memoryStream = new MemoryStream(100000)) { // 100ko by default
using (var stream = await httpMessage.Content.ReadAsStreamAsync().ConfigureAwait(false)) {
await stream.CopyToAsync(memoryStream, 10000, userCancelToken).ConfigureAwait(false); // copy to memory stream 10ko per 10ko
}
string data = "";
if (memoryStream.Length > 0) {
var headers = httpMessage.Content.Headers;
Encoding encoding;
if (headers != null && headers.ContentType != null && headers.ContentType.CharSet != null) {
encoding = Encoding.GetEncoding(headers.ContentType.CharSet);
} else {
encoding = Encoding.UTF8;
}
data = encoding.GetString(memoryStream.GetBuffer(), 0, (int)memoryStream.Length);
}
// Then you do whatever you want with data
}
You can use one of the variants of the various methods that accepts a CancellationToken.
If the cancel happens after the method's Task has completed, the cancellation is ignored and you can continue with e.g. processing the content from the result.
var client = new HttpClient();
var cancel = new CancellationTokenSource();
cancel.CancelAfter(TimeSpan.FromSeconds(10));
var resp = client.GetAsync("http://www.google.co.uk", cancel.Token).Result;
So, in the above, provided we get enough back from the server for the GetAsync to complete, the cancellation has no effect.
Related
I'm using hybrid connections to request data from a listener. If I can write and read to the connection, how can I know that the response I've read from the connection matches the request I've given it? For example:
private HybridConnectionClient _client = new HybridConnectionClient(***);
public override async Task<RelayResponse> SendAsync(RelayRequest request)
{
var stream = await _client.CreateConnectionAsync();
var writer = new StreamWriter(stream) { AutoFlush = true };
var reader = new StreamReader(stream);
var reqestSerialized = JsonConvert.SerializeObject(request);
await writer.WriteLineAsync(reqestSerialized);
string responseSerialized = await reader.ReadLineAsync();
var response = JsonConvert.DeserializeObject<RelayResponse>(responseSerialized);
return response;
}
If the listener on this connection is reading and responding to many requests at the same time, is there anyway to know that the next Readline() we do on the client side to get the response is the one that is associated with the request? Or is that something that has to be managed?
Understanding a bit more about azure relay hybrid connections, I understand this now.
There isn't really any concept of a synchronous request/response in the framework, but if you use a new connection for each request, and respond on that same connection in the listener, you can be sure the response is for the request you sent.
Spawn a new connection for each request, then make sure the response is written to that connection. So looking at Microsoft's listener example code, whenever listener.AcceptConnectionAsync() fires do all the message response on relayConnection, then go back to waiting at await listener.AcceptConnectionAsync();
while (true)
{
var relayConnection = await listener.AcceptConnectionAsync();
if (relayConnection == null)
{
break;
}
ProcessMessagesOnConnection(relayConnection, cts);
}
I have tried to create a simple console application.
We have a call system from 8x8 that provide a web streaming API but their documentation is very limited and nothing in C#.
The api service streams call statuses in near real time and I would like to get that 'stream' and be able to read and process it in realtime if possible. The response or Content Type is 'text/html'. But the actual body of the response can be declared as json - sample below:
{"Interaction":{"attachedData":{"attachedDatum":[{"attachedDataKey":"#pri","attachedDataValue":100},{"attachedDataKey":"callingName","attachedDataValue":999999999999},{"attachedDataKey":"cha","attachedDataValue":99999999999},{"attachedDataKey":"cnt","attachedDataValue":0},{"attachedDataKey":"con","attachedDataValue":0},{"attachedDataKey":"med","attachedDataValue":"T"},{"attachedDataKey":"pho","attachedDataValue":9999999999},{"attachedDataKey":"phoneNum","attachedDataValue":9999999999},{"attachedDataKey":"tok","attachedDataValue":999999999}]},"event":"InteractionCreated","inboundChannelid":9999999999,"interactionEventTS":9999999,"interactionGUID":"int-15b875d0da2-DJOJkDhDsrh3AIaFP8VkICv9t-phone-01-testist","resourceType":0}}
I have seen several posts concerning httpClient and the GetAsync methods but none of these appear to work as they appear to be for calls when a response is made, not something that constantly has a response.
Using fiddler for the call it does not appear to close so the stream is constantly running, so fiddler does not display any data until a separate user or instance connects.
When I use a browser the content is 'streamed' to the page and updates automatically and shows all the content (as above).
The api contains authentication so when another client connects and retrieves data the connected client closes and finally I am able to see the data that was gathering.
This is the code so and does return the big stream when another client connects but ideally I want a real time response and appears to just get stuck in the GETASYNC method:
var response = await client.GetAsync(address, HttpCompletionOption.ResponseHeadersRead);
if (response.IsSuccessStatusCode)
{
var responseContent = response.Content;
string responseString = await responseContent.ReadAsStringAsync();
Console.WriteLine(responseString);
}
Hopefully that's enough information for one of you clever people to help me in my predicament.
I was also having an issue consuming their streaming API and the examples I found that worked with the Twitter and CouchBase streaming API's did not work with 8x8. Both Twitter and CouchBase send line terminators in their pushes so the solution relied on ReadLine to pull in the feed. Since 8x8 does not send terminators you'll need to use ReadBlock or better ReadBlockAsync.
The following code shows how to connect using credentials and consume their feed:
private static async Task StreamAsync(string url, string username, string password)
{
var handler = new HttpClientHandler()
{
Credentials = new NetworkCredential {UserName = username, Password = password},
PreAuthenticate = true
};
// Client can also be singleton
using (var client = new HttpClient(handler))
{
client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
var request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Connection.Add("keep-alive");
using (var response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead))
{
using (var body = await response.Content.ReadAsStreamAsync())
{
using (var reader = new StreamReader(body))
{
while (!reader.EndOfStream)
{
var buffer = new char[1024];
await reader.ReadBlockAsync(buffer, 0, buffer.Length);
Console.WriteLine(new string(buffer));
}
}
}
}
}
}
I'm working with two web API projects that get communicated. The first web API calls the second one using an HttpClient class.
What I would like to do is to set a short timeout (500 ms) when I call the second web API, and if I don't get response in that time, just to skip the next lines that process the result in the client, but continue processing the request at server side(second API).
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Clear();
client.Timeout = this.Timeout; // (500ms)
HttpResponseMessage response = client.PostAsJsonAsync(EndPoint, PostData).Result;
if (response.IsSuccessStatusCode)
{
return response.Content.ReadAsAsync<T>().Result;
}
else
{
throw new CustomException()
}
}
It works in the first API side, however in the second API(server), I get the following exceptions:
"A task was canceled."
"The operation was cancelled."
at System.Threading.CancellationToken.ThrowOperationCanceledException()
at System.Threading.CancellationToken.ThrowIfCancellationRequested()
I think it is caused by the small timeout of the call, that ends when the second API is still processing the result.
How could I avoid this behaviour in the second API and continue processing the request ?
Thanks in advance.
That is the expected behavior. When you set a timeout and the call does not respond in that amount of time, the task is canceled and that exception is thrown.
And by the way, do not use .Result. That will cause blocking. Mark your method async and use await.
The whole thing should look something like this:
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Clear();
client.Timeout = this.Timeout; // (500ms)
try
{
HttpResponseMessage response = await client.PostAsJsonAsync(EndPoint, PostData);
if (response.IsSuccessStatusCode)
{
return await response.Content.ReadAsAsync<T>();
}
else
{
throw new CustomException()
}
}
catch (TaskCanceledException)
{
// request did not complete in 500ms.
return null; // or something else to indicate no data, move on
}
}
I am working with OAuth at the moment. The problem with the current code is it doesn't wait until the user allows the application on the site and gets the proper key and secret. I was using a threading type wait but, sometimes it not long enough...some users are slower then others. I have attached a snippet of my code. What I would like to know is where to insert a while statement, or should I even use that ?
public OAuthToken GetRequestToken(Uri baseUri, string consumerKey, string consumerSecret)
{
var uri = new Uri(baseUri, "oauth/request_token");
uri = SignRequest(uri, consumerKey, consumerSecret);
var request = (HttpWebRequest) WebRequest.Create(uri);
request.Method = WebRequestMethods.Http.Get;
var response = request.GetResponse();
var queryString = new StreamReader(response.GetResponseStream()).ReadToEnd();
var parts = queryString.Split('&');
var token = parts[1].Substring(parts[1].IndexOf('=') + 1);
var secret = parts[0].Substring(parts[0].IndexOf('=') + 1);
return new OAuthToken(token, secret);
}
You should switch over to the newer System.Net.Http and System.Net.Http.WebRequest libraries that come with .NET now. These all use the new async programming stuff that is available with .NET 4.5.
You can call a request (returning you a task object that you can wait on) and automatically pause the thread for the response. The UI won't respond, as normal. That is probably the easiest thing to do if you don't understand how the new async and await keywords work. For more information on them, see http://msdn.microsoft.com/en-us/library/hh191443.aspx
Here is your code doing things with the new libraries:
using System.Net.Http;
public OAuthToken GetRequestToken(Uri baseUri, string consumerKey, string consumerSecret)
{
var uri = new Uri(baseUri, "oauth/request_token");
uri = SignRequest(uri, consumerKey, consumerSecret);
var message = new HttpRequestMessage(new HttpMethod("GET"), uri);
var handler = new WebRequestHandler();
var client = new HttpClient(handler);
// Use the http client to send the request to the server.
Task<HttpResponseMessage> responseTask = client.SendAsync(message);
// The responseTask object is like a wrapper for the other task thread.
// We can tell this task object that we want to pause our current thread
// and wait for the client.SendAsync call to finish.
responseTask.Wait();
// - Once that thread finishes, and the code continues on, we need to
// tell it to read out the response data from the backing objects.
// - The responseTask.Result property represents the object the async task
// was wrapping, we want to pull it out, then use it and get the content
// (body of the response) back.
// - Getting the response actually creates another async task (the
// .ReadAsStringAsync() call) but by accessing the .Result
// property, it is as if we called .ReadAsStringAsync().Wait(); Except that
// by using Result directly, we not only call Wait() but we get the resulting,
// wrapped object back. Hope that didn't confuse you much :)
var queryString = responseTask.Result.Content.ReadAsStringAsync().Result;
// And all your other normal code continues.
var parts = queryString.Split('&');
var token = parts[1].Substring(parts[1].IndexOf('=') + 1);
var secret = parts[0].Substring(parts[0].IndexOf('=') + 1);
return new OAuthToken(token, secret);
}
Why Not use a modal pop up and then call the authentication class on the submit button
I have to send an async POST to a web service and I want to send/retrieve the data asynchronously.
I also need to send the request but only wait a maximum of 1500 ms waiting for a response. If I don't get a response, the program should continue on (this is a service making an external web service call). I want to offload these service calls to IOCP's instead of blocking for a long time and waiting for them to return. I only want to block for 1500 ms total.
Here's what I have so far:
var httpRequest = (HttpWebRequest)WebRequest.Create(#"urltoPostTo");
httpRequest.Method = "POST";
byte[] data = Encoding.ASCII.GetBytes("test-post");
httpRequest.ContentLength = data.Length;
var asyncTask = Task.Factory.FromAsync<Stream>(httpRequest.BeginGetRequestStream, httpRequest.EndGetRequestStream, httpRequest)
.ContinueWith(response =>
{
var localStream = response.Result;
if (localStream != null)
{
localStream.Write(data, 0, data.Length);
localStream.Close();
}
}); //how do I do the continuation for BeginGetResponse and EndGetResponse from here?
I have a couple of requirements that unfortunately I can't change.
I am using Visual Studio 2010 targeting 4.0
I cannot use the Async BCL
I would like to try and use Task.FromAsync
This has already been answered in Implementing extension method WebRequest.GetResponseAsync with support for CancellationToken with a GetResponseAsync extension method that properly handles timeouts and CancellationTokens. When Request.Timeout expires, the method calls Request.Abort before returning the Task in its cancelled state.
The reason for such involved coding is that it's the client's (yours) responsibility to properly handle timeouts, so you can't depend on FromAsync to handle the timeout expiration. Perhaps this is the reason why FromAsync doesn't accept a cancellation token.
Another option is to avoid cancelling the request itself and cancel the continuation. You can use the ContinueWith overload that accepts a CancellationToken and call CancellationTokenSource.CancelAfter to set the cancellation timeout.
This would allow your code to ignore the results and keep running but it wouldn't break the connection to the server and wouldn't stop the background thread from processing any potentially expensive results.
You could write something like this:
var tcs=new CancellationTokenSource();
var asyncTask = Task.Factory.FromAsync<Stream>(httpRequest.BeginGetRequestStream, httpRequest.EndGetRequestStream, httpRequest)
.ContinueWith(response =>
{...},
cts.Token);
cts.CancelAfter(1500);
Note that the call to CancelAfter is done after starting the asynchronous task.
I would prefer Reed Copsey's extension method in a busy site because a high number of cancelled but outstanding requests can easily exhaust the thread pool, consume a lot of memory without a reason and consume potentially expensive connections to external systems.
why not to work with HttpClient?
webApiHttpClient.PostAsJsonAsync(GetFullAPI("api/Subscribe"), obj)
.ContinueWith(res => _logger.InfoFormat("Subscribe result: {0}", res.Result.StatusCode));
Try this method helper.
public static Task<string> Post(string url, Encoding encoding, string content)
{
var httpRequest = (HttpWebRequest)WebRequest.Create(url);
httpRequest.Method = "POST";
byte[] data = encoding.GetBytes(content);
httpRequest.ContentLength = data.Length;
TaskCompletionSource<string> result = new TaskCompletionSource<string>();
Task.Factory.FromAsync<Stream>(httpRequest.BeginGetRequestStream, httpRequest.EndGetRequestStream, httpRequest)
.ContinueWith(requestStreamTask =>
{
try
{
using (var localStream = requestStreamTask.Result)
{
localStream.Write(data, 0, data.Length);
localStream.Flush();
}
Task.Factory.FromAsync<WebResponse>(httpRequest.BeginGetResponse, httpRequest.EndGetResponse, httpRequest)
.ContinueWith(responseTask =>
{
try
{
using (var webResponse = responseTask.Result)
using (var responseStream = webResponse.GetResponseStream())
using (var sr = new StreamReader(responseStream, encoding))
{
result.SetResult(sr.ReadToEnd());
}
}
catch (Exception e)
{
result.SetException(e);
}
}, TaskContinuationOptions.AttachedToParent);
}
catch (Exception e)
{
result.SetException(e);
}
}, TaskContinuationOptions.AttachedToParent);
return result.Task;
}