I have the following C# method that accepts a URL as an input and returns the text data that exists at that location:
public string GetWebData(string uri)
{
string response = string.Empty;
try
{
var request = WebRequest.Create(uri);
request.BeginGetResponse(result =>
{
var httpRequest = (HttpWebRequest)result.AsyncState;
var httpResponse = (HttpWebResponse)httpRequest.EndGetResponse(result);
using (var reader = new StreamReader(httpResponse.GetResponseStream()))
{
response = reader.ReadToEnd();
}
}, request);
}
catch (WebException)
{
response = string.Empty;
}
return response;
}
However, the reader.ReadToEnd(); method returns an empty string. I'm not sure if I'm doing anything wrong, since the method seems to be syntactically identical to all the tutorials I've consulted. What am I doing wrong?
You're returning response immediately - even though the callback which assigns a useful value to response will only fire later. Do you understand how BeginGetResponse works? It would be worth studying the documentation and examples carefully.
Why are you using asynchronous methods if you actually want to return the value as soon as the method finishes? If asynchronous methods are all you've got (e.g. you're on Windows Phone) then you should stick with that asynchronous idiom - don't try to make it synchronous.
As an aside, swallowing exceptions like this is a really bad idea too - you should almost never continue as if everything's okay, and even if you do want to ignore the error as far as the user experience is concerned, you should almost certainly be logging the exception.
Additionally, when you fetch the response you should use a using statement as WebResponse implements IDisposable.
Related
Is there a good way of propagating an object on a successful response or the status code and response body to consumers of a typed HTTP client in ASP.NET Core?
Given the following API service:
public class TestApiService
{
private readonly HttpClient _httpClient;
public TestApiService(HttpClient httpClient)
{
httpClient.BaseAddress = new Uri("https://localhost:5000");
_httpClient = httpClient;
}
public async Task<string> GetVersion()
{
var response = await _httpClient.GetAsync("/api/v1/version");
if (response.IsSuccessStatusCode)
{
return await response.Content.ReadAsStringAsync();
}
return null;
}
}
which is registered with the DI container via:
services.AddHttpClient<TestApiService>();
I would like to return the string value from the TestApiService.GetVersion() method if the response was successful or if the response was not successful return the status code and the response body.
It doesn't appear to be possible to do something like:
public async Task<string> GetVersion()
{
var response = await _httpClient.GetAsync("/api/v1/version");
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
and get the desired outcome because the HttpRequestException thrown from HttpResponseMessage.EnsureSuccessStatusCode() does not include the status code or the response body.
There is an open issue about this on GitHub but I'm not sure if it will get implemented anytime soon or not.
While ActionResult does exist it seems to really be for the controller layer so I'm not sure if using it here is an appropriate use of that class or not or if there is a better way of getting the desired outcome?
It should be possible to create my own exception class and throw that from the service but I would really like to avoid that if there is some built-in mechanism that is usable instead.
Remove response.EnsureSuccessStatusCode() this is basically checking the status and if its not a 200 throwing the exception. Consider using response.IsSuccessStatusCode or check the response status code manually. Either way will prevent the raising of the exception which you don't want.
if (HttpStatusCode.Ok == response.StatusCode)
{
// Read your result
}
else if ( // handle the specific failure case was it a 404 or a 401)
{
string value = await response.Content?.ReadAsStringAsync();
// Read your failed result
return $"{response.StatusCode} {value}".Trim()";
}
The next question is how you handle and communicate failure to the callee's of your service? Do you want your service to be opaque to your client application?
Since your code is only returning a string, have you considered either returning something else such as an encompassing object { Success = true|false, Error = "", ErrorCode = 1234, Data = "value"} or simply throwing an appropriate exception to communicate the nature of the failure. E.g. You might want to throw an appropriate exception, e.g. TestApiException where TestApiException might have the ErrorCode or whatever you need on it.
I was reading the following topic http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html
and decided to write a common utility method in my library to do a GET on remote url via HTTPClient
public static async Task<T> GetAsync<T>(HttpGetObject getObject)
{
string baseUrl = getObject.BaseUrl;
string actionUrl = getObject.ActionRelativeUrl;
string acceptType = getObject.AcceptType;
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(baseUrl);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(acceptType));
AddCustomHeadersToHttpClient(client, getObject);
// HTTP GET
HttpResponseMessage httpResponseMessage = await client.GetAsync(actionUrl).ConfigureAwait(false);
if (httpResponseMessage.IsSuccessStatusCode)
{
T response = await httpResponseMessage.Content.ReadAsAsync<T>().ConfigureAwait(false);
return response;
}
else
{
string message = httpResponseMessage.Content.ReadAsStringAsync().Result;
throw new Exception(message);
}
}
return default(T);
}
I know the "await httpResponseMessage.Content.ReadAsAsync().ConfigureAwait(false)" will prevent the deadlock in the above code
First:
My query is for "string message = httpResponseMessage.Content.ReadAsStringAsync().Result" line, will .Result can cause deadlock or not in that line?
Second:
If I call that code from UI like this:
public static object DoGet()
{
// Build getObject
var task = Utility.GetAsync(getObject);
task.Wait();
var response = task.Result;
return response;
}
Will that cause a deadlock?
Please note that I know to avoid all the mess with async-await, all the methods from UI to DAL must be async-await but I am not in position at this moment to change all that structure, my goal at this moment is to call HttpClient library and do a few GET operations.
So my questions is that will the above code can cause a deadlock?
Third:
Is task.Wait(); even needed in the above code?
In the general case, you should assume that yes, calling .Result or .Wait() on anything awaitable is dangerous and can deadlock (unless you are the library issuing the task, and you understand the full context). It is possible that it will work OK in some specific cases, but you should not rely on that behaviour, even if it works today.
I have a problem with creating new task in Asana from my app.
Post method:
protected static T Post<T>(string route, object action = null, object parameters = null) where T : BaseResult, new()
{
var result = new T();
try
{
var actionUrl = GetActionUrl(route, action);
var request = (HttpWebRequest)WebRequest.Create(actionUrl);
request.Credentials = CredentialCache.DefaultCredentials;
request.Accept = "application/json";
request.Method = WebRequestMethods.Http.Post;
request.Proxy.Credentials = CredentialCache.DefaultCredentials;
request.Headers.Add("Authorization: Bearer " + ApiKey);
if (parameters != null)
{
var contentJSON = JsonConvert.SerializeObject(parameters);
request.ContentType = "application/json";
using (var s = request.GetRequestStream())
using (var sw = new StreamWriter(s, Encoding.UTF8))
sw.Write(contentJSON);
}
var response = (HttpWebResponse) request.GetResponse();
using (var reader = new StreamReader(response.GetResponseStream()))
{
var data = reader.ReadToEnd();
result = JsonConvert.DeserializeObject<T>(data);
}
}
catch (Exception ex)
{
result.IsOk = false;
result.Message = ex.GetMessage();
}
return result;
}
Action URL: https://app.asana.com/api/1.0/workspaces/MyWorkspace/tasks
JSON:{"data":{"name":"TestTask1","notes":"Test note","workspace":"*MyWorkspace*","assignee":"*MyAssignee"}}
But Asana returns me "The remote server returned an error: (400) Bad Request."
If I change request.ContentType to "application/x-www-form-urlencoded", I get no errors, but Asana returns me new task with empty fields.
What my next steps to fix issue should be?
Thank you
If you're using an ApiKey (and not a Personal Access Token), I believe that your Authorization Header should be
"Authorization: Basic " + EncodedAuthInfo
where
EncodedAuthInfo = Convert.ToBase64String(Encoding.Default.GetBytes(ApiKey + ":"))
See How do I connect to the Asana Rest API using c#? or the Using Basic Authentication section in https://asana.com/developers/documentation/getting-started/auth for details on using basic authentication.
I'm also a little confused by what you mean when you say that
JSON = {"data": {"name": "TestTask1"} ...
Is this the HTTP response that you expect?
Anyways, hopefully what I've outlined helps.
Hmm. I think I've got what's blocking you sorted out.
Imagine the scenario where you post to
https://app.asana.com/api/1.0/workspaces/123456/tasks
and you pass in the request body the parameter
"workspace":"789012"
What should the Asana platform do with this data? You've inadvertently specified the workspace twice with conflicting numbers. For this reason, you cannot specify the workspace id in the data when hitting an endpoint which also contains the workspace id in the URL.
The documentation is confusing on this point, because we don't clarify which parameters are found in the URL and which parameters are found in the JSON in the request body. I'm actually fixing this very soon! If this is indeed what's causing the issue, I'm sorry that we were not clear on this.
Personally, I think it might be a better user experience to allow the workspace to be duplicated in the parameter data if and only if it's identical to the one in the URL, but right now, we simply check to see that there is only one value for the workspace id. If there are more than one, even if they are the same one, we return the 400 error code.
You might consider parsing the response body, even on errors. In it, we try to provide fairly decent information about what was wrong about the request. For example, when testing my hunch about your request, what I got back was:
"errors":[{"message":"Duplicate field: workspace", ...}]
If we've done a good job about sending back informative messages, I hope you'll find this even more useful than an Asana sandbox! If this is not the issue, feel free to comment; I'll be happy to dive into this further.
We are using an HttpClient to post json to a restful web service. In one instance, we are running into something that has us baffled. Using tools like postman, fiddler etc, we can post to an endpoint and see that it is working. When we do the same with HttpClient.PostAsJsonAsync, we can verify in the software we are posting to that it received the data just fine. However, our PostAsJsonAsync will always eventually time out rather than give us a response.
We have worked with the team that created the service we are consuming, plus our additional testing on our side, and we have not yet been able to truly time out that service.
Every time we do a post with HttpClient, we then can verify that the target software we post to does indeed get the data. Any time we do a post to that target software from any other tool, we always very quickly see a response with status code of 200. Something about HttpClient is failing to accept the response from this particular service. Does anyone have an idea what we can look at from here?
Here's the code (though it is so cookie cutter that I hardly feel it is needed)
public string PostData(string resourcePath, Object o, Boolean isCompleteUrl = false, int timeoutMinutes = -1)
{
using (var client = new HttpClient())
{
if (timeoutMinutes > 0)
{
client.Timeout = new TimeSpan(0,timeoutMinutes,0);
}
var useUrl = isCompleteUrl ? resourcePath : ApiBase + resourcePath;
var response = client.PostAsJsonAsync(useUrl, o).Result;
if(response.StatusCode == System.Net.HttpStatusCode.OK)
{
return response.Content.ReadAsStringAsync().Result;
}
return "";
}
}
This:
var response = client.PostAsJsonAsync(useUrl, o).Result;
Is causing you code to deadlock. This is often the case when blocking on async API's, and that's why you're experiencing the "I don't see any response coming back" effect.
How is this causing a deadlock? The fact that you are executing this request in an environment that contains a synchronization context, perhaps one which belongs to the UI. It's executing the async request, and when the response arrives, it continues via an IO completion thread which attempts to post the continuation onto that same UI context, which is currently blocked by your .Result call.
If you want to make an HTTP request synchronously, use WebClient instead. If you want to take advantage of the async API properly, then await instead of block with .Result.
I had the same issue and this SO answer fixed it for me.
In a nutshell, you have to use the ConfigureAwait(false) extension to avoid the deadlock:
var response = await client.PostAsJsonAsync(useUrl, o).ConfigureAwait(false);
Is there a reason why you're not following the async await pattern? You're calling an async method, but not awaiting it. You didn't say if the code calling your REST service was a Windows Forms or ASP.NET application, but that .Result is probably causing you issues.
Can you restructure your method like this:
public async Task<string> PostData(string resourcePath, Object o, Boolean isCompleteUrl = false, int timeoutMinutes = -1)
{
using (var client = new HttpClient())
{
if (timeoutMinutes > 0)
{
client.Timeout = new TimeSpan(0,timeoutMinutes,0);
}
var useUrl = isCompleteUrl ? resourcePath : ApiBase + resourcePath;
var response = await client.PostAsJsonAsync(useUrl, o);
if(response.StatusCode == System.Net.HttpStatusCode.OK)
{
return await response.Content.ReadAsStringAsync();
}
return "";
}
}
This is a slight modification to #Justin Helgerson's solution. There are 2 blocking .Result calls in your method; once you go async you should fix them both.
public async Task<string> PostDataAsync(string resourcePath, Object o, Boolean isCompleteUrl = false, int timeoutMinutes = -1)
{
using (var client = new HttpClient())
{
if (timeoutMinutes > 0)
{
client.Timeout = new TimeSpan(0,timeoutMinutes,0);
}
var useUrl = isCompleteUrl ? resourcePath : ApiBase + resourcePath;
var response = await client.PostAsJsonAsync(useUrl, o);
if(response.StatusCode == System.Net.HttpStatusCode.OK)
{
return await response.Content.ReadAsStringAsync();
}
return "";
}
}
Note I've also renamed the method to PostDataAsync in accordance with the TAP pattern.
System.Net.ServicePointManager.Expect100Continue = false;
That one line of code in our case fixed the problem. A developer from a different team offered that suggestion, and it works. I have yet to google it and read up on it enough to offer an explanation of what that is addressing.
In my app I need to do lot of parallel http requests and I have read that it is proper to do it using async/await. In each request I need to get string content from it (often it is html of some site) and my question is: how can I do it in best way?
My current implementation:
public static async Task<string> GetStringContentAsync(HttpWebRequest webRequest)
{
try
{
using (var response = (HttpWebResponse) await webRequest.GetResponseAsync()
.ConfigureAwait(false))
{
var content = await GetStringContentFromResponseAsync(response)
.ConfigureAwait(false);
return content;
}
}
catch (Exception exception)
{
return null;
}
}
private static async Task<string> GetStringContentFromResponseAsync(HttpWebResponse response)
{
using (var responseStream = GetResponseStream(response))
{
if (responseStream == null)
return null;
using (var streamReader = new StreamReader(responseStream))
{
var content = await streamReader.ReadToEndAsync()
.ConfigureAwait(false);
return content;
}
}
}
private static Stream GetResponseStream(HttpWebResponse webResponse)
{
var responseStream = webResponse.GetResponseStream();
if (responseStream == null)
return null;
Stream stream;
switch (webResponse.ContentEncoding.ToUpperInvariant())
{
case "GZIP":
stream = new GZipStream(responseStream, CompressionMode.Decompress);
break;
case "DEFLATE":
stream = new DeflateStream(responseStream, CompressionMode.Decompress);
break;
default:
stream = responseStream;
break;
}
return stream;
}
And example of using:
var httpWebRequest = (HttpWebRequest) WebRequest.Create("http://stackoverflow.com/");
var content = await HttpHelper.GetStringContentAsync(httpWebRequest)
.ConfigureAwait(false);
Is this correct implementation, or we can improve something here? Maybe I'm doing some overhead when using async/await when reading stream?
Reason of my question is that when I'm using my code like this:
for(var i=0;i<1000;i++)
{
Task.Run(()=>{
var httpWebRequest = (HttpWebRequest) WebRequest.Create("http://google.com/");
var content = await HttpHelper.GetStringContentAsync(httpWebRequest)
.ConfigureAwait(false);
});
}
this tasks take to long to execute, but one request to google is very fast. I thought that async requests in this example must be ready almost in same time and this time must be pretty close to "one google request" time.
EDIT:
I forgot to say that I know about ServicePointManager.DefaultConnectionLimit and set it 5000 in my app. So it is not a problem.
I can't use HttpClient because my final goal is to do 100-300 requests at one time from different proxies. And if I understand right, HttpClient can work with only one proxy at one time and can't setup each request separately.
That's a tricky one. Since you know about DefaultConnectionLimit, it's already something good, but there is one more interesting and rather surprising thing:
httpRequest.ServicePoint.ConnectionLeaseTimeout
httpRequest.ServicePoint.MaxIdleTime
Information is here, your latencies might be caused by its default behavior and connections being held to ServicePoint while trying to make next request
Here's the answer answer to your issue: https://msdn.microsoft.com/en-us/library/86wf6409(v=vs.90).aspx
Using synchronous calls in asynchronous callback methods can result in severe performance penalties. Internet requests made with WebRequest and its descendants must use Stream.BeginRead to read the stream returned by the WebResponse.GetResponseStream method.
That means absolutely no synchronous code (including awaits) when reading the response stream. But even that isn't enough, as DNS lookups and TCP connection are still blocking. If you can use .NET 4.0, there's a much more easy to use System.Net.Http.HttpClient class. Otherwise, you can use System.Threading.ThreadPool, which is the workaround I ended up using on 3.5:
ThreadPool.QueueUserWorkItem((o) => {
// make a synchronous request via HttpWebRequest
});