Develop Reference

HttpListenerRequest read InputStream slow when using HttpClient or WebRequest

I have a HttpListener which is waiting for incoming requests. The curious thing is that when I send a request with the HttpClient or WebRequest class, the reading/decoding of the stream takes around 350ms, while sending the same request with Insomnia (https://insomnia.rest/) it only takes 500ticks!!
Can someone explain to me where is my fault?!
HttpClient
private readonly HttpClient _Client = new HttpClient();
private async void HttpClientMethod(string jsonContent)
{
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Post, _BaseUrl);
message.Content = new StringContent(jsonContent);
HttpResponseMessage result = await _Client.SendAsync(message);
string content = await result.Content.ReadAsStringAsync();
Console.WriteLine(content);
}
WebRequest Client
private string WebRequestMethod(string jsonContent)
{
WebRequest request = WebRequest.Create(_BaseUrl);
// Set the Method property of the request to POST.
request.Method = "POST";
// Create POST data and convert it to a byte array.
string postData = jsonContent;
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
WebResponse response = request.GetResponse();
// Display the status.
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
return responseFromServer;
}
Host
private void ReadInputStream(HttpListenerRequest request)
{
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
string text;
using (StreamReader reader = new StreamReader(request.InputStream, request.ContentEncoding))
{
text = reader.ReadToEnd();
}
stopwatch.Stop();
}
HttpClient | WebRequest
Insomnia
Insonmia Body | JSONContent
Sample Project
Just create a new Console App
Program.cs
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace ApiTest
{
class Program
{
static readonly ApiHost _ApiHost = new ApiHost();
static readonly HttpClient _Client = new HttpClient();
static void Main(string[] args) => MainAsync(args).GetAwaiter().GetResult();
static async Task MainAsync(string[] args)
{
_ApiHost.Start();
Console.WriteLine("Host started");
Console.ReadKey();
string jsonContent ="{\"OperationName\":null,\"Query\":\"query {\\r\\n\\t hero{\\r\\n\\t\\t name,\\r\\n\\t\\t id\\r\\n\\t }\\r\\n\\t human(id: \\\"1\\\"){\\r\\n\\r\\n homePlanet,\\r\\n name\\r\\n\\r\\n }\\r\\n }\",\"Variables\":null}";
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Post, new Uri("http://localhost:5000/"));
message.Content = new StringContent(jsonContent);
HttpResponseMessage result = await _Client.SendAsync(message);
string content = await result.Content.ReadAsStringAsync();
Console.WriteLine(content);
Console.ReadKey();
}
}
}
ApiHost.cs
using System;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
namespace ApiTest
{
public class ApiHost
{
public HttpListener Listener = new HttpListener();
private bool _Stop;
public void Start()
{
Listener.Prefixes.Add("http://+:5000/");
Listener.Start();
Task.Run(() =>
{
Semaphore semaphore = new Semaphore(4, 4);
while (!_Stop)
{
semaphore.WaitOne();
Listener.GetContextAsync().ContinueWith(async (contextTask) =>
{
try
{
semaphore.Release();
HttpListenerContext context = await contextTask.ConfigureAwait(false);
Process(context);
}
catch (Exception ex)
{
Console.WriteLine(ex);
throw;
}
});
}
});
}
private void Process(HttpListenerContext ctx)
{
try
{
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
string text;
using (StreamReader reader = new StreamReader(ctx.Request.InputStream, ctx.Request.ContentEncoding))
{
text = reader.ReadToEnd();
}
stopwatch.Stop();
using (Stream output = ctx.Response.OutputStream)
{
using (StreamWriter writer = new StreamWriter(output) { AutoFlush = true })
{
writer.Write(text);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
}
Question:
Why do I have 360ms when using the internal library. When I use Insomnia for example I don't have this delay!

Kaspersky was the problem. After stopping the application it is running without any problems!

HttpWebResponse hangs/freezes when running in a WebAPI

I am using a C# WebAPI project which would call an external API based on its URL. However, when I am trying to retrieve the data back, it hangs/freezes.
The code where it stops is:
var response = (HttpWebResponse)await Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
I don't understand why it is stopping though. Could it be interfering with the API request I am also making? When I run this code as part of a unit test, I would get a response back within seconds. I don't think it is the API service itself, I think it is my code. I have already tried various API URLS. None of them work.
My full code is:
public static async Task<string> CallWebAPi<T>(string url)
{
string returnValue;
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.ContentType = "application/json";
request.Method = "GET";
var response = (HttpWebResponse)await Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
Stream stream = response.GetResponseStream();
StreamReader strReader = new StreamReader(stream);
returnValue = await strReader.ReadToEndAsync();
return returnValue;
}
Any help would be appreciated.

Possible deadlock ConfigureAwait(false), here are a good explanation from Stephen on what cause deadlocks.
var response = (HttpWebResponse)await Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null).ConfigureAwait(false);

As a workaround you can use the synchronous functions, create a Task and await this task:
var response = await Task.Run(() =>
{
return (HttpWebResponse)request.GetResponse();
});

This is how i do my Request. Each Part is in an own Function. Its create the Request and you can get the Response synchronous.
public HttpWebRequest CreateRequest(string Url, string Method, string ContentType, object Content, List<RequestHeader> headers)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url);
request.Method = Method;
if (!string.IsNullOrWhiteSpace(ContentType)) request.ContentType = ContentType;
else if(Content != null) request.ContentType = "application/json";
if (Content != null)
{
var postData = Newtonsoft.Json.JsonConvert.SerializeObject(Content);
var data = Encoding.ASCII.GetBytes(postData);
request.ContentLength = data.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
}
foreach(RequestHeader header in Headers)
{
request.Headers.Add(header.Type, header.Value);
} //class at the end.
return request;
}
public string GetResponse(HttpWebRequest request)
{
var retval = "";
try
{
var response = (HttpWebResponse)request.GetResponse();
retval = ReadResponse(response);
response.Close();
}
catch (Exception ex)
{
resolveException(ex.Message);
}
return retval;
}
public string ReadResponse(HttpWebResponse response)
{
var retval = "";
try
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
var responseText = reader.ReadToEnd();
retval = responseText;
}
}
catch (Exception ex)
{
resolveException(ex.Message);
}
return retval;
}
public class RequestHeader
{
public HttpRequestHeader Type { get; set; }
public string Value { get; set; }
}

you dont need Task.Factory.FromAsync. HttpWebRequest already supports asynchronous operations.
You have defined a generic method CalWebApi<T> but have never used a generic type
if your operation is async, use this.
public async Task<T> CalWebApiAsync<T>(string url)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.ContentType = "application/json";
request.Method = "GET";
using (var response = await request.GetResponseAsync())
{
using (var responseStream = response.GetResponseStream())
{
using (var streamReader = new StreamReader(responseStream))
{
var stringResult = await streamReader.ReadToEndAsync();
T objectResult = Newtonsoft.Json.JsonConvert.DeserializeObject<T>(stringResult);
return objectResult;
}
}
}
}
var result = await CallWebApiAsync<YourType>("exteranlapiurl");
if your operation is not async, use this..
public T CalWebApi<T>(string url)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.ContentType = "application/json";
request.Method = "GET";
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var streamReader = new StreamReader(responseStream))
{
var stringResult = streamReader.ReadToEnd();
T objectResult = Newtonsoft.Json.JsonConvert.DeserializeObject<T>(stringResult);
return objectResult;
}
}
}
}
var result = CallWebApi<YourType>("exteranlapiurl");

Call asynchronous external web service in asp.net MVC controller

In the Asp.net MVC controller (GET method) I am calling external web service - for geolocation of IP - returning json data for IP location. How can I make the call to be async, hence the stack can continue while waiting the response from the service. When the GEO IP request finished I want to be able to make update to the db. Here is the current sync code:
public ActionResult SelectFacility(int franchiseId, Guid? coachLoggingTimeStampId)
{
//...
string responseFromServer = Helpers.GetLocationByIPAddress(userIpAddress);
HomeModels.GeoLocationModel myojb = new HomeModels.GeoLocationModel();
if (!String.IsNullOrEmpty(responseFromServer))
{
JavaScriptSerializer js = new JavaScriptSerializer();
myojb = (HomeModels.GeoLocationModel)js.Deserialize(responseFromServer, typeof(HomeModels.GeoLocationModel));
}
//...
}
public static string GetLocationByIPAddress(string ipAddress)
{
Stream resStream = null;
string responseFromServer = "";
try
{
string url = GeoLocationPath.FreeGeoIP + ipAddress;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
resStream = response.GetResponseStream();
StreamReader reader = new StreamReader(resStream);
responseFromServer = reader.ReadToEnd();
return responseFromServer;
}
catch (Exception ex)
{
//TODO handle this
}
finally
{
if (null != resStream)
{
resStream.Flush();
resStream.Close();
}
}
return responseFromServer;
}
Any suggestion - Thread, AsyncTask ?
Thanks

Make your ASP.NET MVC controller asynchronous:
http://www.asp.net/mvc/tutorials/mvc-4/using-asynchronous-methods-in-aspnet-mvc-4
Then use HttpClient.GetStringAsync and await its result:
public async Task<ActionResult> SelectFacility(
int franchiseId, Guid? coachLoggingTimeStampId)
{
//...
string responseFromServer = await Helpers.GetLocationByIPAddressAsync(
userIpAddress);
//...
}
public static async Task<string> GetLocationByIPAddress(string ipAddress)
{
using (var httpClient = new HttpClient())
return await httpClient.GetStringAsync(
GeoLocationPath.FreeGeoIP + ipAddress);
}

Asynchronously using HttpWebRequest without blocking UI thread

I am trying to use HttpWebRequest and HttpWebResonse asynchronously in a WinForms application without blocking my UI thread.
I saw this similar SO Question which explains how to do it. However, I am not 100% sure the accepted answer is the correct way to do it. The accepted answer is using BeginGetResponse.
According to the MSDN documentation:
The BeginGetResponse method requires some synchronous setup tasks to
complete (DNS resolution, proxy detection, and TCP socket connection,
for example) before this method becomes asynchronous. As a result,
this method should never be called on a user interface (UI) thread
because it might take some time, typically several seconds.
Can someone please provide me with the correct way to do this. Here is my C# function that I'm trying to make work "properly" without blocking my UI thread:
private IDictionary<string, object> CreateWebRequest(string endPoint, string method, string data, bool addAuthHeader)
{
var req = (HttpWebRequest)WebRequest.Create(endPoint);
req.Method = method;
if (addAuthHeader)
{
req.Headers.Add(HttpRequestHeader.Authorization, "Bearer " + _accessToken.AccessToken);
}
if (!string.IsNullOrEmpty(data))
{
var utfenc = new UTF8Encoding();
byte[] buffer = utfenc.GetBytes(data);
req.ContentLength = buffer.Length;
req.ContentType = "application/x-www-form-urlencoded";
using (Stream strm = req.GetRequestStream())
{
strm.Write(buffer, 0, buffer.Length);
strm.Close();
}
}
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)req.GetResponse();
}
catch (WebException e)
{
if (_accessToken == null)
{
throw;
}
var responseError = (HttpWebResponse)e.Response;
if (responseError.StatusCode == HttpStatusCode.Unauthorized)
{
var stackTrace = new StackTrace();
var q = stackTrace.GetFrame(1).GetMethod().Name;
RefreshAccessCode();
req = (HttpWebRequest)WebRequest.Create(endPoint);
req.Method = method;
if (addAuthHeader)
{
req.Headers.Add(HttpRequestHeader.Authorization, "Bearer " + _accessToken.AccessToken);
}
response = (HttpWebResponse)req.GetResponse();
}
}
string jsonResponse = null;
if (response != null)
using (Stream stream = response.GetResponseStream())
{
if (stream != null)
{
using (var reader = new StreamReader(stream))
{
jsonResponse = reader.ReadToEnd();
}
}
}
return DeserializeResponse(jsonResponse);
}

If you're open to something besides WebRequest, I'm a fan of System.Net.WebClient. It uses System.Threading.Tasks instead of BeginGetResponse() and EndGetResonse().
public async Task<Dictionary<string, object>> CreateWebRequest(string endPoint, string method, string data, bool addAuthHeader)
{
WebClient client = new WebClient();
if (addAuthHeader)
{
client.Headers.Add(HttpRequestHeader.Authorization, "Bearer " + _accessToken.AccessToken);
}
byte[] buffer = null;
if (!string.IsNullOrEmpty(data))
{
var utfenc = new UTF8Encoding();
buffer = utfenc.GetBytes(data);
}
else
{
buffer = new byte[0];
}
return await client.UploadDataTaskAsync(endPoint, method, buffer)
.ContinueWith((bytes) =>
{
string jsonResponse = null;
using (var reader = new StreamReader(new MemoryStream(bytes)))
{
jsonResponse = reader.ReadToEnd();
}
return DeserializeResponse(jsonResponse);
});
}
}

To use your existing code, just add a method that is intended to execute on the UI thread, and create your web request on a separate thread. When the web request is complete you call the final method on the UI thread. So, a simple version would be:
//do some stuff that wont block the ui thread here
ThreadPool.QueueUserWorkItem((o) =>
{
IDictionary<string, object> result =
CreateWebRequest("lalala", "lol", "butts", true);
BeginInvoke(OnAsyncWebRequestComplete, result);
}, null);
private void OnAsyncWebRequestComplete(IDictionary<string, object> result)
{
//do some stuff on the UI thread here
}

Easiest way to read the response from WebResponse

private void RespCallback(IAsyncResult asynchronousResult)
{
try
{
WebRequest myWebRequest1 = (WebRequest)asynchronousResult.AsyncState;
// End the Asynchronous response.
WebResponse webResponse = myWebRequest1.EndGetResponse(asynchronousResult);
}
catch (Exception)
{
// TODO:Log the error
}
}
Now having the webResponse object, what is the easiest way to read its contents?

I would simply use the async methods on WebClient - much easier to work with:
WebClient client = new WebClient();
client.DownloadStringCompleted += (sender,args) => {
if(!args.Cancelled && args.Error == null) {
string result = args.Result; // do something fun...
}
};
client.DownloadStringAsync(new Uri("http://foo.com/bar"));
But to answer the question; assuming it is text, something like (noting you may need to specify the encoding):
using (var reader = new StreamReader(response.GetResponseStream()))
{
string result = reader.ReadToEnd(); // do something fun...
}

Here is one way to do it if the response is coming in from XML.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create("https://www.yoururl.com");
WebResponse response = myReq.GetResponse();
Stream responseStream = response.GetResponseStream();
XmlTextReader reader = new XmlTextReader(responseStream);
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Text)
{
Console.WriteLine("{0}", reader.Value.Trim());
}
Console.ReadLine();
}

internal string Get(string uri)
{
using (WebResponse wr = WebRequest.Create(uri).GetResponse())
{
using (StreamReader sr = new StreamReader(wr.GetResponseStream()))
{
return sr.ReadToEnd();
}
}
}