Develop Reference

Xamarin iOS WebException: App crashes after HttpWebRequest is complete

The problem
So we are creating a Xamarin iOS app that will connect to a REST API running on ASP .NET. To perform the HttpRequests it uses a library we share between the iOS app and an android app.
On the first look everything works fine. The iOS app calls an asynchronous Method from our library, receives data from the server and correctly displays it in a table view. However for some reason it doesn't terminate the connection once it's done resulting in a WebException some time later (seemingly random from immediately after the request is performed up until minutes later) saying Error getting response stream (ReadDoneAsync2): ReceiveFailure. Interestingly enough when we call the exact same method in our library from a console application everything works just fine.
Library Code
This is the relevant code from our library (We don't know where the error occurs, as the main UI Thread crashes. Debugging also doesn't reveal anything suspicious. I'll therefore include all code below that will we executed on the client during the API call):
ClubProvider:
public class ClubProvider : BaseProvider
{
/// <summary>
/// Creates a new <see cref="ClubProvider"/> using the specified <paramref name="uri"/>.
/// </summary>
internal ClubProvider(string uri)
{
if (!uri.EndsWith("/"))
{
uri += "/";
}
Uri = uri + "clubs";
}
public async Task<ClubListResponse> GetClubListAsync()
{
return await ReceiveServiceResponseAsync<ClubListResponse>(Uri);
}
}
BaseProvider (where the actual HttpWebRequest is performed)
public abstract class BaseProvider
{
public virtual string Uri { get; private protected set; }
/// <summary>
/// Reads the response stream of the <paramref name="webResponse"/> as a UTF-8 string.
/// </summary>
private async Task<string> ReadResponseStreamAsync(HttpWebResponse webResponse)
{
const int bufferSize = 4096;
using Stream responseStream = webResponse.GetResponseStream();
StringBuilder builder = new StringBuilder();
byte[] buffer = new byte[bufferSize];
int bytesRead;
while ((bytesRead = await responseStream.ReadAsync(buffer, 0, bufferSize)) != 0)
{
builder.Append(Encoding.UTF8.GetString(buffer, 0, bytesRead));
}
return builder.ToString();
}
/// <summary>
/// Performs the <paramref name="webRequest"/> with the provided <paramref name="payload"/> and returns the resulting <see cref="HttpWebResponse"/> object.
/// </summary>
private async Task<HttpWebResponse> GetResponseAsync<T>(HttpWebRequest webRequest, T payload)
{
webRequest.Host = "ClubmappClient";
if (payload != null)
{
webRequest.ContentType = "application/json";
string jsonPayload = JsonConvert.SerializeObject(payload);
ReadOnlyMemory<byte> payloadBuffer = Encoding.UTF8.GetBytes(jsonPayload);
webRequest.ContentLength = payloadBuffer.Length;
using Stream requestStream = webRequest.GetRequestStream();
await requestStream.WriteAsync(payloadBuffer);
}
else
{
webRequest.ContentLength = 0;
}
HttpWebResponse webResponse;
try
{
webResponse = (HttpWebResponse)await webRequest.GetResponseAsync();
}
catch (WebException e)
{
/* Error handling
....
*/
}
return webResponse;
}
/// <summary>
/// Performs the <paramref name="webRequest"/> with the provided <paramref name="payload"/> and returns the body of the resulting <see cref="HttpWebResponse"/>.
/// </summary>
private async Task<string> GetJsonResponseAsync<T>(HttpWebRequest webRequest, T payload)
{
using HttpWebResponse webResponse = await GetResponseAsync(webRequest, payload);
return await ReadResponseStreamAsync(webResponse);
}
/// <summary>
/// Performs an <see cref="HttpWebRequest"/> with the provided <paramref name="payload"/> to the specified endpoint at the <paramref name="uri"/>. The resulting <see cref="IResponse"/> will be deserialized and returned as <typeparamref name="TResult"/>.
/// </summary>
private protected async Task<TResult> ReceiveServiceResponseAsync<TResult, T>(string uri, T payload) where TResult : IResponse
{
HttpWebRequest webRequest = WebRequest.CreateHttp(uri);
webRequest.Method = "POST";
string json = await GetJsonResponseAsync(webRequest, payload);
TResult result = JsonConvert.DeserializeObject<TResult>(json);
return result;
}
/// <summary>
/// Performs an <see cref="HttpWebRequest"/> to the specified endpoint at the <paramref name="uri"/>. The resulting <see cref="IResponse"/> will be deserialized and returned as <typeparamref name="TResult"/>.
/// </summary>
private protected async Task<TResult> ReceiveServiceResponseAsync<TResult>(string uri) where TResult : IResponse
{
return await ReceiveServiceResponseAsync<TResult, object>(uri, null);
}
}
Call from iOS client
public async override void ViewDidLoad()
{
// ...
ServiceProviderFactory serviceProviderFactory = new ServiceProviderFactory("https://10.0.0.40:5004/api");
ClubProvider clubProvider = serviceProviderFactory.GetClubProvider();
ClubListResponse clubListResponse = await clubProvider.GetClubListAsync();
var clublist = new List<ClubProfileListData>();
foreach (var entry in clubListResponse.ClubListEntries)
{
clublist.Add(new ClubProfileListData(entry.Name, entry.City + "," + entry.Country, entry.MusicGenres.Aggregate(new StringBuilder(), (builder, current) => builder.Append(current).Append(",")).ToString()));
}
// ...
}
At first this seems to work just fine but it crashes later on with this error message:
Taking a look at the performed request with Wireshark reveals that the connection is never terminated:
The connection stays open until the app crashes and we close the simulator.
Interestingly enough the app doesn't always crash immediately. We load the received data into a TableView and every once in a while the app doesn't crash after loading the data. It only crashes when we start scrolling through the results. This doesn't make sense to us though as all network streams should be closed by now right? (After all we are using using statements for all ResponseStreams. Therefore all streams should automatically be disposed when returning from the awaited Task :C ) As if it would be trying to stream the data as needed.
Testing the library code using a Console Application
Now the obvious reason for this could be that we forgot to close some stream in our library however the following code succeeds with no error whatsoever:
class Program
{
static void Main(string[] args)
{
new Thread(Test).Start();
while (true)
{
Thread.Sleep(1000);
}
}
public static async void Test()
{
ServiceProviderFactory serviceProviderFactory = new ServiceProviderFactory("https://10.0.0.40:5004/api");
ClubProvider clubProvider = serviceProviderFactory.GetClubProvider();
ClubListResponse clubListResponse = await clubProvider.GetClubListAsync();
foreach (ClubListResponseEntry entry in clubListResponse.ClubListEntries)
{
Console.WriteLine(entry.Name);
}
Console.WriteLine("WebRequest complete!");
Console.ReadLine();
}
}
Taking a look at the captured packets we see that the connection is closed as expected once the request is completed:
The question
So why is this? Why does our library code work as intended in our .NET Core Console Application but fails to disconnect when called by the iOS app? We have the suspicion that this could be due to the async/await calls (as described here). However we do get an exception so we aren't sure if this is really the same bug described in the question linked above. Now before we rewrite all our library code we'd like to eliminate all other possible causes for this weird behavior. Also we successfully use async calls to load some images without crashing the application so we're really just guessing at this point :C
Any help would be greatly appreciated.

Alright further testing revealed that the app crashing actually didn't have anything to do with our API calls whatsoever. The problem actually was us loading the images to be shown next to our datasets asynchronously on demand. Turns out that during testing of the image loading we only had like 20 entries in our ListView. This was fine because iOS could load all images at once. However when loading 1500+ datasets from our API the ListView started buffering, only loading images as needed and that's when the application crashed. Probably because the original image stream wasn't available anymore or something like that.
Also as an interesting side note: iOS does actually close the network connection to the server but only after a 100 second timeout of no packets sent while the Windows .NET Core Console Application closes it immediately. And we never waited this long. Oh well :D

.NET Client - Waiting for an MQTT response before proceeding to the next request

I have a MQTT calls inside a loop and in each iteration, it should return a response from the subscriber so that I could use the value being forwarded after I published. But the problem is I don't know how would I do it.
I hope you have an idea there or maybe if I'm just not implementing it right, may you guide me through this. Thanks.
Here's my code:
// MyClientMgr
class MyClientMgr{
public long CurrentOutput { get; set; }
public void GetCurrentOutput(MyObjectParameters parameters, MqttClient client)
{
MyMessageObject msg = new MyMessageObject
{
Action = MyEnum.GetOutput,
Data = JsonConvert.SerializeObject(parameters)
}
mq_GetCurrentOutput(msg, client);
}
private void mq_GetCurrentOutput(MyMessageObject msg, MqttClient client)
{
string msgStr = JsonConvert.SerializeObject(msg);
client.Publish("getOutput", Encoding.UTF8.GetBytes(msgStr),
MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, false);
client.MqttMsgPublishReceived += (sender, e) =>{
MyObjectOutput output = JsonConvert.DeserializeObject<MyObjectOutput>(Encoding.UTF8.GetString(e.Message));
CurrentOutput = output;
};
}
}
// MyServerMgr
class MyServerMgr
{
public void InitSubscriptions()
{
mq_GetOutput();
}
private void mq_GetOutput()
{
MqttClient clientSubscribe = new MqttClient(host);
string clientId = Guid.NewGuid().ToString();
clientSubscribe.Connect(clientId);
clientSubscribe.Subscribe(new string[] { "getOutput" }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
MqttClient clientPublish = new MqttClient(host);
string clientIdPub = Guid.NewGuid().ToString();
clientPublish.Connect(clientIdPub);
clientSubscribe.MqttMsgPublishReceived += (sender, e) => {
MyMessageObj msg = JsonConvert.DeserializeObject<MyMessageObj>(Encoding.UTF8.GetString(e.Message));
var output = msg.Output;
clientPublish.Publish("getOutput", Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(output)), MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, false);
}
}
}
// MyCallerClass
class MyCallerClass
{
var host = "test.mqtt.org";
var myServer = new MyServerMgr(host);
var myClient = new MyClientMgr();
myServer.InitSubscriptions();
MqttClient client = new MqttClient(host);
for(int i = 0; i < 10; i++)
{
long output = 0;
MyObjectParameters parameters = {};
myClient.GetCurrentOutput(parameters, client) // here I call the method from my client manager
// to publish the getting of the output and assigned
// below for use, but the problem is the value doesn't
// being passed to the output variable because it is not
// yet returned by the server.
// Is there a way I could wait the process to
// get the response before assigning the output?
output = myClient.CurrentOutput; // output here will always be null
// because the response is not yet forwarded by the server
}
}
I have a loop in my caller class to call the mqtt publish for getting the output, but I have no idea how to get the output before it was assigned, I want to wait for the response first before going to the next.
I've already tried doing a while loop inside like this:
while(output == 0)
{
output = myClient.CurrentOutput;
}
Yes, I can get the output here, but it will slow down the process that much. And sometimes it will fail.
Please help me. Thanks.

It looks like you are trying to do synchronous communication over an asynchronous protocol (MQTT).
By this I mean you want to send a message and then wait for a response, this is not how MQTT works as there is no concept of a reply to a message at the protocol level.
I'm not that familiar with C# so I'll just give an abstract description of possible solution.
My suggestion would be to use a publishing thread, wait/pulse (Look at the Monitor class) to have this block after each publish and have the message handler call pulse when it has received the response.
If the response doesn't contain a wait to identify the original request you will also need a state machine variable to record which request is in progress.
You may want to look at putting a time out on the wait in case the other end does not respond for some reasons.

You can use AutoResetEvent class that has WaitOne() and Set() methods. Using WaitOne() after publish will wait until the message is published and using Set() under client_MqttMsgPublishReceived event will release the wait when the subscriber received the message he subscribed for.

Wait Handles and Xamarin Forms

I am currently working on a proof of concept application using the Xamarin free trial, and I have hit a rather interesting little problem... Here is the code I am using within a Portable Class Library:
using System;
using System.Net;
using Newtonsoft.Json;
namespace poc
{
public class CurrentWeatherInformation
{
public string WeatherText { get; set; }
public CurrentWeatherInformation (string cityName)
{
// api.openweathermap.org/data/2.5/weather?q=Leeds
var request = (HttpWebRequest)WebRequest.Create(string.Format("http://api.openweathermap.org/data/2.5/weather?q={0}", cityName));
request.ContentType = "application/json";
request.Method = "GET";
object state = request;
var ar = request.BeginGetResponse (WeatherCallbackMethod, state);
var waitHandle = ar.AsyncWaitHandle as System.Threading.ManualResetEvent;
waitHandle.WaitOne();
}
public void WeatherCallbackMethod(IAsyncResult ar)
{
object state = ar.AsyncState;
var request = state as HttpWebRequest;
var response = request.EndGetResponse(ar);
var data = new System.IO.StreamReader (response.GetResponseStream ()).ReadToEnd ();
this.WeatherText = data;
}
}
}
Essentially, I just want to call against a webservice and get a response, but I note with Xamarin that I am unable to do this using the good old GetResponse() method, and have to use BeginGetResponse() and EndGetResponse() instead, with the old IAsyncResult pattern. Shizzle.
Anyway, my problem is that the code following my waiting on the waitHandle is executing BEFORE the code in the callback, and I don't see why. This is precisely what we have the wait handle for!
Can anyone spot what I am sure will prove to be a simple mistake by a simpleton?

On Windows Phone you are forced to use the async API. When you try to wait for a result of an async method synchronously on main thread you can end up in an infinite loop.
Use async and await when you do expensive things. It's the common pattern for doing asynchronous work.
Take a look at some tutorials:
https://visualstudiomagazine.com/articles/2013/10/01/asynchronous-operations-with-xamarin.aspx
http://developer.xamarin.com/recipes/android/web_services/consuming_services/call_a_rest_web_service/
How to implement Android callbacks in C# using async/await with Xamarin or Dot42?
https://github.com/conceptdev/xamarin-forms-samples/blob/master/HttpClient/HttpClientDemo/GeoNamesWebService.cs

C# and running of HttpListener in background

I created simple HttpListener that listens to port 9090 and depending on a request's URL writes some info to the console.
But I'm stuck :( I thought of multithreading, event based system, but I dind't manage to do anything with it.
Here is the code of my listener that I launched as a separate console app:
string urlTemplate = String.Format("/prefix/{0}/suffix", id);
string prefix = String.Format("http://localhost:9090/");
HttpListener listener = new HttpListener();
listener.Prefixes.Add(prefix);
listener.Start();
Console.WriteLine("Listening to {0}...", prefix);
while (true)
{
HttpListenerContext context = listener.GetContext();
HttpListenerRequest request = context.Request;
//Response object
HttpListenerResponse response = context.Response;
//Construct response
if (request.RawUrl.Contains(urlTemplate) && request.HttpMethod == "POST")
{
string requestBody;
Stream iStream = request.InputStream;
Encoding encoding = request.ContentEncoding;
StreamReader reader = new StreamReader(iStream, encoding);
requestBody = reader.ReadToEnd();
Console.WriteLine("POST request on {0} with body = [{1}]", request.RawUrl, requestBody);
response.StatusCode = (int)HttpStatusCode.OK;
//Return a response
using (Stream stream = response.OutputStream) { }
}
else
{
response.StatusCode = (int)HttpStatusCode.BadRequest;
Console.WriteLine("Invalid HTTP request: [{0}] {1}", request.HttpMethod, request.Url);
using (Stream stream = response.OutputStream) { }
}
}
I decided to use it as an utility for unit tests (maybe somewhere else). So when test starts I need to configure the listener, run it, then make some requests and receive the info (which listener wrote earlier to Console), and at the end of the test stop the listener.
My main idea was to incapsulate this listener to separate class MyHttpListener which has methods: StartListener(), StopListener().
But when I call StartListener() my test freezes because of infinite while loop. I tried to create separate background thread or event based system, but my lack of experience with them, prevents me from doing it. I've already spent a lot of time trying to find the solution, but all for nothing.
Hope you can help me finding the solution for such trivial task.
Thanks in advance.

One of the responder's variant (it seems he deleted his post) looked good, but it didn't work for me. I tried to fix things in order it started working, but at the end that variant gave me an idea how to solve the problem - with multithreading and events :)
Here's what I have now and it works:
public delegate void HttpListenerRequestHandler(object sender, HttpListenerEventArgs e);
public event HttpListenerRequestHandler OnCorrectRequest;
...
if(OnCorrectRequest != null)
OnCorrectRequest(this, new HttpListenerEventArgs(response));
lock (threadLock)
{
Console.WriteLine("POST request on {0} with body = [{1}]", request.RawUrl, requestBody);
}
...
public class HttpListenerEventArgs : EventArgs
{
public readonly HttpListenerResponse response;
public HttpListenerEventArgs(HttpListenerResponse httpResponse)
{
response = httpResponse;
}
}
In order to receive detailed responses from HttpListener in main thread, I use the following:
private HttpListenerResponse response;
public void HttpCallbackRequestCorrect(object sender, HttpListenerEventArgs e)
{
response = e.response;
Console.WriteLine("{0} sent: {1}", sender, e.response.StatusCode);
}
Unfortunately I have the following exception which I don't know how to handle:
System.Net.HttpListenerException: The I/O operation has been aborted because of either a thread exit or an application request
at System.Net.HttpListener.GetContext()

About HttpWebRequest's EndGetResponse

I'm using asynchronous calls for communication to my server. I written some component to collect all unauthorized requests and to resend them after user logs in. I written some test to produce 10 threads that are sending some requests without first being authorized. Than I wait for 20 seconds and do authorization and after that I wait for request to successfully finish. But problem appeared at EndGetResponse method which I call in my callback method. I done that this way:
public void InternalCallback(IAsyncResult result)
{
try
{
RequestState state = (RequestState)result.AsyncState;
IHttpWebRequest request = state.Request;
using (IHttpWebResponse response = responseGetter.GetResponse(request, result))
{
// ...
}
}
// ...
}
So, I made some custom class RequestState which has some higher level callbacks I need and it has request which I'll use to call EndGetResponse method. But this way I got error:
IAsyncResult object was not returned from the corresponding asynchronous method.
I changed this so I now have Request field in my callback class which I set before calling BeginGetResponse and I use that Request field when calling EndGetResponse in my callback.
public void InternalCallback(IAsyncResult result)
{
try
{
using (IHttpWebResponse response = responseGetter.GetResponse(this.Request, result))
{
// ...
}
}
// ...
}
Is this new solution valid one? Can you suggest is this good way to do this or how should I do this?