Develop Reference

HttpClient.GetAsync() gives an AggregateException while fetching data on Azure Function

I am trying to fetch Employee data from Zoho using the URL :
https://people.zoho.com/people/api/forms/P_EmployeeView/records
using the HttpClient's GetAsync(). While executing the code in my local dev environment the code runs smoothly and fetches the required data but as soon as I publish my code to the azure function I get an exception with the following stack trace :
2021-06-01T06:14:45.870 [Error] System.AggregateException: One or more errors occurred. (One or
more errors occurred. (A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because connected host has failed
to respond.))---> System.AggregateException: One or more errors occurred. (A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond.)--->
System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did
not properly respond after a period of time, or established connection failed because connected
host has failed to respond.---> System.Net.Sockets.SocketException (10060): A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond.at
System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken
cancellationToken)--- End of inner exception stack trace ---at
System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken
cancellationToken)at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request,
Boolean allowHttp2, CancellationToken cancellationToken)at
System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request,
CancellationToken cancellationToken)at
System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request,
CancellationToken cancellationToken)at
System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean
doRequestAuth, CancellationToken cancellationToken)at
System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken
cancellationToken)at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request,
CancellationToken cancellationToken)at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1
sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)--- End of
inner exception stack trace ---at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean
includeTaskCanceledExceptions)at System.Threading.Tasks.Task`1.GetResultCore(Boolean
waitCompletionNotification)at System.Threading.Tasks.Task`1.get_Result()at
EmployeeDataRefresh.ZohoClient.GetEmployeeData(ILogger log) in
C:\Projects\ZohoAttendance\Internal-Automation-and-Power-BI-Dashboard-zoho-employee-data-
update\src\EmployeeDataRefresh\EmployeeDataRefresh\ZohoClient.cs:line 39--- End of inner exception
stack trace ---at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean
includeTaskCanceledExceptions)at System.Threading.Tasks.Task`1.GetResultCore(Boolean
waitCompletionNotification)at System.Threading.Tasks.Task`1.get_Result()at
EmployeeDataRefresh.Trigger.DoRefresh(ILogger log) in C:\Projects\ZohoAttendance\Internal-
Automation-and-Power-BI-Dashboard-zoho-employee-data-
update\src\EmployeeDataRefresh\EmployeeDataRefresh\Trigger.cs:line 68at
EmployeeDataRefresh.Trigger.AutoRefreshEmployeeData(TimerInfo myTimer, ILogger log) in
C:\Projects\ZohoAttendance\Internal-Automation-and-Power-BI-Dashboard-zoho-employee-data-
update\src\EmployeeDataRefresh\EmployeeDataRefresh\Trigger.cs:line 30
Here's my code that fetches the data
using(var httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.Authorization
= new AuthenticationHeaderValue("Bearer", _authToken);
Uri myUri = new Uri(_url, UriKind.Absolute);
var response = httpClient.GetAsync(myUri);
log.LogInformation(_authToken);
log.LogInformation("Sending Get Request to Zoho...\n");
var data = await response.Result.Content.ReadAsStringAsync();
log.LogInformation("Data fetched from Zoho...\n");
var employes = JsonConvert.DeserializeObject<List<Employee>>(data);
return employes;
}
I get the error at line
var data = await response.Result.Content.ReadAsStringAsync();
I have put various log statements to debug the issue and the last log statement that gets printed on azure function log is "Sending Get Request to Zoho...".
I have printed the tokens and other required variables to check whether they have correct values and they are getting the correct value so invalid token is definitely not an issue. Can someone suggest what could be the possible reason for this error ?

are you expecting json response type ? and for best practice perhaps you need to supply in your header of httpclient
string contentTypeValue = "application/json";
client.DefaultRequestHeaders.Add("Content-Type", contentTypeValue);
and also practice
httpResponse.EnsureSuccessStatusCode(); // throws if not 200-299
before read result stream.

Here there is no issue with token or any authentication , just clear out asynchronous programming,
Frist,
Instead, by getting the value of the response.Result property, you force the current thread to wait until the asynchronous operation has completed, and second I will recommend
static readonly HttpClient client = new HttpClient();
try
{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _authToken);
Uri myUri = new Uri(_url, UriKind.Absolute);
HttpResponseMessage response = await client.GetAsync(_url);
var data = await response.Result.Content.ReadAsStringAsync();
var employes = JsonConvert.DeserializeObject<List<Employee>>(data);
}
catch (Exception)
{
throw;
}

HttpClient seems to be ignoring the Timeout property

I am trying to load a large amount of data from a server by returning MultiPartContent in the response. However, during the HttpGet method the timeout value I am setting is being ignored. I am trying to set a longer timeout as waiting in this situation is OK.
I have tried various Timeout values between 10 minutes to 2 hours. Realistically the user will only have to wait ~5 minutes the first time they use our app and all other times it will be less.
I have also tried storing a reference of HttpClient to ensure that api.HttpClient isn't re-creating the HttpClient object each time.
var handler = new HttpClientHandler();
var progress = new ProgressMessageHandler(handler);
var api = APIHelpers.GetSession(progress);
progress.HttpReceiveProgress += (e, args) => ProgressChanged.Invoke(e, new SyncEventArgs(GlobalEnums.SyncStage.Downloading, args.ProgressPercentage * 0.01f));
api.HttpClient.Timeout = new TimeSpan(0, 10, 0);
var downloadUri = BuildURI();
var response = await api.HttpClient.GetAsync(downloadUri, HttpCompletionOption.ResponseHeadersRead);
I would expect the application to wait for 10 minutes before throwing a Timeout exception, however the exception is being thrown after 100 seconds (i believe that is the default value?).
Checking the HttpClient.Timeout value after it is set does show that it was set correctly.
The APIHelpers.GetSession() method returns an object of our API with authorisation headers created. The HttpClient object is accessible via an inherited class. This is created by Swashbuckle and Swagger. I have used the api.HttpClient.Timeout = x this way before with success so I don't think this is the issue. It seems to be specific to this scenario in particular.
The exception thrown:
{System.Net.Http.HttpRequestException: An error occurred while sending the request ---> System.Net.WebException: The operation has timed out.
at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <a1ab7fc4639d4d84af41d68234158b1c>:0
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00019] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System/System.Net/HttpWebRequest.cs:1200
at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func`2[T,TResult] endFunction, System.Action`1[T] endAction, System.Threading.Tasks.Task`1[TResult] promise, System.Boolean requiresSynchronization) [0x0000f] in <939d99b14d934342858948926287beba>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.MonoWebRequestHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x003d1] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System.Net.Http/MonoWebRequestHandler.cs:499
--- End of inner exception stack trace ---
at System.Net.Http.MonoWebRequestHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x0046a] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System.Net.Http/MonoWebRequestHandler.cs:503
at Microsoft.Rest.RetryDelegatingHandler+<>c__DisplayClass11_0.<SendAsync>b__1 () [0x000ad] in <6a6c837cafbb4f1faffaba1ff30ca4e3>:0
at Microsoft.Rest.RetryDelegatingHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00150] in <6a6c837cafbb4f1faffaba1ff30ca4e3>:0
at System.Net.Http.Handlers.ProgressMessageHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00087] in <19af20e76ce04547b3fc150a0f8f2d47>:0
at System.Net.Http.HttpClient.SendAsyncWorker (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) [0x0009e] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System.Net.Http/System.Net.Http/HttpClient.cs:281
at ...
EDIT:
I'm just adding some additional things I have tried in an attempt to solve this.
I have tried using the DependencyService in Xamarin.Forms to get native HttpClientHandlers and HttpClient objects for each platform where I can set the timeouts per platform instead of relying on Xamarin.Forms to translate it for me. This had the exact same result.
public HttpMessageHandler GetHttpHandler(double timeoutSeconds)
{
var sessionConfig = NSUrlSessionConfiguration.DefaultSessionConfiguration;
sessionConfig.TimeoutIntervalForRequest = timeoutSeconds;
sessionConfig.TimeoutIntervalForResource = timeoutSeconds;
sessionConfig.WaitsForConnectivity = true;
var sessionHandler = new NSUrlSessionHandler(sessionConfig);
return sessionHandler;
}
I have tried creating a new HttpClient object just for this API call but with no luck here either. (This replaces Apihelpers.GetSession() in my code)

There was a known issue in both Xamarin.iOS and Xamarin.Android that HttpClient.Timeout values greater than 100 seconds are ignored. This is because the underlying native http clients have a timeout set to 100 seconds, so this times out before the .NET HttpClient times out when the HttpClient.Timeout value > 100 seconds. This should be fixed in all of the latest stable versions, so make sure you are updated. If you are using Visual Studio 2017, you won't have the fix, you will need to get VS 2019 for the fixes.

Had the same issue in my Xamarin Android project. Resolved it by changing HttpClient implementation to Android in the Project properties as below:

The operation has timed out ~ System.Net.WebException: The operation has timed out

The operation has timed out ~ System.Net.WebException: The operation has timed out at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context) at System.Net.HttpWebRequest.GetRequestStream()
doing a load testing of http server with multiple requests per second. after few hundred successful requests and responses, new request get stuck and after the timeout value is achieved(timeout = 30 second) request timeout exception is thrown.
all the solutions i found was regarding closing the stream properly which I hope I am doing.
Any help on resolving the timeout problem would be appreciated. thanks.
using (System.IO.Stream requestStream = client.GetRequestStream())
{
requestStream.Write(content, 0, content.Length);
requestStream.Close();
}

WebClient Timeout solution doesn't work (subclassing to set Timeout)

I am trying to use this as a commonly accepted solution for setting a timeouts for WeblClient calls. In my test case of requesting a url from an offline machine: I consistently get 20 second timeouts when set at 1 second.
public class TimeoutWebClient : WebClient
{
public TimeoutWebClient(TimeSpan timeout)
{
_timeout = timeout;
}
private TimeSpan _timeout;
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
request.Timeout = (int)_timeout.TotalMilliseconds;
return request;
}
}
Server based timeouts can't matter in this scenario. What could I be missing?
I did find a snippet that set both HttpWebRequest.ReadWriteTimeout and HttpWebRequest.ServicePoint.MaxIdleTime. But setting these to the timeout value still didn't make a difference and am still getting ~20 second timeouts.
Any other thoughts of what could cause this behavior?

Any chance you are using async download method?
From the MSDN:
The Timeout property affects only synchronous requests made with the GetResponse method. To time out asynchronous requests, use the Abort method.

HttpClient File Download Error - Unable to read data from the transport connection

I've written an application that in part can download files from a specific web service. The code utilizes an HttpClient to make the calls. The problem is that occasionally I will get a failed request with the following exception message:
Unable to read data from the transport connection: The connection was closed.
I did run across these blog posts, in which the author had to revert the protocol version to 1.0, disable keep alive, and limit the number of service point connections:
http://briancaos.wordpress.com/2012/07/06/unable-to-read-data-from-the-transport-connection-the-connection-was-closed/
http://briancaos.wordpress.com/2012/06/15/an-existing-connection-was-forcibly-closed-by-the-remote-host/
I followed those instructions, as best I knew how and still got the error. I also made sure to keep a single instance of the HttpClient around (following the Singleton principle).
What is interesting is that when running Fiddler I've yet to get the error, which makes me think that there is something that can be done on the client side since Fiddler appears to be doing something to keep the connection alive (though the issue is so sporadic this may be a red herring).
A couple more notes:
The error invariably occurs in the middle of a download (never when initiating the request).
The file continues to download up to the point of failure (there are no extended pauses or delays first).
--UPDATE--
The error occurs on the following line:
responseTask.Wait(cancellationTokenSource.Token);
The following is the full exception:
System.AggregateException occurred HResult=-2146233088 Message=One
or more errors occurred. Source=mscorlib StackTrace:
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at Form1.StartDownload() in c:\Projects\Visual Studio 2012\Demo\Demo\Form1.cs:line 88 InnerException:
System.Net.Http.HttpRequestException
HResult=-2146233088
Message=Error while copying content to a stream.
InnerException: System.IO.IOException
HResult=-2146232800
Message=Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Source=System
StackTrace:
at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.EndRead(IAsyncResult
asyncResult)
at System.Net.Http.Handlers.ProgressStream.EndRead(IAsyncResult
asyncResult)
at System.Net.Http.StreamToStreamCopy.BufferReadCallback(IAsyncResult ar)
InnerException: System.Net.Sockets.SocketException
HResult=-2147467259
Message=An existing connection was forcibly closed by the remote host
Source=System
ErrorCode=10054
NativeErrorCode=10054
StackTrace:
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
InnerException:
--UPDATE #2--
I thought I would try changing the completion option from 'content read' to 'headers read'. This also failed with the same exception, albeit in a different location (where the TODO comment is, reading the content stream).
--UPDATE #3--
I can confirm that the web service (which is hosted in IIS) is aborting the connections (the IIS logs show a win32 status code of 1236 - ERROR_CONNECTION_ABORTED). To try and narrow things down, the MinFileBytesPerSec metabase property was set to zero (on the off chance the client stopped pulling down data momentarily) and the connection is still being aborted. I've double checked all the timeouts and buffer sizes I can think of to no avail. Clawing at thin air at the moment. Any ideas would be appreciated.
Client Setup:
private void SetupClient()
{
// In case we're taxing the web server, limit the number
// connections we're allowed to make to one.
ServicePointManager.DefaultConnectionLimit = 1;
// Set up the progress handler so that we can keep track of the download progress.
_progressHandler = new ProgressMessageHandler();
_progressHandler.HttpReceiveProgress += ProgressHandler_HttpReceiveProgress;
// Create our HttpClient.
_client = HttpClientFactory.Create(_progressHandler);
_client.BaseAddress = new Uri("http://localhost");
_client.Timeout = TimeSpan.FromMinutes(30);
_client.DefaultRequestHeaders.TransferEncodingChunked = true;
}
Download Logic:
private void StartDownload()
{
// Create the request.
using (HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, "http://localhost/Download"))
{
// Revert the protocol version and turn off keep alive in accordance with:
// http://briancaos.wordpress.com/2012/07/06/unable-to-read-data-from-the-transport-connection-the-connection-was-closed/
// http://briancaos.wordpress.com/2012/06/15/an-existing-connection-was-forcibly-closed-by-the-remote-host/
request.Version = new Version("1.0");
request.Headers.Add("Keep-Alive", "false");
// Set the cancellation token's timeout to 30 minutes.
int timeoutInMilliseconds = 30 * 60 * 1000;
using (CancellationTokenSource cancellationTokenSource = new CancellationTokenSource(timeoutInMilliseconds))
{
// Making sure that the message isn't "complete" until everything is read in so we can cancel it at anytime.
Task<HttpResponseMessage> responseTask = _client.SendAsync(request, HttpCompletionOption.ResponseContentRead);
responseTask.Wait(cancellationTokenSource.Token);
using (HttpResponseMessage response = responseTask.Result)
{
if (!response.IsSuccessStatusCode)
{
throw new Exception("Request failed!");
}
Task<Stream> streamTask = response.Content.ReadAsStreamAsync();
using (Stream contentStream = streamTask.Result)
{
// TODO: Save to disk.
}
}
}
}
}