I am trying to fetch Employee data from Zoho using the URL :
https://people.zoho.com/people/api/forms/P_EmployeeView/records
using the HttpClient's GetAsync(). While executing the code in my local dev environment the code runs smoothly and fetches the required data but as soon as I publish my code to the azure function I get an exception with the following stack trace :
2021-06-01T06:14:45.870 [Error] System.AggregateException: One or more errors occurred. (One or
more errors occurred. (A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because connected host has failed
to respond.))---> System.AggregateException: One or more errors occurred. (A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond.)--->
System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did
not properly respond after a period of time, or established connection failed because connected
host has failed to respond.---> System.Net.Sockets.SocketException (10060): A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond.at
System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken
cancellationToken)--- End of inner exception stack trace ---at
System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken
cancellationToken)at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request,
Boolean allowHttp2, CancellationToken cancellationToken)at
System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request,
CancellationToken cancellationToken)at
System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request,
CancellationToken cancellationToken)at
System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean
doRequestAuth, CancellationToken cancellationToken)at
System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken
cancellationToken)at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request,
CancellationToken cancellationToken)at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1
sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)--- End of
inner exception stack trace ---at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean
includeTaskCanceledExceptions)at System.Threading.Tasks.Task`1.GetResultCore(Boolean
waitCompletionNotification)at System.Threading.Tasks.Task`1.get_Result()at
EmployeeDataRefresh.ZohoClient.GetEmployeeData(ILogger log) in
C:\Projects\ZohoAttendance\Internal-Automation-and-Power-BI-Dashboard-zoho-employee-data-
update\src\EmployeeDataRefresh\EmployeeDataRefresh\ZohoClient.cs:line 39--- End of inner exception
stack trace ---at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean
includeTaskCanceledExceptions)at System.Threading.Tasks.Task`1.GetResultCore(Boolean
waitCompletionNotification)at System.Threading.Tasks.Task`1.get_Result()at
EmployeeDataRefresh.Trigger.DoRefresh(ILogger log) in C:\Projects\ZohoAttendance\Internal-
Automation-and-Power-BI-Dashboard-zoho-employee-data-
update\src\EmployeeDataRefresh\EmployeeDataRefresh\Trigger.cs:line 68at
EmployeeDataRefresh.Trigger.AutoRefreshEmployeeData(TimerInfo myTimer, ILogger log) in
C:\Projects\ZohoAttendance\Internal-Automation-and-Power-BI-Dashboard-zoho-employee-data-
update\src\EmployeeDataRefresh\EmployeeDataRefresh\Trigger.cs:line 30
Here's my code that fetches the data
using(var httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.Authorization
= new AuthenticationHeaderValue("Bearer", _authToken);
Uri myUri = new Uri(_url, UriKind.Absolute);
var response = httpClient.GetAsync(myUri);
log.LogInformation(_authToken);
log.LogInformation("Sending Get Request to Zoho...\n");
var data = await response.Result.Content.ReadAsStringAsync();
log.LogInformation("Data fetched from Zoho...\n");
var employes = JsonConvert.DeserializeObject<List<Employee>>(data);
return employes;
}
I get the error at line
var data = await response.Result.Content.ReadAsStringAsync();
I have put various log statements to debug the issue and the last log statement that gets printed on azure function log is "Sending Get Request to Zoho...".
I have printed the tokens and other required variables to check whether they have correct values and they are getting the correct value so invalid token is definitely not an issue. Can someone suggest what could be the possible reason for this error ?
are you expecting json response type ? and for best practice perhaps you need to supply in your header of httpclient
string contentTypeValue = "application/json";
client.DefaultRequestHeaders.Add("Content-Type", contentTypeValue);
and also practice
httpResponse.EnsureSuccessStatusCode(); // throws if not 200-299
before read result stream.
Here there is no issue with token or any authentication , just clear out asynchronous programming,
Frist,
Instead, by getting the value of the response.Result property, you force the current thread to wait until the asynchronous operation has completed, and second I will recommend
static readonly HttpClient client = new HttpClient();
try
{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _authToken);
Uri myUri = new Uri(_url, UriKind.Absolute);
HttpResponseMessage response = await client.GetAsync(_url);
var data = await response.Result.Content.ReadAsStringAsync();
var employes = JsonConvert.DeserializeObject<List<Employee>>(data);
}
catch (Exception)
{
throw;
}
Related
i made a program, to retrieve some price from some website, it works pretty good, but if i let my application rest or idle for 1 or 2 minutes then when i want to run a price check againm it would throw an exception "an error occurred while sending the request" , i have tried everything i found in google, like adding SecurityProtocol, running it on a Task or using WebClient, but nothing works.
here is the function that retrieve my http code, i then extract the price from it.
HttpClient client = new HttpClient();
public async Task ListViewScanner(string URL, int SelectedItem)
{
string webData;
try
{
System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12 |
System.Net.SecurityProtocolType.Tls11 |
System.Net.SecurityProtocolType.Tls;
using (HttpResponseMessage response = await client.GetAsync(URL))
{
response.EnsureSuccessStatusCode();
webData = await response.Content.ReadAsStringAsync(); //HERE MY CATCH GETS TRIGGERED
}
}
catch (HttpRequestException e)
{
MessageBox.Show($"Message :{e.Message} ");
webData = "ERROR";
}
}
I commented where i get the exception.
exceptions
SocketException: An existing connection was forcibly closed by the remote host
IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
WebException: The underlying connection was closed: An unexpected error occurred on a send.
HttpRequestException: An error occurred while sending the request.
Use a new instance of HttpClient every time, it's maybe related to the old instance that close the connection.
don't forge to use with a using statement to make sure that is destroyed when the call end.
I am trying to load a large amount of data from a server by returning MultiPartContent in the response. However, during the HttpGet method the timeout value I am setting is being ignored. I am trying to set a longer timeout as waiting in this situation is OK.
I have tried various Timeout values between 10 minutes to 2 hours. Realistically the user will only have to wait ~5 minutes the first time they use our app and all other times it will be less.
I have also tried storing a reference of HttpClient to ensure that api.HttpClient isn't re-creating the HttpClient object each time.
var handler = new HttpClientHandler();
var progress = new ProgressMessageHandler(handler);
var api = APIHelpers.GetSession(progress);
progress.HttpReceiveProgress += (e, args) => ProgressChanged.Invoke(e, new SyncEventArgs(GlobalEnums.SyncStage.Downloading, args.ProgressPercentage * 0.01f));
api.HttpClient.Timeout = new TimeSpan(0, 10, 0);
var downloadUri = BuildURI();
var response = await api.HttpClient.GetAsync(downloadUri, HttpCompletionOption.ResponseHeadersRead);
I would expect the application to wait for 10 minutes before throwing a Timeout exception, however the exception is being thrown after 100 seconds (i believe that is the default value?).
Checking the HttpClient.Timeout value after it is set does show that it was set correctly.
The APIHelpers.GetSession() method returns an object of our API with authorisation headers created. The HttpClient object is accessible via an inherited class. This is created by Swashbuckle and Swagger. I have used the api.HttpClient.Timeout = x this way before with success so I don't think this is the issue. It seems to be specific to this scenario in particular.
The exception thrown:
{System.Net.Http.HttpRequestException: An error occurred while sending the request ---> System.Net.WebException: The operation has timed out.
at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <a1ab7fc4639d4d84af41d68234158b1c>:0
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00019] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System/System.Net/HttpWebRequest.cs:1200
at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func`2[T,TResult] endFunction, System.Action`1[T] endAction, System.Threading.Tasks.Task`1[TResult] promise, System.Boolean requiresSynchronization) [0x0000f] in <939d99b14d934342858948926287beba>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Http.MonoWebRequestHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x003d1] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System.Net.Http/MonoWebRequestHandler.cs:499
--- End of inner exception stack trace ---
at System.Net.Http.MonoWebRequestHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x0046a] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System.Net.Http/MonoWebRequestHandler.cs:503
at Microsoft.Rest.RetryDelegatingHandler+<>c__DisplayClass11_0.<SendAsync>b__1 () [0x000ad] in <6a6c837cafbb4f1faffaba1ff30ca4e3>:0
at Microsoft.Rest.RetryDelegatingHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00150] in <6a6c837cafbb4f1faffaba1ff30ca4e3>:0
at System.Net.Http.Handlers.ProgressMessageHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00087] in <19af20e76ce04547b3fc150a0f8f2d47>:0
at System.Net.Http.HttpClient.SendAsyncWorker (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) [0x0009e] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.14.0.114/src/Xamarin.iOS/mcs/class/System.Net.Http/System.Net.Http/HttpClient.cs:281
at ...
EDIT:
I'm just adding some additional things I have tried in an attempt to solve this.
I have tried using the DependencyService in Xamarin.Forms to get native HttpClientHandlers and HttpClient objects for each platform where I can set the timeouts per platform instead of relying on Xamarin.Forms to translate it for me. This had the exact same result.
public HttpMessageHandler GetHttpHandler(double timeoutSeconds)
{
var sessionConfig = NSUrlSessionConfiguration.DefaultSessionConfiguration;
sessionConfig.TimeoutIntervalForRequest = timeoutSeconds;
sessionConfig.TimeoutIntervalForResource = timeoutSeconds;
sessionConfig.WaitsForConnectivity = true;
var sessionHandler = new NSUrlSessionHandler(sessionConfig);
return sessionHandler;
}
I have tried creating a new HttpClient object just for this API call but with no luck here either. (This replaces Apihelpers.GetSession() in my code)
There was a known issue in both Xamarin.iOS and Xamarin.Android that HttpClient.Timeout values greater than 100 seconds are ignored. This is because the underlying native http clients have a timeout set to 100 seconds, so this times out before the .NET HttpClient times out when the HttpClient.Timeout value > 100 seconds. This should be fixed in all of the latest stable versions, so make sure you are updated. If you are using Visual Studio 2017, you won't have the fix, you will need to get VS 2019 for the fixes.
Had the same issue in my Xamarin Android project. Resolved it by changing HttpClient implementation to Android in the Project properties as below:
I've written an application that in part can download files from a specific web service. The code utilizes an HttpClient to make the calls. The problem is that occasionally I will get a failed request with the following exception message:
Unable to read data from the transport connection: The connection was closed.
I did run across these blog posts, in which the author had to revert the protocol version to 1.0, disable keep alive, and limit the number of service point connections:
http://briancaos.wordpress.com/2012/07/06/unable-to-read-data-from-the-transport-connection-the-connection-was-closed/
http://briancaos.wordpress.com/2012/06/15/an-existing-connection-was-forcibly-closed-by-the-remote-host/
I followed those instructions, as best I knew how and still got the error. I also made sure to keep a single instance of the HttpClient around (following the Singleton principle).
What is interesting is that when running Fiddler I've yet to get the error, which makes me think that there is something that can be done on the client side since Fiddler appears to be doing something to keep the connection alive (though the issue is so sporadic this may be a red herring).
A couple more notes:
The error invariably occurs in the middle of a download (never when initiating the request).
The file continues to download up to the point of failure (there are no extended pauses or delays first).
--UPDATE--
The error occurs on the following line:
responseTask.Wait(cancellationTokenSource.Token);
The following is the full exception:
System.AggregateException occurred HResult=-2146233088 Message=One
or more errors occurred. Source=mscorlib StackTrace:
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at Form1.StartDownload() in c:\Projects\Visual Studio 2012\Demo\Demo\Form1.cs:line 88 InnerException:
System.Net.Http.HttpRequestException
HResult=-2146233088
Message=Error while copying content to a stream.
InnerException: System.IO.IOException
HResult=-2146232800
Message=Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Source=System
StackTrace:
at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.EndRead(IAsyncResult
asyncResult)
at System.Net.Http.Handlers.ProgressStream.EndRead(IAsyncResult
asyncResult)
at System.Net.Http.StreamToStreamCopy.BufferReadCallback(IAsyncResult ar)
InnerException: System.Net.Sockets.SocketException
HResult=-2147467259
Message=An existing connection was forcibly closed by the remote host
Source=System
ErrorCode=10054
NativeErrorCode=10054
StackTrace:
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
InnerException:
--UPDATE #2--
I thought I would try changing the completion option from 'content read' to 'headers read'. This also failed with the same exception, albeit in a different location (where the TODO comment is, reading the content stream).
--UPDATE #3--
I can confirm that the web service (which is hosted in IIS) is aborting the connections (the IIS logs show a win32 status code of 1236 - ERROR_CONNECTION_ABORTED). To try and narrow things down, the MinFileBytesPerSec metabase property was set to zero (on the off chance the client stopped pulling down data momentarily) and the connection is still being aborted. I've double checked all the timeouts and buffer sizes I can think of to no avail. Clawing at thin air at the moment. Any ideas would be appreciated.
Client Setup:
private void SetupClient()
{
// In case we're taxing the web server, limit the number
// connections we're allowed to make to one.
ServicePointManager.DefaultConnectionLimit = 1;
// Set up the progress handler so that we can keep track of the download progress.
_progressHandler = new ProgressMessageHandler();
_progressHandler.HttpReceiveProgress += ProgressHandler_HttpReceiveProgress;
// Create our HttpClient.
_client = HttpClientFactory.Create(_progressHandler);
_client.BaseAddress = new Uri("http://localhost");
_client.Timeout = TimeSpan.FromMinutes(30);
_client.DefaultRequestHeaders.TransferEncodingChunked = true;
}
Download Logic:
private void StartDownload()
{
// Create the request.
using (HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, "http://localhost/Download"))
{
// Revert the protocol version and turn off keep alive in accordance with:
// http://briancaos.wordpress.com/2012/07/06/unable-to-read-data-from-the-transport-connection-the-connection-was-closed/
// http://briancaos.wordpress.com/2012/06/15/an-existing-connection-was-forcibly-closed-by-the-remote-host/
request.Version = new Version("1.0");
request.Headers.Add("Keep-Alive", "false");
// Set the cancellation token's timeout to 30 minutes.
int timeoutInMilliseconds = 30 * 60 * 1000;
using (CancellationTokenSource cancellationTokenSource = new CancellationTokenSource(timeoutInMilliseconds))
{
// Making sure that the message isn't "complete" until everything is read in so we can cancel it at anytime.
Task<HttpResponseMessage> responseTask = _client.SendAsync(request, HttpCompletionOption.ResponseContentRead);
responseTask.Wait(cancellationTokenSource.Token);
using (HttpResponseMessage response = responseTask.Result)
{
if (!response.IsSuccessStatusCode)
{
throw new Exception("Request failed!");
}
Task<Stream> streamTask = response.Content.ReadAsStreamAsync();
using (Stream contentStream = streamTask.Result)
{
// TODO: Save to disk.
}
}
}
}
}
I am calling my ServiceStack service to run a long running process. (ServiceStack 4.011)
I keep getting a Gateway Timeout error after approximately 60 seconds.
I tried to set the timeout to be longer on the client (see below) but it isn't helping. I still get the gateway timeout error after 60 seconds.
Can anyone help me extend the timeout so that I don't get a Gateway Timeout error?
jsonServiceClient client = new JsonServiceClient(ncdServiceAddress);
client.Timeout = new TimeSpan(0,5,0); // 5 min
The full error is below.
NCD.DataPump.Cache_Coverages_Pump.PumpDOT_CarriersToNCD_fromLastUpdatedDate_usingS3' failed: Gateway Timeout
ServiceStack.WebServiceException: Gateway Timeout
at ServiceStack.ServiceClientBase.ThrowWebServiceException[TResponse](Exception ex, String requestUri)
at ServiceStack.ServiceClientBase.ThrowResponseTypeException[TResponse](Object request, Exception ex, String requestUri)
at ServiceStack.ServiceClientBase.HandleResponseException[TResponse](Exception ex, Object request, String requestUri, Func`1 createWebRequest, Func`2 getResponse, TResponse& response)
at ServiceStack.ServiceClientBase.Send[TResponse](Object request)
I've got a web app, that gets data from external services. The request itself happens like the code below - quite straightforward as far as I can see. Create a request, fire it away asynchronously and let the callback handle the response. Works fine on my dev environment.
public static void MakeRequest(Uri uri, Action<Stream> responseCallback)
{
WebRequest request = WebRequest.Create(uri);
request.Proxy = null;
request.Timeout = 8000;
try
{
Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null)
.ContinueWith(task =>
{
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(response.GetResponseStream());
responseStream.Close();
response.Close();
});
} catch (Exception ex)
{
_log.Error("MakeRequest to " + uri + " went wrong.", ex);
}
}
However external test environments and the production environment could, for reasons beyond me, not reach the target URL. Fine, I thought - a request timeout won't really hurt anyone. However, it seemed that every time this request timed out, ASP.NET crashed and IIS was restarted. The event log shows me, among other things, this stacktrace:
StackTrace: at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.get_Result()
at MyApp.AsyncWebClient.<>c__DisplayClass2.<MakeRequest>b__0(Task`1 task)
at System.Threading.Tasks.Task`1.<>c__DisplayClass17.<ContinueWith>b__16(Object obj)
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
InnerException: System.Net.WebException
Message: Unable to connect to the remote server
StackTrace: at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endMethod, TaskCompletionSource`1 tcs)
InnerException: System.Net.Sockets.SocketException
Message: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
StackTrace: at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)
..so it all boils down to a SocketException, it seems to me. And right afterwards (within the same second) another event (which I'm guessing is relevant) is logged:
Exception Info: System.AggregateException
Stack:
at System.Threading.Tasks.TaskExceptionHolder.Finalize()
This is sort of beyond me as I'm no master of async code and threading, but that a timeout from a web requests causes IIS to crash seems very weird. I reimplemented MakeRequest to perform the requests synchronously, which works out great. The request still times out in those environments, but no damage is done and the app continues to run happily forever after.
So I've sortof solved my problem, but why does this happen? Can anyone enlighten me? :-)
Your continuation needs to handle the fact that .Result might reflect an exception. Otherwise you have an unhandled exception. Unhandled exceptions kill processes.
.ContinueWith(task =>
{
try {
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(responseStream);
responseStream.Close();
response.Close();
} catch(Exception ex) {
// TODO: log ex, etc
}
});
your old exception handler only covers the creation of the task - not the callback.