We've been having issues sending certain Docusign envelopes lately, specifically those with large file sizes.
The errors we've been getting are:
Error calling CreateEnvelope: The operation has timed out
And
The request was aborted: The request was canceled.
No inner exception with any additional information in either case.
These errors only occur on our production server; on my local development machine everything works fine, so I can only assume that this is a connectivity issue; that there simply isn't enough time to send the supplied data over the available connection before something times out. What I would like to know is, what is the something that's timing out? Are these errors coming from my end, or Docusign's? If the former, is there any way to increase the timeout? I've got my HTTP execution timeout set to 300 seconds:
<httpRuntime maxRequestLength="30000" requestValidationMode="4.0" executionTimeout="300" targetFramework="4.5" />
... but that doesn't seem to affect anything, it always seems to time out at the default 1 minute 50 seconds.
Is there anything more I can do to prevent these requests from timing out?
Thanks,
Adam
Our issue has been resolved. The timeouts were indeed being caused by something on our end; there is a "Timeout" property which can be set against the EnvelopesApi object before sending; it can also be passed into the constructor when declared. So our fix was as simple as:
EnvelopesApi envelopesApi = new EnvelopesApi();
envelopesApi.Configuration.Timeout = DocusignTimeout;
The crux of our issue was that the Timeout property was not exposed in older versions of eSign. We had upgraded to 2.1.0 (the current version) earlier this week, but something must not have taken, as the metadata still showed our DocuSign.eSign.Client.Configuration class at version 15.4.0.0. Uninstalling the reinstalling eSign and RestSharp packages from NuGet gave us the correct version of this class, and enabled us to set our own timeout.
Hope this is helpful!
Related
I am fetching messages from Exchange in Office365 using Microsoft Graph API.
However, for some folders I seem to get intermittent exceptions.
What we are using:
Microsoft.Graph Version 3.9.0 - Microsoft Graph Client Library for .Net
Microsoft.Graph.Core Version 1.21.0 - Microsoft Graph Core Client Library for .Net
This is the call being used:
'GET /v1.0/users/{id}/mailFolders/{id}/messages'
And this is the error (ServiceUnavailable with UnknownError as inner exception):
Status Code: ServiceUnavailable Microsoft.Graph.ServiceException:
Code: UnknownError Message: Error while processing response. Inner
error:
AdditionalData:
date: 2020-08-04T13:55:33
request-id: ** ClientRequestId: **
Code: UnknownError Message: Error while processing response. Inner error:
AdditionalData:
date: 2020-08-04T13:55:33
request-id: ** ClientRequestId: **
What I've tried:
Throttling:
These are usually the errors we would see with throttling. However, in this case, there seems to be no indication of throttling being applied. There isn't any 'back-off' time returned in the result. Other requests to different folders returns just fine too. By applying our own 'back-off' time (ranging between 5mins-20mins does not seem so make a difference either).
Beta endpoint:
The call posted above shows /v1.0 used. We've also switched to the /beta endpoint, with no difference.
Amount of mails retrieved:
Graph allows us to retrieve up to 999 mails at a time. We've reduced that all the way down to a mail or 2 at a time, but it still returns with the same error.
Delta token:
We've also tried switching over to using the delta token in order to retrieve the mails. This also returns with the same error.
Graph downgrade:
Hoping that there is some difference in the last few versions, we downgraded Graph. There was no difference.
Check local sync issues:
I've noticed in the past (quite a while back), that when doing this call for a folder that has potential local sync issues, this is the same type of error response returned. In this case, there is no reason to believe that these are local sync issues.
Additional:
When setting up the httpProvider, I've removed the default retry handlers as well. I've seen that using the default retry handler, it would automatically catch the 'ServiceException' and do internal retries (not adhering to back offs (not that there is any)), and would result in a tooManyRetries or a timeout (hiding the actual issue). By removing the default retry handler, we can see the actual 'ServiceException' error returned by the server.
When:
Based on our telemetry, this seems to have started happening a lot more frequently since around the 11-13th of June. Before that we did not experience any issues.
There are days that the requests work, but they are few and far in between.
This is quite a big issue for us, so any suggestions would be greatly appreciated. Any specific Microsoft Support channel that I can log this with would also help.
Thanks in advance.
Background
I have an API that makes uses the ADOMD.Net client to retrieve data from Azure Analysis Services (AAS) Model. A connection is created using a connection string that contains an access token obtained as documented here.
Problem
The API works fine, until the AAS Model is paused and resumes which happens every night. After this, the Model returns a 207 Temporary Redirect response.
The remote server returned an error: (307) Temporary Redirect.
Technical Details:
RootActivityId: 4d85e2d9-e1ec-406d-92aa-3f3e33ac4ed4
Date (UTC): 3/9/2020 1:09:05 PM
The location in the header is:
https://asazureweu10-westeurope.asazure.windows.net/webapi/xmla
The original request (but made by the ADOMD.Net client, not me) is made to:
https://asazureweu5-westeurope.asazure.windows.net/webapi/xmla
After I restart the API the client starts responding with the data and functions correctly. This leads me to believe something is being cached, and once the API is restarted the cache is cleared maybe? Is this a Connection pooling issue?
Does anyone know what is happening behind the scenes and know why this is happening?
This is a verified bug which has been reported to MS.
The MS support engineer has said.
The client library indeed caches this the cluster information in the
AppDomain level cache, which has a timeout of 60 minutes. But there is
an invalidation code that invalidates the cache on error code >= 300
and <= 399. In this case, the error is 307 so the invalidation logic
should still work, as long as the connection is closes and then opened
again.
This bug is due to be patched in a late august, early September Nuget release.
Update: This bug has now been patched in Nuget package microsoft.AnalysisServices.AdomdClient.NetCore.retail.amd64, version 19.9.0.1-Preview.
The code snippet I'm using is as below.
// Catch any error redirection errors and retry. Due to internal workings of lib, if model is scaled up or down, or paused and restarted,
// the model will come back on a different cluster, so we need to get a new connection.
catch (AdomdConnectionException ex)
{
if ((int) ((HttpWebResponse) ((WebException) ex.InnerException).Response).StatusCode >= 300
& (int) ((HttpWebResponse) ((WebException) ex.InnerException).Response).StatusCode <= 399)
{
aasConnection = await GetOpenAasConnectionAsync(customData, role);
}
else throw;
}
Recenlty we've been having issues downloading envelope documents.Getting the bellow exception after 6 minutes.
envelopesApi.GetDocument(accountId, envelopeId, documentId)
DocuSign.eSign.Client.ApiException: Error calling GetDocument: The operation has timed out.
The timeout has set for 10min as bellow,
var envelopesApi = new EnvelopesApi();
envelopesApi.Configuration.Timeout = 600000;
envelopesApi.Configuration.ApiClient.RestClient.Timeout = 600000;//also added this
But after receive error, when re-trying through the postman is succeeding.
Also this error is intermittent.
Is there anything that we are missing ?
Thanks,
Dula
Timeouts can occur for a variety of reasons, like internet latency and other TPC/IP issues on the way from DocuSign servers to your box. I would recommend that operations like retrieving large files are done in the background.
You should also suggest you update the SDK to the latest version as some improvements were made in this area.
We have an Azure-based ASP.NET Web Service that accesses an Azure KeyVault. We are seeing two instances in which a method "hangs" on a first try, and then works a minute or so later.
In both instances, a KeyVault access occurs. In both instances the problem started when we started using the KeyVault in these methods.
We have done very careful logging in the first instance, and cannot see anything else in our code that could cause the hang. The KeyVault access is the primary suspect.
In addition, if we run the app from our local servers (from Visual Studio), the KeyVault access works fine on the "first try". It only produces the "hang" error when it runs in production on Azure, and only on that "first try".
By "hang" I mean that in one instance, which is triggered by an external API, it takes at least 60 seconds (we can tell that because the external API times out.) In the other instance, which is triggered by a page request, several minutes can pass and the page just spins, at which point we assume the DB request or something else has timed out.
When I say "a minute or so later", that's as fast as we have timed the retry.
Is there some kind of issue or function where the KeyVault needs to be "warmed up" before it works on the first try?
Update: I'm looking at the code more carefully, and I see at least a couple of places where we can insert still more logging to get a more exact picture of where the failure occurs. I'm going to do that, and then I'll report back here.
Update: See answer below - major newbie error, has been corrected.
Found the problem, and the solution.
Key Vault access needs to be called from an async task, because there is a multi-second delay.
private async Task<string> GetKeyVaultSecretValue(varSecretParms) {
I don't understand the underlying technology, however, apparently, if the call is from within a standard code sequence, the server doesn't like to wait, and so the thread is abandoned/halts.
According to your description, it seems that it dues to WebApp that does not enable Always on .
By default, web apps are unloaded if they are idle for some period of time. This lets the system conserve resources. In Basic or Standard mode, you can enable Always On to keep the app loaded all the time
If possible, please have a try to enable Always on and try it again.
In C# 5 and winform, I used a library created by Telegram Company. In this library there is a function SendDocument(UserId,DocumentStream). I know in this function, they used some HttpWebRequest, and the Timeout property of that is not handled. because sometimes it can't send large documents and after exact 100 seconds(default timeout in DotNet), the function throws an exception The task was canceled.
From the document of Telegram Company, we can send 50 MB files and my example files is about 15 MB.
Ok, Now I want to adjust the timeout of all HttpWebRequest of my server but I don't have any feature for this.
Can I adjust all HttpWebRequest.Timeout property in my server??
This is not directly related to your question, but may help ease your mind.
My advice is, don't bother adjusting the timeout. It is not likely to help. Here is what I have gone through:
I have tried to upload a 20M mp4 video file using Telegram Bot API. From a Raspberry Pi, it took 5 minutes, then returned a 504 Gateway-Timeout error. From a hosted server, it took 1 minute, then returned a 504 Gateway-Timeout error. In both cases, however, the video did eventually reach the recipient 5 minutes later. So, the upload seemed somewhat successful, yet not quite successful.
I tried to fix the problem by streaming the upload. Same problem persisted.
I tried to adjust the HTTP timeout parameter. Same problem persisted.
I tried to use cURL to make the request (instead of using telepot, a Python library I author). Same problem persisted.
I suspect the problem lies with the Telegram servers, so I talked to Bot Support. They got back to me once, saying they have made some improvements and asked if I still have the same problem. But same problem still persists.
So, it seems the problem does lie with the Telegram servers. It's not your code.
I know it's a pretty old question but may be my answer will help somebody. When I tried to send cosiderably large files via my bot I received Telegram.Bot.Exceptions.ApiRequestException: Request timed out and the only solution I found was this issue. Which wasn't really helpfull because if you check source code then you'll see that passing cancellation token does nothing with request timeout. But then I saw that you can pass HttpClient to your bot client instance and make it something like this:
_httpClient = new HttpClient();
_httpClient.Timeout = new TimeSpan(0, 5, 0); // 5 min
_client = new TelegramBotClient(botConfig.Token, _httpClient);
Hope this will help