Develop Reference

"Resource temporarily unavailable" error from System.Net.Sockets

I am working on a .NET API that runs inside of a docker container. At some point it makes a call to a Python Flask API that is also running in a container.
var response = await httpClient.GetAsync("http://service-name:8000/actual/url")
which then produces the following error:
System.Net.Http.HttpRequestException: Resource temporarily unavailable
---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken
cancellationToken)
Has anyone had experience with this before and potentially knows a solution? I cant find much on the web about it at all. I have some seen some mentions of the issue potentially being related to the Flask API not using async methods but that doesnt make sense to me.
The Flask API produces the appropriate responses when accessed through a web browser or Postman using localhost:8000/actual/url and the container logs these responses. I have tried using the localhost URL in the .NET API but that does not work either.
If anymore information is needed please leave a comment and I will do my best to update the post quickly.
-- Christie

TLDR
A reason for the "Resource temporarily unavailable" error is when during name resolution the DNS Server responds with RCODE 2 (Server failure).
Long answer
I noticed the same behavior in a dotnet application running in a dotnet runtime alpine docker container. Here are the results of my investigation:
The error message "Resource temporarily unavailable" corresponds to the EAGAIN error code which gets returned by various functions from the C standard library. At first I suspected the connect() function because the C# stack trace indicates the error happening during the ConnectAsync() call of the c# socket. And indeed the EAGAIN error code appears in the man page of connect() with this description: "No more free local ports or insufficient entries in the routing cache".
I simulated a system with depleted local ports and noticed that a different exception gets thrown in that case, which rules out local port availability as a root cause for the original exception. Regarding the other mentioned cause in the man page it turns out that the routing cache was removed from Linux in 2012. commit
I started to look around for EAGAIN in the source of the musl C lib which is used in the dotnet runtime alpine docker container. After a while I finally noticed the gethostbyname2_r function which is used for resolving a domain name to an ip address via DNS. During System.Net.Sockets.Socket.ConnectAsync() the hostname is still a string and the name resolving happens in native code using the gethostbyname2_r function (or one of its variations).
The final question is: When does gethostbyname2_r return the EAGAIN error code? It's when the RCODE field in the header of the DNS Response has the value 2, which stands for "Server failure". source line 166
To verify this result I ran a simple mock DNS server which always returns the RCODE 2 in the DNS response. The resulting c# exception along with the stack trace matched the original exception exactly.

System.Net.WebException: The remote name could not be resolved

This kind of question has been asked several times, and I understand why it happens, and probably nothing we can do about it except retry.
I do have one question on name resolution though.
I am using AWS .Net SDK for 3.5 .Net. I am uploading a big file (>500MB up to 1.5GB, medical images). I call TransferUtility.Upload() method.
For most part the program works great.
Occasionally we get this error in the middle of the upload. Usually happens when the internet is slow.
I can catch the exception and retry, which means rery from the beginning since exception happens inside the AWS code.
My question is, if the program has resolved the s3 bucket name and has been uploading for a while why would it give me name resolution error instead of just using the cached resolved name?
Does each thread resolve the name independently and one of thread is failing since the network is saturated? Is this a computer setting? This error we were able to reproduce pretty consistently on a Windows 10 machine with Charter as ISP uploading a 800MB file.
The error occurred after about 250MB upload was done.
This is the actual exception
Exception during upload :Amazon.Runtime.AmazonServiceException:
A WebException with status NameResolutionFailure was thrown. --->
System.Net.WebException: The remote name could not be resolved: 'my-bucket.s3.amazonaws.com'

This web exception is telling you the there was an issue with the "Name Resolution". What it doesn't tell you is that the "name" it's referring to is the "EndpointRegion", for example: USEast1, USEast2 etc.
When using the Amazon.S3.Transfer.Transferutility it's crucial that the EndpointRegion you use in the Upload call MATCHES that of the bucket you're uploading into.
In my case using RegionEndpoint.GetBySystemName("USEast1") vs RegionEndpoint.GetBySystemName("US-East-1") was the difference maker.

Another cause for this issue could be DNS resolution. If your system is not able to perform DNS resolves it will give you this same error.

Google.Apis.Admin.Email_Migration_v2 HTTP 410 status code returned for a 503 error

Essentially, while using the .NET version of the Email Migration v2 Google API our application is sending up too many requests per second to a single Google Apps mailbox/user; greater than 1 Request per second. A GoogleApiException is being returned which is fine, and expected, however the body of the error message states a service unavailable (503) error has occurred, but yet the "HttpStatusCode" property of that same GoogleApiException instance is equal to an Http status code of Gone (410), I will include a code snippet and some log output below. At this point, see the questions section at the bottom or read on for better detail.
What steps will reproduce the problem?
Create a process/application that does the following:
Create a Google.Apis.Admin.email_migration_v2.AdminService object, properly initialize it using your OAuth2.0 credentials.
For each message that needs to be sent to Google Apps Create a Google.Apis.Admin.email_migration_v2.MailResource.InsertMediaUpload instance using the AdminService object from above through using AdminService.Mail.Insert() providing proper parameters.
Call MailResource.InsertMediaUpload.UploadAsync while catching any errors that occur.
Do the following:
Begin sending messages at an exponential rate by spooling off hundreds of instances of this process/application all pointing to the same user, using the same OAuth2.0 credentials.
Sit back sip your mountain dew and wait for the [503] errors to roll in...
Once errors start rolling in close down your applications.. no need to hammer the poor Google servers other than for testing the applications exception handling..
What is the expected output? What do you see instead?
Using an instance of the following type: Google.GoogleApiException
One would expect to see the instance's HttpStatusCode property be equivalent to System.Net.HttpStatusCode.ServiceUnavailable instead of System.Net.HttpStatusCode.Gone
What version of the product are you using?
Google.Apis.Admin.Email_Migration_v2 (1.8.1.20)
What is your operating system?
Windows Server 2008 R2 Enterprise (SP1)
What is your IDE?
Visual Studio 2013 Premium
What is the .NET framework version?
4.0.30319
Please provide any additional information below.
Here is a code snippet of the method being called for uploading purposes:
UploadStatus TryUpload(MailResource.InsertMediaUpload insertMediaUpload)
{
try
{
IUploadProgress uploadProgress = insertMediaUpload.UploadAsync(_cancellationToken).Result; // Task.Result locks this thread until completed.
if (uploadProgress != null && uploadProgress.Exception != null)
{
// Display additional information on any of the various exceptions that can be returned by the upload call.
HandleUploadProgressException(uploadProgress);
}
return uploadProgress != null ? uploadProgress.Status : UploadStatus.Failed;
Here is a code snippet from the method that is displaying the output of the exception.
void HandleUploadProgressException(IUploadProgress uploadProgress)
{
if (uploadProgress.Exception is GoogleApiException)
{
GoogleApiException gApiEx = uploadProgress.Exception as GoogleApiException;
throw new vsEventException(vsMapEvent.MapHttpError(gApiEx.HttpStatusCode, vsEventMessages.Id.errGmailUnidentifiableGoogleApiException),
String.Format("GoogleApiException handled in GmailMessenger.HandleUploadProgressException. HttpStatusCode: {0}", gApiEx.HttpStatusCode),
gApiEx);
}
Here is paraphrased output of the GoogleApiException handled by the HandleUploadProgressException method: (note using a custom logging class;
outputting to DebugView)
** Context Info **
Error attempting to write item to Gmail...
** Event Details **
VS-EventID: 30003(errGmailTryUpload) GoogleApiException handled in
GmailMessenger.HandleUploadProgressException. HttpStatusCode: Gone
** Inner Exception Details **
The service admin has thrown an exception: Google.GoogleApiException:
Google.Apis.Requests.RequestError
Service unavailable. Please try again [503]
Errors [ Message[Service unavailable. Please try again] Location[ - ]
Reason[backendError] Domain[global] ]
at
Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task)
at
Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task
task)
at
Google.Apis.Upload.ResumableUpload`1.d__e.MoveNext()
Questions:
Can anyone shed some light on if this is expected behavior or a bug?
If this is the expected result how should the application handle a 410 based error? I know that when most if not all 400 errors are encountered processing of that particalar item should stop, but this does not seem to be a local issue, more a server side issue.
I appreciate any responses returned, I know it can be hard for questions as specific as this.

How to set MaxBufferSize/MaxReceivedMessageSize in Windows Phone 7?

I've been having trouble querying a web server for information from my Windows Phone 7 app, and from my research, I've traced it back to MaxBufferSize/MaxReceivedMessageSize being too low. The problem is, I can't figure out how to change it. Every search result I find talks about the application being a WCF app, and to change the binding in ServiceReferences.ClientConfig. The solution from this post gives this example:
1. edit the ServiceReferences.ClientConfig to accept a large buffer.
<binding name="BasicHttpBinding_MosaicService" maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647">
However, I'm not using a WCF app, and I don't know what the equivalents of the above, and ServiceReferences.ClientConfig are for a regular WP7 application. Can anyone give me some help with this? I've done my best to figure it out myself but I'm getting nowhere.
For reference, in case someone else is having the problem I was (it took a really long time to figure out that this may be the problem), this is what is happening to me: I'm getting a WebException with the message: "The remote server returned an error: NotFound" thrown. And the relevant section of the stacktrace is:
at System.Net.AsyncHelper.BeginOnUI(SendOrPostCallback beginMethod, Object state)
at System.Net.BrowserHttpWebRequest.EndGetResponse(IAsyncResult asyncResult)

I'm not sure how you went from the error message "The remote server returned an error: NotFound" to thinking that your client can't handle the size of the response. Much more likely, the error message is correct and the URL you are requesting can't be found. Drop a breakpoint in your code where you kick off the request. What is the RequestUri property set to on your HttpWebRequest object? Copy that URL into a browser and you'll more than likely see that you get a "page not found". Fix your incorrect URL and all will be well.

What causes the System.Web.HttpException with error code 0x80070057 on Page.Flush while debugging in VS2005?

Here is the complete error message:
An exception of type
'System.Web.HttpException' occurred in
System.Web.dll but was not handled in
user code
Additional information: The remote
host closed the connection. The error
code is 0x80070057.
and the offending code:
char[] buffer = oPage.HTML.HTML.ToCharArray();
Page.Response.Write(buffer, 0, buffer.Length);
Page.Response.Flush();
Page.Response.End();
The oPage.HTML.HTML is a string in a custom page object used by our app. The exception triggers on Page.Flush() and appears to be benign -- I just hit "continue" and everything goes along fine. This never appears at run time.
I have chased many, many Google hits down many rabbit holes and have found nothing. Visual Studio 2005, Vista Ultimate (IIS7).

I've been dealing with this same error for a while now, and my understanding is that when Flush is called, there must be a connection on the other end, otherwise, this error is thrown. It's easy to get into a "fire-and-forget" kind of model when writing web pages, but when the client disconnects (in this debugging case, you're the client), there's nowhere to flush to.
There are two solutions I've found to this:
Wrap Response.Flush and catch the exception.
Check Response.IsClientConnected before you call flush.
I'm not 100% sure about the second one...I'm still in the process of checking that one out.
Good luck!