Develop Reference

gRPC - bidirectional stream goes to TRANSIENT_FAILURE if idle for too long

I'm trying to get a fairly simple test scenario to work - I'd like to create a long-lived bidirectional streaming rpc that may sit idle for long periods of time (electron app with local server).
A Node gRPC client starts a C# gRPC server locally and initiates a bidirectional stream. The streaming service receives each message, waits 50 ms, and sends it back.
The Node client test code is set up to send 5 messages, wait 30 seconds, and then send 5 more messages. The first 5 messages successfully roundtrip. The second 5 messages eventually roundtrip, but not until 5 minutes later. The server side code is not hit during this time.
I'm sure I'm being a baboon here, but I don't understand why the connection seems to be dying so fast. I'm also not sure what options could help here, if any. It seems like keepalive is intended for tracking whether the TCP connection is still alive, but doesn't actually help keep it alive. idleTimeout doesn't seem relevant either, because we're going to TRANSIENT_FAILURE status according to the enum documentation here.
This discussion from 2016 is close to what I'm trying to do, but the solution was a RYO heartbeat. This grpc-dotnet issue seems to rely on a heartbeat-type solution specific to ASP.NET, which is not currently used.
gRPC server logs:
After the first 5 messages are sent:
transport 000001A7B5A63090 set connectivity_state=4
Start BDP ping err..."Endpoint read failed" (paraphrasing)
5 minutes later right before the second set of 5 messages comes through:
W:000001A7B5AC8A10 SERVER [ipv6:[::1]:57416] state IDLE -> WRITING [RETRY_SEND_PING]
Node library is #grpc/grpc-js
tl;dr How can I keep the connection healthy & working in the case of downtime?

how to set httpclient connect timeout?

I'm struggling with setting connect timeout with c# HttpClient or its siblings (HttpClientHandler,WebRequestHandler,...) . There's a timeout property in HttpClient, but it seems to be a timeout from the beginning of the request until receiving the response. I want to have a method which specify that for example if you don't received ACK from the net socket within 10 seconds for example , then break up and do the next.
I saw that there may be something similar in WinHttpHandler class, but it seems to be deleted or not available in recent version. compare the first link vs second :
1.WinHttpHandler MSDN
2.WinHttpHandler Microsoft Docs
I really need this, because I must differentiate asap between IP's which have a working web servers (maybe slow) vs which don't have a web server at all.

I use HttpWebRequest.Timeout in my project to verdict the connection time before establish tcp connection. And use HttpWebRequest.ReadOrWriteTimeout to verdict whole response timeout.
Ps: The HttpClient seems cut off some useful properties.

How can a slow internet on client side be a reason for server time out exception in asp.net?

I have a .Net application up and running.
We have had a fluctuating connection yesterday. While testing in such scenarios we had received multiple server time out exception emails like below.
Server Time Out
Type : System.Web.HttpException, System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Message : Request timed out.
Source :
Help link :
WebEventCode : 3001
ErrorCode : -2147467259
Data : System.Collections.ListDictionaryInternal
TargetSite :
HResult : -2147467259
Stack Trace : The stack trace is unavailable.
Additional Info:
IMPORTANT: Above exception occurred while doing a ajax post by a button placed with in update panel.
My question here is why would a slow internet on client side raise such server time out exception?
Isn't server timeout exception is related to such cases where server cannot execute the request in underlying time mention in HttpRuntime setting? May be due to some lengthy operation or some long database execution which takes longer than the time mentioned in setting under HttpRuntime.
If server is not able to connect to the client due to clients fluctuating internet, then Client Disconnected exception would be raised which we did yesterday. But I am not able to conclude the reason for this server timeout exception.
I already know that increasing the execution timeout will fix the issue, but I have to provide technical explanation for the reason as to why such exception of Server Timeout raised.
My best guess here is that the ajax request would be doing some continuous communication with server for executing of single request server and would raise timeout exception if it does not receives some required further communication messages due to client's bad internet. I have search over internet for the same to support my guess but in vain.
Also to provide environmental details, there is a load balancer serving the request.
Any help would be highly appreciated.

It is because (as you write) the connection of client to server is slow, so if the server (or client) sending data to this server, connection can´t handle it, so you get timeout error, because the data can´t been transfered in defined time.
You also write, that this is caused by sending Ajax request, so maybe try to increase execusion timeout in web configuration file (web.config):
<httpRuntime executionTimeout = "number(in seconds)"/>
More about executionTimeout here and here about Ajax requests.

Firstly, I think the cause of this error is because of execution time required by your application request connecting with the remote server is exceeding the currently set ASP.NET request execution timeout value. As per the MSDN Exception Document, default value is set to 110 seconds, in that it is remarked like:
The ExecutionTimeout property indicates the maximum number of seconds
a request is allowed to execute before being automatically shut down
by ASP.NET.
So based on error detail with event code 3001 occurs because no response was received during the time-out period for a request. You can use IIS troubleshooting failed request mechanism to figure it out exact issue like any poor performance/deadlocks when making calls from your ASP.NET application.
Secondly, it is not related to user's internet connectivity issue otherwise you get exception with status like connection-closed or keep alive failure. See this article for detail. The browser is going to wait for a 60 minutes(which is very long period of time that server isn't going to answer any request)for server to response.
And at any case when the browser abandons any request, it is going to close the socket and you'll get an error page from the browser. You don't get anything related to sever-end.

Azure ASP .net WebApp The request timed out

I have deployed an ASP .net MVC web app to Azure App service.
I do a GET request from my site to some controller method which gets data from DB(DbContext). Sometimes the process of getting data from DB may take more than 4 minutes. That means that my request has no action more than 4 minutes. After that Azure kills the connection - I get message:
500 - The request timed out. The web server failed
to respond within the specified time.
This is a method example:
[HttpGet]
public async Task<JsonResult> LongGet(string testString)
{
var task = Task.Delay(360000);
await task;
return Json("Woke", JsonRequestBehavior.AllowGet);
}
I have seen a lot of questions like this, but I got no answer:
Not working 1
Cant give other link - reputation is too low.
I have read this article - its about Azure Load Balancer which is not available for webapps, but its written that common way of handling my problem in Azure webapp is using TCP Keep-alive. So I changed my method:
[HttpGet]
public async Task<JsonResult> LongPost(string testString)
{
ServicePointManager.SetTcpKeepAlive(true, 1000, 5000);
ServicePointManager.MaxServicePointIdleTime = 400000;
ServicePointManager.FindServicePoint(Request.Url).MaxIdleTime = 4000000;
var task = Task.Delay(360000);
await task;
return Json("Woke", JsonRequestBehavior.AllowGet);
}
But still get same error.
I am using simple GET request like
GET /Home/LongPost?testString="abc" HTTP/1.1
Host: longgetrequest.azurewebsites.net
Cache-Control: no-cache
Postman-Token: bde0d996-8cf3-2b3f-20cd-d704016b29c6
So I am looking for the answer what am I doing wrong and how to increase request timeout time in Azure Web app. Any help is appreciated.
Azure setting on portal:
Web sockets - On
Always On - On
App settings:
SCM_COMMAND_IDLE_TIMEOUT = 3600
WEBSITE_NODE_DEFAULT_VERSION = 4.2.3

230 seconds. That's it. That's the in-flight request timeout in Azure App Service. It's hardcoded in the platform so TCP keep-alives or not you're still bound by it.
Source -- see David Ebbo's answer here:
https://social.msdn.microsoft.com/Forums/en-US/17305ddc-07b2-436c-881b-286d1744c98f/503-errors-with-large-pdf-file?forum=windowsazurewebsitespreview
There is a 230 second (i.e. a little less than 4 mins) timeout for requests that are not sending any data back. After that, the client gets the 500 you saw, even though in reality the request is allowed to continue server side.
Without knowing more about your application it's difficult to suggest a different approach. However what's clear is that you do need a different approach --
Maybe return a 202 Accepted instead with a Location header to poll for the result later?

I just changed my Azure Web Site from Shared Enviroment to Standard, and it works.

responding policy-file-request of flash from c#

i have developed a server application with c# and a client application with flash action script 3.0. Flash socket asking for a policy file when called from a browser with a message
<policy-file-request/>
everything is normal so far. My server is waiting for this message and sending to client a policy file string which is like this:
public const String POLICY_FILE = "<?xml version=\"1.0\"?>\n" +
"<!DOCTYPE cross-domain-policy SYSTEM \"http://www.adobe.com/xml/dtds/cross-domain-policy.dtd\">\n" +
"<cross-domain-policy>" +
"<allow-access-from domain=\"*\" to-ports=\"*\" />" +
"</cross-domain-policy>\u0000";
this string is being sent this way:
if (message.Contains("policy-file-request"))
{
client.Send(Encoding.ASCII.GetBytes(Statics.POLICY_FILE));
return;
}
I'm pretty sure that this was working but i really don't know what happened and started not working. When flash client receives this message from server, connection was succesfull and everything was going how it had to go. But now the flash client waits 20 seconds (timeout of flash socket) and throws security exception
[SecurityErrorEvent type="securityError" bubbles=false cancelable=false eventPhase=2 text="Error #2048"]
I'm stuck and can't move forward. I'm listening to port 963, server machine fully qualified name is "mypc.domain.local" which can be accessible across my network. there is also an IIS running on this machine and the flash application is hosted here.
http://mypc.domain.local:90/page.html
this is the way, i call my flash application and
mypc.domain.local:963
is the address of server running. i am also working on this machine. i tried calling the page http://localhost:90/page.html or http://127.0.0.1:90/page.html and also tried the connection to server as localhost:963 or 127.0.0.1:963. same result on every combination.
What is wrong here? what could have been changed causing my working code broke down?
Thanks.

It's hard to tell without more code, but based on what you've shown, it appears that when that request comes in, you respond with the contents of the policy file, which isn't an actual valid HTTP response. My guess for the 20 second timeout would be that it's still waiting for the HTTP headers.
If possible, try to use the HTTP classes already in the BCL instead of doing http 'manually', but if you have to do the socket stuff yourself, then use something like Fiddler during debugging since it's great for identifying violations of the HTTP protocol.