I am having consumption plan in azure, do not have any gateway or proxy setup.
While testing I got this error also its not logging in my app insights, it neglects that request, as like it did not reached/requested to it.
How can fix this ?
502 - Web server received an invalid response while acting as a gateway or proxy server
Cause:
This problem is often caused by application level issues, such as:
1.requests taking a long time
2.application using high memory/CPU
3.application crashing due to an exception.
Steps:
1.Observe and monitor application behavior
2.Collect data
3.Mitigate the issue
Have a look of this Offcial doc.
Related
I'm encountering the following problem in production with an ASP.Net Core website that runs InProc on IIS. The installed .Net Core runtime is 2.2.8.
Following a push notification on Signalr, my clients are sending about a hundred requests to a particular URL, which is bound to a controller method that looks like this:
[HttpPost()]
[Route("Api/GetData")]
public string[] GetData()
After serving a few requests, the server suddenly stops responding for a few minutes, before serving all the responses at once. No errors can be seen in the app logs, and the IIS logs shows huge response times for the requests.
I have enabled Failed Request Tracing in IIS to analyse requests that take longer than 9 seconds and I see that the requests that take forever to respond are actual promptly processed by IIS.
E.g:
IIS log: 47952ms
2020-05-12 12:18:15 169.XXX.XXX.XXX POST /Api/GetData - 443 -
195.XXX.XXX.XXX Mozilla/5.0+(Windows+NT+6.1;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko
200 0 0 79479 945 47952
IIS Failed Request Tracing logs:
I haven't used IIS Failed Request Tracing before and I'm not sure how I should interpret this. I first though of a lock in the code, but I haven't found it and I'm not sure how it would also affect requests to other URLs? What's your thoughts?
A maybe important note is that the clients are behind a proxy, so the server sees the same IP for them all. We have never encountered this issue before on another server.
UPDATE: We managed to reproduce the problem with a version of GetData() that simply returns a static array of string. We found that the size of the array mattered. We reproduced the issue with a response of 88KB, but not with a small one (2.5K). Looking at the .net core logs, it seems that the result of the method is computed promptly, but the response itself is hanging (we haven't figured out why). Upgrading to .Net Core 3.1 solved the problem...
I'm keeping the question open in case somebody can provide an explanation.
Upon calling a 3rd party SOAP endpoint i'm getting a 401 Unauthorised error - This is from my production server only. Works fine on every other machine I've tested on.
Production is running on Windows server 2008
I'm using the wsdl url to setup the request. I've noticed it seems to throw back the 401 so fast, that it doesn't seem to even be sending it to the remote machine. Is there perhaps some machine specific config that could be causing this?
I'm unsure of how to debug the situation as i can't replicate it on my local machine
There is no code i can think of that might help with this issue, since it is literally a single call of a SOAP wsdl
this.UseDefaultCredentials = true;
Placed the above manually inside the initialization of the class. From scouring the internet it seems occasionally windows server will use weird credentials and fail to bypass its own checks resulting in a 401 immediately.
Forcing useDefaultCredentials fixes this issue.
I downloaded the SignalR Getting Started solution and it works well. However if I add Azure Service Bus backplane then the client can't connect any more.
First it tries to connect to WebSocket
SCRIPT12030: WebSocket Error: Network Error 12030, The connection with the server was terminated abnormally
Then falls back to long polling and it doesn't work either.
SCRIPT7002: XMLHttpRequest: Network Error 0x2eff, Could not complete the operation due to error 00002eff.
Running it locally, debugging and there is no exception thrown on the server side.
The only difference between the working and non-working status is in the Startup.cs if I comment out the UseServiceBus part then it works:
//GlobalHost.DependencyResolver.UseServiceBus(connectionString, "Blabla");
app.MapSignalR();
Obviously I can't leave it like this for Azure deployment.
I tried upgrading / downgrading packages, both SignalR and WindowsAzure.ServiceBus, all the same.
Any ideas?
I have the same error, i have seen that the response for https://localhost/signalr/connect?transport=webSockets.... is 400 the problem is that my service bus Pricing plan is basic but only Standard tire to support Topics. So you have to go to azure portal and under the "Pricing tire" section of service bus you have to specifically go to the settings and scaled it up from Basic to Standard.
I have an Azure web role that accesses an external WCF based SOAP web service (port 80) for various bits of data. The response from this service is highly erratic. I routinely get the following error.
There was no endpoint listening at
http://www.myexternalservice.com/service.svc that could accept the message. This is
often caused by an incorrect address or SOAP action.
To isolate the problem I created a simple console app to repetitively call this service in 1 second intervals and log all responses.
using (var svc = new MyExternalService())
{
stopwatch.Start();
var response = svc.CallService();
stopwatch.Stop();
Log(response, stopwatch.ElapsedMilliseconds);
}
If I RDP to one of my Azure web instances and run this app it takes 10 to 20 attempts before it gets a valid response from the external service. These first attempts are always accompanied by the above error. After this "warm up period" it runs fine. If I stop the app and then immediately restart, it has to go back through the same "warm up" period.
However, if I run this same app from any other machine I receive valid responses immediately. I have run this logger app on servers running in multiple data centers (non Azure), desktops on different networks, etc... These test runs are always very stable.
I am not sure why this service would react this way in the Azure environment. Unfortunately, for the short term I am forced to call this service but my users cannot tolerate this inconsistency.
A capture of network traffic on the Azure server indicates a large number of SynReTransmit's in 10 second intervals during the same time I experience the connection errors. Once the "warm up" is complete the SynReTransmit's no longer occur.
The Windows Azure data center region where the Windows Azure application is deployed might not be near the external Web Service. The local machine you're trying (which works fine) might be close to the web service. That’s why there might be huge latency in Azure which would likely cause it to fail.
Success accessing WSDL from a browser in Azure VM might be due to browser caching. Making a function call from browser would tell you if it is actually making a connection.
We found a solution for this problem although I am not completely happy with it. After exhausting all other courses of action we changed the load balancer to Layer-7 Load Balancing from Layer-4 Load Balancing. While this fixed the problem of lost requests I am not sure why this made a difference.
I am load testing my webservice
and get a
System.ServiceModel.CommunicationException
when I use 10 threads to communicate to it (without any sleep in between) - basically testing 10 conenctions at a time - through a windows application
An error occurred while receiving the HTTP response to http://localhost/XXX/XXXService.asmx. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.
Why would this happen and how to best resolve it
Also is this the way how asp.net application will interact with it too - is threading the best way to do the load testing
To begin with, you might want to try using WCF for new web services development. Microsoft now considers ASMX web services to be "legacy technology", and suggests that all new development use WCF.
Next, you might want to find out what happened. The exception isn't lying to you. There was an HTTP protocol error. You could look at the network traffic using Fiddler, to see what the problem is.
You might also look in the server event log.