I am sending 1000 Request asynchronously to API with timeout to each request is 10 seconds.
But the trigger execute only 400 - 500 requests and ignoring rest of all.
My questions is “Is Http Trigger execute all request in parallel or sequentially or there is any limit for parallel threads in Http Trigger”.
Is Http Trigger execute all request in parallel or sequentially or there is any limit for parallel threads in Http Trigger.
It should be paralleled executed, in your case it seems that there is no enough resource for dealling with the request in your service plan.
For Azure function there are 2 different modes: Consumption plan and Azure App Service plan. We may could get more info from Azure document.
The Consumption plan automatically allocates compute power when your code is running, scales out as necessary to handle load, and then scales down when code is not running.
In the App Service plan, your function apps run on dedicated VMs on Basic, Standard, and Premium SKUs, similar to Web Apps. Dedicated VMs are allocated to your App Service apps, which means the functions host is always running.
It seems that you are using App service plan, if it is that case, please have a try scale up or scale out your service plan.
Related
I have an Azure HttpTrigger function which processes POST requests and scales out on heavy load. The issue is that the caller of the function only waits 3 sec. for a HTTP 200 status code.
But when an azure function scales out it takes 4-6 sec. until the request gets processed. If the caller sends a request during the scale out it is possible that he cancels the request and my service is never able to process it. Which is a worst case scenario.
Is there a way to prevent that? My ideal scenario would be an immediate HTTP 202 answer to the caller. But I'm afraid that this is not possible during a scale out process.
A Scale-out will require your app to be loaded onto another instance so some delay occurs/incur for those requests because of time taken to load your app onto the new instance.
As described in the Official Documentation:
Consumption Plan is the true serverless hosting plan since it enables scaling to zero when idle state, but some requests might have additional latency at startup.
To get constant low latency with autoscaling, you should move to the premium hosting plan which avoids cold starts with perpetually warm instances.
Our web application (hosted in a Web App in Azure) experiences spikes in HTTP Queue Length. Each time there is a spike in HTTP Queue Length, the web application crashes and we either have to wait for Azure to restart the web app itself, or we restart the web app ourselves. This happens very often.
The web application does use SignalR, and a Web Job is running that calls a method on the Hub which then broadcasts data to connected clients. There is only ever a handful of users at this stage, so we have not implemented a SignalR backplane.
Here is an example of the spikes in HTTP Queue Length
Note, we tried having the web application in its very own Application Service Plan (P3) and it still exhibited the same behaviour. The memory percentage was much lower than that shown here though, around the 20-40 percent, but still crashed with regular spikes in HTTP Queue Length. Thus, I don't believe it's a memory issue that is causing the issue.
After a while of trying to diagnose this issue, we decided to then host the application (same code) into a VM (still in Azure) and change the URL to point to the VM instead of the web app. The new VM is only very basic, with only 3.5GB Memory.
Since moving to a VM, the application is performing great, no crashes and performs much better than in a Web App with a large dedicated service plan.
So it is difficult to say it is the code, when we running perfmon and other indicators, all memory and queue lengths seem to quickly drop down after serving requests. Whereas in a WebApp, this seemed to continually grow until it crashed.
Just wondering if anyone else has experienced this behaviour with Web Apps? We are going to continue hosting in a VM, but originally preferred hosting within a Web App as PaaS is more appealing.
In case it helps, more information on the tech stack is:
HTML5, C#, Web API 2, Kendo MVVM, SignalR, Azure SQL Server, Web Jobs processing Service Bus Topics.
Kind regards,
Stefan
We have an WebApi json rest service written in C# for .Net 4.0 running in AWS. The service has a /log endpoint which receives logs and forwards the logs onto logstash via tcp for storage.
The /log endpoint uses Task.Factory.StartNew to send the logs to logstash async and returns StatusCode.OK immediately. This is because we don't want to client to wait for the log to be sent to logstash.
All exceptions are observed and handled, also we don't care if logs are lost because the service is shutdown or recycled from time to time as they are not critical.
At first the flow of logs was very low, probably 20 or 30 per hour during peek time. However we have recently started sending larger amounts of logs through, can be well over a thousand per hour. So the question now is that by using Task.Factoring.StartNew are we generating a large number of threads, i.e. 1 per request to the /log endpoint or is this managed somehow by a thread pool?
We use nLog for internal logging but are wondering if we can pass the logs from the /log endpoint to nlog to take advantage of its async batching features and have it send the logs to logstash? We have a custom target that will send logs to a tcp port.
Thanks in advance.
A Task in .NET does not equal one thread. It's safe to create as many as you need (almost). .NET will manage how many threads are created. .NET will not start more tasks than the hardware can handle.
I am using a WebApi service controller, hosted by IIS,
and i'm trying to understand how this architecture really works:
When a WebPage client is sending an Async requests simultaneously, are all this requests executed in parallel at the WebApi controller ?
At the IIS app pool, i've noticed the queue size is set to 1,000 default value - Does it mean that 1,000 max threads can work in parallel at the same time at the WebApi server?
Or this value is only related to ths IIS queue?
I've read that the IIS maintains some kind of threads queue, is this queue sends its work asynchronously? or all the client requests sent by the IIS to the WebApi service are being sent synchronously?
The queue size you're looking at specifies the maximum number of requests that will be queued for each application pool (which typically maps to one w3wp worker process). Once the queue length is exceeded, 503 "Server Too Busy" errors will be returned.
Within each worker process, a number of threads can/will run. Each request runs on a thread within the worker process (defaulting to a maximum of 250 threads per process, I believe).
So, essentially, each request is processed on its own thread (concurrently - at least, as concurrently as threads get) but all threads for a particular app pool are (typically) managed by a single process. This means that requests are, indeed, executed asynchronously as far as the requests themselves are concerned.
In response to your comment; if you have sessions enabled (which you probably do), then ASP.NET will queue the requests in order maintain a lock on the session for each request. Try hitting your sleeping action in Chrome and then your quick-responding action in Firefox and see what happens. You should see that the two different sessions allow your requests to be executed concurrently.
Yes, all the requests will be executed in parallel using the threads from the CLR thread pool subject to limits. About the queue size set against the app pool, this limit is for IIS to start rejecting requests with a 503 - Service unavailable status code. Even before this happens, your requests will be queued by IIS/ASP.NET. That is because threads cannot be created at will. There is a limit to number of concurrent requests that can run which is set by MaxConcurrentRequestsPerCPU and a few other parameters. For 1000 threads to execute in parallel in a true sense, you will need 1000 CPU cores. Otherwise, threads will need to be time sliced and that adds overhead to the system. Hence, there are limits to number of threads. I believe it is very difficult to comprehensively answer your questions through a single answer here. You will probably need to read up a little bit and a good place to start will be http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx.
I have an Azure web role that accesses an external WCF based SOAP web service (port 80) for various bits of data. The response from this service is highly erratic. I routinely get the following error.
There was no endpoint listening at
http://www.myexternalservice.com/service.svc that could accept the message. This is
often caused by an incorrect address or SOAP action.
To isolate the problem I created a simple console app to repetitively call this service in 1 second intervals and log all responses.
using (var svc = new MyExternalService())
{
stopwatch.Start();
var response = svc.CallService();
stopwatch.Stop();
Log(response, stopwatch.ElapsedMilliseconds);
}
If I RDP to one of my Azure web instances and run this app it takes 10 to 20 attempts before it gets a valid response from the external service. These first attempts are always accompanied by the above error. After this "warm up period" it runs fine. If I stop the app and then immediately restart, it has to go back through the same "warm up" period.
However, if I run this same app from any other machine I receive valid responses immediately. I have run this logger app on servers running in multiple data centers (non Azure), desktops on different networks, etc... These test runs are always very stable.
I am not sure why this service would react this way in the Azure environment. Unfortunately, for the short term I am forced to call this service but my users cannot tolerate this inconsistency.
A capture of network traffic on the Azure server indicates a large number of SynReTransmit's in 10 second intervals during the same time I experience the connection errors. Once the "warm up" is complete the SynReTransmit's no longer occur.
The Windows Azure data center region where the Windows Azure application is deployed might not be near the external Web Service. The local machine you're trying (which works fine) might be close to the web service. That’s why there might be huge latency in Azure which would likely cause it to fail.
Success accessing WSDL from a browser in Azure VM might be due to browser caching. Making a function call from browser would tell you if it is actually making a connection.
We found a solution for this problem although I am not completely happy with it. After exhausting all other courses of action we changed the load balancer to Layer-7 Load Balancing from Layer-4 Load Balancing. While this fixed the problem of lost requests I am not sure why this made a difference.