I have ASP.NET Web API app (backend for mobile) published on Azure. Most of requests are lightweight and processed fast. But every client makes a lot of requests and do it rapidly on every interaction with mobile application.
The problem is that web application can't process even small (10/sec) amount of requests. Http queue growth but CPU doesn't.
I ran load testing with 250 requests/second and avg response time growth from ~200ms to 5s.
Maybe problem in my code? Or it's hardware restrictions? Can I increase count of processed requests at one time?
First it really matters what instances do you use (specially if you use small and extra small instances), how many instances do you use - dont expect too much from 1 core and 2Gb RAM on server.
Use caching (WebApi.OutputCache.V2 to decrease servers processing efforts, Azure Redis Cache as fast cache storage), Database also can be a bottleneck.
If you`ll have same results after adding both more instances to server and caching - then you should take a look at your code and find bottlenecks there.
And thats only general recommendatins, there is no code in a question.
Related
I have a social-style web-app, and I want to add "live" updates to votes under posts.
At first I thought that a client-side loop with /GET polling the server every few seconds would be an inferior choice, and that all cool kids use websockets or server sent events.
But now I found that websocket would be limited at 65k live connections (even less in practice).
Is there a way to push vote updates to a big number of users realtime?
The app has around ~2 million daily users, so I'd expect 200-300k simultaneous socket connections.
The stack is ASP.NET Core backend hosted on an Ubuntu machine with nginx reverse proxy.
At current state all load is easily handled by a single machine, and I don't really want to add multiple instances just to be able to work with signalR.
May be there is a simple solution that I'm missing out?
We have several OData API's using Entity Framework and AutoMapper. These connect to an on-premise SQL database through a VNet. The GET requests of this API are not asynchronous per example found here. The scaling is set to S2. We have enabled always on.
Sometimes the requests complete in 500 ms. Sometimes the very same requests take 40 seconds. We have tried scaling out but this offers no tangible benefit. We have tried making the GET function on the controllers async. We have tried disabling authentication. We have tried looking at the application insights call stack in the profiler but sometimes the code hangs on one call, while other times on another. We even found a 39 second call to String.Replace(). We've tried Kudu but can't seem to get any knowledge from it.
On top of this I alone succeed in bringing the server to its knees simply by spamming F5 on a relatively simple request, locking the CPU at 100%. S2 seems pretty high already, and we are stunned that the server apparently cannot handle it. And it's also not always the case that low CPU usage on the server equals fast requests. Sometimes these requests also take an extraordinary amount of time.
We have tried looking at the application insights data but grow even more confused as some data suggests one thing is at fault while other data suggests it is not.
CPU usage on the app service plan is high.
CPU usage in the live metrics usually remains low.
This suggests that SQL is at fault. But we have almost ruled that out since if we spam an API on one app service plan and send the same single request to another app service plan we get the result immediately.
This suggests that the code or server is at fault.
How can we diagnose this issue and find the bottleneck?
I do have my OWIN application hosted as a Windows Service and I am getting a lot of timeout issues from different clients. I have some metrics in place around the request/response time however the numbers are very different. For example I can see the client is taking around one minute to perform a request that looks like in the server is taking 3-4 seconds. I am then assuming that the number of requests that can be accepted has reached the limit and subsequent requests that come in would get queued up. Am I right? If that's the case, is there any way I can monitor the number of incoming requests at a given time and how big is the queue (as in number of requests pending to get served)?
I am playing around with https://msdn.microsoft.com/en-us/library/microsoft.owin.host.httplistener.owinhttplistener.setrequestprocessinglimits(v=vs.113).aspx but doesn't look to have any effect.
Any feedback is much appreciated.
THanks!
HttpListener is built on top of Http.Sys so you need to use its performance counters and ETW traces to get this level of information.
https://msdn.microsoft.com/en-us/library/windows/desktop/cc307239%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
http://blogs.msdn.com/b/wndp/archive/2007/01/18/event-tracing-in-http-sys-part-1-capturing-a-trace.aspx
My cloud service writes data to two azure table storages in two separate data centers. When I test the service with larger load the table storage in the same data center as the service keeps it's latency (around 10ms), but the operations to the other data center (I think it's East US 2 from West Europe) are getting slower and slower until they take around 5 seconds.
I checked the storage monitoring and there are no throttling errors, so I am pretty sure I am not breaking the max number of requests. I can't see any network errors or anything like that for the storage, so I guess the delay must be result of something else.
My cloud service is not cpu intensive. I tried to set no retry policy, but there are no errors, those table operations are successful, they just take a lot of time. Each request has a different partition key. The cloud service is Asp.NET Web Api running as a worker role, each table operation is run async.
What can be the issue here? Is Azure throttling requests between datacenters?
Requests do not get throttled so long as your application is within the storage performance and scalability target (http://azure.microsoft.com/en-us/documentation/articles/storage-scalability-targets/).
Please see troubleshooting guidance section under Monitor, diagnose, and troubleshoot Microsoft Azure Storage MSDN article (http://azure.microsoft.com/en-us/documentation/articles/storage-monitoring-diagnosing-troubleshooting/#troubleshooting-guidance) to understand the source of high latency.
By default the http connection limit is set to two. Increasing this limit means more than two requests will be processed at the same time, which will increase the performance. http://tk.azurewebsites.net/2012/12/10/greatly-increase-the-performance-of-azure-storage-cloudblobclient/
I've had a fairly good search on google and nothing has popped up to answer my question. As I know very little about web services (only started using them, not building them in the last couple of months) I was wondering whether I should be ok to call a particular web service as frequently as I wish (within reason), or should I build up requests to do in one go.
To give you an example, my app is designed to make job updates, which for certain types of updates will call the web service. It seems like my options are that I could create a datatable in my app of updates that require the web service and pass the whole datatable to the web service and then write a method in the web service to process the datatable's updates. Alternatively I could iterate through my entire table of updates (which includes other updates than those requiring the web service) and call the web service as when an update requires it.
At the moment it seems like it would be simpler for me to pass each update rather than a datatable to the web service.
In terms of data being passed to the web service each update would contain a small amount of data (3 strings, max 120 characters in length). In terms of numbers of updates there would probably be no more than 200.
I was wondering whether I should be ok to call a particular web service as frequently as I wish (within reason), or should I build up requests to do in one go.
Web services or not, any calls routed over the network would benefit from building up multiple requests, so that they could be processed in a single round-trip. In your case, building an object representing all the updates is going to be a clear winner, especially in setups with slower connections.
When you make a call over the network, these things need to happen when a client communicates to a server (again, web services or not):
The data associated with your call gets serialized on the client
Serialized data is sent to the server
Server deserializes the data
Server processes the data, producing a response
Server serializes the response
Server sends serialized response back to the client
The response is deserialized on the client
Steps 2 and 6 usually cause a delay due to network latency. For simple operations, latency often dominates the timing of the call.
The latency on fastest networks used for high-frequency trading is in microseconds; on regular ones it is in milliseconds. If you are sending 100 packages one by one on a network with 1ms lag (2ms per roundtrip), you are wasting 200ms just on the network latency! This one fifth of a second, a lot of time by the standards of today's CPUs. If you can eliminate it simply by restructuring your requests, it's a great reason to do it.
You should usually favor coarse-grained remote interfaces over a fine-grained ones.
Consider adding a 10ms network latency to each call - what would be the delay for 100 updates?