I have a web app that connects to an external API.
That API has a limit of 3 connections per second.
I have a method that gets employee data for a whole factory.
It works fine, but I've found that if a particular factory has a lot of employees, I hit the API connection limit and get an error.
(429) API calls exceeded...maximum 3 per Second
So I decided to use await Task.Delay(1000) to set a 1 second delay, every time this method is used.
Now it seems to have reduced the number of errors I get, but I am still getting a few limit errors.
Is there another method I could use to ensure my limit is not reached?
Here is my code:
public async Task<YourSessionResponder> GetAll(Guid factoryId)
{
UserSession.AuthData sessionManager = new UserSession.AuthData
{
UserName = "xxxx",
Password = "xxxx"
};
ISessionHandler sessionMgr = new APIclient();
YourSessionResponder response;
response = await sessionMgr.GetDataAsync(sessionManager, new ListerRequest
{
FactoryId = factoryId;
await Task.Delay(1000);
return response;
}
I call it like this:
var yourEmployees = GetAll(factoryId);
I have a web app that connects to an external API.
Your current code limits the number of outgoing requests made by a single incoming request to your API. What you need to do is limit all of your outgoing requests, app-wide.
It's possible to do this using a SemaphoreSlim:
private static readonly SemaphoreSlim Mutex = new(1);
public async Task<YourSessionResponder> GetAll(Guid factoryId)
{
...
YourSessionResponder response;
await Mutex.WaitAsync();
try
{
response = await sessionMgr.GetDataAsync(...);
await Task.Delay(1000);
}
finally
{
Mutex.Release();
}
return response;
}
But I would take a different approach...
Is there another method I could use to ensure my limit is not reached?
Generally, I recommend just retrying on 429 errors, using de-correlated jittered exponential backoff (see Polly for an easy implementation). That way, when you're "under budget" for the time period, your requests go through immediately, and they only slow down when you hit your API limit.
From a comment on the question:
I am calling it like this: var yourEmployees = GetAll(factoryId);
Then you're not awaiting the task. While there's a 1-second delay after each network operation, you're still firing off all of the network operations in rapid succession. You need to await the task before moving on to the next one:
var yourEmployees = await GetAll(factoryId);
Assuming that this is happening in some kind of loop or repeated operation, of course. Otherwise, where would all of these different network tasks be coming from? Whatever high-level logic is invoking the multiple network operations, that logic needs to await one before moving on to the next.
Related
I have written a little winforms application that sends http requests to every ip address within my local network to discover a certain device of mine. On my particular subnet mask thats 512 addresses. I have written this using backGroundWorker but I wanted to tryout httpClient and the Async/Await pattern to achieve the same thing. The code below uses a single instance of httpClient and I wait until all the requests have completed. This issue is that the main thread gets blocked. I know this because I have a picturebox + loading gif and its not animating uniformly. I put the GetAsync method in a Task.Run as suggested here but that didn't work either.
private async void button1_Click(object sender, EventArgs e)
{
var addresses = networkUtils.generateIPRange..
await MakeMultipleHttpRequests(addresses);
}
public async Task MakeMultipleHttpRequests(IPAddress[] addresses)
{
List<Task<HttpResponseMessage>> httpTasks = new List<Task<HttpResponseMessage>>();
foreach (var address in addresses)
{
Task<HttpResponseMessage> response = MakeHttpGetRequest(address.ToString());
httpTasks.Add(response);
}
try
{
if (httpTasks.ToArray().Length != 0)
{
await Task.WhenAll(httpTasks.ToArray());
}
}
catch (Exception ex)
{
Console.WriteLine("\thttp tasks did not complete Exception : {0}", ex.Message);
}
}
private async Task<HttpResponseMessage> MakeHttpGetRequest(string address)
{
var url = string.Format("http://{0}/getStatus", address);
var cts = new System.Threading.CancellationTokenSource();
cts.CancelAfter(TimeSpan.FromSeconds(10));
HttpResponseMessage response = null;
var request = new HttpRequestMessage(HttpMethod.Get, url);
response = await httpClient.SendAsync(request, cts.Token);
return response;
}
I have read a similar issue here but my gui thread is not doing much. I have read here that I maybe running out of threads. Is this the issue, how can I resolve it?
I know its the Send Async because if I replace the code with the simple task below there is no blocking.
await Task.Run(() =>
{
Thread.Sleep(1000);
});
So one of the issues here is that you are creating 500+ tasks one after another in quick succession with a timeout set outside the task creation.
Just because you ask to run 500+ tasks, doesn't mean 500+ tasks are all going to run at the same time. They get queued up and run when the scheduler deems it's possible.
You set a timeout at the time of creation of 10 seconds. But they could sit in the scheduler for 10 seconds before they even get executed.
You want to have your Http requests to timeout organically, you can do that like this when you create the HttpClient:
private static readonly HttpClient _httpClient = new HttpClient
{
Timeout = TimeSpan.FromSeconds(10)
};
So, by moving the timeout to the HttpClient, your method should now look like this:
private static Task<HttpResponseMessage> MakeHttpGetRequest(string address)
{
return _httpClient.SendAsync(new HttpRequestMessage(HttpMethod.Get, new UriBuilder
{
Host = address,
Path = "getStatus"
}.Uri));
}
Try using that method and see if it improves your lock-up issue in Debug mode.
As far as the issue you were having: It's locking up because you are in Debug mode and the debugger is trying to say "hey, you got an exception" 500 times all at the same time because they were all spawned at the same time. Run it in Release mode and see if it still locks up.
What I would consider doing is batching out your operations. Do 20, then wait until those 20 finish, do 20 more, so on and so forth.
If you'd like to see a slick way of batching tasks, let me know and I would be more than happy to show you.
On .NET Framework, the number of connections to a server is controlled by the ServicePointManager Class.
For a client, the default connection limit is 2 on client processes.
No matter how many HttpClient.SendAsync invocations you do, only 2 will be active at the same time.
But you can manage the connections yourself.
On .NET Core here isn't the concept of service point manager and the equivalent default limit is int.MaxValue.
using: Asp.net Core, Entityframework Core, ABP 4.5
I have a user registration and initialization flow. But it takes a long time. I want to parallelize this. This is due to updating from the same entity, but with a different field.
My goal:
1. The endpoint should respond as soon as possible;
2. Long initialization is processed in the background;
Code-before (minor details omitted for brevity)
public async Task<ResponceDto> Rgistration(RegModel input)
{
var user = await _userRegistrationManager.RegisterAsync(input.EmailAddress, input.Password, false );
var result = await _userManager.AddToRoleAsync(user, defaultRoleName);
user.Code = GenerateCode();
await SendEmail(user.EmailAddress, user.Code);
await AddSubEntities(user);
await AddSubCollectionEntities(user);
await CurrentUnitOfWork.SaveChangesAsync();
return user.MapTo<ResponceDto>();
}
private async Task AddSubEntities(User user)
{
var newSubEntity = new newSubEntity { User = user, UserId = user.Id };
await _subEntityRepo.InsertAsync(newSubEntity);
//few another One-to-One entities...
}
private async Task AddSubEntities(User user)
{
List<AnotherEntity> collection = GetSomeCollection(user.Type);
await _anotherEntitieRepo.GetDbContext().AddRangeAsync(collection);
//few another One-to-Many collections...
}
Try change:
public async Task<ResponceDto> Rgistration(RegModel input)
{
var user = await _userRegistrationManager.RegisterAsync(input.EmailAddress, input.Password, false );
Task.Run(async () => {
var result = await _userManager.AddToRoleAsync(user, defaultRoleName);
});
Task.Run(async () => {
user.Code = GenerateCode();
await SendEmail(user.EmailAddress, user.Code);
});
Task.Run(async () => {
using (var unitOfWork = UnitOfWorkManager.Begin())
{//long operation. defalt unitOfWork out of scope
try
{
await AddSubEntities(user);
}
finally
{
unitOfWork.Complete();
}
}
});
Task.Run(async () => {
using (var unitOfWork = UnitOfWorkManager.Begin())
{
try
{
await AddSubCollectionEntities(user);
}
finally
{
unitOfWork.Complete();
}
}
});
await CurrentUnitOfWork.SaveChangesAsync();
return user.MapTo<ResponceDto>();
}
Errors:
here I get a lot of different errors related to competition. frequent:
A second operation started on this context before a previous operation completed. This is usually caused by different threads using the same instance of DbContext.
In few registratin calls: Cannot insert duplicate key row in object 'XXX' with unique index 'YYY'. The duplicate key value is (70). The statement has been terminated.
I thought on the server every request in its stream, but apparently not.
or all users are successfully registered, but they don’t have some sub-entity in the database. it’s much easier not to register the user than to figure out where he was initialized incorrectly =(
how to keep the user entity “open” for updating and at the same time “closed” for changes initiated by other requests? How to make this code thread safe and fast, can anyone help with advice?
Using Task.Run in ASP.NET is rarely a good idea.
Async methods run on the thread pool anyway, so wrapping them in Task.Run is simply adding overhead without any benefit.
The purpose of using async in ASP.NET is simply to prevent threads being blocked so they are able to serve other HTTP requests.
Ultimately, your database is the bottleneck; if all these operations need to happen before you return a response to the client, then there's not much you can do other than to let them happen.
If it is possible to return early and allow some operations to continue running on the background, then there are details here showing how that can be done.
Task.Run is not the same as parallel. It takes a new thread from the pool and runs the work on that thread, and since you're not awaiting it, the rest of the code can move on. However, that's because you're essentially orphaning that thread. When the action returns, all the scoped services will be disposed, which includes things like your context. Any threads that haven't finished, yet, will error out as a result.
The thread pool is a limited resource, and within the context of a web application, it equates directly to the throughput of your server. Every thread you take is one less request you can service. As a result, you're more likely to end up queuing requests, which will only add to processing time. It's virtually never appropriate to use Task.Run in a web environment.
Also, EF Core (or old EF, for that matter) does not support parallelization. So, even without the other problems described above, it will stop you cold from doing what you're trying to do here, regardless.
The queries you have here are not complex. Even if you were trying to insert 100s of things at once, it should still take only milliseconds to complete. If there is any significant delay here, you need to look at the resources of your database server and your network latency, first.
More likely than not, the slow-down is coming from the sending of the email. That too can likely be optimized, though. I was in a situation once where it was taking emails 30 seconds to send, until I finally figured out that it was an issue with our Exchange server, where an IT admin had idiotically introduced a 30 second delay on purpose. Regardless, it is generally always preferable to background things like sending emails, since they aren't core to your app's functionality. However, that means actually processing them in background, i.e. queue them and process them via something like a hosted service or an entirely different worker process.
General Information
I want to improve the performance of a program issuing multiple HTTP requests to the same external API endpoint. Therefore, I have created a console application to perform some tests. The method GetPostAsync sends an asynchronous HTTP request to the external API and returns the result as a string.
private static async Task<string> GetPostAsync(int id)
{
var client = new HttpClient();
var response = await client.GetAsync($"https://jsonplaceholder.typicode.com/posts/{id}");
return await response.Content.ReadAsStringAsync();
}
Additionally, I have implemented the methods below to compare the execution time of multiple calls to await and Task.WhenAll.
private static async Task TaskWhenAll(IEnumerable<int> postIds)
{
var tasks = postIds.Select(GetPostAsync);
await Task.WhenAll(tasks);
}
private static async Task MultipleAwait(IEnumerable<int> postIds)
{
foreach (var postId in postIds)
{
await GetPostAsync(postId);
}
}
Test Results
Using the integrated Stopwatch class, I have measured the timings of the two methods and interestingly enough, the approach using Task.WhenAll performed way better than its counterpart:
Issue 50 HTTP requests
TaskWhenAll: ~650ms
MultipleAwait: ~4500ms
Why is the method using Task.WhenAll so much faster and are there any negative effects (i.e exception handling) when choosing this approach over the other?
Why is the method using Task.WhenAll so much faster
It is faster because you are not awaiting GetPostAsync. So actually every time you await client.GetAsync($"https://jsonplaceholder.typicode.com/posts/{id}"); the control will be returned to the caller which then can make another HTTP request. If you consider that HTTP request is much longer than creating the new client you effectively have the parallelism by running multiple HTTP requests in parallel. The WhenAll will just create a suspension point and wait for all tasks to finish.
With the multiple await approach, you make HTTP requests sequentially one by one by await GetPostAsync(postId) from foreach loop. You start the task but at the same time, you make a suspension point and wait for it to finish.
are there any negative effects (i.e exception handling, etc.) when
choosing this approach over the other?
There are no negative effects, using await/async pattern handling exception become just, as usual, using try-catch block. WhenAll will aggregate all exception from each task which is in Faulted state.
I’m creating an API that serves as the bridge between the app and 2 other APIs. I want to know if what is the best way to do this. I’m using HttpClient. The app has almost a thousand users so if I use synchronous calls does that mean that if a user calls the API, then the other users have to wait until the 1st user gets the response before it proceeds to the other API requests? Is there a better way of doing an API like this?
Here is a sample of my code using synchronous:
[HttpGet]
[Route("api/apiname")]
public String GetNumberofP([FromUri]GetNumberofPRequest getNPRequest){
var request = JsonConvert.SerializeObject(getNPRequest);
string errorMessage = "";
try{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token.gettoken());
var response = httpClient.GetAsync("api/MobileApp/GetNumberP?"
+ "strCardNumber=" + getNPRequest.strCardNumber
+ "&strDateOfBirth=" + getNPRequest.strDateOfBirth).Result;
return response;
}
catch (Exception e){
throw utils.ReturnException("GetNumberofP", e, errorMessage);
}
}
if I use synchronous calls does that mean that if a user calls the API, then the other users have to wait until the 1st user gets the response before it proceeds to the other API requests
No. When a request comes into the pipeline, a new thread is spawned by the framework. So if 1,000 requests come in at the same time, the 1,000th user will not have to wait for the other 999 requests to finish.
You are better off using async code for this anyway. For any I/O like network requests, you're usually better off for performance letting a background thread do the waiting. Side note, you never want to call .Result because that forces the async code to become blocking and effectively becomes synchronous.
t's always easy to turn a synchronous call into an asynchronous one, but the other way around is fraught with danger. You should make your API asynchronous.
[HttpGet]
[Route("api/apiname")]
public Task<string> GetNumberofP([FromUri]GetNumberofPRequest getNPRequest)
{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token.gettoken());
return httpClient.GetAsync($"api/MobileApp/GetNumberP?strCardNumber={getNPRequest.strCardNumber}&strDateOfBirth={getNPRequest.strDateOfBirth}");
}
You should also consider creating a new httpClient for each call.
It seems you're missing the async and await keywords.
public async String GetNumberofP([FromUri]GetNumberofPRequest getNPRequest){
(...)
var response = await httpClient.GetAsync();
We are working on a project developed in UWP(frontend) and REST-MVC-IIS(backend).
I was thinking on a theoretical scenario which might ensue:
From what I know, there is no way to guarantee the order in which requests will be processed and served by IIS.
So in a simple scenario, let's just assume this:
UI:
SelectionChanged(productId=1);
SelectionChanged(productId=2);
private async void SelectionChanged(int productId)
{
await GetProductDataAsync(productId);
}
IIS:
GetProductDataAsync(productId=1) scheduled on thread pool
GetProductDataAsync(productId=2) scheduled on thread pool
GetProductDataAsync(productId=2) finishes first => send response to client
GetProductDataAsync(productId=1) finishes later => send response to client
As you can see, the request for productId=2 for whatever reason finished faster then the first request for productId=1.
Because the way async works, both calls will create two continuation tasks on the UI which will override each other if they don't come in the correct order since they contain the same data.
This can be extrapolated to almost any master-detail scenario, where it can happen to end up selecting a master item and getting the wrong details for it (because of the order in which the response comes back from IIS).
What I wanted to know is if there are some best practice to handle this kind of scenarios... lot's of solutions come to mind but I don't want to jump the gun and go for one implementation before I try to see what other options are on the table.
As you presented your code await GetProductDataAsync(productId=2); will always run after await GetProductDataAsync(productId=1); has completed. So, there is no race condition.
If your code was:
await Task.WhenAll(
GetProductDataAsync(productId=1),
GetProductDataAsync(productId=2))
Then there might be a race condition. And, if that's a problem, it's not particular to async-await but due to the fact that you are making concurrent calls.
If you wrap that code in another method and use ConfigureAwait(), you'll have only one continuation on the UI thread:
Task GetProductDataAsync()
{
await Task.WhenAll(
GetProductDataAsync(productId=1).ConfigureAwait(),
GetProductDataAsync(productId=2).ConfigureAwait()
).ConfigureAwait();
}
I think I get what you're saying. Because of the async void eventhandler, nothing in the UI is awaiting the first call before the second. I am imagining a drop down of values and when it changes, it fetches the pertinent data.
Ideally, you would probably want to either lock out the UI during the call or implement a cancellationtoken.
If you're just looking for a way to meter the calls, keep reading...
I use a singleton repository layer in the UWP application that handles whether or not to fetch the data from a web service, or a locally cached copy. Additionally, if you want to meter the requests to process one at a time, use SemaphoreSlim. It works like lock, but for async operations (oversimplified simile).
Here is an example that should illustrate how it works...
public class ProductRepository : IProductRepository
{
//initializing (1,1) will allow only 1 use of the object
static SemaphoreSlim semaphoreLock = new SemaphoreSlim(1, 1);
public async Task<IProductData> GetProductDataByIdAsync(int productId)
{
try
{
//if semaphore is in use, subsequent requests will wait here
await semaphoreLock.WaitAsync();
try
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("yourbaseurl");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
string url = "yourendpoint";
HttpResponseMessage response = await client.GetAsync(url);
if (response.IsSuccessStatusCode)
{
var json = await response.Content.ReadAsStringAsync();
ProductData prodData = JsonConvert.DeserializeObject<ProductData>(json);
return prodData;
}
else
{
//handle non-success
}
}
}
catch (Exception e)
{
//handle exception
}
}
finally
{
//if any requests queued up, the next one will fire here
semaphoreLock.Release();
}
}
}