Develop Reference

Waiting a second between each returned response

I have a web app that connects to an external API.
That API has a limit of 3 connections per second.
I have a method that gets employee data for a whole factory.
It works fine, but I've found that if a particular factory has a lot of employees, I hit the API connection limit and get an error.
(429) API calls exceeded...maximum 3 per Second
So I decided to use await Task.Delay(1000) to set a 1 second delay, every time this method is used.
Now it seems to have reduced the number of errors I get, but I am still getting a few limit errors.
Is there another method I could use to ensure my limit is not reached?
Here is my code:
public async Task<YourSessionResponder> GetAll(Guid factoryId)
{
UserSession.AuthData sessionManager = new UserSession.AuthData
{
UserName = "xxxx",
Password = "xxxx"
};
ISessionHandler sessionMgr = new APIclient();
YourSessionResponder response;
response = await sessionMgr.GetDataAsync(sessionManager, new ListerRequest
{
FactoryId = factoryId;
await Task.Delay(1000);
return response;
}
I call it like this:
var yourEmployees = GetAll(factoryId);

I have a web app that connects to an external API.
Your current code limits the number of outgoing requests made by a single incoming request to your API. What you need to do is limit all of your outgoing requests, app-wide.
It's possible to do this using a SemaphoreSlim:
private static readonly SemaphoreSlim Mutex = new(1);
public async Task<YourSessionResponder> GetAll(Guid factoryId)
{
...
YourSessionResponder response;
await Mutex.WaitAsync();
try
{
response = await sessionMgr.GetDataAsync(...);
await Task.Delay(1000);
}
finally
{
Mutex.Release();
}
return response;
}
But I would take a different approach...
Is there another method I could use to ensure my limit is not reached?
Generally, I recommend just retrying on 429 errors, using de-correlated jittered exponential backoff (see Polly for an easy implementation). That way, when you're "under budget" for the time period, your requests go through immediately, and they only slow down when you hit your API limit.

From a comment on the question:
I am calling it like this: var yourEmployees = GetAll(factoryId);
Then you're not awaiting the task. While there's a 1-second delay after each network operation, you're still firing off all of the network operations in rapid succession. You need to await the task before moving on to the next one:
var yourEmployees = await GetAll(factoryId);
Assuming that this is happening in some kind of loop or repeated operation, of course. Otherwise, where would all of these different network tasks be coming from? Whatever high-level logic is invoking the multiple network operations, that logic needs to await one before moving on to the next.

HttpClient SendAsync blocks main thread

I have written a little winforms application that sends http requests to every ip address within my local network to discover a certain device of mine. On my particular subnet mask thats 512 addresses. I have written this using backGroundWorker but I wanted to tryout httpClient and the Async/Await pattern to achieve the same thing. The code below uses a single instance of httpClient and I wait until all the requests have completed. This issue is that the main thread gets blocked. I know this because I have a picturebox + loading gif and its not animating uniformly. I put the GetAsync method in a Task.Run as suggested here but that didn't work either.
private async void button1_Click(object sender, EventArgs e)
{
var addresses = networkUtils.generateIPRange..
await MakeMultipleHttpRequests(addresses);
}
public async Task MakeMultipleHttpRequests(IPAddress[] addresses)
{
List<Task<HttpResponseMessage>> httpTasks = new List<Task<HttpResponseMessage>>();
foreach (var address in addresses)
{
Task<HttpResponseMessage> response = MakeHttpGetRequest(address.ToString());
httpTasks.Add(response);
}
try
{
if (httpTasks.ToArray().Length != 0)
{
await Task.WhenAll(httpTasks.ToArray());
}
}
catch (Exception ex)
{
Console.WriteLine("\thttp tasks did not complete Exception : {0}", ex.Message);
}
}
private async Task<HttpResponseMessage> MakeHttpGetRequest(string address)
{
var url = string.Format("http://{0}/getStatus", address);
var cts = new System.Threading.CancellationTokenSource();
cts.CancelAfter(TimeSpan.FromSeconds(10));
HttpResponseMessage response = null;
var request = new HttpRequestMessage(HttpMethod.Get, url);
response = await httpClient.SendAsync(request, cts.Token);
return response;
}
I have read a similar issue here but my gui thread is not doing much. I have read here that I maybe running out of threads. Is this the issue, how can I resolve it?
I know its the Send Async because if I replace the code with the simple task below there is no blocking.
await Task.Run(() =>
{
Thread.Sleep(1000);
});

So one of the issues here is that you are creating 500+ tasks one after another in quick succession with a timeout set outside the task creation.
Just because you ask to run 500+ tasks, doesn't mean 500+ tasks are all going to run at the same time. They get queued up and run when the scheduler deems it's possible.
You set a timeout at the time of creation of 10 seconds. But they could sit in the scheduler for 10 seconds before they even get executed.
You want to have your Http requests to timeout organically, you can do that like this when you create the HttpClient:
private static readonly HttpClient _httpClient = new HttpClient
{
Timeout = TimeSpan.FromSeconds(10)
};
So, by moving the timeout to the HttpClient, your method should now look like this:
private static Task<HttpResponseMessage> MakeHttpGetRequest(string address)
{
return _httpClient.SendAsync(new HttpRequestMessage(HttpMethod.Get, new UriBuilder
{
Host = address,
Path = "getStatus"
}.Uri));
}
Try using that method and see if it improves your lock-up issue in Debug mode.
As far as the issue you were having: It's locking up because you are in Debug mode and the debugger is trying to say "hey, you got an exception" 500 times all at the same time because they were all spawned at the same time. Run it in Release mode and see if it still locks up.
What I would consider doing is batching out your operations. Do 20, then wait until those 20 finish, do 20 more, so on and so forth.
If you'd like to see a slick way of batching tasks, let me know and I would be more than happy to show you.

On .NET Framework, the number of connections to a server is controlled by the ServicePointManager Class.
For a client, the default connection limit is 2 on client processes.
No matter how many HttpClient.SendAsync invocations you do, only 2 will be active at the same time.
But you can manage the connections yourself.
On .NET Core here isn't the concept of service point manager and the equivalent default limit is int.MaxValue.

Should I use synchronous/asynchronous API calls when creating an API that calls 2 other APIs?

I’m creating an API that serves as the bridge between the app and 2 other APIs. I want to know if what is the best way to do this. I’m using HttpClient. The app has almost a thousand users so if I use synchronous calls does that mean that if a user calls the API, then the other users have to wait until the 1st user gets the response before it proceeds to the other API requests? Is there a better way of doing an API like this?
Here is a sample of my code using synchronous:
[HttpGet]
[Route("api/apiname")]
public String GetNumberofP([FromUri]GetNumberofPRequest getNPRequest){
var request = JsonConvert.SerializeObject(getNPRequest);
string errorMessage = "";
try{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token.gettoken());
var response = httpClient.GetAsync("api/MobileApp/GetNumberP?"
+ "strCardNumber=" + getNPRequest.strCardNumber
+ "&strDateOfBirth=" + getNPRequest.strDateOfBirth).Result;
return response;
}
catch (Exception e){
throw utils.ReturnException("GetNumberofP", e, errorMessage);
}
}

if I use synchronous calls does that mean that if a user calls the API, then the other users have to wait until the 1st user gets the response before it proceeds to the other API requests
No. When a request comes into the pipeline, a new thread is spawned by the framework. So if 1,000 requests come in at the same time, the 1,000th user will not have to wait for the other 999 requests to finish.
You are better off using async code for this anyway. For any I/O like network requests, you're usually better off for performance letting a background thread do the waiting. Side note, you never want to call .Result because that forces the async code to become blocking and effectively becomes synchronous.

t's always easy to turn a synchronous call into an asynchronous one, but the other way around is fraught with danger. You should make your API asynchronous.
[HttpGet]
[Route("api/apiname")]
public Task<string> GetNumberofP([FromUri]GetNumberofPRequest getNPRequest)
{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token.gettoken());
return httpClient.GetAsync($"api/MobileApp/GetNumberP?strCardNumber={getNPRequest.strCardNumber}&strDateOfBirth={getNPRequest.strDateOfBirth}");
}
You should also consider creating a new httpClient for each call.

It seems you're missing the async and await keywords.
public async String GetNumberofP([FromUri]GetNumberofPRequest getNPRequest){
(...)
var response = await httpClient.GetAsync();

SendAsync and CopyToAsync not working when downloading a large file

I have a small app that receives a request from a browser, copy the header received and the post data (or GET path) and send it to another endpoint.
It then waits for the result and sends it back to the browser. It works like a reverse proxy.
Everything works fine until it receives a request to download a large file. Something like a 30MB will cause an strange behaviour in the browser. When the browser reaches around 8MB it stops receiving data from my app and, after some time, it aborts the download. Everything else works just fine.
If I change the SendAsync line to use HttpCompletionOption.ResponseContentRead it works just fine. I am assuming there is something wrong waiting for the stream and/or task, but I can't figure out what is going on.
The application is written in C#, .net Core (latest version available).
Here is the code (partial)
private async Task SendHTTPResponse(HttpContext context, HttpResponseMessage responseMessage)
{
context.Response.StatusCode = (int)responseMessage.StatusCode;
foreach (var header in responseMessage.Headers)
{
context.Response.Headers[header.Key] = header.Value.ToArray();
}
foreach (var header in responseMessage.Content.Headers)
{
context.Response.Headers[header.Key] = header.Value.ToArray();
}
context.Response.Headers.Remove("transfer-encoding");
using (var responseStream = await responseMessage.Content.ReadAsStreamAsync())
{
await responseStream.CopyToAsync(context.Response.Body);
}
}
public async Task ForwardRequestAsync(string toHost, HttpContext context)
{
var requestMessage = this.BuildHTTPRequestMessage(context);
var responseMessage = await _httpClient.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, context.RequestAborted);
await this.SendHTTPResponse(context, responseMessage);
}
EDIT
Changed the SendHTTPResponse to wait for responseMessage.Content.ReadAsStreamAsync using await operator.

Just a guess but I believe the issue lies with the removal of the transfer encoding:
context.Response.Headers.Remove("transfer-encoding");
If the http request you are making with _httpClient returns the 30MB file using Chunked encoding (target server doesn't know the file size) then you would need to return the file to the browser with Chunked encoding as well.
When you buffer the response on your webservice (by passing HttpCompletionOption.ResponseContentRead) you know the exact message size you are sending back to the browser so the response works successfully.
I would check the response headers you get from responseMessage to see if the transfer encoding is chunked.

You are trying to stream a file but you are doing it not exactly right. If you do not specify,ResponseHeadersRead, response will never come back unless the server ends the request because it will try to read the response till the end.
HttpCompletionOption enumeration type has two members and one of them is ResponseHeadersRead which tells the HttpClient to only read the headers and then return back the result immediately.
var response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var stream = await response.Content.ReadAsStreamAsync();
using (var reader = new StreamReader(stream)) {
while (!reader.EndOfStream) {
//Oh baby we are streaming
//Do stuff copy to response stream etc..
}
}

Figure 3 shows a simple example where one method blocks on the result of an async method. This code will work just fine in a console application but will deadlock when called from a GUI or ASP.NET context. This behavior can be confusing, especially considering that stepping through the debugger implies that it’s the await that never completes. The actual cause of the deadlock is further up the call stack when Task.Wait is called.
Figure 3 A Common Deadlock Problem When Blocking on Async Code
public static class DeadlockDemo
{
private static async Task DelayAsync()
{
await Task.Delay(1000);
}
// This method causes a deadlock when called in a GUI or ASP.NET context.
public static void Test()
{
// Start the delay.
var delayTask = DelayAsync();
// Wait for the delay to complete.
delayTask.Wait();
}
}
The root cause of this deadlock is due to the way await handles contexts. By default, when an incomplete Task is awaited, the current “context” is captured and used to resume the method when the Task completes. This “context” is the current SynchronizationContext unless it’s null, in which case it’s the current TaskScheduler. GUI and ASP.NET applications have a SynchronizationContext that permits only one chunk of code to run at a time. When the await completes, it attempts to execute the remainder of the async method within the captured context. But that context already has a thread in it, which is (synchronously) waiting for the async method to complete. They’re each waiting for the other, causing a deadlock.
Note that console applications don’t cause this deadlock. They have a thread pool SynchronizationContext instead of a one-chunk-at-a-time SynchronizationContext, so when the await completes, it schedules the remainder of the async method on a thread pool thread. The method is able to complete, which completes its returned task, and there’s no deadlock. This difference in behavior can be confusing when programmers write a test console program, observe the partially async code work as expected, and then move the same code into a GUI or ASP.NET application, where it deadlocks.
The best solution to this problem is to allow async code to grow naturally through the codebase. If you follow this solution, you’ll see async code expand to its entry point, usually an event handler or controller action. Console applications can’t follow this solution fully because the Main method can’t be async. If the Main method were async, it could return before it completed, causing the program to end. Figure 4 demonstrates this exception to the guideline: The Main method for a console application is one of the few situations where code may block on an asynchronous method.
Figure 4 The Main Method May Call Task.Wait or Task.Result
class Program
{
static void Main()
{
MainAsync().Wait();
}
static async Task MainAsync()
{
try
{
// Asynchronous implementation.
await Task.Delay(1000);
}
catch (Exception ex)
{
// Handle exceptions.
}
}
}
LEARN MORE HERE

try these.
using (HttpResponseMessage responseMessage= await client.SendAsync(request))
{
await this.SendHTTPResponse(context, responseMessage);
}
or
using (HttpResponseMessage responseMessage=await _httpClient.SendAsync(requestMessage,
HttpCompletionOption.ResponseHeadersRead, context.RequestAborted))
{
await this.SendHTTPResponse(context, responseMessage)
}

How can I test to make sure this code is really running async?

I have adapted the following code to attempt to load from a web api from the asp.net code by Wasson.
static public async Task<IEnumerable<T>> ExecuteAsync(HttpClient client, String endPoint)
{
IEnumerable<T> result = null;
HttpResponseMessage response = client.GetAsync(endPoint).Result;
response.EnsureSuccessStatusCode();
if (response.IsSuccessStatusCode)
{
var tResult = await response.Content.ReadAsStringAsync();
result = JsonConvert.DeserializeObject<IEnumerable<T>>(tResult);
}
return result;
}
As I have it designed, I think this procedure will run asynchronously, but I am not sure how to test it to make sure it does.
I am new to using asynchronous coding, so any suggestions would be helpful.
The calling procedure is as follows:
public virtual IEnumerable<T> Fill()
{
IEnumerable<T> result = null;
try
{
using (var client = CreateClient("", new MediaTypeWithQualityHeaderValue("application/json")))
{
String _endPoint = "api/" + typeof(T).Name + "/Get";
result = (ExecuteAsync(client, _endPoint)).Result;
}
}
catch (Exception ex)
{
LogError(ex.Message);
}
return result;
}
Is this combination likely to run asynchronously as I want it to?
The reason I want to be able to run it asynchronously is that the Fill routines fetch large amounts of data from a database. It is more a case of a large number of individual records rather than a large size for each record, although the size of some of the records may also be an issue.
What I don't want is the user sitting there waiting for the entire page to load while the communication with the database is occurring.
It doesn't really matter in this case whether I am directly connecting to the database or going through the web api I am using: It takes a long time in some cases either way.
I am also concerned about deadlocks: One large fetch coming in while another is pending.
I initially patterned the code after Mike Wasson's sample code, but found that the code hanged indefinitely even though it compiled.
Does this help frame the situation a little better?

Not sure what you're after here mate.
The code you posted will still block the thread it's on since you're calling Result which will block.
result = (ExecuteAsync(client, _endPoint)).Result;
You could make it asynchronous by letting the async and await keywords spread back to whatever method initially created the request.
public virtual async Task<IEnumerable<T>> Fill()
{
...
result = await ExecuteAsync(client, _endPoint);
...
}
However I'm not sure this really is what you're after since you mention
What I don't want is the user sitting there waiting for the entire page to load while the communication with the database is occurring.
By default, making things asynchronous will not solve this for you. An operation that takes 15 seconds to complete will take 15 seconds to complete regardless of it being synchronous or asynchronous. The difference is that if it's asynchronous then you're being more resource effective (meaning, you'll free up the thread to preform other duties while waiting for something to complete).
However, you can use tasks to achive parallelity. Take for example:
public void DoWork()
{
Thread.Sleep(10000);
}
public void DoMoreWork()
{
Thread.Sleep(10000);
}
If we called these two like this
DoWork();
DoMoreWork();
Then that would take 20000 ms or 20 seconds
And if we make it asynchronous
public async Task DoWork()
{
await Task.Delay(10000);
}
public async Task DoMoreWork()
{
await Task.Delay(10000);
}
and call it like this
await DoWork();
await DoMoreWork();
Then it will STILL take 20 seconds to complete. However, since DoWork and DoMoreWork can operate independent from eachothers we could run them both at the same time by doing
await Task.WhenAll(DoWork(), DoMoreWork());
This would (in a perfect world) result in both tasks being done in 10 seconds halving the time the user has to wait.
Hope I shed some light on the issue :)