In the following code I receive 200 in the first request and 500 for all after that. I am new to .Net Core C# so I learned I should assign certain HttpClient attributes only once i.e. timeout.
However, my automated tests are successful first time but 500 after that. I am using HttpClientFactory.
if (httpClient.Timeout == null)
{
httpClient.Timeout = TimeSpan.FromSeconds(Foo.timeout);
}
if (httpClient.BaseAddress == null) {
httpClient.BaseAddress = new Uri(Foo.url);
}
response = await httpClient.PostAsync("", content);
I found the issue with the code. I should have created a new instance of the HttpClient using the HttpClientFactory for every Http request:
_httpClient = _httpClientFactory.CreateClient(command.ClientId);
public class Foo
{
private readonly IHttpClientFactory _httpClientFactory;
public Foo(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<HttpRequestMessage> Bar()
{
var httpClient = _httpClientFactory.CreateClient("Foo");
using var response = await httpClient.PostAsync("example.com", new StringContent("Foo"));
return response.RequestMessage;
// here as there is no more reference to the _httpclient, the garbage
//collector will clean
// up the _httpclient and release that instance. Next time the method is
//called a new
// instance of the _httpclient is created
}
}
Related
this is a problem that I have only when trying to write a test.
In a Net 5 solution, I test GetResult() in MyController:
public class MyController : ControllerBase
{
private readonly IHttpClientFactory _httpClientFactory;
private readonly IConfiguration _config;
public MyController(IHttpClientFactory httpClientFactory, IConfiguration config)
{
_httpClientFactory = httpClientFactory;
_config = config;
}
public async Task<IActionResult> GetResult(int id)
{
var firstResult = await GetFirstResult(id);
var secondResult = await GetSecondResult(id);
return Ok("")
}
private async Task<int> GetFirstResult(int id)
{
using (var httpClient = _httpClientFactory.CreateClient("MySetting"))
{
var response = await httpClient.GetAsync($"MyUrl1{id}");
return (response.IsSuccessStatusCode ? 0 : 1);
}
}
private async Task<int> GetSecondResult(int id)
{
using (var httpClient = _httpClientFactory.CreateClient("MySetting"))
{
var response = await httpClient.GetAsync($"MyUrl2{id}");
return (response.IsSuccessStatusCode ? 0 : 1);
}
}
My test:
[Test]
public async Task Get_Should_Return_OK_String()
{
var httpClientFactory = new Mock<IHttpClientFactory>();
var client = new HttpClient();
client.BaseAddress = new Uri("https://sthing/server.php");
httpClientFactory.Setup(_ => _.CreateClient(It.IsAny<string>))).Returns(client);
var config = InitConfiguration();
var controller = new MyController(httpClientFactory.Object, config);
var result = await controller.GetResult(1);
Assert.NotNull(result);
}
An exception is thrown in GetSecondResult() at the line of return(response...).
Message: "Cannot access a disposed object.
Object name: 'System.Net.Http.HttpClient'."
I am aware of this case
Why is this HttpClient usage giving me an "Cannot access a disposed object." error?
but there is no use of httpClientFactory.
Is there a way to pass false to the constructor of the client through the factory?
Why would it work out of the test anyway?
First of all, .NET 5 goes out of support today. You should migrate to .NET 6, the current Long-Term-Support version. This isn't a sudden change, .NET Core's lifecycle was announced several years ago. For the most part, all you need to do is change net5.0 to net6.0 in your projects and update NuGet packages.
As for the error, it's caused by the using block :
using (var httpClient = _httpClientFactory.CreateClient("MySetting"))
This isn't needed (it's actually discouraged), HttpClient instances and sockets are pooled and recycled by the HttpClientFactory.
The test code is configured to return the same HttpClient instance every time :
var client = new HttpClient();
client.BaseAddress = new Uri("https://sthing/server.php");
httpClientFactory.Setup(_ => _.CreateClient(It.IsAny<string>)))
.Returns(client);
The call to GetFirstResult() disposes this instance and any subsequent use throws
To fix this just don't use using. HttpClient is meant to be reused anyway:
var httpClient = _httpClientFactory.CreateClient("MySetting");
I am writing a simple unit test for this small service that simply calls external APIs:
public class ApiCaller : IApiCaller
{
private readonly IHttpClientFactory _httpFactory;
public ApiCaller(IHttpClientFactory httpFactory)
{
_httpFactory = httpFactory;
}
public async Task<T> GetResponseAsync<T>(Uri url)
{
using (HttpClient client = _httpFactory.CreateClient())
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.Timeout = TimeSpan.FromSeconds(20);
using (HttpResponseMessage response = await client.GetAsync(url))
{
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(responseBody);
}
}
}
}
My first question is: it doesn't seem to be very common practice mocking and therefore testing such services and I am wondering if there is some specific explanation.
Second, I tried to write a simple unit test but I cannot Mock the GetAsync call since HttpClient doesn't implement any interface.
public class ApiCallerTest
{
private readonly ApiCaller _target;
private readonly Mock<IHttpClientFactory> _httpClientFactory;
public ApiCallerTest()
{
_httpClientFactory = new Mock<IHttpClientFactory>();
_target = new ApiCaller(_httpClientFactory.Object);
}
[Fact]
public void WhenACorrectUrlIsProvided_ServiceShouldReturn()
{
var client = new HttpClient();
_httpClientFactory.Setup(x => x.CreateClient(It.IsAny<string>())).Returns(client);
var httpMessageHandler = new Mock<HttpMessageHandler>();
}
}
The code below is what you should use regardless of the method in the HttpClient class you use (GetAsync, PostAsync, etc.). All these methods are created for the convenience of the programmer. What they do is use the SendAsync method of the HttpMessageHandler class.
var mockHttpMessageHandler = new Mock<HttpMessageHandler>();
// Setup Protected method on HttpMessageHandler mock.
mockHttpMessageHandler.Protected()
.Setup<Task<HttpResponseMessage>>(
"SendAsync",
ItExpr.IsAny<HttpRequestMessage>(),
ItExpr.IsAny<CancellationToken>()
)
.ReturnsAsync((HttpRequestMessage request, CancellationToken token) =>
{
HttpResponseMessage response = new HttpResponseMessage();
// configure your response here
return response;
});
And then you use it this way:
var httpClient = new HttpClient(mockHttpMessageHandler.Object);
var result = await httpClient.GetAsync(url, cancellationToken);
You can also take a look here How to create mock for httpclient getasync method?
Setup your Mock HttpMessageHandler first and pass it to the constructor of your HttpClient. Then you can setup a Mock for the GetAsync method on the handler like this:
var httpMessageHandler = new Mock<HttpMessageHandler>();
// Setup Protected method on HttpMessageHandler mock.
httpMessageHandler.Protected()
.Setup<Task<HttpResponseMessage>>(
"GetAsync",
ItExpr.IsAny<string>(),
ItExpr.IsAny<CancellationToken>()
)
.ReturnsAsync((HttpRequestMessage request, CancellationToken token) =>
{
HttpResponseMessage response = new HttpResponseMessage();
// Setup your response for testing here.
return response;
});
var client = new HttpClient(httpMessageHandler.Object);
This is modified from a unit test I use to mockSendAsync, but it should be very similar.
The API I'm calling from my ASP.NET Web API app requires two tokens i.e. accessToken and userToken.
The following code is not working because it takes only the second token, not both. Looks like the second line is over-writing the first one.
How do I add multiple tokens to my request header?
_client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("APIAccessToken", "token1");
_client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("UserToken", "token2");
UPDATE:
Here's the way I set this up and it's not working. Basically, my API calls seem to go nowhere. I get no errors. Just no response.
First, I have the HttpClientAccessor that looks like this:
public static class HttpClientAccessor
{
private static Lazy<HttpClient> client = new Lazy<HttpClient>(() => new HttpClient());
public static HttpClient HttpClient
{
get
{
client.Value.BaseAddress = new Uri("https://api.someurl.com");
client.Value.DefaultRequestHeaders.Accept.Clear();
client.Value.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.Value.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
client.Value.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
return client.Value;
}
}
}
I then have my ApiClient that will perform my API calls which looks like this:
public class MyApiClient
{
HttpClient _client;
public MyApiClient()
{
_client = HttpClientAccessor.HttpClient;
}
public async Task Get()
{
try
{
HttpResponseMessage response = await _client.GetAsync("/myendpoint"); // This is where it gets lost
var data = await response.Content.ReadAsStringAsync();
}
catch(Exception e)
{
var error = e.Message;
}
}
}
This is my controller action:
public class MyController : Controller
{
private readonly MyApiClient _client;
public MyController()
{
_client = new MyApiClient();
}
public IActionResult SomeAction()
{
_client.Get().Wait();
}
}
You are confusing the standard authorization header with custom headers
According to the linked documentation
Request Header
Add the generated tokens to the request headers "APIAccessToken" and "UserToken"
Example Request
APIAccessToken: zjhVgRIvcZItU8sCNjLn+0V56bJR8UOKOTDYeLTa43eQX9eynX90QntWtINDjLaRjAyOPgrWdrGK12xPaOdDZQ==
UserToken: 5sb8Wf94B0g3n4RGOqkBdPfX+wr2pmBTegIK73S3h7uL8EzU6cjsnJ0+B6vt5iqn0q+jkZgN+gMRU4Y5+2AaXw==
To get headers like above, add them to the client like below
_client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
_client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
Based on shown update, the client is adding the headers every time the client is called. This should be in the value factory of the lazy client.
public static class HttpClientAccessor {
public static Func<HttpClient> ValueFactory = () => {
var client = new HttpClient();
client.BaseAddress = new Uri("https://someApiUrl");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
return client;
};
private static Lazy<HttpClient> client = new Lazy<HttpClient>(ValueFactory);
public static HttpClient HttpClient {
get {
return client.Value;
}
}
}
The controller action also needs to be refactored to avoid deadlocks because of the mixing of async and blocking calls like .Wait() or .Result.
public class MyController : Controller {
private readonly MyApiClient _client;
public MyController() {
_client = new MyApiClient();
}
public async Task<IActionResult> SomeAction() {
await _client.Get();
//... code removed for brevity
}
}
I want to set a default header for every method in the UserHttpClient but I don`t want that every method is doing that, I want to do it in a general way.
The problem I see with the current implementation is, that when I call one method the _client gets disposed thus at the next call within a Http Request the _client is not initialized, as this happens within the constructor.
The UserHttpClient is registered via DI as per Http Request.
I also do not want to create a private/base method where I pass the _client and do the header addition there.
How would you solve that problem?
public class UserHttpClient : IUserRemoteRepository
{
private readonly string baseUrl = ConfigurationManager.AppSettings["baseUrl"];
private readonly string header = ConfigurationManager.AppSettings["userHeader"];
private readonly HttpClient _client;
public ServiceProductDataProvider(string toolSystemKeyHeader)
{
_client = new HttpClient();
_client.DefaultRequestHeaders.Add(header, token);
}
public async Task<List<UserDto>> GetUsers(UserRequestDto dto)
{
using (_client)
{
// do stuff
var users = await _client.GetAsync("url here");
}
}
public async Task<UserDto> GetUser(Guid userId)
{
using (_client)
{
// do stuff
var users = await _client.GetAsync("url here");
}
}
}
The class UserHttpClient has a member that is IDisposable (private readonly HttpClient _client;). That means that the UserHttpClient should also implement IDisposable:
public void Dispose()
{
_client.Dispose();
}
Then, the class/code that is using UserHttpClient is responsible for Disposing it after it's done with it. If the instance is injected, then the DI framework you use probably handles disposing it automatically at the end of the request. What's left for you then is to simply remove the using blocks from the implementation:
public async Task<List<UserDto>> GetUsers(UserRequestDto dto)
{
// do stuff
var users = await _client.GetAsync("url here");
}
---- EDIT ----
You could also work around the issue by not reusing the HttpClient:
private string _toolSystemKeyHeader;
public ServiceProductDataProvider(string toolSystemKeyHeader)
{
_toolSystemKeyHeader = toolSystemKeyHeader
}
private HttpClient GetClientInstance()
{
HttpClient _client = new HttpClient();
_client.DefaultRequestHeaders.Add(header, _toolSystemKeyHeader); //?? in your original code, the toolSystemKeyHeader is not used, but I guess it is the token..?
return _client;
}
And:
public async Task<List<UserDto>> GetUsers(UserRequestDto dto)
{
using (var _client = GetClientInstance())
{
// do stuff
var users = await _client.GetAsync("url here");
}
}
I discovered that a single HttpClient could be shared by multiple requests. If shared, and the requests are to the same destination, multiple requests could reuse the connections. WebRequest needs to recreate the connection for each request.
I also looked up some documentation on other ways to use HttpClient in examples.
The following article summarizes the high-speed NTLM-authenticated connection sharing: HttpWebRequest.UnsafeAuthenticatedConnectionSharing
Possible implementations that I tried out are shown below
A)
private WebRequestHandler GetWebRequestHandler()
{
CredentialCache credentialCache = new CredentialCache();
credentialCache.Add(ResourceUriCanBeAnyUri, "NTLM", CredentialCache.DefaultNetworkCredentials);
WebRequestHandler handler = new WebRequestHandler
{
UnsafeAuthenticatedConnectionSharing = true,
Credentials = credentialCache
};
return handler;
}
using (HttpClient client = new HttpClient(GetWebRequestHandler(), false))
{
}
B)
using (HttpClient client = new HttpClient)
{
}
C)
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("some uri string")
I would appreciate any help in making me understand which approach I should take so as to achieve max performance, minimizing connections and making sure security is not impacted.
If you use either of them with async it should be good for the performance point of view as it will not block the resources waiting for the response and you will get good throughput.
HttpClient is preferred over HttpWebRequest due to async methods available out of the box and you would not have to worry about writing begin/end methods.
Basically when you use async call (using either of the class), it will not block the resources waiting for the response and any other request would utilise the resources to make further calls.
Another thing to keep in mind that you should not be using HttpClient in the 'using' block to allow reuse of same resources again and again for other web requests.
See following thread for more information
Do HttpClient and HttpClientHandler have to be disposed?
This is my ApiClient which creates the HttpClient for only once. Register this object as singleton to your dependency injection library. It's safe to reuse because it's stateless. Do NOT recreate HTTPClient for each request. Reuse Httpclient as much as possible
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
//You need to install package Newtonsoft.Json > https://www.nuget.org/packages/Newtonsoft.Json/
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
public class MyApiClient : IDisposable
{
private readonly TimeSpan _timeout;
private HttpClient _httpClient;
private HttpClientHandler _httpClientHandler;
private readonly string _baseUrl;
private const string ClientUserAgent = "my-api-client-v1";
private const string MediaTypeJson = "application/json";
public MyApiClient(string baseUrl, TimeSpan? timeout = null)
{
_baseUrl = NormalizeBaseUrl(baseUrl);
_timeout = timeout ?? TimeSpan.FromSeconds(90);
}
public async Task<string> PostAsync(string url, object input)
{
EnsureHttpClientCreated();
using (var requestContent = new StringContent(ConvertToJsonString(input), Encoding.UTF8, MediaTypeJson))
{
using (var response = await _httpClient.PostAsync(url, requestContent))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
public async Task<TResult> PostAsync<TResult>(string url, object input) where TResult : class, new()
{
var strResponse = await PostAsync(url, input);
return JsonConvert.DeserializeObject<TResult>(strResponse, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
public async Task<TResult> GetAsync<TResult>(string url) where TResult : class, new()
{
var strResponse = await GetAsync(url);
return JsonConvert.DeserializeObject<TResult>(strResponse, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
public async Task<string> GetAsync(string url)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.GetAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public async Task<string> PutAsync(string url, object input)
{
return await PutAsync(url, new StringContent(JsonConvert.SerializeObject(input), Encoding.UTF8, MediaTypeJson));
}
public async Task<string> PutAsync(string url, HttpContent content)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.PutAsync(url, content))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public async Task<string> DeleteAsync(string url)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.DeleteAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public void Dispose()
{
_httpClientHandler?.Dispose();
_httpClient?.Dispose();
}
private void CreateHttpClient()
{
_httpClientHandler = new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
};
_httpClient = new HttpClient(_httpClientHandler, false)
{
Timeout = _timeout
};
_httpClient.DefaultRequestHeaders.UserAgent.ParseAdd(ClientUserAgent);
if (!string.IsNullOrWhiteSpace(_baseUrl))
{
_httpClient.BaseAddress = new Uri(_baseUrl);
}
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(MediaTypeJson));
}
private void EnsureHttpClientCreated()
{
if (_httpClient == null)
{
CreateHttpClient();
}
}
private static string ConvertToJsonString(object obj)
{
if (obj == null)
{
return string.Empty;
}
return JsonConvert.SerializeObject(obj, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
private static string NormalizeBaseUrl(string url)
{
return url.EndsWith("/") ? url : url + "/";
}
}
The usage;
using ( var client = new MyApiClient("http://localhost:8080"))
{
var response = client.GetAsync("api/users/findByUsername?username=alper").Result;
var userResponse = client.GetAsync<MyUser>("api/users/findByUsername?username=alper").Result;
}
There is a problem in your implementation 'A'. The lifetime of the instance returned from GetWebRequestHandler() is shortlived (maybe just for the sake of the example?). If this was done on purpose, it negates the passing of false for the 2nd param of HttpClient constructor. The value of false tells the HttpClient to not dispose the underlying HttpMessageHandler (which helps with scaling since it does not close the port for the request). This is, of course, assuming that the lifetime of the HttpMessageHandler is long enough for you to take advantage of the benefit of not opening/closing ports (which is a big impact on the scalability of your server). Thus, I have a recommendation below of option 'D'.
There is also an option 'D' that you do not list above - to make the Httpclient instance static and re-used across all api calls. This is much more efficient from a memory allocation and GC perspective - as well as the opening of ports on the client. You don't have the overhead of allocating memory for and creating HttpClient instances (and all of its underlying objects) and thus avoid the cleanup via GC for them too.
Please refer to my answer provided about a similar question - What is the overhead of creating a new HttpClient per call in a WebAPI client?