I have an application where we communicate with hundreds of HTTPs endpoints. The application is a proxy of sorts.
When testing with polly, I've noticed that if one endpoint, say api.endpoint1.com fails, the calls to api.endpoint2.com and api.endpoint3.com will also be in an open/blocked state.
This makes sense as I've only defined one policy, but what is the recommended approach to handling this scenario so that calls to unrelated endpoints are not blocked due to another having performance issues?
Do I create a collection of Policy's, one for each endpoint or is there a way to supply a context key of sorts(i.e. the hostname) to scope the failures to a given host endpoint?
I've reviewed Polly's docs regarding context keys and it appears these are a way to exchange data back and forth and not what I'm looking for here.
var policy = Policy
.Handle<TimeoutException>()
.CircuitBreaker(1, TimeSpan.FromSeconds(1));
//dynamic, large list of endpoints.
var m = new HttpRequestMessage(HttpMethod.Post, "https://api.endpoint1.com")
{
Content = new StringContent("some JSON data here", Encoding.UTF8,"application/json")
};
policy.Execute(() => HTTPClientWrapper.PostAsync(message));
Yes, your best bet is to create a separate policy per endpoint. This is better than doing it per host because an endpoint may be slow responding for a reason that's specific to that endpoint (e.g., stored procedure is slow).
I've used a Dictionary<string, Policy> with the endpoint URL as the key.
if (!_circuitBreakerPolices.ContainsKey(url))
{
CircuitBreakerPolicy policy = Policy.Handle<Exception>().AdvancedCircuitBreakerAsync(
onBreak: ...
);
_circuitBreakerPolicies.Add(url, policy);
}
await _circuitBreakerPolicies[url].ExecuteAsync(async () => ... );
Here is my alternative solution which does not maintain a collection of policies (either via an IDictionary or via an IConcurrentPolicyRegistry) rather it takes advantage of named typed clients. (Yes you have read correctly named and typed HttpClients)
The named and typed clients
Most probably you have heard (or even used) named or typed clients. But I'm certain that you haven't used named and typed clients. It is a less documented feature of HttpClientFactory + HttpClient combo.
If you look at the different overloads of the AddHttpClient extension method then you can spot this one:
public static IHttpClientBuilder AddHttpClient<TClient,TImplementation>
(this IServiceCollection services, string name, Action<HttpClient> configureClient)
where TClient : class where TImplementation : class, TClient;
It allows us to register a typed client and give a logical name to it. But how can I get the proper instance? That's where the ITypedHttpClientFactory comes into the picture. It allows us to create a typed client from a named client. Wait what??? I hope you will understand this sentence at the end of this post. :)
The typed client
For the sake of simplicity let me use this typed client as an example:
public interface IResilientClient
{
Task GetAsync();
}
public class ResilientClient: IResilientClient
{
private readonly HttpClient client;
public ResilientClient(HttpClient client)
{
this.client = client;
}
public Task GetAsync()
{
//TODO: implement it properly
return Task.CompletedTask;
}
}
The named and typed clients registration
Let suppose you have a list of downstream system urls (urls). Then you can register multiple typed client instances with different unique names and base urls
foreach (string url in urls)
{
builder.Services
.AddHttpClient<IResilientClient, ResilientClient>(url,
client => client.BaseAddress = new Uri(url))
.AddPolicyHandler(GetCircuitBreakerPolicy());
}
Here I have used the url as the unique name
So, we can get the appropriate instance based on the downstream url
The policy definition
private IAsyncPolicy<HttpResponseMessage> GetCircuitBreakerPolicy()
=> Policy<HttpResponseMessage>
.Handle<TimeoutException>()
.CircuitBreakerAsync(1, TimeSpan.FromSeconds(1));
I have modified the policy to support async: .CircuitBreakerAsync
I've also amended it to be suitable with the AddPolicyHandler: Policy<HttpResponseMessage>
It is defined as a function so each registered named typed client will have a different Circuit Breaker instance
The usage
This is be a bit clumsy, but I think it is okay. So, wherever you want to use one of the named typed clients you have to inject two interfaces:
IHttpClientFactory: To be able to create a named HttpClient
ITypedHttpClientFactory<ResilientClient>: To be able to create a typed client from the named HttpClient
public XYZService(
IHttpClientFactory namedClientFactory,
ITypedHttpClientFactory<ResilientClient> namedTypedClientFactory)
{
var namedClient = namedClientFactory.CreateClient(xyzUrl);
var namedTypedClient = namedTypedClientFactory.CreateClient(namedClient);
}
Please note that you have to use ResilientClient concrete class as the type parameter not the interface IResilientClient
If you would use the interface then you would receive the following runtime error:
InvalidOperationException: A suitable constructor for type 'IResilientClient' could not be located. Ensure the type is concrete and all parameters of a public constructor are either registered as services or passed as arguments. Also ensure no extraneous arguments are provided.
Summary
With the named and typed client feature of AddHttpClient we can register multiple instances of the same typed client
With the IHttpClientFactory we can retrieve a registered named client which has the proper BaseAddress and decorated with a Circuit Breaker
With the ITypedHttpClientFactory we can convert the named client into a typed client to be able to hide low-level API usage
Related sample application's github repository
Related
This question already has answers here:
Configure HttpClientFactory to use data from the current request context
(2 answers)
Closed 7 months ago.
Is there any way to implement an interceptor so that I can call a microservice api from another microservices where all the http call will be intercepted & added jwt token in the header? So that, I don't have to set authorization token each time I request.
In angular there is a concept of HttpInterceptor. Where each http requests will be intercepted ans added jwt token in the header. Is there any way to achieve that in asp.net core web api?
A simple approach are named clients. Here you add a preconfigured HttpClient instance to your service registry and can access this instance wherever you need it:
builder.Services.AddHttpClient("GitHub", httpClient =>
{
httpClient.BaseAddress = new Uri("https://api.github.com/");
// using Microsoft.Net.Http.Headers;
// The GitHub API requires two headers.
httpClient.DefaultRequestHeaders.Add(
HeaderNames.Accept, "application/vnd.github.v3+json");
httpClient.DefaultRequestHeaders.Add(
HeaderNames.UserAgent, "HttpRequestsSample");
});
Use the httpClient.Default* properties.
See: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/http-requests?view=aspnetcore-6.0#named-clients
The drawback is, that you don't know how requests must look like when using the instance. One approach to solve this is to create a so-called typed client. Basically a wrapper around HttpClient adding the right configuration and enforcing the correct request and response types.
public class CatalogService : ICatalogService
{
private readonly HttpClient _httpClient;
private readonly string _remoteServiceBaseUrl;
public CatalogService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<Catalog> GetCatalogItems(int page, int take, int? brand, int? type)
{
var uri = API.Catalog.GetAllCatalogItems(_remoteServiceBaseUrl, page, take, brand, type);
var responseString = await _httpClient.GetStringAsync(uri);
var catalog = JsonConvert.DeserializeObject<Catalog>(responseString);
return catalog;
}
}
(example from the docs here: https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests)
If you have static values (e.g. a fixed client secret), you can use options to inject them as described here: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-6.0
If you get your token from another service, obviously just inject this other service.
You can also configure the HttpClient when adding your typed client to the service registry:
services.AddHttpClient<ICatalogService, CatalogService>(client =>
{
client.BaseAddress = new Uri(Configuration["BaseUrl"]);
});
Example from: https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests#multiple-ways-to-use-ihttpclientfactory
Configuring the client here is similar to angular injectors in the way, that it separates configuration and client implementation. Example: You could use the same CatalogService class with a catalog server that expects an Authorization header and a server that expects "MySpecialAuth" header without having to take care of those differences in the CatalogService class.
To show the differences in the flow...
Angular iterceptors: use shared http client > then intercept requests based on e.g. domain
C# clients: preconfigure clinet > then use configured instance
Side notes:
You typically should not create new instances of HttpClient manually. Inject them using and use IHttpClientFactory in the background.
You can create a custom HttpClient using inheritence which adds this Header in each request, and use this across your application
I have a Web Api project which relies heavily on Azure Cosmos DB. Until now, having one Cosmos DB account (one connection string) was sufficient. Now a new requirement is to be able to connect to a different Cosmos (two connection strings) depending on an incoming parameter.
For customerId X we should fetch documents from Cosmos DB 1 and for another customer Y we have to look in Cosmos DB 2.
Until now my Startup.cs file registered a singleton instance of CosmosClient. Which in turn gets instantiated like this
cosmosClient = new CosmosClient(endpointUrl, primaryKey);
And this worked really well. The Web Api was easily able to process all requests. But now that we have to new up a CosmosClient per request, performance is really bad.
So my question is; Is there a way to have multiple instances of the same singleton? As in; can we create a single instance of the combination Class+EndPointUrl? (Would that still be a singleton?)
Right now, we are newing up thousands of CosmosClients every minute. And we really need just one more compared to what we had earlier.
There's multiple ways to do this, but an easy implementation would be to create a wrapper around each CosmosClient you use. The only use of the wrapper will be to allow you to use various instances of the CosmosClient and differentiate them by their types.
//Create your own class for each client inheriting the behaviour of CosmosClient
public class ContosoCosmosClient : CosmosClient
{
public ContosoCosmosClient(string connectionString, CosmosClientOptions clientOptions = null) : base(connectionString, clientOptions)
{
}
public ContosoCosmosClient(string accountEndpoint, string authKeyOrResourceToken, CosmosClientOptions clientOptions = null) : base(accountEndpoint, authKeyOrResourceToken, clientOptions)
{
}
public ContosoCosmosClient(string accountEndpoint, TokenCredential tokenCredential, CosmosClientOptions clientOptions = null) : base(accountEndpoint, tokenCredential, clientOptions)
{
}
}
//In Startup.up add a Singleton for each client
services.AddSingleton(new ContosoCosmosClient(...));
services.AddSingleton(new FabrikamCosmosClient(...));
Then in your business logic you can add both clients and depending on your logic choose which client you want to use:
public class MyService
{
public MyService(ContosoCosmosClient contosoClient, FabrikamCosmosClient fabrikamClient)
{
//...
}
}
Thanks for all comments and answers.
In the end, is this case, the best solution was the approach that was suggested by Mr. T. https://devblogs.microsoft.com/cosmosdb/httpclientfactory-cosmos-db-net-sdk/
I'm now still using one CosmosClient, Scoped. Which allows dynamic use of endpoints.
By injecting the IHttpClientFactory and setting the CosmosClientOptions like this;
{
HttpClientFactory = () => _httpClientFactory.CreateClient("cosmos")
});
we are now making full use of the HttpClient and its ability to reuse ports.
I'm using BotFramework version(v4) integrated with LUIS. In ConfigureServices(IServiceCollection services) method in startup.cs file I'm assigning storage and LUIS in the middleware.Below is the sample code.
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton(configuration);
services.AddBot<ChoiceBot>(options =>
{
options.CredentialProvider = new ConfigurationCredentialProvider(configuration);
var (luisModelId, luisSubscriptionKey, luisUri) = GetLuisConfiguration(configuration, "TestBot_Dispatch");//
var luisModel = new LuisModel(luisModelId, luisSubscriptionKey, luisUri);
var luisOptions = new LuisRequest { Verbose = true };
options.Middleware.Add(new LuisRecognizerMiddleware(luisModel, luisOptions: luisOptions));
//azure storage emulater
//options.Middleware.Add(new ConversationState<Dictionary<string, object>>(new AzureTableStorage("UseDevelopmentStorage=true", "conversationstatetable")));
IStorage dataStore = new AzureTableStorage("DefaultEndpointsProtocol=https;AccountName=chxxxxxx;AccountKey=xxxxxxxxx;EndpointSuffix=core.windows.net", "TableName");
options.Middleware.Add(new ConversationState<Dictionary<string,object>>(new MemoryStorage()));
options.Middleware.Add(new UserState<UserStateStorage>(dataStore));
}
}
My bot will be getting requests from users of different roles such as (admin,sales,etc..).I want to change the table storage connection-string passed to middleware based on the role extracted from the incoming request. I will get user role by querying DB from the user-name which is extracted from the current TurnContext object of an incoming request. I'm able to do this in OnTurn method, but as these are already declared in middleware I wanted to change them while initializing in the middleware itself.
In .NET Core, Startup logic is only executed once at, er, startup.😊
If I understand you correctly, what you need to be able to do is: at runtime, switch between multiple storage providers that, in your case, are differentiated by their underlying connection string.
There is nothing "in the box" that enables this scenario for you, but it is possible if use the correct extension points and write the correct plumbing for yourself. Specifically you can provide a customized abstraction at the IStatePropertyAccessor<T> layer and your upstream code would continue to work at that level abstraction and be none-the-wiser.
Here's an implementation I've started that includes something I'm calling the ConditionalStatePropertyAccessor. It allows you to create a sort of composite IStatePropertyAccessor<T> that is configured with both a default/fallback instance as well as N other instances that are supplied with a selector function that allows them to look at the incoming ITurnContext and, based on some details from any part of the turn, indicate that that's the instance that should be used for the scope of the turn. Take a look at the tests and you can see how I configure a sample that chooses an implementation based on the ChannelId for example.
I am a little busy at the moment and can't ship this right now, but I intend to package it up and ship it eventually. However, if you think it would be helpful, please feel free to just copy the code for your own use. 👍
I have an application (IJobInit) that uses a list from JSON settings to create multiple instances of a class (IJob). This class does some work using two other dependencies, IInputClient and IOutputClient. It uses M.Extensions.DependencyInjection to create a container which is handed off to AutoFac to create an IContainer.
IJobInit(IContainer container)
I would like IInputClient to be configured different for each instance of IJob. Speficially, I'd like to pass in a secret for it to use. The result would be:
IInputClient(HttpClient client)
where HttpClient is configured using ConfigureHttpClient such that IJob does not know that it is pre-authenticated. This would also be suitable:
IInputClient(ISecretProvider secretsProvider, string secretName)
The end result is three instances of IJob with IInputClient configured differently.
IJob(IInputClient inputClient1, IOutputClient outputClient)
IJob(IInputClient inputClient2, IOutputClient outputClient)
IJob(IInputClient inputClient3, IOutputClient outputClient)
How do I achieve this? I was looking at Autofac scopes but those controlwhen an instance is created without any control over its configuration (unless I missed it).
A colleague suggested that I could host each instance of IJob in its own process with its own configuration which is possible but I'm trying to host all the jobs in a single Azure Function and use the list in config to create the inner jobs.
Thanks!
I'm not totally happy with this solution but it works for now.
private async Task<IInputClient> GetClientAsync(string secretId)
{
HttpClient httpClient = this.httpClientFactory.CreateClient();
string secret = await this.secretsProvider.GetSecretAsync(secretId);
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(Encoding.ASCII.GetBytes(string.Concat(":", secret))));
return this.scope.Resolve<IInputClient>(new TypedParameter(typeof(HttpClient), httpClient));
}
Need a way for one service on a well-known Endpoint to return strings which are relative addresses. The client can then connect to Endpoints using these relative addresses.
Clearly this resembles REST in some ways, but in this case running a Windows Service using NetNamedPipeBinding for IPC, so no need for HTTP.
Don't want to create the Endpoint ahead of time since there will be a potentially large number of relative addresses, only some of which the client would be interested in.
All Contracts are known in advance.
Tried to find a solution with AddressFilterMode but wasn't sure how to provision new Binding so that client connected to it, UriTemplate but don't want to use the HTTP framework. Haven't looked into RoutingService because constrained to .Net 3.5.
Pseudocode for client would be something like that below...
namespace Testing
{
class RunTest
{
static void Test()
{
NetNamedPipeBinding namedpipe = new NetNamedPipeBinding();
ChannelFactory<Contracts.IRoot> factoryRoot =
new ChannelFactory<Contracts.IRoot>(
namedpipe
, new EndpointAddress("net.pipe://localhost/root");
);
Contracts.IRoot root = factoryRoot.CreateChannel();
ICommunicationObject commsRoot = root as ICommunicationObject;
commsRoot.Open();
// Service examines address and creates Endpoint dynamically.
string address = root.SomeFunctionWhichGetsARelativeAddress();
// IBar service routes endpoint requests internally based on
// "address" variable.
ChannelFactory<Contracts.IBar> factoryBar =
new ChannelFactory<Contracts.IBar>(
namedpipe
, new EndpointAddress("net.pipe://localhost/root/IBar/" +
address)
);
Contracts.IBar bar = factoryBar.CreateChannel();
bar.DoSomething();
}
} // Ends class RunTest
} // Ends namespace Testing
Message Filters are the way to go. You can use “Prefix” or create a custom.
WCF Addressing In Depth
From the Message Filters section of the article:
...it uses message filters to determine the matching endpoint, if one
exists. You can choose which message filter to use or you can provide
your own. This flexibility allows you to break free from the
traditional dispatching model when using Windows Communication
Foundation to implement things other than traditional SOAP—for
instance, the techniques described here enable you to implement
REST/POX-style services on the Windows Communication Foundation
messaging foundation.
Nice question, by the way. I learned something trying to figure this out.
AddressFilterMode.Prefix might suffice. The actual Endpoint used can be inspected in Service methods via
OperationContext.Current.IncomingMessageHeaders.To
Helper code can parse the endpoint and do any necessary internal processing from there.
Hopefully there's some extensibility on the server side which can simplify that code.
Pseudocode for host:
namespace Services
{
[System.ServiceModel.ServiceBehavior(AddressFilterMode =
System.ServiceModel.AddressFilterMode.Prefix)]
class BarService : Contracts.IBar
{
#region IBar Members
public void DoSomething()
{
System.Uri endpoint = System.ServiceModel.OperationContext.Current.IncomingMessageHeaders.To;
Console.WriteLine("DoSomething endpoint: {0}", endpoint);
}
} // Ends class BarService
} // Ends namespace Services
class RunHost
{
static void HostIBar()
{
System.Uri uriBase = new System.Uri("net.pipe://localhost");
System.ServiceModel.ServiceHost hostBar =
new System.ServiceModel.ServiceHost(
typeof(Services.BarService),
uriBase);
hostBar.AddServiceEndpoint(
typeof(Contracts.IBar) // Type implementedContract
, namedpipeBinding // System.ServiceModel.Channels.Binding binding
, "root/IBar" //string address
);
hostBar.Open();
Console.WriteLine("Press <ENTER> to stop...");
Console.ReadLine();
}
}
Correction: I'd originally said that this wouldn't treat "net.pipe://localhost/root/IBar/1" and "net.pipe://localhost/root/IBar/2" as distinct endpoints, but it does. Each causes its own WCF Service instance to be created and called.
An additional change was to encode the data in URL style query parameters and not embed it in the path. E.g.: "net.pipe://localhost/root/IBar?something=1&somethingelse=11" and "net.pipe://localhost/root/IBar?something=2&somethingelse=22" using HttpUtility.ParseQueryString