To access DocumentDB/CosmosDB I'm using package Microsoft.Azure.DocumentDB.Core(v1.3.2). I have noticed when I create and initialise DocumentClient class:
var documentClient = new DocumentClient(new Uri(endpointUrl), primaryKey);
await documentClient.OpenAsync();
There is a number of requests fired to the endpoint to get information about indexes and other information. To be exact there are 9 HTTP requests going out on .OpenAsync(). This makes the creation and activation of the client a very costly operation in terms of performance - takes up to a second to get all the requests back home.
So to mitigate this costly operation I'm making DocumentClient to be a singleton and keep the reference around for the lifetime of the application.
Application is Asp.Net Core MVC and this might keep the reference of this object in memory for days.
Question: is it OK to keep this object as a singleton for that long? if not, what should be the strategy to dispose it? Or is there a way to make the initialisation cheaper (i.e. don't make these initial requests?).
We've wondered that for ourselves as well and found this:
From the docs
SDK Usage Tip #1: Use a singleton DocumentDB client for the lifetime of your application Note that each DocumentClient instance is thread-safe and performs efficient connection management and address caching when operating in Direct Mode. To allow efficient connection management and better performance by DocumentClient, it is recommended to use a single instance of DocumentClient per AppDomain for the lifetime of the application.
I suppose this is still valid now you can address CosmosDB with it as well.
Related
Let's say I have a simple service that's registered as a Transient in Startup, and I use Flurl like so:
public async Task DoStuff()
{
string url = "some valid Url";
await url
.AppendPathSegment("notifications")
.WithHeader("a header", headervalue1)
.WithHeader("another header", headervalue2)
.PostJsonAsync(data);
}
This service will be used a lot throughout our app. Can I count on Flurl to handle the requests efficiently so that my app doesn't exhaust the number of sockets available under heavy loads?
According to their docs: yes - the default usage, as you are showing, makes uses of the implementation guidelines provided by Microsoft:
Quote:
Flurl.Http is built on top of the System.Net.Http stack. If you're familiar with HttpClient, you probably already know this advice:
HttpClient is intended to be instantiated once and re-used throughout the life of an application. Especially in server applications, creating a new HttpClient instance for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors.
Flurl.Http adheres to this guidance by default. Fluent methods like this will create an HttpClient lazily, cache it, and reuse it for every call to the same host*:
Sources:
https://flurl.dev/docs/client-lifetime/
https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-6.0#remarks
Keep in mind though - "heavy load" can still mean you'll hit certain limits like:
reaching maximum server connections - i.e.: possible server overload
reaching maximum client socket usage - i.e.: initiating too many concurrent connections
For more info see:
https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.maxconnectionsperserver?view=net-6.0
This means you'll still need to do a sanity check on the amount of connections you'll be expecting.
I've got a .Net Core project that needs to connect to around 4 different API services, I'm no expert with any of the HttpClient code, but from what I found, was that you'd generally only want to reuse one instance of your HttpClient. From what I can tell the general consensus is to use the HttpClientFactory in .Net Core by registering it in your Startup class and then requesting it using DI.
Now most of my default headers and such are all generally the same besides the BaseAddress url, how should I go about this when connecting to 4 diff API services? Should I register 4 different named clients or have one client with all the default information pre-set and then manually configure it as needed e.g. configuring the address?
General questions would be as I'm fairly new to this is, it's been said to re-use one instance of an HttpClient.
If I create 4 different named clients for each API service, wouldn't this create 4 instances of the HttpClient when I call the .CreateClient() method?
The .CreateClient() creates a new instance every time it's called, doesn't this defeat the purpose of having one instance of the HttpClient if say I need to make 3 different calls to one API service, each of those calls will call a .CreateClient() to establish some sort of connection and that will create 3 instances of the HttpClient?
Any help for clarity would be appreciated,
Thanks!
The purpose of using IHttpClientFactory is not to reuse instances of HttpClient. Instead, it is to reuse (by pooling) instances of HttpMessageHandler (actually HttpClientHandler, which is derived from the abstract HttpMessageHandler) that is the underlying object that manages HTTP connections & sockets. This diagram from Microsoft Docs shows it well.
You were worried that frequent calls to IHttpClientFactory.CreateClient() will create the same problem as frequent calls to new HttpClient(). However, this is not the case. As explained by Microsoft docs, the reason that frequent calls to new HttpClient() will result in socket exhaustion is that this constructor will create a new instance of HttpMessageHandler:
However, the issue isn't really with HttpClient per se, but with the default constructor for HttpClient, because it creates a new concrete instance of HttpMessageHandler, which is the one that has sockets exhaustion and DNS changes issues mentioned above.
You can see from the source code of IHttpClientFactory that it does not use the parameterless constructor of HttpClient in CreateClient(). Instead, it gets the HttpMessageHandler from a pool and inject it into the created HttpClient.
Whether you are using typed or named clients, you should use the HttpClient instance as if it's a transient object: it is cheap to create and you don't need to cache it for long periods of time.
We are creating range of dotnet core 2.0 microservices based on the servicestack framework. We want to use http-header based correlation tokens, so we can track a request in our distributed logging system (Seq).
We would like to use IoC to setup a a class holding a threadsafe JsonServiceClient for performance reasons, but how can we ensure that headers placed on one thread will not leak into another concurrent request? Client code example:
public TResponse Get(IReturn requestDto)
...
_serviceClient.AddHeader("r-id", theReqId); // how can we make these specific for the thread request only?
var responseFromDownstreamService = _serviceClient.Get(requestDto);
If you’re modifying the service client instance the dependency needs to be transient so each thread receives a new instance they can mutate without modifying the same instance used by other threads.
I have implemented REST service using WebAPI2, service implemeted to manage different sessions which are created and joined by different clients which are accessing service.
Session contains information about access of application functionality and information of participants which have joined same session.
Each client get session information and access list from server for synchronization purpose on every second. According to access changed, client functionality will changed(Enable/Disable).
I am using MemoryCache class to store session info in WebAPI service as below.
public static class SessionManager{
private static object objForLock = new object();
public static List<Session> SessionCollection
{
get
{
lock (objForLock)
{
MemoryCache memoryCache = MemoryCache.Default;
return memoryCache.Get("SessionCollection") as List<Session>;
// return HttpContext.Current.Application["SessionCollection"] as List<Session>;
}
}
set
{
lock (objForLock)
{
MemoryCache memoryCache = MemoryCache.Default;
memoryCache.Add("SessionCollection", value, DateTimeOffset.UtcNow.AddHours(5));
//HttpContext.Current.Application["SessionCollection"] = value;
}
}
}
}
My problem is regarding inconsistent behavior of cache.
When clients send synchronization call, it will gives inconsistent results. For some requests, clients gets proper data and for some requests client gets null data alternative after some requests.
I have add debugger and monitor the object for null result, then "memoryCache.Get("SessionCollection")" also null. After some consecutive request it will be proper again. I am not getting why this object is not persistent.
Alternative, I have tried "HttpContext.Current.Application["SessionCollection"]" as well, But same issue is there.
I have read about "app pool recycle", it recycle all cache after particulate time. If my cached object is recycled by app pool recycle, then how can I get this object again?
Please some can help me to get out of this issue. Thanks in advance.
You should store client specific information in Session instead of Cache. Cache should be for the whole application (shared)
However, it's not recommended as web api is built with RESTful in mind and RESTful services should be stateless (APIs do not cache state). Stateless applications have many benefits:
Reduce Memory Usage
Better scalability: Your application scales better. Image what happens if you store information of millions of client at the same time.
Better in load balancing scenario: every server can handle every client without losing state.
Session expiration problem.
In case you want to store client state, you could do it anyway. Please try the suggestions in the following post: ASP.NET Web API session or something?
In general, caching state locally on the web server is bad (both Session and local MemoryCache). The cache could lose for many reasons:
App pool recycle.
Load balancing environment
Multiple worker processes in IIS
Regarding your requirements:
Each client get session information and access list from server for
synchronization purpose on every second. According to access changed,
client functionality will changed(Enable/Disable).
I'm not sure if you want to update the other clients with new access list immediately when a client sends synchronization call. If that's the case, SignalR would be a better choice.
Otherwise, you could just store the updated access list somewhere (shared cache or even in database) and update the other clients whenever they reconnect with another request.
#ScottHanselman said about a bug in .NET 4 here. I hope this fix help you:
The temporary fix:
Create memory cache instance under disabled execution context flow
using (ExecutionContext.SuppressFlow()) {
// Create memory cache instance under disabled execution context flow
return new YourCacheThing.GeneralMemoryCache(…);
}
The Hotfix is http://support.microsoft.com/kb/2828843 and you can request it here: https://support.microsoft.com/contactus/emailcontact.aspx?scid=sw;%5BLN%5D;1422
Just a caution, MemoryCache will keep data in memory in single server. So if you have multiple web servers(in front of load balancers), that cache will not be available to other servers. You also use the cache name - "SessionCollection". That data will be shared to all clients. If you need to store data in cache unique to each client, you need to return a token (guid) to the client and use that token to get/update data in cache in subsequent requests.
Try introducing a class level variable. So your code will look like below. (Some code remove for clarity)
private readonly MemoryCache _memCache = MemoryCache.Default;
....
return _memCache.Get("SessionCollection") as List<Session>;
...
_memCache .Add("SessionCollection", value, DateTimeOffset.UtcNow.AddHours(5));
I have a self hosted WCF service that I am using in a silverlight application. I am trying to store a list of user guids in an IDictionary object. Each time a user hits the service, it updates the users datetime so I can keep track of which users have active "sessions". The problem is, every time I am hitting the service, the list is empty. It appears to be dropping the values on each soap request?
Can you store information in a self hosted service that will be available across multiple service requests?
Thanks in advance!
It's on a per instance basis. I.e session-less by default.
Have a look at this
When a service contract sets the
System.ServiceModel.ServiceContractAttribute.SessionMode property to
System.ServiceModel.SessionMode.Required, that contract is saying that
all calls (that is, the underlying message exchanges that support the
calls) must be part of the same conversation.
If you need to store things in between requests you will need to create either a static dictionary with the appropriate locking to store these requests as they come in, or store this info in a database (or other external store) and check to see if it exists there in each method call. The reason for this is that the service class is instantiated on every client request.
Since you are already updating the users datetime when a user hits the service it would be better do a lookup to see if this is an active user or not by comparing to the datetime field. This has the advantage of being accurate on every call (the dictionary could get out of sync with the db if the service is restarted). Databases already have mechanisms in place to deal with concurrency, so rather than rolling your own locking solution around a singleton object you can push the complexity to the data store.
If the second solution is not fast enough (and you have profiled the app and determined it's the bottleneck), then the other option is to use some kind of cache solution in front of the db so that data can first be checked in memory before going to the db. This cache object would need to be static like the dictionary and has the same pitfalls around locking as any other multi-threaded application.
EDIT: If this hosted WCF service is being used as session storage for the users of the silverlight application and the data is not being stored in an external data store, then you better be sure that tracking if they are active is not mission critical. This data cannot be guaranteed to be correct as described.
Based on the accepted answer if your service faults and needs to be rebooted (since this is self hosted it is advised that you monitor the faulted event) you have to dispose of the service host and instantiate a new one. The only way the Guid data can be kept is if it is rebound to the service in between restarts (assuming the host app itself isn't restarted which is a different issue).
private Dictionary<Guid,string> _session;
Service service = new Service(_session);
_serviceHost = new ServiceHost(service, GetUriMethodInHostApp());
Better would be to store this externally and do a lookup as #marc_s suggests. Then this complexity goes away.
You need to change the InstanceContextMode. You can do so by adding the following compiler directive to your WCF class:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
This will run the WCF service as a singleton of sorts. See more on WCF Instance Context Mode
And then you should construct your service host with your singleton object. Here's code from a working example where I'm doing something similar:
private ServiceHost serviceHost;
if (serviceHost != null)
serviceHost.Close();
if (log.IsInfoEnabled)
log.Info("Starting WCF service host for endpoint: " + ConfiguredWCFEndpoint);
// Create our service instance, and add create a new service host from it
ServiceLayer.TagWCFService service = new ServiceLayer.TagWCFService(ApplicationName,
ApplicationDescription,
SiteId,
ConfiguredUpdateRateMilliseconds);
serviceHost = new ServiceHost(service, new Uri(ConfiguredWCFEndpoint));
// Open the ServiceHostBase to create listeners and start listening for messages.
serviceHost.Open();
As others have politely noted, this can have "consequences" if you're not familiar with how it works or if it's not a good fit for your particular application.
If you don't what to involve locking and thread-safe specific code, you can use a NoSQL database to store your session data, something like MongoDB or RavenDB
Like #marc_s, I think that using the Singleton mode is a risky thing, you have to be very careful in making your own thread-safe session mechanism.