Develop Reference

Best practices to implement dotnet core web api with the option to use horizontal scaling in cloud

We have created a dotnet core web api project which is using SQL Server database. Now, we are planning to deploy this project to Microsoft Azure.
While the deployment of this application, we are also considering to enable autoscaling option (horizontal scaling).
Before, we do it. We want to have some questions that we want to clarify.
Should we need to add some additional code in our application which allows autoscaling to work properly?
Properly in a sense, as there can be more than one instance of the application running because of horizontal scaling. We are using database and more than one instance is running will it case race condition (i.e., two resources accessing the same data at a time). I mean we can add a transaction (or use locking) in our code to avoid these kinds of scenarios?
I want to know that is there any best practices to follow while implementing that kind of application?
Thank you and waiting for your answers!

Consider the following points when designing an autoscaling strategy:
The system must be designed to be horizontally scalable. Avoid making
assumptions about instance affinity; do not design solutions that
require that the code is always running in a specific instance of a
process. When scaling a cloud service or web site horizontally, do
not assume that a series of requests from the same source will always
be routed to the same instance. For the same reason, design services
to be stateless to avoid requiring a series of requests from an
application to always be routed to the same instance of a service.
When designing a service that reads messages from a queue and
processes them, do not make any assumptions about which instance of
the service handles a specific message because autoscaling could
start additional instances of a service as the queue length grows.
The Competing Consumers pattern describes how to handle this
scenario.
If the solution implements a long-running task, design this task to
support both scaling out and scaling in. Without due care, such a
task could prevent an instance of a process from being shutdown
cleanly when the system scales in, or it could lose data if the
process is forcibly terminated. Ideally, refactor a long-running task
and break up the processing that it performs into smaller, discrete
chunks. The Pipes and Filters pattern provides an example of how you
can achieve this. Alternatively, you can implement a checkpoint
mechanism that records state information about the task at regular
intervals, and save this state in durable storage that can be
accessed by any instance of the process running the task. In this
way, if the process is shutdown, the work that it was performing can
be resumed from the last checkpoint by using another instance.
For more information, follow the doc : https://github.com/Huachao/azure-content/blob/master/articles/best-practices-auto-scaling.md

Regarding this:
Properly in a sense, as there can be more than one instance of the application running because of horizontal scaling. We are using database and more than one instance is running will it case race condition (i.e., two resources accessing the same data at a time). I mean we can add a transaction (or use locking) in our code to avoid these kinds of scenarios?
Please keep in mind that, even if the app is running on a single machine, requests will still be handled concurrently. This means that even on a single machine 2 requests can cause the same entry in the database to be updated. So the above questions about race conditions apply to single instance web apps as well.
Try to avoid locking: the whole point of (horizontal) scaling is to gain performance benefits. By using locks you effectively remove this benefits as only one process at a time can use the locked resource.
Other points of considerations are:
If you are using an in-memory cache you might want to swap it out for a distributed cache.
The guidance at the MS docs

Design question about background processing web service [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
the title of the question may not clear enough, allow me to explain the background here:
I would like to design a web service that generates PDF and submit it to printer, here is the workflow:
User submit a request to the web service, probably the request will be one off so that user wouldn't suffer from waiting the job complete. User may received a HTTP200 and continue their work.
Once web service received the request, the web service generates the PDF, and submit it to designated printer and this process could take some time and CPU resources. As I don't want the drain all resource on that server, I may use producer consumer pattern here, there might be a queue to to queue client jobs, and process them one by one.
My Questions is that:
I'm new to C#, what is the proper pattern to queue and process them? Should I use ConcurrentQueue and ThreadPool to archive it?
What is the proper way to notify user about the job is success/fail? Instead of using callback service, is async an ideal way? My concern is that there may be lots of jobs in the queue and I don't want client suffer from waiting it complete.
The web service is placed behind a load balancer, how can I maintain a 'process queue' among them? I've tried using Hangfire and it seems okay, however I'm looking for alternative?
How can I know the number of jobs in the Queue/ how may thread is currently running? The webservice will be deployed on IIS, is there a Native way to archive it, or should I implement a web service call to obtain them?
Any help will be appreciated, thanks!

WCF supports the idea of a fire-and-forget methods. You just mark your contract interface method as one way, and there will be no waiting for a return:
[OperationContract( IsOneWay = true )]
void PrintPDF( PrintRequest request );
The only downside, of course, is that you won't get any notification from the server that you're request was successful or even valid. You'd have to do some kind of periodic polling to see what's going on. I guess you could put a Guid into the PrintRequest, so you could interrogate for that job later.
If you're not married to wcf, you might consider signalR...there's a comprehensive sample app of both a server and simple wpf client here. It has the advantage that either party can initiate an exchange once the connection has been established.
If you need to stick with wcf, there's the possibility of doing dualHttp. The client connects with an endpoint to callback to...and the server can then post notifications as work completes. You can get a feel for it from this sample.
Both signalR and wcf dualHttp are pretty straightforward. I guess my preference would be based on the experience of the folks doing the work. signalR has the advantage of playing nicely with browser-based clients...if that ever turns into a concern for you.
As for the queue itself...and keeping with the wcf model, you want to make sure your requests are serializable...so if need be, you can drain the queue and restart it later. In wcf, that typically means making data contracts for queue items. As an aside, I never like to send a boatload of arguments to a service, I prefer instead to make a data contract for method parameters and return types.
Data contracts are typically just simple types marked up with attributes to control serialization. The wcf methods do the magic of serializing/deserializing your types over the wire without you having to do much thinking. The client sends a whizzy and the server receives a whizzy as it's parameter.
There are caveats...in particular, the deserialization doesn't call your constructor (I believe it uses MemberwiseClone instead) ...so you can't rely on the constructor to initialize properties. To that end, you have to remember that, for example, collection types that aren't required might need to be lazily initialized. For example:
[DataContract]
public class ClientState
{
private static object sync = new object( );
//--> and then somewhat later...
[DataMember( Name = "UpdateProblems", IsRequired = false, EmitDefaultValue = false )]
List<UpdateProblem> updateProblems;
/// <summary>Problems encountered during previous Windows Update sessions</summary>
public List<UpdateProblem> UpdateProblems
{
get
{
lock ( sync )
{
if ( updateProblems == null ) updateProblems = new List<UpdateProblem>( );
}
return updateProblems;
}
}
//--> ...and so on...
}
Something I always do is to mark the backing variable as the serializable member, so deserialization doesn't invoke the property logic. I've found this to be an important "trick".
Producer/consumer is easy to write...and easy to get wrong. Look around on StackOverflow...you'll find plenty of examples. One of the best is here. You can do it with ConcurrentQueue and avoid the locks, or just go at it with a good ol' simple Queue as in the example.
But really...you're so much better off using some kind of service bus architecture and not rolling your own queue.
Being behind a load balancer means you probably want them all calling to a service instance to manage a single queue. You could roll your own or, you could let each instance manage its own queue. That might be more processing than you want going on on your server instances...that's your call. With wcf dual http, you may need your load balancer to be configured to have client affinity...so you can have session-oriented two-way communications. signalR supports a message bus backed by Sql Server, Redis, or Azure Service Bus, so you don't have to worry about affinity with a particular server instance. It has performance implication that are discussed here.
I guess the most salient advice is...find out what's out there and try to avoid reinventing the wheel. By all means, go for it if you're in burning/learning mode and can afford the time. But, if you're getting paid, find and learn the tools that are already in the field.
Since you're using .Net on both sides, you might consider writing all your contracts (service contracts and data contracts) into a .DLL that you use on both the client and the service. The nice thing about that is it's easy to keep things in sync, and you don't have to use the (rather weak) generated data contract types that come through WSDL discovery or the service reference wizard, and you can spin up client instances using ChannelFactory<IYourServiceContract>.

WCF timeout handling

We are currently developing a software solution which has a client and a number of WCF services that it consumes. The issues we are having is WCF services timing out after a period of inactivity. As far as I understand, there are 2 ways to resolve this:
Increase timeouts (as far as I understood, this is generally not recommended. Eg. setting timeout to infinite/weeks is considered bad practice)
Periodically ping the WCF services from the Client (I'm not sure that I'm a huge fan of his as it will add redundant, periodic calls)
Handle timeout issues and attempt to reconnect (this is slow and requires a lot of manual code)
Reliable Sessions - some sources mention that this is the in-built WCF pinging and message reliability mechanism, but other sources mention that this will still time out.
What is the recommended/best way of resolving this issue? Is there any official reading material on this? I could not find all that much info myself
Thanks!

As i can see, you have to use a combination of your stated points.
You are right, increasing the timeouts is bad practice and can give you a lot of problems.
If you don't want to use Reliable Sessions, then Ping is the only applicable way to hold the connection.
You need to handle this things, no matter if a timeout occurs, the connection is lost or a exception is thrown. There are a plenty of possibilities that your connection can fault.
Reliable Sessions are a good way not to implement a ping, but technically, it does nearly the same. WCF automatically sends an "I am still here" Request.
The conclusion of this is, that you need point 3 and point 2 or 4. To reduce the manually code for point 3, you can use Proxies or a wrapper around your ServiceClient, that establishes a new connection if the old one is faulted during a request. Point 4 is easy to implement, because you only need some small additions to your binding in your config. And the traffic overhead is not that big. Point 2 is the most expensive way, you need to handle a Thread/Task that only pings the server and the service needs to be extended. But as you stated before, Reliable Sessions can fail, and Pings should bring you on the safe side.

You should ask yourself what is your WCF endpoint is doing? Is the way you have your command setup the most optimal?
Perhaps it'd be better to have your endpoint that takes a long time be based on a polling system that allows there to be a quick query instead of waiting on the results of the endpoints actions.
You should also consider data transfer as a possible issue. Is the amount of data you're transferring back a lot?
To get a more pointed answer, we'd need to know more about the specific endpoint as well as any other responsibilities there are for the service.

Problems with singleton pattern inside WCF service method

I will go ahead and preface this by saying: I am somewhat new to WCF.
I'm working on a server-side routine that's responsible for doing a great deal of business logic. It's accessible from a client via WCF.
My main WCF method calls off to several other private methods. Instead of passing around all of the "lookup data" I need for the business logic to each private method, I decided to use a singleton instance of a class named DataProvider that contains all of this "lookup data".
At the end of the routine, I "release" the DataProvider's lookup data so the next time the routine is executed, the latest lookup data will be used.
So, here's a simplified example:
public void Generate()
{
try
{
//populate singleton DataProvider with it's lookup data...
DataProvider.Instance.LoadLookupData();
//do business logic...
}
finally
{
//release provider's lookup data...
DataProvider.Release();
}
}
This works great until I have two different clients that execute the method at (or near) the same time. Problems occur because they share the same singleton instance and the task who finishes first will release the DataProvider before the other completes.
So...
What are my options here?
I'd like to avoid passing around all of the lookup data so the singleton pattern (or some derivative) seems like a good choice. I also need to be able to support multiple clients calling the method at the same time.
I believe the WCF service is configured as "Per-Call". I'm not sure if there's a way to configure a WCF service so that the static memory is not shared between service invocations.
Any help would be appreciated.

By default WCF is using "Per-Call", which means new instance of the WCF service is created for each client's call. Now since you implemented singleton even though new instance of the WCF is created it still calls your singleton.
If you would like to create lookup that is created for each call (like you have now) you should not do it as singleton. This way each client that calls your method will have new instance of the lookup, I think that was your intention.
However if you have lookup that is not changing that fast, I would recommend to share it between all calls, this will improve performance of your WCF service. You will need to declare your WCF service as
InstanceContextMode = InstanceContextMode.Single
ConcurrencyMode = ConcurrencyMode.Multiple
What this does is creating Singleton automatically for you by WCF, so you don't have to do it yourself, second it will support > 1 concurrent user (ConcurrencyMode.Multiple).
Now if you have your lookup that is changing and it needs to be reloaded after some period of time, I still would recommend using
InstanceContextMode = InstanceContextMode.Single
ConcurrencyMode = ConcurrencyMode.Multiple
but inside in your code cache it and then expire your cache at specific time or relative time (1 hours).
Here are some links that might help you:
3 ways to do WCF instance management (Per call, Per session and Single)
Hope this will help.

The static variables in a WCF service are always shared between instances regardless of the WCF InstanceContextMode setting. It seems you would be better off using a caching pattern for your look up data. The answers to this caching question provide some alternatives to rolling your own although they are a bit dated.
Also, if you decide that making the whole service instance a singleton (InstanceContextMode=Single) is the easiest solution be aware that you'll generally kill service scalability unless you also make your code multi-threaded (ConcurrencyMode=Multiple). If you can knock out thread-safe code in your sleep then a singleton service might be for you.

simplest is to use a synchronization mechanism - have you looked at lock(...) - this will act as a gatekeeper a lot like a critical section (if you have come across those in windows programming)
define a static object in your class
i.e.
static object lockObject = new object();
and use it in Generate method
i.e.
void Generate()
{
lock(lockObject)
{
...
}
}

C# Thoughts on this design

I have this problem domain where I need to able to run a background process that would:
Run a filter to get an obj collection (time consuming operation)
Pass the obj coll through a set of rules...maybe thru a rule interface
Be able to expose any changes that the rules caused to any interested listeners.
Each filter may have many rules and there can be more than one filter.
Would would be the practical way to approach this? I'm thinking:
Have a WCF app hosted in a Windows Service that would expose callback for rule changes
Let the service do the grunt work of running filter->rules. Will this need to be a separate threaded work ?
Any thoughts or references to existing frameworks, design patterns, etc. are welcome.
thanks
Sunit

If your background process needs to be instantly (24/7/365) accessible from remote machines, the Windows service makes a lot of sense to me. Assuming you are familiar with C#, it is trivial to create a Windows service. You can follow the step-by-step here. Once you've got the Windows service going, it's easy to host the WCF service by creating the System.ServiceModle.ServiceHost instance in the OnStart callback of the Windows service. As far as WCF patterns and good practices, I'll refer you to Juval Lowy's website, IDesign.net. The site has a lot of free WCF-related downloads just by providing your email address.

You have a couple options, the two most obvious are either the client calls a method that starts the job and polls the server for status, or, setup a callback.
Either way the job should be run on a seperate thread so it doesn't block the service.
If you go with the poll for status route, put the actual result in the returning status.
If you go with the callback, use the WSDualHttpBinding and setup a callback. This looks a little scary to setup but it's really not that bad.
I'll let someone else chime in for actual patterns or frameworks, I'm just not sure. Also, checkout MSMQ, this might be another viable solution.

You could use WWF to take care of the rules. You should be able to host WWF as a service.