I have a quartz.net scheduler implementation (and jobs). The scheduler is inside my website/application itself and runs jobs async at specified time intervals.
It is all very well on local and testing . As soon as i move this code to live (which has multiple load balancers running instances of my website), the jobs start firing as many times as there are load balancers.
How do i handle this? How do i make sure that there is just one scheduler that runs jobs across all my load balancers.
Alternatively is there a way for me to find out when was the previous job fired (on any of the load balancers!)
thanks
If you use a database as your job store, point all your schedulers at the same database and set them up as clustered, then your jobs will be shared across the servers and you won't have them firing multiple times.
Alternatively you could also move your scheduler out into it's own windows service and then share it across your web servers.
Related
I am having an asp.mvc application which resides on a server.From this application, I want to start a process which is a bit long-running operation and will be resource intensive operation.
So what I want to do is I want to have some user agent like 3 which will be there on 3 machines and this user agent will use resources of their respective machines only.
Like in Hadoop we have master nodes and cluster in which tasks are run on the individual cluster and there is 1 master node keeping track of all those clusters.
In Azure, we have virtual machines on which tasks are run and if require Azure can automatically scale horizontally by spinning up the new instance in order to speed up the task.
So I want to create infrastructure like this where I can submit my task to 3 user agents from the mvc application and my application will keep track of this agents like which agent is free, which is occupied, which is not working something like this.
I would like to receive progress from each of this user agent and show on my MVC application.
Is there any framework in .net from which I can manage this background running operations(tracking, start, stop etc..) or what should be the approach for this?
Update : I don't want to put loads of server for this long running operations and moreover I want to keep track of this long running process too like what they are doing, where is error etc.
Following are the approach which I am thinking and I don't know which will make more sense:
1) Install Windows Service in the form of agents of 2-3 computer on premises to take advantage of resp resources and open a tcp/ip connection with this agents unless and until the long running process is complete.
2) Use hangfire to run this long running process outside of IIS thread but I guess this will put load on server.
I would like to know possible problems of above approaches and if there are any better approaches than this.
Hangfire is really a great solution for processing background tasks, and we have used used it extensively in our projects.
We have setup our MVC application on separate IIS servers which is also a hangfire client and just enqueues the jobs needs to be executed by hangfire server. Then we have two hangfire server instances, which are windows service application. So effectively there is no load on the MVC app server to process the background jobs, as it is being processed by separate hangfire servers.
One of the extremely helpful feature of hangfire is its out of the box dashboard, that allows you to monitor and control any aspect of background job processing, including statistics, background job history etc.
Configure the hangfire in application as well as in hangfire servers
public void Configuration(IAppBuilder app)
{
GlobalConfiguration.Configuration.UseSqlServerStorage("<connection string or its name>");
app.UseHangfireDashboard();
app.UseHangfireServer();
}
Please note that you use the same connection string across. Use app.UseHangfireServer() only if you want to use the instance as hangfire server, so in your case you would like to omit this line from application server configuration and use only in the hangfire servers.
Also use app.UseHangfireDashboard() in instance which will serve your hangfire dashboard, which would be probably your MVC application.
At that time we have done it using Windows Service, but if had to do it now, I would like to go with Azure worker role or even better now Azure Web Jobs to host my hangfire server, and manage things like auto scaling easily.
Do refer hangfire overview and documentation for more details.
Push messages to MSMQ from your MVC app and have your windows services listen (or loop) on new messages entering the queue.
In your MVC app create a ID per message queued, so make restful API calls from your windows services back to the mvc app as you make progress on the job?
Have a look at Hangfire, this can manage background tasks and works across VMs without conflict. We have replaced windows services using this and it works well.
https://www.hangfire.io
Give a try to http://easynetq.com/
EasyNetQ is a simple to use, opinionated, .NET API for RabbitMQ.
EasyNetQ is a collection of components that provide services on top of the RabbitMQ.Client library. These do things like serialization, error handling, thread marshalling, connection management, etc.
To publish with EasyNetQ
var message = new MyMessage { Text = "Hello Rabbit" };
bus.Publish(message);
To subscribe to a message we need to give EasyNetQ an action to perform whenever a message arrives. We do this by passing subscribe a delegate:
bus.Subscribe<MyMessage>("my_subscription_id", msg => Console.WriteLine(msg.Text));
Now every time that an instance of MyMessage is published, EasyNetQ will call our delegate and print the message’s Text property to the console.
The performance of EasyNetQ is directly related to the performance of the RabbitMQ broker. This can vary with network and server performance. In tests on a developer machine with a local instance of RabbitMQ, sustained over-night performance of around 5000 2K messages per second was achieved. Memory use for all the EasyNetQ endpoints was stable for the overnight run
how does Quartz.net work in the background. suppose i have created a job which would get triggered next year. so what's happening in the background for one year. . Is there a process or thread which keeps running continuously for one year.does it implement timer in the background?
You need to start the Quartz scheduler. As long as the scheduler is running, Quartz will keep track of schedules and start jobs according to their schedule. But as Quartz is a library hosted inside your application, you have to start the Quartz scheduler yourself.
In scenarios where you don't have an application that is always active (like a website), you have to find some way to let the Quartz scheduler run independent of that (for instance in a scheduler service dedicated to running your jobs). I've found a few references to using Quartz.net in ASP.Net projects, but I have no personal experience with running Quartz within a website. I don't know how Quartz would react to IIS shutting down or reusing the application pool your site is running in. But if your job can handle being aborted, it may be a viable option. You'd have to start the scheduler in the Application_Start event in your Global class.
No matter how you run Quartz, you will want to use a non-volatile job store. Quartz can store the scheduled jobs in memory, but you'd have to re-schedule all jobs after each application start. So it is highly recommended to use another job store that can persist jobs between application restarts.
The Quartz.net quick-start tutorial has bits of code to show you how to use Quartz in your application.
I want to Scheduling in Asp.net
I have following options to implement this
To write SQLServer JOB(I dont want to do this.Dont want to go outside of .Net environment)
Second option is I will write windows service and this window service will call asp.net
webservice then this webservice calls asp.net method
(I also dont need to do this because my hosting provider might not be allow me to install
window service)
Third option is I call my scheduling method in Application_Start event in global class
(Drawback is, webserver will kill thread any time )
To call Scheduling Code in Page_Load event of Home Page(Might be nobody visits my website for hours
,Also page execution might be slow due to scheduling code)
I also found some online services that calls your page at given interval,some are given below
http://www.cronservice.co.uk/new/
http://scheduler.codeeffects.com
Anybody give me bettor solution of this and also explain why it is bettor?
Thanks in Advance
The ASP.NET application isn't the right place to implement scheduling. I would suggest creating a service or a scheduled task that runs in short intervals.
You don't have many options in a shared hosting environment. My host (WinHost) allows remote access to their database, so I was able to create an executable that ran on a local server with Task Scheduler.
The performance isn't great since the database is accessed over the internet, but it's still better than attempting to run pseudo scheduled tasks with ASP.NET.
Some hosts also offer a service that will request a url within your site on a scheduled basis. However, this didn't work for me because the task I had to run took longer than the request timeout.
There is no one solution that fits all. SQL jobs and windows jobs (scheduled thru windows task scheduler) are very widely used. In one of my previous work places they had jobs that ran on multiple platforms (mainframe,windows,sql server). Failure in some of these jobs, would cost in thousands by the day. So they employed something called ESP. This software monitored jobs on all platforms and sent a message to the control room in case of a failure.
If you throw some more light on the requirement, we might be able to help you better.
ASP.NET is not the right place to house your Scheduled Tasks. I'm using Quartz.net when I have to create Scheduled Tasks.
Create a page that launches your task and place it at the URL http://www.mydomain.com/runtask.
Create a scheduled task on your home PC that sends a request to http://www.mydomain.com/runtask.
You'll need to keep your home PC on all the time.
Ideally I would go with number 1 as you get full control/history/error reporting etc. You can write an SSIS job in .NET and have SQL server schedule it.
However, I have had a similar problem with shared hosting that is very restrictive. What I did was create a page which runs the process on page load (using validation in the querystring for security). I then used a machine I have which is always on to schedule a Windows Task Scheduler (this is part of Windows as standard) to call a bit of VB script that opens the browser and then shuts it.
I have several important Quartz events that MUST go off at specific times of the night. Lately I have been noticing that not all the events are run. I have a feeling that overnight our server load is very light (ie. zero users) and that the web server kind of goes to sleep, and hence so does Quartz. Does this seem plausible? I am using Quartz.net within the web server, and not as a separate service.
Yes, that is plausible. In general, it is considered a bad practice to have IIS run scheduled tasks, that is a job better left for a Windows Service, or the built-in Windows Scheduled tasks (which has been much improved for Windows Server 2008).
Your worker process might have been shut down because there are no load. By default, IIS shuts down worker processes after 20 minutes without ingoing requests (you can alter this in the Application Pool settings). Also, worker processes are likely to be restarted after a certain amount of time or requests, or if they consume too much memory.
A quick-fix for your specific problem might be to use the Windows Task Scheduler to request the site periodically to keep it alive - or have it request a URL that triggers your task at the predefined time.
ASP.Net, by default, will shut down AppDomain's after a period of inactivity.
The recommended course of action is to implement timed events either in a Windows Service or as an executable launched from Window's Scheduled Tasks.
It is also possible to change the IIS configuration so that it will not shut down your AppDomain. How exactly this is accomplished varies between versions of IIS, but instructions can easily be found by searching.
My application that runs on Windows Azure processes incoming requests from a user (which are put into an Azure Queue) and assigns them to real-world people.
The people have a certain amount of time to handle the request. If none of the people assigned handle the request, I need to move on to a new set of people. Basically, I want to queue these tasks to be handled at a certain time, and then handle them again. If one of the users handles the task, I need to dequeue it so it isn't handled again by the worker.
You need to use a scheduled task. There are two good libraries out there that you could use:
Quartz.Net and
Castle scheduler.
With a scheduledler, such a task becomes easy.
You just create a job that runs when the processing time expires. There you would check for any unprocessed requests and if there are any left, you notify the next set of people and set another scheduled start to fire after processing time expires.
Let me know if you need further detail.
I've used Quartz.net in an azure webrole successfully in a production app.