.net 4.5, asp.net mvc: What is the best way to run long-lasting process (1-2 minutes) from ASP.NET application giving it should be run in a single-threaded environment, I mean the process is initiated for one user at a time only, executions for all other users have to wait till the current execution is done? The scenario is the following: user clicks button that run some sort of long-lasting calculations, http response returned to the user immediately, then user has to request status of the calculations with separate request manually. Asp.net http session abortion should not lead to the process termination, it should keep going. The process might be run on the same or separate server.
I'll show you how to perform this task with http://hangfire.io – incredibly easy way to perform fire-and-forget, delayed and recurring tasks inside ASP.NET applications. No Windows Service required.
First, install the package through NuGet. If you have any problems, please see the Quick Start guide in the official documentation.
PM> Install-Package Hangfire
Open your OWIN Startup class and add the following lines:
public void Configure(IAppBuilder app)
{
GlobalConfiguration.Configuration.UseSqlServerStorage("connection_string");
app.UseHangfireDashboard();
app.UseHangfireServer();
}
Then write the method that will do the long-running work (I applied attribute to perform only one method at a time):
[DisableConcurrentExecution]
public void LongRunning()
{
// Some processing stuff
}
And then call a method in background as fire-and-forget to respond user immediately:
public ActionResult Perform()
{
BackgroundJob.Enqueue(() => LongRunning());
return View();
}
If you want to notify a user about job completion, consider using SignalR and append the LongRunning method correspondingly.
.Net 4.5.2 adds QueueBackgroundWorkItem that you can use to schedule a task. If you don't control the server (when it's rebooted), the 90 second default delay of appPool shut down won't work (unless you can detect the task didn't complete and run it on another server). For more details see "QueueBackgroundWorkItem to reliably schedule and run background processes in ASP.NET"
I would suggest using a product such as NServiceBus to offload the processing and run it in single threaded mode. The advantage to this is that all requests will be processed in order and the processing can be offloaded from the web server as you don't really want long running processes to happen on a web server.
If you control the server, and need more simplicity that a full framework like Hangfire, you can make a console app (.exe), and make any thing..., then you can call the .exe with Process.Start Method, you can call the .exe from SQL Server to, service, etc.
Related
I'm working on a CRM application and my client wants to download some information within last 6 months. At the moment I have created an API endpoint which returns FileContentResult object and that will open a new tab in browser and automatically download an Excel file.
But this process is time consuming (since it has over 500K data) and users don't wait in the same page until the process is done. So, once an user change between pages I get issues and sometimes the application return timeout error since the API response is slow.
Now, I'm planning to enhance that same function/API endpoint by introducing some silent job. Which means once user click on "Download" button, process will start and it will send a message stating that "Your download process has been started. You will receive an email with the report within next 15minutes". In this way, users don't have to wait and they can do something else in the system.
Currently, I'm using async task and awaits until the job is done.
public async Task<FileContentResult> ExportData()
{
//...
//... process data and create excel file
//...
//...
return new FileContentResult(*some byte array*, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")
{
FileDownloadName = $"Data.xlsx"
};
}
and I'm calling this method by
await exportService.ExportData();
My concern is what are the things I should change here in order to avoid any impact on other processes and run this as a background job. Once I get the result, I will send an email with an attachment.
Please help me with your valuable ideas. Thanks in advance
what are the things I should change here in order to avoid any impact on other processes and run this as a background job. Once I get the result, I will send an email with an attachment.
You should use a basic distributed architecture, as I describe on my blog. Specifically:
Instead of creating the report in your ASP.NET app, your ASP.NET app should just create a message indicating that the report should be created, and place that message into a durable queue.
Have a separate, independent process read the messages from that queue, generate the report, and send the email.
I want to start below potentially long running thread in it's own AppDomain to prevent the webserver from aborting it during recycling. It compiles fine, however during runtime I get this cryptic error
Type is not resolved for member 'MyCore.MyWebService,MyCore,
Version=5.0.0.0, Culture=neutral, PublicKeyToken=null'.
How do I find out what member is not resolved?
Are there any better ways running a long standing thread in a MVC business service layer, that does not get aborted by the server recycling mechanism?
Here is the code:
namespace MyCore
{
[Serializable]
public class MyWebService : IMyWebService
{
AppDomain domain = AppDomain.CreateDomain("Domain");
Thread.CurrentThread.Name = "MVCThread";
domain.SetData("lDatabaseID", lDatabaseID);
domain.DoCallBack(() =>
{
long lID = Convert.ToInt64(AppDomain.CurrentDomain.GetData("lDatabaseID"));
Thread thread = new Thread(
(() =>
{
PopulateTables(lID );
}));
thread.Name = "DomThread";
thread.Start();
});
}
}
IIS is heavily optimised to respond very quickly to hundreds of small simultaneous requests and just isn't the right tool for what you're attempting. You can try to work around that but in the long term you'll be better off building a tool that is designed for long-running tasks. You've then got a pre-packaged solution the next time this problem arises.
The basic idea is to create an external application that does your background processing with some way to pass tasks to it and get results back. I like using the database to communicate as most web applications that need baground processing already use a database. Add a 'tasks' table with {status, startedDateTime, finishedDateTime, parameters, etc}, then write an external application that will periodically look for a new task, complete it and update the database. Your web site can poll the database for status or your application could make an AJAX call to notify the web site when a job has completed (a small iframe in the web site header that shows waiting / completed tasks can be useful if someone will be waiting for the job to complete and is easy to do).
EDIT: Before you do the above review HangFire (which works inside IIS, as a Windows Service or as a console app). Same principles, but a pre-packaged solution. Note that I haven't implemented this yet but it looks good.
Although it's a bit of work to set up, handing this task off to a Windows Service is a good approach if you might have multiple tasks and need them responded to quickly. There are a lot of tutorials on the web that will help you create a Windows Service, such as http://www.codeproject.com/Articles/106742/Creating-a-simple-Windows-Service but you'll have to build a simple task executor on top of that so if that's the way you want to go I'd look for a pre-built task engine (I couldn't find one quickly but I'm probably using the wrong search phrase).
But that's overkill if turn-around time isn't important and a better approach for you might be to create a small console application that will be started every five minutes by task scheduler. It would connect to the database, execute any waiting tasks then shut down again. This is easier to debug and install than a Windows service and achieves the same goal of moving the task execution out of IIS.
Remember that you still have to detect and handle Windows shutdown so that you don't get half-finished orphaned jobs - at the very least just tag that task as aborted and exit cleanly.
Alright after having mucked with Hangfire, I finally got it to work in .Net 4.0 and MVC 3. Had to install Common.Logging.Core 2.2.0, since the NuGet installed the wrong version (3.3.0)
In my Initial controller I added the following
namespace Core.Controllers
{
...
public void Configuration(IAppBuilder app)
{
app.UseHangfire(config =>
{
config.UseSqlServerStorage(ConnectionString.GetTVConnectionString());
config.UseServer();
});
}
...
}
ConnectionString.GetTVConnectionString() gets the connection string from the config file.
Up top I added the following
[assembly: OwinStartup(typeof(Core.Controllers.BaseController))]
In the code that starts the background thread I added the following, passing in a long instead of the class and having the job load the POCO class from the db.
BackgroundJob.Enqueue(() => PopulateTables(lDatabaseID, JobCancellationToken.Null));
The Enqueue() function returns a job id, that later can be used to cancel the job if needed, through the BackgroundJob.Delete(jobid) function.
In the job method I then have this
while (idxMin < max)
{
try
{
cancellationToken.ThrowIfCancellationRequested();
....
}
catch (JobAbortedException jobEx)
{
....
}
}
It's important to use dependency injection, so my class had a parameter less constructor added that re-reads the connection string rather than have it passed in.
public MyWebService ()
: this(ConnectionString.GetTVConnectionString())
{
}
public MyWebService (string sConnStr)
{
msConnStr = sConnStr;
}
After that it seems to run pretty well. A number of tables are added to the database specified in the connection string. So far it seems like the jobs survive recycling on the webserver.
I need some guidance on creating and running a Cron Job in asp.net(C#.net) to run every 30 minutes.i have created a class in that i have written code for getting tweets, facebook feeds.
i have created another page in that i have one button to download tweets and save in database.
If i want to get tweets i have to click on sync button every time.
i want to create cron job so that the database will get automatically synchronized with new tweets,facebook feeds.
Thanks
You can follow any one of the following approaches
Create a console app with the logic to fetch the tweets and feeds, and use a Task scheduler to run it for every 30 mins.
You could build a windows service, which polls the feeds within a timer and updates the db.
You could checkout this scheduler which is a rough equivalent to cron jobs. Personally haven't tried it. Check out this SO
If your intended 30-minute scheduled task is meant to be a discrete transactional action (like, for instance, your example of synchronizing some database values), then you may want to take a look at the Revalee open source project.
You can use it to schedule web callbacks at specific times. In your case, you could schedule a web callback (30 minutes in the future). When your ASP.NET application receives the callback, it can schedule the next 30 minute callback as well as perform whatever tasks you need it to handle every half-hour. When your ASP.NET application launches for the very first time, then you would schedule the first web callback. Since your web application is being called back, you do not need to worry about your web application unloading (which it will do periodically on IIS, for example).
For example using Revalee, you might do the following:
Register a future (30 minutes from now) callback when your application launches via the ScheduleThirtyMinuteCallback() method (see below).
private DateTimeOffet? previousCallbackTime = null;
private void ScheduleThirtyMinuteCallback()
{
// Schedule your callback 30 minutes from now
DateTimeOffset callbackTime = DateTimeOffset.Now.AddMinutes(30.0);
// Your web service's Uri, including any query string parameters your app might need
Uri callbackUrl = new Uri("http://yourwebapp.com/Callback.aspx");
// Register the callback request with the Revalee service
RevaleeRegistrar.ScheduleCallback(callbackTime, callbackUrl);
previousCallbackTime = callbackTime;
}
When the web scheduled task activates and calls your application back, you perform whatever action you need to do every 30 minutes and you schedule the next callback too. You do this by adding the following method call (CallbackMonitor()) to your Callback.aspx page handler.
private void CallbackMonitor()
{
if (!previousCallbackTime.HasValue
|| previousCallbackTime.Value <= DateTimeOffset.Now.AddMinutes(-30.0))
{
// Perform your "30 minutes have elapsed"-related tasks
// ...do your work here...
// Schedule subsequent 30 minute callback
ScheduleThirtyMinuteCallback();
}
}
To be clear, the Revalee Service is not an external 3rd party online scheduler service, but instead a Windows Service that you install and fully control on your own network. It resides and runs on a server of your own choosing, most likely your web server (but this is not a requirement), where it can receive callback registration requests from your ASP.NET application.
If, however, your 'run every 30 minutes' task is a long running task, then you probably do not want to embed that functionality within your ASP.NET application.
I hope this helps.
Disclaimer: I was one of the developers involved with the Revalee project. To be clear, however, Revalee is free, open source software. The source code is available on GitHub.
I would like to run a time consuming script (to update a database from a 3rd party API) at regular intervals. I was wondering what the best practice for this would be:
What should the 'script' be - an ASP.NET service? Bearing in mind I am on shared hosting, this may not be possible (but I would like to know).
How could the script be scheduled to run at regular intervals/at set time automatically?
Thanks in advance!
Some options for this:
Use a separate thread that keeps running all the time - and does the update on time (and then sleeps).
Use a timer and trigger the update event.
Use a Cache expiration trigger, but test this so that it keeps running without users visiting the site.
I would suggest checking out http://www.beansoftware.com/ASP.NET-Tutorials/Scheduled-Tasks.aspx for more details on these methods.
There is no way you can guarantee that something runs e.g. every night in a normal IIS setup. Batch jobs are thus a pain to handle. The only "mode" of execution for IIS is requests. If your application has no requests it doesn't run at all so IIS does not spend any resources on executing code in your application, i.e. it can unload it entirely.
If you have your own host, you would typically create a windows service to run your background tasks. I believe the same is possible in Azure. But for a standard sharesd IIS host, you basically can't setup a scheduled background task.
One of the simplest hacks is to setup a protected service that executes the job when it gets a request. Then you can make sure an external caller calls into your service at the required intervals.
What you can do is add a System.Timers.Timer in Global.asax.
System.Threading.Timer timer = new System.Threading.Timer(new TimerCallback(TimerElapsed), null, new Timespan(0), new Timespan(24, 0, 0));
// This will run every 24 hours.
private void TimerElapsed(object o)
{
// Do stuff.
}
In IISManager, enable HTTP-Keep Alives for your application.
In IIS Manager, select Http Response Headers Module, open it and in the Actions Pane, select Set Common Headers and in there select Enable Http Keep Alives.
Also, check for a setting of your application pool -
Select the application pool of your application, select Advanced Settings from the right Actions Tab.
In there there is a setting called - Idle Timeout (minutes)
By default it is 20 Minutes. Make it something like 60 Minutes or increase it even more and check.
I would like my ASP.NET MVC app to execute a query once per day. What is the recommended way to do this?
My first thought is to put a timer in Global.asax that goes off every 24 hours, then call my query from the Elapsed handler. Any pitfalls with doing it this way? Is there a better way?
Edit
Let me add a little detail to what I'm trying to do. I'd specifically like the query to execute at midnight every day. If a day is missed (say due to sever maintenance or upgrading the app), that wouldn't be a major issue.
Edit 2
Couple more details:
The query is actually an INSERT, not a SELECT. The purpose is to add a "renewal" record for any member that is due to renew his/her membership at the end of the month.
I'm using SQL Server Compact (it's a very small database).
Does it have to originate in the Web layer? Who'd be there to consume the HTML? Typically, periodic SQL queries are scheduled within the database. In case of MS SQL Server - via the SQL Agent job facility. SQL Server can even send e-mail.
RE: edit2: Should've told so right away. SQL Server Compact is not the same as SQL Server - for one, it does not have SQL Agent IIRC. Still, invoking the Web layer is an overkill. I'd use a Windows Scripting Host file (.js) in conjuction with Windows task scheduler. WSH files can connect to databases via ADO and do whatever they want - inserts, selects, anything.
To detect missed scheduled runs, introduce an extra table with a log of scheduled runs. Then on subsequent runs you can analyse the date of the last run and act accordingly.
Edit2: so no administrative access. You should really tell all those details in the question. In this case, I would go through the Web layer after all, but the scheduling would be on MY end - where I do have control. Have Task Scheduler run on your end and invoke an HTTP URL on the server. To invoke URLs, you can use something like the free CURL utility. Running IE in scheduled manner has the disadvantage of leaving the window open.
IIS is not a scheduling engine.
Edit3 re:comment: sorry, I've misunderstood the nature of your setup. My own experiences have clouded my judgement :) Can you just run a check during every logon operation, and if it's been a while since the last maintenance operation, run it right then and there? How long does the maintenance take? If it's ~1min+, makes sense to run it in a worker thread, so that the logging-on user is not made wait.
Scheduling daily maintenance is a good idea in general, and it is implemented fairly often, but it seems you simply don't have the capability.
I do this very thing in my web apps, but use Asynchronous HTTP Handlers (http://msdn.microsoft.com/en-us/library/ms227433.aspx#Y512); I believe this would be recommended. I just start it off on application start and shut it down on application end (Global.asx).
The thing to remember is that you'll probably have to store the last time the query ran in the database because you'll loose track of that when your application pool recycles.
I'm doing this by putting some fake information in "Cache" and put the time period i want then handel the "_onCacheRemove" event do whatever i wanna do then recreate the "CacheItem" again:
e.g.
I put my tasks in Enum with the time that i wanna to rerun this task in seconds:
public enum ScheduledTasks
{
CleanGameRequests = 120,
CleanUpOnlineUsers = 6
}
then deal with them at "Application_Start" :
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
RegisterRoutes(RouteTable.Routes);
// Adding the tasks i want at App_Start
// so if the application restarted my task will refreshed.
AddTask(ScheduledTasks.CleanGameRequests);
AddTask(ScheduledTasks.CleanUpOnlineUsers);
}
// event to handel
private static CacheItemRemovedCallback _onCacheRemove;
private void AddTask(ScheduledTasks task)
{
_onCacheRemove = new CacheItemRemovedCallback(CacheItemRemoved);
HttpRuntime.Cache.Insert(task.ToString(), (int)task, null,
DateTime.Now.AddSeconds((int)task), Cache.NoSlidingExpiration,
CacheItemPriority.NotRemovable, _onCacheRemove);
}
public void CacheItemRemoved(string key, object time, CacheItemRemovedReason r)
{
var task = (ScheduledTasks)Enum.Parse(typeof(ScheduledTasks), key);
switch (task)
{
case ScheduledTasks.CleanGameRequests:
// Do the concept that you wanna to do.
GameRequest.CleanUp();
break;
case ScheduledTasks.CleanUpOnlineUsers:
OnlineUsers.CleanUp();
break;
default:
break;
}
// Don't forget to recreate the "CacheItem" again.
AddTask(task);
}
Note: You may make your time management as you want. In my case i
wanna these tasks to run every period
of time regardless of what time it is.
In your case you should check the time
before then recreate the CacheItem
again.
Hope this helped :)
Unless you have very active site chances are that IIS will bring your application down and there will be no process to execute your task.
Alternatives:
just do that during/immediately after request that is close enough by time
have external task that will trigger the operation on your site via GET/POST.
reconfigure IIS to never recycle/stop your app pool. Than your timer has chance to execute.
use some external service on the server to schedule the task ("at" or even SQL tasks).