I am currently experiencing difficulties with running a long HTTP request in a Web Job. Same application runs fine when started on local machine. Since I've tried multiple ways of trying to make this work I am now curious if this is possible at all.
My testing code below:
static void Main(string[] args)
{
/// [...]Getting configuration and using it in "SendPayload"
_stopwatch = new Stopwatch();
_stopwatch.Start();
Task.Run(async () => { await SendPayloadAndAwait(configuration); });
while (_isWaiting)
{
if (_stopwatch.ElapsedMilliseconds % 5000 == 0)
{
Console.WriteLine(".");
}
}
}
public static async Task<string> SendPayloadAndAwait (RequestModel targetRequest)
{
/// [...]Preparing client and payload
Console.WriteLine("Start");
Response= curClient.SendAsync(req).Result;
var endContent = Response.Content.ReadAsStringAsync().Result;
Console.WriteLine("End");
_isWaiting = false;
return "Done";
}
The "End" is never printed in Web Job console, it is an infinite (up to the point of timeout) loop of ".". Is such scenario somehow prohibited?
The way I solved at a previous company was by having the long running job in a Web API which was implemented as an Azure Function was
First call (i.e. /api/StartJob)
creates a job entry (in azure table)
starts backgound thread
returns a 202 [Accepted] status with the job entries Id
Background thread performs long running task updating job entry with it's progress
Client Loop (while percentage done < 100)
Requests status (i.e. /api/JobStatus)
If percentage done = 100 exit loop
This was very successful and reliable providing we had a paid for plan
The free plans kill the session after approx 5 minutes (by design)
You will need to enable the "Always on" setting to make sure your long-running job doesn't get terminated.
The web app times out after 20 minutes of inactivity, and only the requests to the actual web app resets this timer. The web jobs are executed asynchronously in a separate process, so they don't keep the web app "awake".
Here's a note from the web jobs documentation:
A web app can time out after 20 minutes of inactivity. and only requests to the actual web app can reset the timer. Viewing the app's configuration in the Azure portal or making requests to the advanced tools site (https://<app_name>.scm.azurewebsites.net) doesn't reset the timer. If you set your web app to run continuous or scheduled (timer-trigger) WebJobs, enable the Always on setting on your web app's Azure Configuration page to ensure that the WebJobs run reliably. This feature is available only in the Basic, Standard, and Premium pricing tiers.
Related
I am trying to find the appropriate way to setup a long running api task using Asp.Net Web API. The tasks I am running could take up to 3 minutes to complete. I have found several links such as QueueBackgroundWorkItem or this post here but am unsure on a few things.
Question:
So I guess I am most confused about the overall structure for this and the appropriate way to even go about it. My task seems to long to use a QueueBackgroundWorkItem but I don't want to implement a fire and forget API call.
What I would like:
I would like to be able to fire a web API task, return a OK response, but continue to poll for the returned data after firing the task. I just have no idea how to even begin implementing a queue system or even how to poll for that data from the Web API.
Other Information:
Using .Net Framework 4.5.2
Front end is asp.net MVC web app
Web API method is called through an AJAX call on my front
Currently send about 30 requests at a time with a long time out. Then as they return I update my UI with returned data from the API.
My current code:
This is what I am doing now which works fine. The only issue is I have to set the timeout to some ridiculous amount on my UI application. Which is why I am trying to find the correct way to implement a long running task/polling/queueing mechanism for long running tasks. Overall just unsure what is out there for solving the problem below.
public NewProjectLogDTO CreateProject(string modelNumber, string orderName = "")
{
try
{
string EplanPort = ConnectToEPLAN();
SetContext(modelNumber, orderName);
AddSchematicToDB(modelNumber, orderName); //accesses DB
ExecuteCreateProjectAction(); //Really long running task about 3 minutes long
DecreasePortCounter(EplanPort);
AddMacrosToDb(modelNumber, orderName); //accesses DB
return GetNewProjectLogFromContext(modelNumber,orderName); //return data from long running task
}
catch (Exception ex)
{
}
}
So I am using Quartz Scheduler for my ASP.NET MVC application in order to schedule a job for writing a file. This job can be triggered hourly, daily, weekly etc depending on user requirements when he configures the job from a webpage.
Code in this link:
How to check if a particular job is running in quartz scheduler c#
However I discarded my stop() method to make the configured job run forever. I am able to get the jobs running successfully locally, but I can see the file job executing only after I refresh the web application running at IIS server.
At first, I was calling the start() of my job scheduler at Application_start() of Global.asax.cs file in order to resume the job when web application starts.
public class MvcApplication : System.Web.HttpApplication
{
protected void Application_Start()
{
using (var db = new M3DBContext())
{
//db.Configurations is a table in my db
var file_loc = (from c in db.Configurations where c.ConfigurationKey == "file_location" select c.ConfigurationValue).Single();
var freq = (from c in db.Configurations where c.ConfigurationKey == "frequency" select c.ConfigurationValue).Single();
var delim = (from c in db.Configurations where c.ConfigurationKey == "delimiter" select c.ConfigurationValue).Single();
var datetime = (from c in db.Configurations where c.ConfigurationKey == "datetime" select c.ConfigurationValue).Single();
//System.Diagnostics.Debug.WriteLine(file_loc + " " + freq + " " + delim + " " + datetime);
JobScheduler js = new JobScheduler(datetime,freq);
if (!(file_loc.Equals("default") || freq.Equals("default") || delim.Equals("default") || datetime.Equals("1970-1-1")))
{
js.start();
}
}
}
But the main requirement is to make the scheduled job run forever at configured intervals in spite of web application being open/close/refreshed/unrefreshed. My job doesn't run when the web applciation is closed and the IIS is still on.
How can I make my application's quartz scheduler to run forever in IIS inspite of web application is opened/closed/refreshed/unrefreshed?
Application_Start is only run after the first HTTP request and, indeed, after a IIS recycling, JobScheduler will not be run until a new request triggers the web server. You cannot change this behaviour.
However, you can make IIS run a default request after the recycling by enabling preloadEnabled on IIS setting of your web site. It will trigger Application_Start.
This is the exact reason why scheduling things in a web host is dicey at best, and honestly not recommended. If you have jobs that need to run on a regular schedule, regardless of web traffic, the best option is always a stand-alone service to handle the background jobs. It can live on the same server, but services are designed with high uptime, and are tailor made to stay alive and execute things in the background.
Having said that, if you for some reason have to have this as a part of your web application, then you will need to turn on the feature to have the application always running. As mentioned above you will want to turn on preload and turn on always on mode of IIS https://blogs.msdn.microsoft.com/vijaysk/2012/10/09/iis-8-whats-new-application-pool-settings/
The only alternative to this is have an outside service access a page(any page) of the application on a regular basis, or on server startup. This could be accomplished as a scheduled task/cron job with a simple curl command to load a page.
In my ASP MVC 5 app I have this database related operation that I've to perform once in month maybe and it takes about 30 - 60 minutes.
I start this action like this:
Repository dbForCategories = new Repository();
dbForCategories.Database.CommandTimeout = 60000;
var t = Task.Factory.StartNew(async delegate
{
var crs = new ProductCategoryRelationsSetter(dbForCategories, categoryService);
crs.AddProductsCategoriesRelations(oneLineTree);
}, CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default);
After about 5 minutes of working in background Im getting logged out of application. I think that the app resets because some static variables Im using are reset.
In elmah i don't have any errors. I put my code in Try Catch block.
Only hope in you guys:)
As #David mentioned, it's probably best to go the Windows Service route:
Write the item to a database table
Let a Windows Service poll the table every month (or however often you need it to).
If it finds a new item in the queue, let the Windows Service perform the operation.
Why not do Background threads in ASP.NET?
The server may recycle Application pool at some point and will not know to wait for your Task on a separate thread. (Discussed in this SO question)
.NET 4.5.2 onward. You can fire and forget short tasks
For interest sake you can use HostingEnvironment.QueueBackgroundWorkItem (see here) and the server will respect the background item and not recyle the App pool while it's busy, BUT ONLY for up to 90 seconds. anything longer and Windows Service is your best bet.
HostingEnvironment.QueueBackgroundWorkItem(ct => yourAsyncWork(ct));
Hangfire is wonderful for stuff like this.
https://www.nuget.org/packages/Hangfire/
I need some guidance on creating and running a Cron Job in asp.net(C#.net) to run every 30 minutes.i have created a class in that i have written code for getting tweets, facebook feeds.
i have created another page in that i have one button to download tweets and save in database.
If i want to get tweets i have to click on sync button every time.
i want to create cron job so that the database will get automatically synchronized with new tweets,facebook feeds.
Thanks
You can follow any one of the following approaches
Create a console app with the logic to fetch the tweets and feeds, and use a Task scheduler to run it for every 30 mins.
You could build a windows service, which polls the feeds within a timer and updates the db.
You could checkout this scheduler which is a rough equivalent to cron jobs. Personally haven't tried it. Check out this SO
If your intended 30-minute scheduled task is meant to be a discrete transactional action (like, for instance, your example of synchronizing some database values), then you may want to take a look at the Revalee open source project.
You can use it to schedule web callbacks at specific times. In your case, you could schedule a web callback (30 minutes in the future). When your ASP.NET application receives the callback, it can schedule the next 30 minute callback as well as perform whatever tasks you need it to handle every half-hour. When your ASP.NET application launches for the very first time, then you would schedule the first web callback. Since your web application is being called back, you do not need to worry about your web application unloading (which it will do periodically on IIS, for example).
For example using Revalee, you might do the following:
Register a future (30 minutes from now) callback when your application launches via the ScheduleThirtyMinuteCallback() method (see below).
private DateTimeOffet? previousCallbackTime = null;
private void ScheduleThirtyMinuteCallback()
{
// Schedule your callback 30 minutes from now
DateTimeOffset callbackTime = DateTimeOffset.Now.AddMinutes(30.0);
// Your web service's Uri, including any query string parameters your app might need
Uri callbackUrl = new Uri("http://yourwebapp.com/Callback.aspx");
// Register the callback request with the Revalee service
RevaleeRegistrar.ScheduleCallback(callbackTime, callbackUrl);
previousCallbackTime = callbackTime;
}
When the web scheduled task activates and calls your application back, you perform whatever action you need to do every 30 minutes and you schedule the next callback too. You do this by adding the following method call (CallbackMonitor()) to your Callback.aspx page handler.
private void CallbackMonitor()
{
if (!previousCallbackTime.HasValue
|| previousCallbackTime.Value <= DateTimeOffset.Now.AddMinutes(-30.0))
{
// Perform your "30 minutes have elapsed"-related tasks
// ...do your work here...
// Schedule subsequent 30 minute callback
ScheduleThirtyMinuteCallback();
}
}
To be clear, the Revalee Service is not an external 3rd party online scheduler service, but instead a Windows Service that you install and fully control on your own network. It resides and runs on a server of your own choosing, most likely your web server (but this is not a requirement), where it can receive callback registration requests from your ASP.NET application.
If, however, your 'run every 30 minutes' task is a long running task, then you probably do not want to embed that functionality within your ASP.NET application.
I hope this helps.
Disclaimer: I was one of the developers involved with the Revalee project. To be clear, however, Revalee is free, open source software. The source code is available on GitHub.
.net 4.5, asp.net mvc: What is the best way to run long-lasting process (1-2 minutes) from ASP.NET application giving it should be run in a single-threaded environment, I mean the process is initiated for one user at a time only, executions for all other users have to wait till the current execution is done? The scenario is the following: user clicks button that run some sort of long-lasting calculations, http response returned to the user immediately, then user has to request status of the calculations with separate request manually. Asp.net http session abortion should not lead to the process termination, it should keep going. The process might be run on the same or separate server.
I'll show you how to perform this task with http://hangfire.io – incredibly easy way to perform fire-and-forget, delayed and recurring tasks inside ASP.NET applications. No Windows Service required.
First, install the package through NuGet. If you have any problems, please see the Quick Start guide in the official documentation.
PM> Install-Package Hangfire
Open your OWIN Startup class and add the following lines:
public void Configure(IAppBuilder app)
{
GlobalConfiguration.Configuration.UseSqlServerStorage("connection_string");
app.UseHangfireDashboard();
app.UseHangfireServer();
}
Then write the method that will do the long-running work (I applied attribute to perform only one method at a time):
[DisableConcurrentExecution]
public void LongRunning()
{
// Some processing stuff
}
And then call a method in background as fire-and-forget to respond user immediately:
public ActionResult Perform()
{
BackgroundJob.Enqueue(() => LongRunning());
return View();
}
If you want to notify a user about job completion, consider using SignalR and append the LongRunning method correspondingly.
.Net 4.5.2 adds QueueBackgroundWorkItem that you can use to schedule a task. If you don't control the server (when it's rebooted), the 90 second default delay of appPool shut down won't work (unless you can detect the task didn't complete and run it on another server). For more details see "QueueBackgroundWorkItem to reliably schedule and run background processes in ASP.NET"
I would suggest using a product such as NServiceBus to offload the processing and run it in single threaded mode. The advantage to this is that all requests will be processed in order and the processing can be offloaded from the web server as you don't really want long running processes to happen on a web server.
If you control the server, and need more simplicity that a full framework like Hangfire, you can make a console app (.exe), and make any thing..., then you can call the .exe with Process.Start Method, you can call the .exe from SQL Server to, service, etc.