Develop Reference

Long running Web API task

I am trying to find the appropriate way to setup a long running api task using Asp.Net Web API. The tasks I am running could take up to 3 minutes to complete. I have found several links such as QueueBackgroundWorkItem or this post here but am unsure on a few things.
Question:
So I guess I am most confused about the overall structure for this and the appropriate way to even go about it. My task seems to long to use a QueueBackgroundWorkItem but I don't want to implement a fire and forget API call.
What I would like:
I would like to be able to fire a web API task, return a OK response, but continue to poll for the returned data after firing the task. I just have no idea how to even begin implementing a queue system or even how to poll for that data from the Web API.
Other Information:
Using .Net Framework 4.5.2
Front end is asp.net MVC web app
Web API method is called through an AJAX call on my front
Currently send about 30 requests at a time with a long time out. Then as they return I update my UI with returned data from the API.
My current code:
This is what I am doing now which works fine. The only issue is I have to set the timeout to some ridiculous amount on my UI application. Which is why I am trying to find the correct way to implement a long running task/polling/queueing mechanism for long running tasks. Overall just unsure what is out there for solving the problem below.
public NewProjectLogDTO CreateProject(string modelNumber, string orderName = "")
{
try
{
string EplanPort = ConnectToEPLAN();
SetContext(modelNumber, orderName);
AddSchematicToDB(modelNumber, orderName); //accesses DB
ExecuteCreateProjectAction(); //Really long running task about 3 minutes long
DecreasePortCounter(EplanPort);
AddMacrosToDb(modelNumber, orderName); //accesses DB
return GetNewProjectLogFromContext(modelNumber,orderName); //return data from long running task
}
catch (Exception ex)
{
}
}

can there be a stale EF model in IIS application pool?

We made a very minor change to our Entity Framework Code First model by adding a column to an already existing Entity called "EntityLog". After deployment we verified that the data was going into the EntityLog along with the data for that new column.
We are using WebBackgrounder, which schedules a web based background job as part of the web application and runs every few minutes and detects if there is something to be recorded to the EntityLog and logs it. It has an Execute method which creates an async Task which in turn creates a repository to the EF model.
Around 10 hours after deployment the application started throwing exceptions when the WebBackgrounder scheduler would try to log to EntityLog saying the new column was invalid. The web application itself was fine, and we could trigger actions in it that also logged to Entity Log, including the new column. It's just the Background scheduler that kept failing. And it didn't consistently fail. It succeeded sometimes.
I couldn't really figure out what was going on except from the stack trace I could see what exactly it was trying to put in the Log. It stopped throwing exceptions around 15-16 hours after that and I could see that it had finally logged to the EntityLog.
The whole web application runs under a single application pool. So it's not as if the Background job is running in another application pool. The weird part to me is that the Background job had succeeded in putting other messages into the Database earlier. My question is, could there have been a worker process in the application pool that had a stale DBContext that it kept thinking the new column in the table was invalid? Has anybody else experienced this?
EDIT: I was talking about async tasks provided in .net System.Threading.Tasks:
public override Task Execute()
{
return new Task(() =>
{
try
{
_logger.Debug("Background processing started");
using (var repository = RepositoryFactory.CreateRepository())
{
var cacheProvider = new CacheProvider(new MemoryCacheAdapter());
using (var managerContainer = new ManagerContainer(repository, cacheProvider))
{
try
{

Creating new AppDomain calling method in same class as the AppDomain in made

I want to start below potentially long running thread in it's own AppDomain to prevent the webserver from aborting it during recycling. It compiles fine, however during runtime I get this cryptic error
Type is not resolved for member 'MyCore.MyWebService,MyCore,
Version=5.0.0.0, Culture=neutral, PublicKeyToken=null'.
How do I find out what member is not resolved?
Are there any better ways running a long standing thread in a MVC business service layer, that does not get aborted by the server recycling mechanism?
Here is the code:
namespace MyCore
{
[Serializable]
public class MyWebService : IMyWebService
{
AppDomain domain = AppDomain.CreateDomain("Domain");
Thread.CurrentThread.Name = "MVCThread";
domain.SetData("lDatabaseID", lDatabaseID);
domain.DoCallBack(() =>
{
long lID = Convert.ToInt64(AppDomain.CurrentDomain.GetData("lDatabaseID"));
Thread thread = new Thread(
(() =>
{
PopulateTables(lID );
}));
thread.Name = "DomThread";
thread.Start();
});
}
}

IIS is heavily optimised to respond very quickly to hundreds of small simultaneous requests and just isn't the right tool for what you're attempting. You can try to work around that but in the long term you'll be better off building a tool that is designed for long-running tasks. You've then got a pre-packaged solution the next time this problem arises.
The basic idea is to create an external application that does your background processing with some way to pass tasks to it and get results back. I like using the database to communicate as most web applications that need baground processing already use a database. Add a 'tasks' table with {status, startedDateTime, finishedDateTime, parameters, etc}, then write an external application that will periodically look for a new task, complete it and update the database. Your web site can poll the database for status or your application could make an AJAX call to notify the web site when a job has completed (a small iframe in the web site header that shows waiting / completed tasks can be useful if someone will be waiting for the job to complete and is easy to do).
EDIT: Before you do the above review HangFire (which works inside IIS, as a Windows Service or as a console app). Same principles, but a pre-packaged solution. Note that I haven't implemented this yet but it looks good.
Although it's a bit of work to set up, handing this task off to a Windows Service is a good approach if you might have multiple tasks and need them responded to quickly. There are a lot of tutorials on the web that will help you create a Windows Service, such as http://www.codeproject.com/Articles/106742/Creating-a-simple-Windows-Service but you'll have to build a simple task executor on top of that so if that's the way you want to go I'd look for a pre-built task engine (I couldn't find one quickly but I'm probably using the wrong search phrase).
But that's overkill if turn-around time isn't important and a better approach for you might be to create a small console application that will be started every five minutes by task scheduler. It would connect to the database, execute any waiting tasks then shut down again. This is easier to debug and install than a Windows service and achieves the same goal of moving the task execution out of IIS.
Remember that you still have to detect and handle Windows shutdown so that you don't get half-finished orphaned jobs - at the very least just tag that task as aborted and exit cleanly.

Alright after having mucked with Hangfire, I finally got it to work in .Net 4.0 and MVC 3. Had to install Common.Logging.Core 2.2.0, since the NuGet installed the wrong version (3.3.0)
In my Initial controller I added the following
namespace Core.Controllers
{
...
public void Configuration(IAppBuilder app)
{
app.UseHangfire(config =>
{
config.UseSqlServerStorage(ConnectionString.GetTVConnectionString());
config.UseServer();
});
}
...
}
ConnectionString.GetTVConnectionString() gets the connection string from the config file.
Up top I added the following
[assembly: OwinStartup(typeof(Core.Controllers.BaseController))]
In the code that starts the background thread I added the following, passing in a long instead of the class and having the job load the POCO class from the db.
BackgroundJob.Enqueue(() => PopulateTables(lDatabaseID, JobCancellationToken.Null));
The Enqueue() function returns a job id, that later can be used to cancel the job if needed, through the BackgroundJob.Delete(jobid) function.
In the job method I then have this
while (idxMin < max)
{
try
{
cancellationToken.ThrowIfCancellationRequested();
....
}
catch (JobAbortedException jobEx)
{
....
}
}
It's important to use dependency injection, so my class had a parameter less constructor added that re-reads the connection string rather than have it passed in.
public MyWebService ()
: this(ConnectionString.GetTVConnectionString())
{
}
public MyWebService (string sConnStr)
{
msConnStr = sConnStr;
}
After that it seems to run pretty well. A number of tables are added to the database specified in the connection string. So far it seems like the jobs survive recycling on the webserver.

MVC 3 - Do some task at interval time

I am developing an web app with ASP.NET MVC3.
The requirement is: At some time interval (0:00AM or 10:00pm), the app must automatically do some task such as: Change order status in DB, send notify email to customer, clear temp folder....
I'm thinking of using a thread in Global.aspx, let it run and sleep for 24 hours, but I don't think that is a good resolution.
#Thank to patryk-wiek comment, I find a solution with " Task Scheduler MVC" keywork here.
https://github.com/jgeurts/FluentScheduler
Is that a good solution or not? I think writing a window service is a bit overkill for me?

Don't do it in your web application as you may not be in control of when the host process times out and goes!
Create a windows service to perform these tasks.
http://msdn.microsoft.com/en-us/library/d56de412.aspx

you can use RX for the same by creating a window service for the same.. as i have attached one sample code below which would suscibe first and will do the operation in regular interval of 2 minutes.:-
Observable.Generate(
true,
_ => true,
i =>
{
////your code...
return i;
},
i => i,
_ => TimeSpan.FromMinutes(2)).Subscribe();

Don't do it inside your web app, becasue the AppPool can be terminated at any time. There are ways how to prevent this, but it's not very clean solution.
Use windows scheduler, WCF service or some workflow solution for log running tasks.

Would like to execute a query once per day in ASP.NET MVC

I would like my ASP.NET MVC app to execute a query once per day. What is the recommended way to do this?
My first thought is to put a timer in Global.asax that goes off every 24 hours, then call my query from the Elapsed handler. Any pitfalls with doing it this way? Is there a better way?
Edit
Let me add a little detail to what I'm trying to do. I'd specifically like the query to execute at midnight every day. If a day is missed (say due to sever maintenance or upgrading the app), that wouldn't be a major issue.
Edit 2
Couple more details:
The query is actually an INSERT, not a SELECT. The purpose is to add a "renewal" record for any member that is due to renew his/her membership at the end of the month.
I'm using SQL Server Compact (it's a very small database).

Does it have to originate in the Web layer? Who'd be there to consume the HTML? Typically, periodic SQL queries are scheduled within the database. In case of MS SQL Server - via the SQL Agent job facility. SQL Server can even send e-mail.
RE: edit2: Should've told so right away. SQL Server Compact is not the same as SQL Server - for one, it does not have SQL Agent IIRC. Still, invoking the Web layer is an overkill. I'd use a Windows Scripting Host file (.js) in conjuction with Windows task scheduler. WSH files can connect to databases via ADO and do whatever they want - inserts, selects, anything.
To detect missed scheduled runs, introduce an extra table with a log of scheduled runs. Then on subsequent runs you can analyse the date of the last run and act accordingly.
Edit2: so no administrative access. You should really tell all those details in the question. In this case, I would go through the Web layer after all, but the scheduling would be on MY end - where I do have control. Have Task Scheduler run on your end and invoke an HTTP URL on the server. To invoke URLs, you can use something like the free CURL utility. Running IE in scheduled manner has the disadvantage of leaving the window open.
IIS is not a scheduling engine.
Edit3 re:comment: sorry, I've misunderstood the nature of your setup. My own experiences have clouded my judgement :) Can you just run a check during every logon operation, and if it's been a while since the last maintenance operation, run it right then and there? How long does the maintenance take? If it's ~1min+, makes sense to run it in a worker thread, so that the logging-on user is not made wait.
Scheduling daily maintenance is a good idea in general, and it is implemented fairly often, but it seems you simply don't have the capability.

I do this very thing in my web apps, but use Asynchronous HTTP Handlers (http://msdn.microsoft.com/en-us/library/ms227433.aspx#Y512); I believe this would be recommended. I just start it off on application start and shut it down on application end (Global.asx).
The thing to remember is that you'll probably have to store the last time the query ran in the database because you'll loose track of that when your application pool recycles.

I'm doing this by putting some fake information in "Cache" and put the time period i want then handel the "_onCacheRemove" event do whatever i wanna do then recreate the "CacheItem" again:
e.g.
I put my tasks in Enum with the time that i wanna to rerun this task in seconds:
public enum ScheduledTasks
{
CleanGameRequests = 120,
CleanUpOnlineUsers = 6
}
then deal with them at "Application_Start" :
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
RegisterRoutes(RouteTable.Routes);
// Adding the tasks i want at App_Start
// so if the application restarted my task will refreshed.
AddTask(ScheduledTasks.CleanGameRequests);
AddTask(ScheduledTasks.CleanUpOnlineUsers);
}
// event to handel
private static CacheItemRemovedCallback _onCacheRemove;
private void AddTask(ScheduledTasks task)
{
_onCacheRemove = new CacheItemRemovedCallback(CacheItemRemoved);
HttpRuntime.Cache.Insert(task.ToString(), (int)task, null,
DateTime.Now.AddSeconds((int)task), Cache.NoSlidingExpiration,
CacheItemPriority.NotRemovable, _onCacheRemove);
}
public void CacheItemRemoved(string key, object time, CacheItemRemovedReason r)
{
var task = (ScheduledTasks)Enum.Parse(typeof(ScheduledTasks), key);
switch (task)
{
case ScheduledTasks.CleanGameRequests:
// Do the concept that you wanna to do.
GameRequest.CleanUp();
break;
case ScheduledTasks.CleanUpOnlineUsers:
OnlineUsers.CleanUp();
break;
default:
break;
}
// Don't forget to recreate the "CacheItem" again.
AddTask(task);
}
Note: You may make your time management as you want. In my case i
wanna these tasks to run every period
of time regardless of what time it is.
In your case you should check the time
before then recreate the CacheItem
again.
Hope this helped :)

Unless you have very active site chances are that IIS will bring your application down and there will be no process to execute your task.
Alternatives:
just do that during/immediately after request that is close enough by time
have external task that will trigger the operation on your site via GET/POST.
reconfigure IIS to never recycle/stop your app pool. Than your timer has chance to execute.
use some external service on the server to schedule the task ("at" or even SQL tasks).