We have a data transfer between 2 systems that can take upto a few hours. The code for doing this transfer is written in C#.
We would like to trigger the transfer with a WCF web service call.
Ideally we would like the web service call to return at once, with a message "OK job started", and then the job to run on the server untill it is complete.
What is the best way to do this?
The possible problems we see are:
web service timing out before job finished
job stops after returning result
Although not entirely similar to your predicament, I had a similar scenario with my MVC application. There are lots of "Jobs" to do that involve importing data, batch emails, financial processes etc.
Basically, I created a windows service which had a job manager, and obviously the various jobs that could be done. It also ran a light HttpServer. This allowed the main MVC application to talk to the service. One sends a request to start jobs and get the status of all jobs or a particular job (when a job is started it is given a unique ID).
So if I was going to implement it in your case, I'd add a download job which did the actual work, and instigate it from the MVC App via a JSON call. The status of the download could be queried at any time by using the ID passed back from the "StartJob" JSON call. Your main web request is therefore handled and over immediately.
I'd write a console application that does this job and let the Web Service call that application.
Running heavy processes within the web server itself is never a good idea.
You may use background thread on the WebServce call.
Related
I am developing a Windows Service Application for sending bulk emails to the customers. In my case, I need to manually start and stop the service from the ASP.Net MVC web application passing some parameters.
Since this is on demand and there is no schedule for this activity, I want the service to perform a long-running task (Sending bulk emails to 100K customers) only once, once started and then stop itself once completed.
Is this approach correct?
If yes, how I could track its progress in MVC application?
Thank You,
This approach is incorrect. A "Windows Service" is a kind of application that waits for some client to call it, or an event to happen. (It is called daemon in Unix/Linux jargon)
In your case, "the service" should be an ordinary program that does batch processing. So, just start a separate process with Process.Start(). It doesn't block anything in your ASP.NET MVC application, it just does its job and terminate.
There are two ways to track the progress. You can call Process objects methods like WaitFor() with a timeout to see if it completes its job. Also, your "service" application would probably write its progress into a file, and the MVC application can read it anytime to see what's happening.
I'm learning basic web application development using Microsoft WebAPI. I've created a drinks ordering service where users order drinks, post it to the server, and the server stores them for later.
Users post their order to a RESTful endpoint on an OrdersController. The endpoint stores the order in a list and checks the list against a condition. E.g, "Are number of orders > 5?"). If satisfied, the server sends a push notification to the users' mobile phones using the Google Cloud Messaging service.
I wish to expand the type of conditions that could trigger a push message. For example, "If no orders have been received in the last 5 minutes, send a push notification". In other words, I would need to check the condition more often than just in response to receiving a new order request.
What is the best way to accomplish this? My initial thought was just to create a Timer which runs the condition checking method at intervals, but a search of stack overflow has suggested that this kind of approach might be a bad idea.
I have several ideas depending on the hosting environment:
If it's your server (virtual or otherwise)
Create a seperate application that runs as a windows service or a windows application that handles that process seperately from the webAPI.
If running on Azure as a Cloud Application you can create a web worker role that is scheduled to run every so many minutes and could then query and process items.
If you only have a hosting enviornment for MVC/WebAPI then you could look into the cache/callback trick. You basically add an entry to the cache when a callback to a method.
a. Create a controller method that adds a cache entry with a callback to a method
b. The method does whatever work it need to do then calls the controller action so another entry is placed in the cache.
Each time the cache times out, it calls the callback method which you can use to process information and then call the controller method to start the process over again.
When I was experimenting with this, I created a small scheduling project that looped through a list of tasks. Each task was responsible for determing if they any processing was needed. Amazingly enough it worked well and the server never shuts down the process.
Keep in mind if your hosting in the cloud that this will simulate activity and memory usage which could cost you some amount of money, although I would think it would be trival.
I've heard they are scheduling solutions, possible even a monitoring service that can call a WebAPI endpoint every so many minutes which could work much like the callback method. You could check for the need to process each time the monitoring service calls the endpoint.
You can use Reactive Extensions: it lets you program time-related events in a declarative manner without having to bother with handling timeouts and tracking. You can even test your setup by controlling time itself!
For example, to trigger an event after some inactivity, you could use the Timeout method of an observable. The Buffer method can let group trigger an event after x drinks have been ordered, etc
In general what you describe sounds like a backend task/responsibility. You can consider this library HangFire.io to safely perform this operation from ASP.NET - http://hangfire.io/
The other way you could simulate event firing from with ASP.NET for simple one-off cases is , is you can you can use the built-in cache and its expiration event.
Once a day, I want my ASP.NET MVC4 website, which may be running on multiple servers, to email a report to me. This seems like a pretty common thing to want to do, but I'm having a tough time coming up with a good way to do it.
Trying to run a timer on a server to do this work is problematic for a couple of reasons. If I have multiple servers then I'd have the timer running on all of them (in case a server goes down); I'd need to coordinate between them, which gets complicated. Also, trying to access the database via Entity Framework from a background thread adds the complication that I must employ a locking strategy to serialize construction/disposal of the DbContext object between the periodic background thread and the "foreground" Controller thread.
Another option would be to run a timer on the servers, but to have the timer thread perform a GET to a magic page that generates and emails the report. This solves the DbContext problem because the database accesses happen in a normal Controller action, serialized with all of the other Controller accesses to the database. But I'm still stuck with the problem of having potentially more than one timer running, so I'd need some smarts in the Controller action to ignore redundant report requests.
Any suggestions? How is this sort of thing normally done?
You should not be doing this task from your web application as Phil Haack nicely explains it in his blog post.
How is this sort of thing normally done?
You could perform this task from a Windows Service or even a console application that is scheduled to run at regular intervals using the Windows Scheduler.
The proper solution is to create a background service that runs independently of your website. However, if that is not an option there is a hack where you can use the cache as explained in Easy Background Tasks in ASP.NET by Jeff Atwood.
A few options:
If you are hosting on Azure as a Website, check out WebJobs which was released recently in preview (http://azure.microsoft.com/en-us/documentation/articles/web-sites-create-web-jobs/)
If you don't want the pain of extracting out your email logic outside of the website, expose that functionality at a url (with a handler, mvc action, etc.) and then run a Windows Scheduled task that hits that url on a schedule.
Write a simple console app that is executed similarly via a Windows Scheduled task.
Or write a simple Windows Service that internally is looping and checking the time and when reached, hits that url, runs that exe, or has it's own code to send you the email.
I would recommend running Quartz.NET as a Windows Service:
Quartz.NET - Enterprise Job Scheduler for .NET Platform
There's boilerplate code for a Windows Service in the download.
I have a data loading application that has to be executed multiple times per day at irregular intervals. I am planning to write a service to kick off the downloads and import the data to a database server. Are there advantages to using a standard service over a webservice or vice versa?
I think you're missing the point here.
Web Services typically are used for a form of communication or remote execution. You call a remote function on a web-service to either adjust the behavior of the machine it's running on or to retrieve data from it.
Windows Services are background processes that run on a machine without any "logged on user" being required. They can perform tasks and do things while the user is at the login screen, or do elevated operations. You can talk to services to adjust their behavior or retrieve information, but it's general purpose is different than a webservice.
The biggest notable difference here is that web-services must be called, they don't run on their own.
For your application I would suggest using a Windows (Standard) Service, as you can have it execute code once per day. I would only use a web-service if you've got something else to automate the calls to the web-service and you require a response from the server detailing it's execution result (success/fail/warning/etc...)
You could also consider writing a normal (windows or console) application that is triggered by a Windows Scheduled Task. What you've described doesn't necessarily sound like something that would require a service.
Sounds like a good use of a windows service to me. Off the top of my head, I'd use a windows service if:
1. Work is performed on a scheduled basis (regular or irregular intervals) in the background;
2. No interaction is needed - work is just done in the background and kicks off based on polling or some other type of trigger (message dropped in queue, database value trigger, scheduled timespan, etc.);
3. Needs to be monitored (either starts/stops along with logging) and you can take advantage of WMI, perfmon and event log with little effort.
A web service is better for tasks that are interactive (like if you wanted to initiate the download based upon a request received).
Sounds like a windows service is the approach you should take.
Hope this helps. Good luck!
If "irregular interval" does mean, the application is invoked by another application, I would use a web service.
If the action is scheduled, I would use a windows service.
If you are working with SQL Server (scheduled or not), I would also consider SQL Server Integration Services.
Scenario: A WCF service receives an XDocument from clients, processes it and inserts a row in an MS SQL Table.
Multiple clients could be calling the WCF service simultaneously. The call usually doesn't take long (a few secs).
Now I need something to poll the SQL Table and run another set of processes in an asynchronous way.
The 2nd process doesn't have to callback anything nor is related to the WCF in any way. It just needs to read the table and perform a series of methods and maybe a Web Service call (if there are records of course), but that's all.
The WCF service clients consuming the above mentioned service have no idea of this and don't care about it.
I've read about this question in StackOverflow and I also know that a Windows Service would be ideal, but this WCF Service will be hosted on a Shared Hosting (discountasp or similar) and therefore, installing a Windows Service will not be an option (as far as I know).
Given that the architecture is fixed (I.E.: I cannot change the table, it comes from a legacy format, nor change the mechanism of the WCF Service), what would be your suggestion to poll/process this table?
I'd say I need it to check every 10 minutes or so. It doesn't need to be instant.
Thanks.
Cheat. Expose this process as another WCF service and fire a go command from a box under your control at a scheduled time.
Whilst you can fire up background threads in WCF, or use cache expiry as a poor man's scheduler those will stop when your app pool recycles until the next hit on your web site and the app pool spins up again. At least firing the request from a machine you control means you know the app pool will come back up every 10 minutes or so because you've sent a request in its direction.
A web application is not suited at all to be running something at a fixed interval. If there are no requests coming in, there is no code running in the application, and if the application is inactive for a while the IIS can decide to shut it down completely until the next request comes in.
For some applications it isn't at all important that something is run at a specific interval, only that it has been run recently. If that is the case for your application then you could just keep track of when the table was last polled, and for every request check if enough time has passed for the table to be polled again.
If you have access to administer the database, there is a scheduler in SQL Server. It can run queries, stored procedures, and even start processes if you have permission (which is very unlikely on a shared hosting, though).
If you need the code on a specific interval, and you can't access the server to schedule it or run it as a service, or can't use the SQL Server scheduler, it's simply not doable.
Make you application pool "always active" and do whatever you want with your threads.