My application that runs on Windows Azure processes incoming requests from a user (which are put into an Azure Queue) and assigns them to real-world people.
The people have a certain amount of time to handle the request. If none of the people assigned handle the request, I need to move on to a new set of people. Basically, I want to queue these tasks to be handled at a certain time, and then handle them again. If one of the users handles the task, I need to dequeue it so it isn't handled again by the worker.
You need to use a scheduled task. There are two good libraries out there that you could use:
Quartz.Net and
Castle scheduler.
With a scheduledler, such a task becomes easy.
You just create a job that runs when the processing time expires. There you would check for any unprocessed requests and if there are any left, you notify the next set of people and set another scheduled start to fire after processing time expires.
Let me know if you need further detail.
I've used Quartz.net in an azure webrole successfully in a production app.
Related
I have an Azure hosted MVC Web App where a user can request a report to be generated by pressing a button. This report is a compute intensive, long-running process that I only want to run at night. I have experience using Queue Triggered WebJobs to process background tasks; however, this job will require more resources than my Web App Service plan has and I don't want to run this compute process along side my Web App. My hope is that I can write a queue message for each request and then have something check that queue each night to see if it has any messages. If it does, create/start a new Worker Role instance of sufficient power/memory to handle the job, process the queue message(s), then shut down/and deallocate the worker to prevent ongoing charges.
I can't figure out the best way to check the queue before starting the Worker Role and only create/start the Worker if there is work to be done since it will be a largish instance I want to minimize uptime to keep costs down.
You can use create a triggered WebJob that uses a TimerTrigger that is set to wake up once a day at some early hour like 2:00AM. the method triggered by the TimerTrigger can then peek at the queue to see if a message exists. If one or more messages exist, kick off a worker role that actually dequeues and processes messages.
You could write a Web Job using a Queue Trigger so it's automatically triggered when a new message pops into the queue. Then you can host the Web Job in it's own App Service Plan, separate from the Web App, so it has it's own dedicated resources allocated.
Since you mention that you want to keep costs down, I would actually recommend instead you use an Azure Function. The Azure Function can be setup with a Queue Trigger as well, with the added benefit of only paying for your Azure Function when it is running using the "Consumption Plan" pricing option.
Here's a link that outlines how Azure Functions pricing works:
https://buildazure.com/2016/10/11/how-azure-functions-pricing-works/
Well, Rob's comment got me on to the Service Management library which then took me to Azure Automation and Runbooks. I think that will end up being the solution. I can create a Runbook using PowerShell cmdlets that will peek the storage queue and if it finds any messages it can create/deploy a new instance of the Worker Role, which will look at the queue on startup and start processing any messages. The tricky part looks to be shutting down the Worker Role. The Self-Destruct/suicidal cloud service examples I have found only seem to kill off multiple instances, but not the last instance. Since I will only have one instance running, I don't think I can have it kill itself and get it into a Unallocated state. I think the solution is to use Runbooks again. Have the Worker Role write a "finished" message to another queue and have a scheduled Runbook watch this queue every x minutes. If it finds a message, stop and deallocate the worker role. Given that Functions has a hard run time limit, I can't use that. And cloud services gives me more resource options (more fine-tuned VM types) than anything I could get on WebJobs/Web App Service plans.
I'm learning basic web application development using Microsoft WebAPI. I've created a drinks ordering service where users order drinks, post it to the server, and the server stores them for later.
Users post their order to a RESTful endpoint on an OrdersController. The endpoint stores the order in a list and checks the list against a condition. E.g, "Are number of orders > 5?"). If satisfied, the server sends a push notification to the users' mobile phones using the Google Cloud Messaging service.
I wish to expand the type of conditions that could trigger a push message. For example, "If no orders have been received in the last 5 minutes, send a push notification". In other words, I would need to check the condition more often than just in response to receiving a new order request.
What is the best way to accomplish this? My initial thought was just to create a Timer which runs the condition checking method at intervals, but a search of stack overflow has suggested that this kind of approach might be a bad idea.
I have several ideas depending on the hosting environment:
If it's your server (virtual or otherwise)
Create a seperate application that runs as a windows service or a windows application that handles that process seperately from the webAPI.
If running on Azure as a Cloud Application you can create a web worker role that is scheduled to run every so many minutes and could then query and process items.
If you only have a hosting enviornment for MVC/WebAPI then you could look into the cache/callback trick. You basically add an entry to the cache when a callback to a method.
a. Create a controller method that adds a cache entry with a callback to a method
b. The method does whatever work it need to do then calls the controller action so another entry is placed in the cache.
Each time the cache times out, it calls the callback method which you can use to process information and then call the controller method to start the process over again.
When I was experimenting with this, I created a small scheduling project that looped through a list of tasks. Each task was responsible for determing if they any processing was needed. Amazingly enough it worked well and the server never shuts down the process.
Keep in mind if your hosting in the cloud that this will simulate activity and memory usage which could cost you some amount of money, although I would think it would be trival.
I've heard they are scheduling solutions, possible even a monitoring service that can call a WebAPI endpoint every so many minutes which could work much like the callback method. You could check for the need to process each time the monitoring service calls the endpoint.
You can use Reactive Extensions: it lets you program time-related events in a declarative manner without having to bother with handling timeouts and tracking. You can even test your setup by controlling time itself!
For example, to trigger an event after some inactivity, you could use the Timeout method of an observable. The Buffer method can let group trigger an event after x drinks have been ordered, etc
In general what you describe sounds like a backend task/responsibility. You can consider this library HangFire.io to safely perform this operation from ASP.NET - http://hangfire.io/
The other way you could simulate event firing from with ASP.NET for simple one-off cases is , is you can you can use the built-in cache and its expiration event.
I have written a web application in asp.net. It has some user roles. There are 3 roles which are Manager, Accountant and Employee. The employees write their expenses in a form and send it to Manager. When manager approves it, it'll be sent to Accountant to pay it. I need to have an idea that when manager doesn't approve the employee's expense in 48 hours, it should send an automatic e-mail to Manager's mail.
I thought that I can write another small console application to handle that by checking every hour. But it would waste resources and decrease performance.
I need a good idea to handle that. How should I do?
There are several options, but if I were you I would go with first or second options.
Console App & scheduler
I would create that console application that every time is run perform the check for you.
Then I will have it run using Windows Scheduler in a daily basis (at 00:05) or a hourly basis if you prefer so. This way Windows Scheduler daemon will launch it every hour and the rest of the time your app is not running.
Check this Microsoft link to see how a scheduled task is created in windows.
Restful Web Service & scheduler
As suggested in #marapet answer, having a restful web service that allow you to perform this action instead of a console application would give you the advantage of having all code in your web application.
Similar as previous one, you should only invoke the restful uri to have your action done. As possible disadvantage, you have to get sure that that uri is not accessible to end users. In usual architecture (Web Server --> Application Server --> DB) this restful service should be in the Application Servers, far away from end user access.
Windows Service
Another option is creating a Windows Service that runs all the time and check the time itself so every hour perform the job (maybe using Quartz or similar). But this does not meet your performance requirements.
The performance hit will be small anyway as your service should check every minute to see if an hour has pass and is time to do its job.. a task pretty easy.
The advantage is that a windows service is easier to control and monitor than a Scheduled tasks
DB job
Yet another option... If your app uses SQL Server you can have a t-sql job that runs daily or hourly. I wouldn't recommend this option unless you really have performance problems.
The problem with this is that you would be splitting the logic and responsibilities of your code. A future developer or admin would find hard to maintain your app.
If you'd like to keep the logic within the web application for simplicity (depending on the total size of your solution, this may or may not be desired):
For a given URL, have the web app check for due approvals and sends emails out if needed. Be sure to keep track of emails sent in order to prevent sending the same email multiple times.
Call this URL in a regular interval. You may use a scheduled task or a third party url monitoring service to do this.
You may call the URL with a simple VBScript (or wget, or curl, or powershell, or whatever is fastest for you), which in turn you can automate by using the task scheduler (see also).
An example script in vbscript for calling an URL:
Function LoadUrl(url)
Dim objRequest
Set objRequest = CreateObject("MSXML2.ServerXMLHTTP.6.0")
objRequest.open "POST", url , false
objRequest.Send
LoadUrl = objRequest.responseText
Set objRequest = Nothing
End Function
Checking every hour won't affect performance. Even checking every minute is probably fine, depending on your database. The simplest option is a console program fired as a Scheduled Task. You can also try a Windows Service but they're a bit trickier.
Also give some thought how you'll count the 48 hours. If an employee puts in expenses just before the weekend then 48 hours will probably elapse every time and you'll end up with a manager having lots of emails in their Inbox on Monday morning. That could cause some friction :)
I'm designing an ASP.NET 4.0 Web application where administrators may create an auction with an expiration. The expiration time would be stored in a database. How can I ensure that the auction ends at the predetermined time, considering the fact that an application instance may not be running when it is time? The application will be hosted with IIS7. I am considering Windows service, but I am wondering what other options are out there.
You can choose from two scenarios here:
Lazy with the web application
Active with a service
Lazy Scenario
Unless you have to interact instantly with the winner of an auction you could wait until the application starts and then let the application determine if there are any auctions that are expired. Collect them at that moment and handle them accordingly.
Active Scenario
Create a service that picks the first expiring auction from the DB. Store that DateTime and let the service sleep till the auction expires. Raise the service at that DateTime and process the expiring auction. Then let the service look for the next auction(s) to expire and let the service sleep again. And on and on.. I think The Windows Workflow Foundation contains all tools and requirements for this practice.
Something in between
Activate a scheduler that wakes up you web-app every hour/half hour/15 minutes and do the lazy stuff. Use a scheduler like HostMonitor.
A windows service that handles the timed events would be the best practice. Using a System.Threading.Timer in ASP.NET is bad juju; it MIGHT work in your case, because the change being made doesn't affect the UI directly, but I wouldn't bet on it working flawlessly. If you need to schedule events in the background of a web app, use a server app.
One thing to keep in mind; you may have hundreds or thousands of open auctions at a time. If you set a timer on every auction when it's opened, you'll have hundreds or thousands of sleeping threads managed by this server. I'd look only for auctions that would end before the next time you'd normally poll, and set timers for those auctions only. That would get you down to maybe a few dozen waiting threads. If you're writing the next eBay, though, even this will choke when you get hundreds of thousands, or millions, of auctions, and several hundred or thousand begin and end every minute.
If you want it to be 100% reliable, a Windows service that perform all scheduled logic is the way to go. As you say, you cannot trust a web application since it may not even be running.
I'm using c# to communicate with twitter and i need to code a schedule system to send twitter messages at a user defined date.
The messages to be sent are in a database with the date and time of deliver.
Which is the best method to check the db for scheduled messages and send it when the time arrives?
How accurate do you need the timing to be? Could you get away with polling the database every 5 minutes, saying "Tell me all the messages which need to be delivered before current time + 5 minutes" and then sending them all? (And marking them as sent in the database, of course.) Does it matter if they're a bit early or late?
You can do the scheduling on the C# side to make it more accurate, but unless you really need to I'd stick with a very simple solution.
(Depending on your database there may be clever ways of getting callbacks triggered etc... but again, I'd stick with the simplest solution which works.)
In addition to the windows service option (or background thread), you could just set up a scheduled task to run an app that polls the DB and sends the tweets once every defined interval.
Windows schedules can be setup using C# if needed and are really easy to set up manually.
There are several ways to do this, but I guess the best way is to set up a Windows Service that will periodically poll (frequency is up to you) the DB for any scheduled tweets that hasn't been sent.
Needless to say you'll need to handle scenarios such as the Internet connection or DB being down, etc.
In fact the solution consists in using a windows service but it can't communicate directly with the ASP.NET MVC app. I've added a Web Service that handles the task and a System.Threading.Timer in Windows Service to periodically call the Web Service.