I have a website that allows a user to upload a spreadsheet of items to a database. When the spreadsheet is uploaded it will be assessed for keywords that match individual user interests and email those users an alert to visit the website. However, when the spreadsheet is successfully uploaded the administrator needs to ftp into their server and upload any related images. This can take up to one hour to perform.
I will be executing the email alerts on a new thread and was wondering if, when that thread is called, I can safely call Thread.Sleep(360000) from within it to pause execution for one hour. Would this create a hog on the servers resources? If so is there another way to do this?
I can't create a scheduled task as uploading can happen at any time/day of the week.
You can't be 100% sure that your application pool won't be recycled, your server rebooted, or any other number of events that could cause your thread to disappear out from under you.
You'll have to have some form of scheduled task, that checks for "things older than X" and runs every minute/couple of minutes and processes any as that approach will allow you to recover from pretty much anything (eg. Server looses power for an hour, your scheduled task catches up on all email alerts that should have happened during that time).
Write the necessary information to a queue or file and write a service that processes the queue. It's a much more robust way to handle such tasks. However, I'm curious about your comment:
However, when the spreadsheet is
successfully uploaded the
administrator needs to ftp into their
server and upload any related images.
This can take up to one hour to
perform.
How do you know the task will be completed in one hour? What signals the completion of the task? Why can't you fire off an email at that point, rather than pre-scheduling the alert? I could be missing something, but it seems like you might not be thinking this through very well.
What if the server explodes (replace with more likely event, like "unexpectedly shuts down") after half an hour?
Maybe you should think of a completely different mechanism (store some information in a database, check periodically for requests completed more than one hour ago...).
EDIT: maybe Windows Workflow Foundation could provide you some help for what you are trying to do.
Related
I have written a web application in asp.net. It has some user roles. There are 3 roles which are Manager, Accountant and Employee. The employees write their expenses in a form and send it to Manager. When manager approves it, it'll be sent to Accountant to pay it. I need to have an idea that when manager doesn't approve the employee's expense in 48 hours, it should send an automatic e-mail to Manager's mail.
I thought that I can write another small console application to handle that by checking every hour. But it would waste resources and decrease performance.
I need a good idea to handle that. How should I do?
There are several options, but if I were you I would go with first or second options.
Console App & scheduler
I would create that console application that every time is run perform the check for you.
Then I will have it run using Windows Scheduler in a daily basis (at 00:05) or a hourly basis if you prefer so. This way Windows Scheduler daemon will launch it every hour and the rest of the time your app is not running.
Check this Microsoft link to see how a scheduled task is created in windows.
Restful Web Service & scheduler
As suggested in #marapet answer, having a restful web service that allow you to perform this action instead of a console application would give you the advantage of having all code in your web application.
Similar as previous one, you should only invoke the restful uri to have your action done. As possible disadvantage, you have to get sure that that uri is not accessible to end users. In usual architecture (Web Server --> Application Server --> DB) this restful service should be in the Application Servers, far away from end user access.
Windows Service
Another option is creating a Windows Service that runs all the time and check the time itself so every hour perform the job (maybe using Quartz or similar). But this does not meet your performance requirements.
The performance hit will be small anyway as your service should check every minute to see if an hour has pass and is time to do its job.. a task pretty easy.
The advantage is that a windows service is easier to control and monitor than a Scheduled tasks
DB job
Yet another option... If your app uses SQL Server you can have a t-sql job that runs daily or hourly. I wouldn't recommend this option unless you really have performance problems.
The problem with this is that you would be splitting the logic and responsibilities of your code. A future developer or admin would find hard to maintain your app.
If you'd like to keep the logic within the web application for simplicity (depending on the total size of your solution, this may or may not be desired):
For a given URL, have the web app check for due approvals and sends emails out if needed. Be sure to keep track of emails sent in order to prevent sending the same email multiple times.
Call this URL in a regular interval. You may use a scheduled task or a third party url monitoring service to do this.
You may call the URL with a simple VBScript (or wget, or curl, or powershell, or whatever is fastest for you), which in turn you can automate by using the task scheduler (see also).
An example script in vbscript for calling an URL:
Function LoadUrl(url)
Dim objRequest
Set objRequest = CreateObject("MSXML2.ServerXMLHTTP.6.0")
objRequest.open "POST", url , false
objRequest.Send
LoadUrl = objRequest.responseText
Set objRequest = Nothing
End Function
Checking every hour won't affect performance. Even checking every minute is probably fine, depending on your database. The simplest option is a console program fired as a Scheduled Task. You can also try a Windows Service but they're a bit trickier.
Also give some thought how you'll count the 48 hours. If an employee puts in expenses just before the weekend then 48 hours will probably elapse every time and you'll end up with a manager having lots of emails in their Inbox on Monday morning. That could cause some friction :)
I have a windows service that listens to a message queue and processes request from this message queue. I have an initial application that places messages in the message queue. The service itself actually creates some files on the fly and when these files are created I need the initial application to grab these files and essentially paint them on the screen. (the files are an html format) For the time being I have the application and the service running on my local machine so the files are just located on the c: drive of my machine.
My issue is that I need some way to signal my initial application that the service has picked up and processed my msg successfully or unsuccessfully. I have been researching how to do this and have come up empty handed. I have also been brainstorming ideas as to what I could do to make this work. One non viable solution was to insert a record into a database table when sending a msg to the queue and upon processing finished within the service update the queue record, all the while having the application query the table checking for a status of processing complete. I don't like this idea at all as I feel like it would be too intensive, and I believe that there has to be a better way of doing this. I am going to continue to research how this can be done, but I am definitely hoping someone has a better solution to this than my sql query running over and over again. Note in the past I have tried to query the event viewer for processes, and I know how to do this, but this will take far too long for the timing I am looking to accomplish.
I ended up going with the route of inserting record into database prior to call windows service. Then when the windows service finished with its process it updated this db record. All the while my page is calling a webservice via javascript which checks the db to see if the flag on the record has been set to done processing. This allows me to determine when the windows service has finished processing my msg from the queue.
I have some code that pulls data from SQL DB, then loops through the records to generate a string, which will eventually be written to a text file.
The code runs fine on my local, from VS, but on the live server, after about a minute and half, I get "No Data Received" error (chrome). The code stops in middle of looping through the DataTable. Hosting support said "The connection was reset" error was thrown.
I"m not sure if this is a timeout issue or what. I've set the executionTimeout in my web.config (with debug = false) and it didn't seem to help. I also checked the Server.ScriptTimeout property, and it does match the executionTimeout value set in the web.config.
Additionally, a timeout would normally give "Page not available" message.
Any suggestions are appreciated.
after about a minute and half
There's your problem. This is a web application? A minute and a half is a very long time for a web application to respond to a request. Long enough that it's not really worth engaging in various trickery to make it kind of sort of work.
You'll want to offload this process to be more asynchronous with the web application itself. The nature of web applications is that they should receive a request and respond in a timely manner. What you have here is a long-running process which can't respond in a timely manner. The web application can facilitate interactions with the data, but shouldn't directly handle the processing thereof in the request/response directly.
How does the web application interact with the process? Does it just start it, or does it provide information for the process to begin? I would recommend that the process itself be handled by something like a Windows Service or perhaps a Console Application. The more de-coupled from the web application, the better. Now, since I don't know anything about the process itself, I'm making a few assumptions about its behavior...
The web application can receive a request to start the process, along with any information needed for the process. It can store this in a database with a status value (pending, queued, etc.) and then respond to the user (in a timely manner) that the request has been received and the process has been queued. The web application can have a page which checks the status so that the user can see how the process is doing (if it's started, how many records it's gone through, etc.).
The offline application (Windows Service, et al) would just monitor that database for newly-queued data to be processed. When it sees it, it updates the status (running, processing, etc.) and provides any relevant feedback during the process (number of records processed, etc.) by updating that data. So the offline application and the web application are both interacting with the same data, but not in a manner which blocks the thread of the web application and prevents a response to the user.
When the process is finished, the status is again updated. The web application can show that it's finished and provide a link to download the results. The offline process could even perhaps send an email to the user when it's done, or maybe the web application can have some kind of notification system (I'm picturing the little notification icons in Facebook) which would alert the user to new activity.
This way the thread isn't blocked, the user can continue to interact with the application (if there's even anything with which to interact), etc. And you get other added benefits, too. For example, results of the process are thus saved in the database and automatically historically tracked.
It sounds like it's the browser that's timing out waiting for a response, not on the server. You can't control what the browser has set for this. What you can do is send a response of some kind to the browser, so that it knows you're still around and haven't crashed in some way.
For this to work, you can't wait until you finish building the entire string. You need to re-think your code so that instead of appending to a string, you are writing each addition to an output stream. This has the added advantage of being a much more efficient way to create your text file. For purposes keeping the browser alive, you can write out anything, as long as some data is coming back for the browser to read. Html comments can work for this. You also need to periodically flush your response stream, so that your data isn't sitting buffered on your web server. Otherwise you might still timeout.
Of course, the real solution here is to re-think your design, such that your operation doesn't take 90 seconds plus in the first place. But until you can do that, hopefully this is helpful.
it does sound like a timeout, Could you try and return the information via a View, this would certainly speed things up.(if possible).
When i had this error, i was able to resolve it by adding in the Web.config file:
<system.web>
<httpRuntime executionTimeout="600" maxRequestLength="51200" />
</system.web>
What if I have website with a button. User clicks on the button and starts a long running process. After a few hours (or minutes) user update webpage and see results. What is the best (and any other) way to implement long running operation on website?
From the user experience point of view I suggest you implement the job like you would implement order handling in an online shop.
When the user starts the job he should be able to track the state of the job. Is the job complete? Did the job fail? What is the progress of the job? He should probably also be able to cancel the job and perhaps modify properties of the job. You could implement an notification mechanism using e-mail or an SMS and the user should be able to control that.
By using JavaScript/AJAX you can provide a more interactive user experience where the job status web page is automatically refreshed at regular intervals in the background without forcing the user to refresh his browser.
The user should be able to leave his computer and later connect to the website from another computer and still be able to get information about pending and completed jobs. This requires some form of user login.
I suggest that you separate the job handling code from the website code. You could expose the job handling user a web service or another similar technology. The website should query the service and display the results providing a user interface on top of the job service.
How long? If it's really going to take hours, you don't want that code running in the web server. Have the server spawn an external process or start a service or something along that line, and put the long running code in there.
You could have that code put status updates somewhere (like in the database) as it runs, and the website can check that when the user comes back to see how it's doing.
I suggest you read up on Threading in asp.Net. For website performance, time consuming processes can be put on a different thread.
A different approach is executing the task with AJAX. This way, you can present the user interface to the user, and start an asynchronous process which handles the request.
I'm using c# to communicate with twitter and i need to code a schedule system to send twitter messages at a user defined date.
The messages to be sent are in a database with the date and time of deliver.
Which is the best method to check the db for scheduled messages and send it when the time arrives?
How accurate do you need the timing to be? Could you get away with polling the database every 5 minutes, saying "Tell me all the messages which need to be delivered before current time + 5 minutes" and then sending them all? (And marking them as sent in the database, of course.) Does it matter if they're a bit early or late?
You can do the scheduling on the C# side to make it more accurate, but unless you really need to I'd stick with a very simple solution.
(Depending on your database there may be clever ways of getting callbacks triggered etc... but again, I'd stick with the simplest solution which works.)
In addition to the windows service option (or background thread), you could just set up a scheduled task to run an app that polls the DB and sends the tweets once every defined interval.
Windows schedules can be setup using C# if needed and are really easy to set up manually.
There are several ways to do this, but I guess the best way is to set up a Windows Service that will periodically poll (frequency is up to you) the DB for any scheduled tweets that hasn't been sent.
Needless to say you'll need to handle scenarios such as the Internet connection or DB being down, etc.
In fact the solution consists in using a windows service but it can't communicate directly with the ASP.NET MVC app. I've added a Web Service that handles the task and a System.Threading.Timer in Windows Service to periodically call the Web Service.