Develop Reference

Calling external services which randomly timeout kills the ASP.NET MVC application

So I have this web application (ASP.NET MVC 4 Web Site) which has at least 2,000 online users at any time. One of the most popular pages in my application contains data about user, and this data is not located in my repository, it is contained in some external vendor which is integrated into my system. So whenever this page is drawn I have to make a call to those services (currently there are 17) and than draw the page according to the data given by them. The data is subject to change in any given moment so I cannot cache it. Everything is working OK most of the time and the CPU utilization is 5% - 30% (depending on the number of online users of course). For each service call I have timeouts of 5000 milliseconds (for service references I set the SendTimeout and for the raw HttpWebRequests' I set the TimeOut property to be equal to 5000 milliseconds) Now suppose that one service is down, the CPU utilization of my server goes unxpectidly low like 3% - 8% and the application is lagging, I mean it takes some time to load pages (any page), for instance, if in a normal mood the response from my application would have taken (150-250ms) now it takes 1-3 seconds. I'm out of ideas of what to do. I cannot decrease the timeout because some services are taking 3-4 seconds sometimes so the 5 second timeout is the lease I can give. What can I do to prevent the late response ? I know it's bit general question. Any suggestion would be appreciated. Thanks in advance.

It looks like you have a threading problem. Too many threads are waiting for response from the external service and they can not process other requests.
What I recommand you is to use Async Controller: http://www.asp.net/mvc/overview/performance/using-asynchronous-methods-in-aspnet-mvc-4

Suggestion 1
What if you replicate this data to your server?
Meaning you can have another service, that works separately and synchronize data of external service with your server... and your websites always point to your local data... Right, this is some kind of caching, and web pages can show kind of old data... but you can set replication service to check data as often as you need...
Suggestion 2
Another suggestion that come to mind, can you use push notification instead? All the web pages open and wait, where server checks the data, and notify all the clients with the fresh data... in this case only one thread will be busy with external data, and all the opened users will have fresh data as soon as it is available. As a starting point, check SignalR

Scheduling job idea needed by the web application in asp.net

I have written a web application in asp.net. It has some user roles. There are 3 roles which are Manager, Accountant and Employee. The employees write their expenses in a form and send it to Manager. When manager approves it, it'll be sent to Accountant to pay it. I need to have an idea that when manager doesn't approve the employee's expense in 48 hours, it should send an automatic e-mail to Manager's mail.
I thought that I can write another small console application to handle that by checking every hour. But it would waste resources and decrease performance.
I need a good idea to handle that. How should I do?

There are several options, but if I were you I would go with first or second options.
Console App & scheduler
I would create that console application that every time is run perform the check for you.
Then I will have it run using Windows Scheduler in a daily basis (at 00:05) or a hourly basis if you prefer so. This way Windows Scheduler daemon will launch it every hour and the rest of the time your app is not running.
Check this Microsoft link to see how a scheduled task is created in windows.
Restful Web Service & scheduler
As suggested in #marapet answer, having a restful web service that allow you to perform this action instead of a console application would give you the advantage of having all code in your web application.
Similar as previous one, you should only invoke the restful uri to have your action done. As possible disadvantage, you have to get sure that that uri is not accessible to end users. In usual architecture (Web Server --> Application Server --> DB) this restful service should be in the Application Servers, far away from end user access.
Windows Service
Another option is creating a Windows Service that runs all the time and check the time itself so every hour perform the job (maybe using Quartz or similar). But this does not meet your performance requirements.
The performance hit will be small anyway as your service should check every minute to see if an hour has pass and is time to do its job.. a task pretty easy.
The advantage is that a windows service is easier to control and monitor than a Scheduled tasks
DB job
Yet another option... If your app uses SQL Server you can have a t-sql job that runs daily or hourly. I wouldn't recommend this option unless you really have performance problems.
The problem with this is that you would be splitting the logic and responsibilities of your code. A future developer or admin would find hard to maintain your app.

If you'd like to keep the logic within the web application for simplicity (depending on the total size of your solution, this may or may not be desired):
For a given URL, have the web app check for due approvals and sends emails out if needed. Be sure to keep track of emails sent in order to prevent sending the same email multiple times.
Call this URL in a regular interval. You may use a scheduled task or a third party url monitoring service to do this.
You may call the URL with a simple VBScript (or wget, or curl, or powershell, or whatever is fastest for you), which in turn you can automate by using the task scheduler (see also).
An example script in vbscript for calling an URL:
Function LoadUrl(url)
Dim objRequest
Set objRequest = CreateObject("MSXML2.ServerXMLHTTP.6.0")
objRequest.open "POST", url , false
objRequest.Send
LoadUrl = objRequest.responseText
Set objRequest = Nothing
End Function

Checking every hour won't affect performance. Even checking every minute is probably fine, depending on your database. The simplest option is a console program fired as a Scheduled Task. You can also try a Windows Service but they're a bit trickier.
Also give some thought how you'll count the 48 hours. If an employee puts in expenses just before the weekend then 48 hours will probably elapse every time and you'll end up with a manager having lots of emails in their Inbox on Monday morning. That could cause some friction :)

Long running operation on website

What if I have website with a button. User clicks on the button and starts a long running process. After a few hours (or minutes) user update webpage and see results. What is the best (and any other) way to implement long running operation on website?

From the user experience point of view I suggest you implement the job like you would implement order handling in an online shop.
When the user starts the job he should be able to track the state of the job. Is the job complete? Did the job fail? What is the progress of the job? He should probably also be able to cancel the job and perhaps modify properties of the job. You could implement an notification mechanism using e-mail or an SMS and the user should be able to control that.
By using JavaScript/AJAX you can provide a more interactive user experience where the job status web page is automatically refreshed at regular intervals in the background without forcing the user to refresh his browser.
The user should be able to leave his computer and later connect to the website from another computer and still be able to get information about pending and completed jobs. This requires some form of user login.
I suggest that you separate the job handling code from the website code. You could expose the job handling user a web service or another similar technology. The website should query the service and display the results providing a user interface on top of the job service.

How long? If it's really going to take hours, you don't want that code running in the web server. Have the server spawn an external process or start a service or something along that line, and put the long running code in there.
You could have that code put status updates somewhere (like in the database) as it runs, and the website can check that when the user comes back to see how it's doing.

I suggest you read up on Threading in asp.Net. For website performance, time consuming processes can be put on a different thread.
A different approach is executing the task with AJAX. This way, you can present the user interface to the user, and start an asynchronous process which handles the request.

What is the preferred method to manage a timed event with a Web application?

I'm designing an ASP.NET 4.0 Web application where administrators may create an auction with an expiration. The expiration time would be stored in a database. How can I ensure that the auction ends at the predetermined time, considering the fact that an application instance may not be running when it is time? The application will be hosted with IIS7. I am considering Windows service, but I am wondering what other options are out there.

You can choose from two scenarios here:
Lazy with the web application
Active with a service
Lazy Scenario
Unless you have to interact instantly with the winner of an auction you could wait until the application starts and then let the application determine if there are any auctions that are expired. Collect them at that moment and handle them accordingly.
Active Scenario
Create a service that picks the first expiring auction from the DB. Store that DateTime and let the service sleep till the auction expires. Raise the service at that DateTime and process the expiring auction. Then let the service look for the next auction(s) to expire and let the service sleep again. And on and on.. I think The Windows Workflow Foundation contains all tools and requirements for this practice.
Something in between
Activate a scheduler that wakes up you web-app every hour/half hour/15 minutes and do the lazy stuff. Use a scheduler like HostMonitor.

A windows service that handles the timed events would be the best practice. Using a System.Threading.Timer in ASP.NET is bad juju; it MIGHT work in your case, because the change being made doesn't affect the UI directly, but I wouldn't bet on it working flawlessly. If you need to schedule events in the background of a web app, use a server app.
One thing to keep in mind; you may have hundreds or thousands of open auctions at a time. If you set a timer on every auction when it's opened, you'll have hundreds or thousands of sleeping threads managed by this server. I'd look only for auctions that would end before the next time you'd normally poll, and set timers for those auctions only. That would get you down to maybe a few dozen waiting threads. If you're writing the next eBay, though, even this will choke when you get hundreds of thousands, or millions, of auctions, and several hundred or thousand begin and end every minute.

If you want it to be 100% reliable, a Windows service that perform all scheduled logic is the way to go. As you say, you cannot trust a web application since it may not even be running.

Schedule method invocation C#

I'm using c# to communicate with twitter and i need to code a schedule system to send twitter messages at a user defined date.
The messages to be sent are in a database with the date and time of deliver.
Which is the best method to check the db for scheduled messages and send it when the time arrives?

How accurate do you need the timing to be? Could you get away with polling the database every 5 minutes, saying "Tell me all the messages which need to be delivered before current time + 5 minutes" and then sending them all? (And marking them as sent in the database, of course.) Does it matter if they're a bit early or late?
You can do the scheduling on the C# side to make it more accurate, but unless you really need to I'd stick with a very simple solution.
(Depending on your database there may be clever ways of getting callbacks triggered etc... but again, I'd stick with the simplest solution which works.)

In addition to the windows service option (or background thread), you could just set up a scheduled task to run an app that polls the DB and sends the tweets once every defined interval.
Windows schedules can be setup using C# if needed and are really easy to set up manually.

There are several ways to do this, but I guess the best way is to set up a Windows Service that will periodically poll (frequency is up to you) the DB for any scheduled tweets that hasn't been sent.
Needless to say you'll need to handle scenarios such as the Internet connection or DB being down, etc.

In fact the solution consists in using a windows service but it can't communicate directly with the ASP.NET MVC app. I've added a Web Service that handles the task and a System.Threading.Timer in Windows Service to periodically call the Web Service.