I'm currently in the process of building an ASP.NET MVC web application in c#.
I want to make sure that this application is built so that it can scale out in the future without the need for major re-factoring.
I'm quite keen on using some sort of queue to post any writes to my database base to and have a process which polls that queue asynchronously to perform the update. Once this data has been posted back to the database the client then needs to be updated with the new information. The implication here being that the process to write the data back to the database could take a short while based on business rules executing on the server.
My question is what would be the best way to handle the update from the client\browser perspective.
I'm thinking along the lines of posting the data back to the server and adding it to the queue and immediately sending a response to the client then polling at some frequency to get the updated data. Any best practices or patterns on this would be appreciated.
Also in terms of reading data from the database would you suggest using any particular techniques or would reading straight from db be sufficient given my scenario.
Update
Thought I'd post an update on this as it's been a while. We've actually ended up using Windows Azure but the solution is applicable to other platforms.
What we've ended up doing is using the Windows Azure Queue to post messages\commands to. This is a very quick process and returns immediately. We then have a worker role which processes these messages on another thread. This allows us to minimize any db writes\updates on the web role in theory allowing us to scale more easily.
We handle informing the user via emails or even silently depending on the type of data we are dealing with.
Not sure if this helps but why dont you have an auto refresh on the page every 30 seconds for example. This is sometimes how news feeds work on sports websites, saying the page will be updated every x minutes.
<meta http-equiv="refresh" content="120;url=index.aspx">
Why not let the user manually poll the status of the request? This is how your typical e-commerce app is implemented. When you purchase something online, the order is submitted to a queue for fullfillment. After it's submitted, the user is presented with a "Thank you for your order" page and a link where they can check the status of the order. The user can visit the link anytime to check the status, no need for an auto-poll mechanism.
Is your scenario so different from this?
Sorry in my previous answer I might have misunderstood. I was talking of a "queue" as something stored in a SQL DB, but it seems on reading your post again you are may be talking about a separate message queueing component like MSMQ or JMS?
I would never put a message queue in the front end, between a user and backend SQL DB. Queues are good for scaling across time, which is suitable between backend components, where variances in processing times are acceptable (e.g. order fulfillment)... when dealing with users, this variance is usually not acceptable.
While I don't know if I agree with the logic of why, I do know that something like jQuery is going to make your life a LOT easier. I would suggest making a RESTful web API that your client-side code consumes. For example, you want to post a new order to the system and have the client responsive? Make a post to www.mystore.com/order/create and have that return the new URI to access the order (i.e. order#) as a URI (www.mystore.com/order/1234). That response is then stored in the client code and a jQuery call is setup to poll for a response or stop polling on an error.
For further reading check out this Wikipedia article on the concept of REST.
Additionally you might consider the Reactive Extensions for .NET and within that check out the RxJS sub-project which has some pretty slick ways of handling with the polling problem without causing you to write the polling code yourself. Fun things to play with!
Maybe you can add a "pending transactions" area to the UI. When you queue a transaction, add it to the user's "pending transactions" list.
When it completes, show that in the user's "pending transactions" list the next time they request a new page.
You can make a completed transaction stay listed until the user clicks on it, or for a predetermined length of time.
Related
I've read a good bit about threading with C#, but to be upfront I haven't done anything in production using it.
I have an application that has to process a bunch of documents and then send the documents via email. This may take 60 seconds to accomplish. I don't want the user of my web application to have to wait for these things to process to move on to other parts of the site.
On a button click the SendEmail function is called. What can I do to this code to make it so that my users can continue browsing the site without discontinuing the processing I need to do within the EmailPDFs function?
[Authorize]
public ActionResult SendEmail(decimal? id, decimal? id2)
{
EmailPDFs(..., ..., ...);
}
Thanks so much!
This is really the kind of thing that message queues are designed to handle. Fire off a message, and a process on a potentially separate server picks it up and processes it. When it's done, it sends a message back to a queue on your server, where a process on your server picks it up and notifies you that it's complete. You then notify your user that the work is finished.
Modern message queue systems can be backed by databases (such as Mongo, MySql, or SQL Server), and are extremely robust. The great thing about them is that they allow you to move long-running or CPU-intensive processes off onto other servers so that your web site remains nice and snappy.
You could try to add multi-threading and parallelism to your web application, by using TaskFactory and all that other stuff (for many folks, this is the route they take), but it doesn't make it very easy to separate your application if you need to, and break those big, resource-hogging pieces off if it becomes necessary.
I urge you to consider a queue-based solution.
Update:
For samples and information on how to implement this type of solution, see the following:
Reliable Messaging with MSMQ and .NET on MSDN
C#: A Message Queuing Service Application on MSDN
Also, consider glancing at this StackOverflow question for a quick crash course on the bare minimimum amount of code required.
A final note: MSMQ is built into certain flavors of Windows, and can be added to it through the Add/Remove Programs feature of the Control Panel. However, how you install it will depend on your specific flavor and version of Windows. A simple Google search will help you to find the appropriate instructions.
Good luck!
I am currently working on a project in ASP.NET MVC 4 and came along a module where a progress bar is needed. The question I am having right now is "What is the best way to implement an async progress bar?".
After some lookup I came across the following method:
Create a startEvent() and getProgress() in C# code.
Use javascript setTimeout() to call the getProgress() method asynchronously.
(Example: https://www.devexpress.com/Support/Center/Example/Details/E4244)
My remark with this method is that that causes the code to be dependent on the timeout you choose. So it would take some fiddling to find the best and most performant timeout.
Now the method that I would most likely have used before I researched the matter is the following:
In code behind, create a method handleItem(int index) which takes an index and does everything you want to do with the item at that index.
Determine the number of items you want to handle and pass that to your javascript.
In javascript, initiate a for loop that loops from 0 to the amount - 1, and for each index, it initiates an ajax-call to handleItem(i).
On that ajax-call's complete-statement, you can update the progress bar with the new amount.
My questions here are the following:
Does this expose too much of the program logic?
Does this create too much overhead seeing as every call goes to the server and back?
Are there any other reasons why I should refrain from using this method?
Thanks in advance
Koen Morren
This is not a recommended strategy, because the client drives the process. If there is any discontinuation of connectivity or maybe the user closes the browser, the process will stop.
Generally, if you use straight HTTP you will need to poll (aka pull) from javascript. The pseudo code is pretty much this:
Call Creates Task ID and sends it to client
Client queries the status of task with given ID
Another possibility are WebSockets, which allow your client to listen for changes that are pushed by the server.
There are many options to store the progress of a given state. You can index the progress by the HttpContext, task id, or some user id, or even store it in a database and use SqlDependency to get notifications of when the status is changed.
In summary, polling has more lag than push mechanisms. Clients should not drive an asynchronous process, but they should be either notified or provided some mechanisms on the status of an async process.
Unlike ASP.NET, there is few way to push data from server to client in MVC, WebSockets or SingnalR like api(s) can work for you.
The ajax approach is good and give you reliable mechanism to update data no matter user go to other page or closes the browser, every time ajax launched it will update UI. So there is nothing wrong there just have a fair interval in javascript.
Does this expose too much of the program logic?
Code will be written only in class file to calculate current %age.
2.Does this create too much overhead seeing as every call goes to the server and back?
No, ajax are light-weight calls
3.Are there any other reasons why I should refrain from using this method?
This method will allow user to freely navigate to other resources as ajax will work independently.
I have a web application which has lots of dropdowns and grids on it.
Currently when value of one dropdown changes , the items in the related drop down gets populated(heavy DB operation) and infact whole page is posted to the server.
Question is can Ajax be recommended in such a case? I have heard that Ajax is only good for light operations.
Yes it can be, just give some sort of feedback to the user that the operation takes a while. Display a message or loading image while the operation is being performed, and then hide it when the operation is done.
You can use ajax but I would like to offer an alternate solution...
Use ajax/post/whatever to start the process then request feedback at regular intervals using one of:
JS polling (setInterval)
Server-Send-Events
SignalR
This way, the user can navigate away while waiting for the operation to complete. When "action completed" feedback is received, you can alert the user with a tidy looking in-page notification or email if more appropriate.
The answer is of course you can use AJAX for heavy DB operations, but maybe the better question is SHOULD you use it; the answer is it depends. Scissors are good for cutting paper, but not the yard; right tool for the right job.
Stepping back for a second, it sounds like your development technology choice may be part of your problem, as much as it is whether or not to use AJAX for heavy DB calls. My point is, if you are trying to avoid full postbacks for the page, then you might consider a more client-based implementation (such as KnockoutJS, Backbone, Ember, AngularJS, etc.) which can use calls to the server just to get the data it needs for a particular portion of the screen (via view models).
I'm building the back end to a site which will have multiple "widgets" on the front end which need to update in real time.
Right now I simply have a load method which populates all the widgets with data, on page load obviously. My question is how to handle the real time aspect of further updates.
I thought of having just multiple ajax calls, which could query a service every second or so, and return the latest data, but that seems inefficient.
Is there a way to "push" data to a View from a Controller?
maybe you can have a look at this project : https://github.com/SignalR/SignalR
ASP.NET SignalR is a new library for ASP.NET developers that makes it
incredibly simple to add real-time web functionality to your
applications. What is "real-time web" functionality? It's the ability
to have your server-side code push content to the connected clients as
it happens, in real-time.
SignalR also provides a very simple, high-level API for doing server
to client RPC (call JavaScript functions in your clients' browsers
from server-side .NET code) in your ASP.NET application, as well as
adding useful hooks for connection management, e.g. connect/disconnect
events, grouping connections, authorization.
(Excerp from http://signalr.net/ )
Hope it helps.
I think your best bet is to periodically poll the server:
$(document).ready(function() {
setTimeout("getUpdate()", 30000);
function getUpdate()
{
// Make an ajax call here
}
});
This will ask for an update every 30 seconds.
It depends on how often the data on the front end needs to be updated. Most pages aren't going to need constant updating. I don't know that there is a "best practice" threshold, but I think a good starting point would be 15-20 second updates using Ajax. Make your Ajax calls fast and lean - maybe just return blank if there are no updates. If you need faster updates than that, look into something called long polling. Long polling is basically where you trigger an ajax call to the server, and the connection sits open until there is data to be sent. Long polling will take more server resources, because you will have open connections and threads running while they are waiting for data to be ready. With ASP.NET you'll also have to worry about killing long polling threads, because by default those threads wouldn't be killed when the browser closes connection (for example if someone navigates away from the page.)
You can also use Web Sockets, if its running in a browser that support HTML5
I am writing one simple web page with bunch of textboxes and a button control. Now when user finished editing the values on this text boxes user has to click the button and this button invoke heavily process intensive algorithm on server side code based on the data received from client (Textboxes)
And it could some time takes up to 30 to 45 minutes to complete the whole operation so the current thread is still inside the button click event handler function.
That background task only provides one event, and the web page subscribes to it to get some text data after each stage of processing
I was wandering if there is any way I can keep user up-to-date with what is the current progress on that background task. I have div element to print the current status information
So I am looking for some sort of reverse mechanism then "get" and "post".
I have read some articles on the Comet (programming) but I can't find any easy or definitive answer
Thanks in advance
Perhaps the simplest way is to submit the job, and get an id back via a straightforward page POST. That id corresponds to the long running job on the server side.
The returned page can then automatically refresh via the HTTP meta refresh mechanism, and the server can return status etc. in the refreshed page. The page will continually refresh with the id every (say) 30s until the job is complete.
It's a brute-force mechanism, but it's straightforward. Plus it has the advantage of allowing the user to bookmark the page and go away/come back. Given that your job is running for 30/45 mins, that could be important.
You are on the right track with 'Comet' (aka Ajax-Push or reverse-Ajax) - Comet is essentially an umbrella term for the ability for the server to 'push' something back to the client without the client triggering the event. Some Ajax frameworks are starting to build this in, and it is not something i would suggest trying to implement yourself close to the metal, because there are a lot of different things to consider (different webapp servers, threading models, etc). You can see that StackOverflow has some of this functionality, when it notifies you of a new answer coming in if you are writing one yourself for a given question.
Now all that being said, in your case, given the length of time of the server side processing, I would agree with Brian's answer that you should give the running job an id and use a more simple refresh mechanism to check it. If you wanted to add the look and feel of a server side push, you could query the server via standard ajax on an interval to check if the job is done rather than a simple refresh, and change the page when it is done through that Ajax call. If your job is able to report progress, then standard ajax could refresh your div with that progress, and there is no need for a server side push.
I agree completely with Brian Agnew's suggestion, with one possible improvement. Since you're essentially doing server-push operations, it might be worth considering a comet server. Now, I say that with a caveat - if you're really only managing jobs that complete every 30-45 minutes, then it may well be overkill. However, it has the advantage that you can push the results to the user when they've completed, and if the user is not longer connected, you can do something different (such as send them a notification email).
It depends:
If you need to be instantly notified when the process ends Comet (long polling) is for you
If some delay is acceptable (for instance the system notifies you 1 minute after the process finishes) then AJAX timers (polling) is your you.
Take a look to three examples of both techniques based on ItsNat framework:
Comet
AJAX Timers
Asynchronous Tasks (similar to
Comet, one shot)
I'm sorry, is Java