I am working on a ASP.net application written in C# with Sql Server 2000 database. We have several PDF reports which clients use for their business needs. The problem is these reports take a while to generate (> 3 minutes). What usually ends up happening is when the user requests the report the request timeout kills the request before the web server has time to finish generating the report, so the user never gets a chance to download the file. Then the user will refresh the page and try again, which starts the entire report generation process over and still ends up timing out. (No we aren't caching reports right now; that is something I am pushing hard for...).
How do you handle these scenarios? I have an idea in my head which involves making an aysnchronous request to start the report generating and then have some javascript to periodically check the status. Once the status indicates the report is finished then make a separate request for the actual file.
Is there a simpler way that I am not seeing?
Using the filesystem here is probably a good bet. Have a request that immediately returns a url to the report pdf location. Your server can then either kick off an external process or send a request to itself to perform the reporting. The client can poll the server (using http HEAD) for the PDF at the supplied url. If you make the filename of the PDF derive from the report parameters, either by using a hash or directly putting the parameters into the name you will get instant server side caching too.
I would consider making this report somehow a little bit more offline from the processing point of view.
Like creating a queue to put report requests into, process the reports from there and after finish, it can send a message to the user.
Maybe I would even create a separate Windows Service for the queue handling.
Update: sending to the user can be email or they can have a 'reports' page, where they can check their reports' status and download them if they are ready.
What about emailing the report to the user. All the asp page should do is send the request to generate the report and return a message that the report will be emailed after is has finished running.
Your users may not accept this approach, but:
When they request a report (by clicking a button or a link or whatever), you could start the report generation process on a separate thread, and re-direct the user to a page that says "thank you, your report will be emailed to you in a few minutes".
When the thread is done generating the report, you could email the PDF directly (probably won't work because of size), or save the report on the server and email a link to the user.
Alternatively, you could go into IIS and raise the timeout to > 3 minutes.
Here is some of the things I would do if I would be presented this problem:
1- Stop those timeout! They are a total waste of resources. (bring up the timeout value of asp pages)
2- Centralize all the db access in one single point, then gather stats about what reports ran when by whom and how much time it took. Investigate why it takes so long, is it because of report complexity? data range? server load? (you could actually all write that on a .csv file on the server and import this file periodically in the sql server to analyze later).
Eventually, it's going to be easier for you to "cache" reports if you go through this single access point (example, same query same date will return same PDF previously generated)
3- I know this really wasn't the question but have you tried diving into those queries to see why they are so long to run? Query tuning maybe?
4- Email/SMS/on screen message when report is ready seems great... if your user generally send a batch of report to be generated maybe a little dashboard indicating progression of "their" queue could be built in the app. A little ajax control would periodically refresh the status..
Hint: If you used that central db access and you have sufficient information about what runs when why and how-long you will eventually be able to roughly estimates the time it will take for a report to run.
If the response time is mission critical, should certain users be limited in the data range (date range for example) during some hours of the day?
Good luck and please post more details about your scenario if you want to get more accurate hints...
Query tuning is probably your best place to start. Though I don't know you are generating the report, that step shouldn't really take all that long. A poorly performing query on the other hand could absolutely kill your performance.
Depending on what you find in looking at the query, you may need to add some indexes, or possibly even set up a table to store the information for your report in a denormalized way, to make it available faster. This denormalized table could then be refreshed (through a SQL Server Job) every hour, or with whatever frequency your requirements dictate (within reason).
If its' a relatively static report, without varying user input parameters, then caching the report run earlier in the day would be a good idea as well, but its' hard to say any more about this without knowing your situation.
For a problem like this you really need to start at the database unless you have reason to suspect your report generating code of being the culprit. There are various band-aids you could use that might help for a while, but if your db is the root cause then those solutions will not scale well, and you'll likely run into similar problems (or worse) some time in the future.
Related
I have a task that basically I don't know how to accomplish it and I want some advice from you guys : I have a database that contains many binary images and a website that has a button.
When I press it a select statement is executed to search all the newly inserted rows after a specific date and send it to the website to process. My boss asked me if there is any way I can do that process manually by using a web service or anything else like a trigger in the database that automatically searches for new rows after a specific date and sends the new rows to the web site.
I don't know if I explained what I need. My main question is it possible to do this and what is the best way to achieve it. one more thing if it is achievable can I trigger it to work every day at a specific time ? Thank you
As I understood you question, these are the requirements:
You need to automate the process that is now manually triggered by clicking the button on the web page
The simplest way to do this in my opinion would be to create a small Console application and install it on a suitable server. Then on the server create a Windows task scheduler (Google it unless you're already familiar with it) task that runs the console application on a suitable schedule.
You could for instance have it run once every night at say 10 minutes past midnight and it would fetch all images from the past day and "send them to be processed".
First of all you have to understand website is not a repository of records but it works based on user requests. So in your website if you want to change contents on web pages, try AJAX reqeusts. If you are looking for some options to update details in other systems periodically think of eitehr Windows services, or cron jobs or may be SSIS jobs etc.
Use Cursor and Trigger but u have to store date in database which u want display on particular date
We are currently using ReportViewer control embedded in an aspx file to render RDLC files for an MVC Web Application. The data is fetched from an SQL Server using nhibernate, the ReportViewer is running in Local Processing mode. The various reports contain calculations within the RDLC including sums, counts and grouping.
We are finding the reports using this method run terribly slowly, usually causing the web server to display the Service 'Unavailable Error'.
It appears that the live servers do not have enough resources to run these reports, and even in an environment where the servers have much more memory and CPU speed, while the reports do not time out, they take a very long time to render even a single report.
Does anyone have recommendations on this matter. How can we reduce the amount of processing time for these reports?
From observing the resource usage, the calculations carried out internally by report viewer on the data are taking a great deal of time. As an example of the data we are looking at about 3000 individual reports, using over 40000 records to calculate from.
Short story:
My site pre generates pages based on user submited data, sometimes this cache has to be cleared when this happens it would kill a super computer unless i controled the amount of stats being generated at once.
The problem:
Now comes the search engine bots that hit the site constantly ( due to the sheer amount of pages, its pretty constants that search engines bot crawl ). The problem here is that they will use up all my "generate" slots, and real users will be left with a page saying "bla bla, please wait".
Posible solution:
Can i basicly return a 503 to the bots, without having them give me negative ranking for having a unstable site?
Or did someone come up with some other solution?
How critical is it that the cache is cleared immediately? If your cache supports it, you could instead mark all the cached pages as 'dirty' and regenerate them when a real user next visits; if a bot visits in the meantime, serve them the stale page.
I'm working on a project for work that has me running a SQL reporting services report all day long. I would like to automate this process and have gotten pretty far in the process. I can navigate to the page, search for what I want, select the output of the file and click export. The problem I run into is the File Download dialog box that pops regardless of file type or mime settings. I am able to get around it now by using an AutoIt function but its flaky at best. I really want to just download the file to a specific location. My question.. Is there a way to intercept whatever clicking the export button does and get some sort of link to the file? Or is there a way to run the function that link does in asp programmaticly and grabbing the file that way? (Sorry if terminology is off not real familiar with ASP).
Edit... I know there are better ways of doing this with reporting services such as subscriptions etc (thanks for the responses btw :)... But this is what I'm stuck with until I can get the admin to fix a few things. In the company I'm at that involves getting about 20 different departments involved and it will most likely not get fixed. But ultimately my question is whether it is possible to do what im asking in C#. Can I intercept where the code is sending me when I click export and grab the file? Is that even possible?
Check if you can use URL Access directly against the Reporting Services server: http://msdn.microsoft.com/en-us/library/ms153586.aspx.
If you can modify the web application, try changing the value of ReportViewer.ExportContentDisposition (http://msdn.microsoft.com/en-us/library/microsoft.reporting.webforms.reportviewer.exportcontentdisposition.aspx) to "AlwaysInline". Alternatively, you can use the Render method (http://msdn.microsoft.com/en-us/library/ms252199.aspx) of ReportViewer.ServerReport or ReportViewer.LocalReport to render into a Response OutputStream.
Otherwise, you need to peek into the JavaScript in the ReportViewer's IFRAME using IE Developer Tools (F12 in IE8). It needs to perform an additional callback to the server to have it render the report in the selected format. Adapt the actions this script performs.
Why not just use the report scheduling feature to automate it? You can enter subscriptions for the report and it can be delivered on a schedule. There are two delivery methods with SSRS out of the box - email and file share. You can even write your own custom delivery extension.
How much access do you have to SSRS? If you can get pretty much carte blanche, you could setup SSRS to export the report at intervals to a specific location. If SSRS can't hit that location, you could set up SSRS to email the report--from there it is an easy jump to making email dump to location. Either way, there are better angles than "auto click OK" screen-scraping UI automation.
I'd look into the web services. If they can't get scheduling right, they probably didn't get around to turning those off either. Setup a command line app to suck the reports off the services and life will be grand.
It sounds as if you need to treat the reporting service as if it is completely external. Perhaps WinBatch would help you script interaction with the windows? http://www.windowware.com/
Maybe not specific to reports but still...
In my asp.net mvc web application there's a section for reports that show 5 columns of data that map almost directly to a table in the db.
The problem is, in some cases, the length of that report may exceed 40,000 records (I know, nobody can really process 40,000 records of data but the report is what it is) and as you can expect, it times out and throws an error.
The question is, what's a good way to process and deliver a report of that size? I thought about creating a small little console app that would build the report outside of the webserver but I'm kind of at a loss as to what direction to look into?
Does the report need to have up-to-the-minute data? If not, you can look at generating the report as a PDF at night (or whenever your server isn't busy) and just providing a link to the PDF. A scheduled task that runs a console app as you suggested could create the report and output it to a file. A lot of reporting tools like Crystal Reports will allow you to export the report to a PDF or an Excel spreadsheet. For that matter, you could generate the report on a completely different machine and then copy it over to the web server. This could allow you to update the report every hour (or whatever) without putting such a load on your web server.
Generating the report while the user waits is probably not a good idea (not to mention SQL / IIS timeouts etc)
You could get the user request a report, then have a windows service pick up these requests, generate the report and email the user? (or have some kind of ajax polling script on the site to notify users when their reports are ready?)
You could extend this to scheduling of the same report at recurring intervals etc.
I would look into SQL Reporting Services (assuming this is running on SQL Server). There's several delivery options which may be better suited to your application's needs (you can schedule a PDF or Excel document to show up in someone's mailbox every night, for example).
There's also a great article from the StackOverflow team that allows background processes within ASP.NET if you can simply generate this report every so often instead of on-demand (maybe every 5-10 minutes?)
https://blog.stackoverflow.com/2008/07/easy-background-tasks-in-aspnet/
Another frequent usage I've seen to handle large scale reporting is setting up a windows service that does the physical generation of the report which dumps the completed binary into a database or file store somewhere and then updates data to show the report is complete with the information needed for the application to link the completed report.
Then you could have your do report button fire off a request to initiate a report and then move them to a processing report page where it lists all the reports queued up / processing for them.
It's doubtful that a user would ever actually look at all of a 40,000 row report. So why not show just the top 1,000 most current rows ordered backwards? If you're using a reporting solution that supports on-demand reports, you could always drill down to second report that shows the next 1,000 most current rows. Just a thought...