I have some code that pulls data from SQL DB, then loops through the records to generate a string, which will eventually be written to a text file.
The code runs fine on my local, from VS, but on the live server, after about a minute and half, I get "No Data Received" error (chrome). The code stops in middle of looping through the DataTable. Hosting support said "The connection was reset" error was thrown.
I"m not sure if this is a timeout issue or what. I've set the executionTimeout in my web.config (with debug = false) and it didn't seem to help. I also checked the Server.ScriptTimeout property, and it does match the executionTimeout value set in the web.config.
Additionally, a timeout would normally give "Page not available" message.
Any suggestions are appreciated.
after about a minute and half
There's your problem. This is a web application? A minute and a half is a very long time for a web application to respond to a request. Long enough that it's not really worth engaging in various trickery to make it kind of sort of work.
You'll want to offload this process to be more asynchronous with the web application itself. The nature of web applications is that they should receive a request and respond in a timely manner. What you have here is a long-running process which can't respond in a timely manner. The web application can facilitate interactions with the data, but shouldn't directly handle the processing thereof in the request/response directly.
How does the web application interact with the process? Does it just start it, or does it provide information for the process to begin? I would recommend that the process itself be handled by something like a Windows Service or perhaps a Console Application. The more de-coupled from the web application, the better. Now, since I don't know anything about the process itself, I'm making a few assumptions about its behavior...
The web application can receive a request to start the process, along with any information needed for the process. It can store this in a database with a status value (pending, queued, etc.) and then respond to the user (in a timely manner) that the request has been received and the process has been queued. The web application can have a page which checks the status so that the user can see how the process is doing (if it's started, how many records it's gone through, etc.).
The offline application (Windows Service, et al) would just monitor that database for newly-queued data to be processed. When it sees it, it updates the status (running, processing, etc.) and provides any relevant feedback during the process (number of records processed, etc.) by updating that data. So the offline application and the web application are both interacting with the same data, but not in a manner which blocks the thread of the web application and prevents a response to the user.
When the process is finished, the status is again updated. The web application can show that it's finished and provide a link to download the results. The offline process could even perhaps send an email to the user when it's done, or maybe the web application can have some kind of notification system (I'm picturing the little notification icons in Facebook) which would alert the user to new activity.
This way the thread isn't blocked, the user can continue to interact with the application (if there's even anything with which to interact), etc. And you get other added benefits, too. For example, results of the process are thus saved in the database and automatically historically tracked.
It sounds like it's the browser that's timing out waiting for a response, not on the server. You can't control what the browser has set for this. What you can do is send a response of some kind to the browser, so that it knows you're still around and haven't crashed in some way.
For this to work, you can't wait until you finish building the entire string. You need to re-think your code so that instead of appending to a string, you are writing each addition to an output stream. This has the added advantage of being a much more efficient way to create your text file. For purposes keeping the browser alive, you can write out anything, as long as some data is coming back for the browser to read. Html comments can work for this. You also need to periodically flush your response stream, so that your data isn't sitting buffered on your web server. Otherwise you might still timeout.
Of course, the real solution here is to re-think your design, such that your operation doesn't take 90 seconds plus in the first place. But until you can do that, hopefully this is helpful.
it does sound like a timeout, Could you try and return the information via a View, this would certainly speed things up.(if possible).
When i had this error, i was able to resolve it by adding in the Web.config file:
<system.web>
<httpRuntime executionTimeout="600" maxRequestLength="51200" />
</system.web>
Related
I realise this question has been asked in different variations but with newer features to .net (e.g. async await) I wonder what solution is the best.
I have a C# .Net Web Forms app that has a long running task: The task handles a user request where they upload a csv data file, serialises into object, and imports to a database. The task can take a few minutes and the browser regularly times out - this causes usability issues.
I have seen many solutions whereas the user will upload the data and then the task is carried out in the background. The page will then call the server intermittently to request the status of the task, thus keeping the user informed of the progress.
I would like to know how this is achieved? The options I see on the table:
Windows Service
Web Service - how is this hosted: IIS or a windows service?
Async, Await - is this a possibility?
I think you could take two different approaches.
The first would be a pull approach, you would be keeping the state of the process per user in the server, perhaps in session, and having the process update that state, then the client can pull the actual state via ajax regularly. The ajax call is made from the client's browser, and the function can be put in the same web page that the client is viewing, there's no need to separate it if it's going to be used only from there.
The second could be a push approach, which is a bit more complex but gives you other possibilities. You would need to use a library like signalr https://www.asp.net/signalr, that allows you to communicate from the server to the client's browser, call JS functions, and push the updated state to the client's form. That could create a more functional two-way communication and a better user experience in exchange of a bit more complexity.
So I have this web application (ASP.NET MVC 4 Web Site) which has at least 2,000 online users at any time. One of the most popular pages in my application contains data about user, and this data is not located in my repository, it is contained in some external vendor which is integrated into my system. So whenever this page is drawn I have to make a call to those services (currently there are 17) and than draw the page according to the data given by them. The data is subject to change in any given moment so I cannot cache it. Everything is working OK most of the time and the CPU utilization is 5% - 30% (depending on the number of online users of course). For each service call I have timeouts of 5000 milliseconds (for service references I set the SendTimeout and for the raw HttpWebRequests' I set the TimeOut property to be equal to 5000 milliseconds) Now suppose that one service is down, the CPU utilization of my server goes unxpectidly low like 3% - 8% and the application is lagging, I mean it takes some time to load pages (any page), for instance, if in a normal mood the response from my application would have taken (150-250ms) now it takes 1-3 seconds. I'm out of ideas of what to do. I cannot decrease the timeout because some services are taking 3-4 seconds sometimes so the 5 second timeout is the lease I can give. What can I do to prevent the late response ? I know it's bit general question. Any suggestion would be appreciated. Thanks in advance.
It looks like you have a threading problem. Too many threads are waiting for response from the external service and they can not process other requests.
What I recommand you is to use Async Controller: http://www.asp.net/mvc/overview/performance/using-asynchronous-methods-in-aspnet-mvc-4
Suggestion 1
What if you replicate this data to your server?
Meaning you can have another service, that works separately and synchronize data of external service with your server... and your websites always point to your local data... Right, this is some kind of caching, and web pages can show kind of old data... but you can set replication service to check data as often as you need...
Suggestion 2
Another suggestion that come to mind, can you use push notification instead? All the web pages open and wait, where server checks the data, and notify all the clients with the fresh data... in this case only one thread will be busy with external data, and all the opened users will have fresh data as soon as it is available. As a starting point, check SignalR
I have a windows service that listens to a message queue and processes request from this message queue. I have an initial application that places messages in the message queue. The service itself actually creates some files on the fly and when these files are created I need the initial application to grab these files and essentially paint them on the screen. (the files are an html format) For the time being I have the application and the service running on my local machine so the files are just located on the c: drive of my machine.
My issue is that I need some way to signal my initial application that the service has picked up and processed my msg successfully or unsuccessfully. I have been researching how to do this and have come up empty handed. I have also been brainstorming ideas as to what I could do to make this work. One non viable solution was to insert a record into a database table when sending a msg to the queue and upon processing finished within the service update the queue record, all the while having the application query the table checking for a status of processing complete. I don't like this idea at all as I feel like it would be too intensive, and I believe that there has to be a better way of doing this. I am going to continue to research how this can be done, but I am definitely hoping someone has a better solution to this than my sql query running over and over again. Note in the past I have tried to query the event viewer for processes, and I know how to do this, but this will take far too long for the timing I am looking to accomplish.
I ended up going with the route of inserting record into database prior to call windows service. Then when the windows service finished with its process it updated this db record. All the while my page is calling a webservice via javascript which checks the db to see if the flag on the record has been set to done processing. This allows me to determine when the windows service has finished processing my msg from the queue.
A client came to me with a request to have a web app that does a lot of processing in the backend (reads from a file, writes to a web service). My question is that since this "process" (when the user clicks 'Go') may take hours, how do I make it so the processing continues after the user closes the web page? Please let me know if this does not make any sense and I can give more information. Any help would be greatly appreciated, thanks!
You have to create MS Windows Service for it.
You have provide for that Service some database table which client is going to use by your Website functionality.
This is, basically, a "batch job" requirement, and that's how I suggest that you approach it.
The client would use the web-page, not to perform the work, but rather to manage the list of batch jobs that are performing the work .. scheduling them, starting or stopping them, viewing their status and current output, and so on. (Yup, just like they did it in the 1970's, but without "//FOOBAR JOB (123,456)" ... thank god.
The user goes to the web-page and enters a request. A back-end service process on the host (the batch job manager...) now starts the job, perhaps on one computer; perhaps on several at a time. The user can, meanwhile, log off the web-site and do whatever he pleases. Then, he can come back at any time, go back to whatever the job-monitoring web page may be, and see how things are going. Stop the job, suspend/resume, what have you.
There are lots of batch-job monitoring tools out there already, for all sorts of environments, both free and commercial. So, it's not like you have to build all this stuff; you merely have to identify what off-the-shelf package works best for you and for your client.
The best possible solution will be to do the work in a windows service, and use your web app just to trigger the processing.
Scenario: A WCF service receives an XDocument from clients, processes it and inserts a row in an MS SQL Table.
Multiple clients could be calling the WCF service simultaneously. The call usually doesn't take long (a few secs).
Now I need something to poll the SQL Table and run another set of processes in an asynchronous way.
The 2nd process doesn't have to callback anything nor is related to the WCF in any way. It just needs to read the table and perform a series of methods and maybe a Web Service call (if there are records of course), but that's all.
The WCF service clients consuming the above mentioned service have no idea of this and don't care about it.
I've read about this question in StackOverflow and I also know that a Windows Service would be ideal, but this WCF Service will be hosted on a Shared Hosting (discountasp or similar) and therefore, installing a Windows Service will not be an option (as far as I know).
Given that the architecture is fixed (I.E.: I cannot change the table, it comes from a legacy format, nor change the mechanism of the WCF Service), what would be your suggestion to poll/process this table?
I'd say I need it to check every 10 minutes or so. It doesn't need to be instant.
Thanks.
Cheat. Expose this process as another WCF service and fire a go command from a box under your control at a scheduled time.
Whilst you can fire up background threads in WCF, or use cache expiry as a poor man's scheduler those will stop when your app pool recycles until the next hit on your web site and the app pool spins up again. At least firing the request from a machine you control means you know the app pool will come back up every 10 minutes or so because you've sent a request in its direction.
A web application is not suited at all to be running something at a fixed interval. If there are no requests coming in, there is no code running in the application, and if the application is inactive for a while the IIS can decide to shut it down completely until the next request comes in.
For some applications it isn't at all important that something is run at a specific interval, only that it has been run recently. If that is the case for your application then you could just keep track of when the table was last polled, and for every request check if enough time has passed for the table to be polled again.
If you have access to administer the database, there is a scheduler in SQL Server. It can run queries, stored procedures, and even start processes if you have permission (which is very unlikely on a shared hosting, though).
If you need the code on a specific interval, and you can't access the server to schedule it or run it as a service, or can't use the SQL Server scheduler, it's simply not doable.
Make you application pool "always active" and do whatever you want with your threads.