Request timing out on large operation? - c#

Trying to complete a web process, however, I am receiving a 'request time out' error. I'm not sure what I can do to get around this.
I modified the method to create a new connection for every number being passed in the for loop, but it seems to be yielding the same result.
I am more of a desktop developer, not overly versed in ASP.Net, so any light that could be shed on my issue would be great.
I've looked up info on ASP background workers, which doesn't seem to be a great way to go, and I've increased the server settings to allow a higher timeout, but still timeout if a huge number of parts are provided.
I'm also trying to avoid a separate process that is scheduled on the server to execute a submitted list of numbers. If more info is needed to make sense of this, just let me know.
Also, when I attempt to run the application locally (debug) there are never issues, only when it's placed on the live site.
Here is the exact error received:
Server Error in '/' Application.
Request timed out.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.Web.HttpException: Request timed out.
Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
Stack Trace:
[HttpException (0x80004005): Request timed out.]
And here is the code:
protected void btnSearch_Click(object sender, EventArgs e)
{
// Clear our reporting panel.
panelHolder.Controls.Clear();
// Store each line in a new array item.
string[] searchlines = txtboxSearch.Text.Replace("\n", "|").Split('|');
// Create a table row containing our main table headers.
panelHolder.Controls.Add(new LiteralControl("<table style=\"width:100%;\">" +
" <tr> " +
" <td></td> " +
" <td>Number</td> " +
" <td>Comparison</td> " +
" </tr>"));
// Variable to hold the row counts.
int j = 0;
// Store our current web members name for use in our tracking data.
string MemberName = Member.GetCurrentMember().Text;
// This table will be used solely for storing our excel exported data.
System.Data.DataTable dt = new System.Data.DataTable();
// Locate our part comparison results for every line of data supplied by our users.
for (int i = 0; i < searchlines.Count(); i++)
{
using (SqlConnection con = new SqlConnection(dbConnection))
{
// If this array item is not blank we will need to collect information about it.
if (searchlines[i].Trim() != string.Empty)
{
// Determine if data collection (reporting) is turned on.
Boolean isReporting = DataCollection();
using (SqlDataReader dr = Connect.ExecuteReader("SelectNumbers]", con,
new SqlParameter("#Number", searchlines[i].Trim()),
new SqlParameter("#CurrentMember", MemberName),
new SqlParameter("#DataCollection", isReporting)))
{
if (dr.HasRows)
{
while (dr.Read())
{
// Add our table rows containing our returned data set.
panelCompetitorHolder.Controls.Add(new LiteralControl("<tr><td>" + Convert.ToString(i + 1) + "</td>"));
AddTableData(dr, "Part Number");
AddTableData(dr, "Comparison");
// Go to our next line item.
j += 1;
}
}
}
}
}
}
// Add our table to the panel control.
panelHolder.Controls.Add(new LiteralControl("</table>"));
}

Your issue may lie in the fact that IIS assumes a maximum period of time for a given request to be processed. By default, the value is 90 seconds. Here's a few ways you can set a different amount of time:
Via Web.config - Check/add this entry on your web.config file:
<system.web>
<httpRuntime executionTimeout="N" />
</system.web>
Programatically - You may add this to your server-side code:
Server.ScriptTimeout = N;
Where N is, in both options, the desired amount of seconds for the request timeout.
Additionally, your values may be superseded/ignored if there's an entry present at the server's applicationhost.config or machine.config files, as described here:
http://www.iis.net/configreference/system.applicationhost/sites/sitedefaults/limits
If that's the case you may have to alter the corresponding entries - or, in lieu of that, alter your codebehind content.

Related

How To DEALLOCATE PREPARE Statement In C# Using Mysql.Data Client?

hope you guys are fine?
OK.. i am using MySQL.Data client/library to access and use MySQL database. I was using happily it for sometimes on quite a few project. But suddenly facing a new issue that causing me hold on my current project. :(
Because current project makes some (looks like it's a lot) db queries. and i am facing following exception :
Can't create more than max_prepared_stmt_count statements (current value: 16382)
i am closing and disposing the db engine/connection every time i am done with it. But getting damn confused why i am still getting this error.
​here is the sample code just to give you idea.. (trimmed out unnecessary parts)
//this loop call an API with pagination and get API response
while(ContinueSalesOrderPage(apiClient, ref pageNum, days, out string response, window) == true)
{
//this handle the API date for the current page, it's normally 500 entry per page, and it throws the error on 4th page
KeyValueTag error = HandleSalesOrderPageData(response, pageNum, out int numOrders, window);
}
private KeyValueTag HandleSalesOrderPageData(string response, int pageNum, out int numOrders, WaitWindow window)
{
numOrders = json.ArrayOf("List").Size;
//init db
DatabaseWriter dbEngine = new DatabaseWriter()
{
Host = dbHost,
Name = dbName,
User = dbUser,
Password = dbPass,
};
//connecting to database
bool pass = dbEngine.Connect();
//loop through all the entry for the page, generally it's 500 entries
for(int orderLoop = 0; orderLoop < numOrders; orderLoop++)
{
//this actually handle the queries, and per loop there could be 3 to 10+ insert/update query using prepared statements
KeyValueTag error = InsertOrUpdateSalesOrder(dbEngine, item, config, pageNum, orderLoop, numOrders, window);
}
//here as you can see, i disconnect from db engine, and following method also close the db connection before hand
dbEngine.Disconnect();
}
//code from DatabaseWriter class, as you see this method close and dispose the database properly
public void Disconnect()
{
_CMD.Dispose();
_engine.Close();
_engine.Dispose();
}
so, as you can see i close/dispose the database connection on each page processing, but still it shows me that error on 4th page. FYI, 4th page data is not the matter i checked that. If i skip the page and only process the 4th page, it process successfully.
and after some digging more in google, i found prepare statement is saved in database server and that needs to be close/deallocate. But i can't find any way to do that using MySQL.Data Client :(
following page says:
https://dev.mysql.com/doc/refman/8.0/en/sql-prepared-statements.html
A prepared statement is specific to the session in which it was created. If you terminate a session without deallocating a previously prepared statement, the server deallocates it automatically. 
but that seems incorrect, as i facing the error even after closing connection on each loop
so, i am at dead end and looking for some help here? 
thanks in advance
best regards
From the official docs, the role of max_prepared_stmt_count is
This variable limits the total number of prepared statements in the server.
Therefore, you need to increase the value of the above variable, so as to increase the maximum number of allowed prepared statements in your MySQL server's configuration
Open the my.cnf file
Under the mysqld section, there is a variable max_prepared_stmt_count. Edit the value accordingly(remember the upper end of this value is 1048576)
Save and close the file. Restart MySQL service for changes to take place.
You're probably running into bug 77421 in MySql.Data: by default, it doesn't reset connections.
This means that temporary tables, user-declared variables, and prepared statements are never cleared on the server.
You can fix this by adding Connection Reset = True; to your connection string.
Another fix would be to switch to MySqlConnector, an alternative ADO.NET provider for MySQL that fixes this and other bugs. (Disclaimer: I'm the lead author.)

MemoryMappedViewStream error "does not have appropriate access"

I am trying to create a MMF-backed collection that dynamically expands up to 2GB. I wanted to be able to read existing items at the same time new items are being added. It works great on my development machine but I am getting an error on some machines:
The thread tried to read from or write to a virtual address for which it does not have the appropriate access.
Has anyone seen this error before? I am likely doing something wrong with the way I am handling MemoryMappedFiles, I'm creating a 2GB MMF with the DelayAllocatePages flag so it doesn't use it all right away:
public const long ONE_GIGABYTE = 1073741824;
long maxCapacity = 2 * ONE_GIGABYTE;
mmStorage = MemoryMappedFile.CreateNew(mmfName, maxCapacity, MemoryMappedFileAccess.ReadWrite, MemoryMappedFileOptions.DelayAllocatePages, null, HandleInheritability.Inheritable);
Then I write data to it a chunk at a time, using an internal collection to keep track of where each chunk is located.
lock ( writeLock ) {
// get the most recently added item
var lastItem = _itemLocations[_itemLocations.Count - 1];
// calculate next items offset
long newOffset = lastItem.Offset + lastItem.Length;
// add next items data
using ( var mmStream = mmStorage.CreateViewStream(newOffset, itemBytes.Length, MemoryMappedFileAccess.ReadWrite) ) {
Trace.WriteLine(string.Format("Writing {0} bytes at location {1}", itemBytes.Length, newOffset));
mmStream.Write(itemBytes, 0, itemBytes.Length);
}
// add location info to list
_itemLocations.Add(new ItemLocation()
{
Offset = newOffset,
Length = itemBytes.Length
});
}
On the remote machine the first write goes ok, but the second write is causing the exception I mentioned which kills the program completely.
Writing 5973 bytes at location 0
Writing 5901 bytes at location 5973
The thread tried to read from or write to a virtual address for which it does not have the appropriate access.
Update
I have tried changing
MemoryMappedFileOptions.DelayAllocatePages
to
MemoryMappedFileOptions.None
and it stops throwing the exception, but it also allocates the full 2GB right away. I'd prefer if I could grow the MMF as needed but I guess it wont work on all machines. I'm not sure why DelayAllocatePages works on some machines and not others.

SSIS: addrows stops after adding one row out of many

I'm trying to process an xml file I'm getting from a vendor. I managed to get some c# code to read in all 26 items in the xml. This code I placed into a script component in SSIS and fed that into a Union All task. I then placed a dataviewer so I can verify what I received. I use this code to add the rows to the output buffer:
Roles roles = GetWebServiceResult(wUrl);
MessageBox.Show("We have read in " + roles.Items.Length + " items");
//Add each role entry to the output buffer.
for (int i = 0; i < roles.Items.Length; i++)
{
MessageBox.Show("Adding item " + (i + 1) + " to the output");
Transfer role = getRole(roles.Items[i]);
Output0Buffer.AddRow();
Output0Buffer.roleKey = role.roleKey;
Output0Buffer.text = role.text;
Output0Buffer.Item = role.Item;
Output0Buffer.Users = role.Users;
}
When I run this I get a popup saying that there are 26 items to process, but I only get one more popup after that, telling me that item #1 has been added. the job then stops with no errors, but I only have one row of output in the dataviewer. I don't understand why this is happening when I know that there are 25 more items to add.
Additional: On a whim, I took out the Output0Buffer code and it went through all 26 items.
I figured it out. I ran it using Ctrl-F5 and studied the output in the console. Turns out a column wasn't big enough. I made that column larger and everything works. I would have thought that error would have stopped the processing.

Is it possible that SqlException is thrown and property Number == 0?

Is it possible that SqlException will be thrown when trying to execute sql procedure and property Number will be 0?
Specific situation: domain controller is not available/not responding and no real communication with sql server occurred.
No, it should not be possible that SqlException.Number is 0, except for the following cases:
Read-only routing failure
Server had severe error processing query
Processed cancellation while parsing results
Failed to create user instance
Assuming you're not using read-only routing of SQL Server 2016, then none of these cases could apply to the scenario you described.
Remaining cases are either a non-zero number from sysmessages, or a Win32 error code (which, since 0 == ERROR_SUCCESS, would also never be zero).
In practice you will see something along the lines of "Cannot create SSPI context", or "A transport-level error has occurred when sending the request to the server", or a "An existing connection was forcibly closed by the remote host", and those have nonzero Win32 error codes too.
Reference: https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlerror.number(v=vs.110).aspx
It may be worth noting that SqlException.Number is synonym for the Number property of the first entry in its Errors array, and that array might contain more than one item.
Answering my own question. Yes, it is possible. We added aditional code to catch:
catch (SqlException ex)
{
for (int i = 0; i < ex.Errors.Count; i++)
{
this.logger.Error("Index #" + i + "\n" +
"Message: " + ex.Errors[i].Message + "\n" +
"Error Number: " + ex.Errors[i].Number + "\n" +
"LineNumber: " + ex.Errors[i].LineNumber + "\n" +
"Source: " + ex.Errors[i].Source + "\n" +
"Procedure: " + ex.Errors[i].Procedure + "\n");
}
if (ex.Number == 0)
{
this.logger.Warn("Exception was caught but ex.Number == 0! Changing to -69.");
oResults.Add("_ExitCode", "-69");
}
}
and this was logged today.
2017-01-17 08:05:08,282 [r074008903] ERROR ProcessLogger Index #0
Message: A severe error occurred on the current command. The results, if any, should be discarded.
Error Number: 0
LineNumber: 0
Source: .Net SqlClient Data Provider
Procedure:
2017-01-17 08:05:08,282 [r074008903] ERROR ProcessLogger Index #1
Message: A severe error occurred on the current command. The results, if any, should be discarded.
Error Number: 0
LineNumber: 0
Source: .Net SqlClient Data Provider
Procedure:
2017-01-17 08:05:08,283 [r074008903] WARN ProcessLogger Exception was caught but ex.Number == 0! Changing to -69.
Because in other place _ExitCode was used to determine is something was wrong in this specific situation we had false information.
It is possible.
I receive 0 when trying to connect SQL which is starting inside ubuntu container. According to this: https://learn.microsoft.com/en-us/sql/relational-databases/errors-events/database-engine-events-and-errors?view=sql-server-ver15
that is undocumented and, probably, is a bug.
Here is my situation, for example:

The file size exceeds the limit allowed and cannot be saved on EventLog

Getting this exception The file size exceeds the limit allowed and cannot be saved when writing to event viewer using EventLog
Code used:
string cs = "LoggingService";
EventLog elog = new EventLog();
if (!EventLog.SourceExists(cs))
{
EventLog.CreateEventSource(cs, cs);
}
elog.Source = cs;
elog.EnableRaisingEvents = true;
elog.WriteEntry(message);
Stack trace:
System.ComponentModel.Win32Exception (0x80004005): The file size exceeds the limit allowed and cannot be saved
at System.Diagnostics.EventLogInternal.get_OldestEntryNumber()
at System.Diagnostics.EventLogInternal.StartRaisingEvents(String currentMachineName, String currentLogName)
at System.Diagnostics.EventLogInternal.set_EnableRaisingEvents(Boolean value)
Things tried:
http://support.microsoft.com/kb/328380#top
When I put this statement elog.Clear() before elog.EnableRaisingEvents = true;
I am getting different exception
System.ComponentModel.Win32Exception (0x80004005): Access is denied
at System.Diagnostics.EventLogInternal.Clear()
The above code is executed by a web service which runs under LocalSystem which has full permission on the computer.
OS: windows server 2008R2 and .NET 4.0
There is a method EventLog.ModifyOverflowPolicy that modifies log policy.
I believe EventLog.ModifyOverflowPolicy(OverflowAction.OverwriteAsNeeded,0) could be helpful.
Event Viewer - Properties - Maximum log size was 1028. I bumped it and it started working fine.
But the same code was working from another window service before I made the above the change.
which make me think what could have happened
The message I was trying to log was just a line (less then 255 characters)

Categories

Resources