I have a WebAPI service which connects to an Oracle database using Oracle.ManagedDataAccess.dll. Each time after a reset of the application pool (or a deployment) there is a long delay on the first OracleConnection.Open() statement. It's typically around 8 seconds. Subsequent calls are around ~0.5 seconds each.
After reading lots of suggestions regarding server OS and networking issues, i have narrowed it down to the oracle client itself. If I remote debug my code, set a breakpoint on the open statement, and then run Sysinternals Process Monitor i can confirm that the first open statement produces 544 entries, second and subsequent tests produce 2 entries.
The entries are quite random, but mostly relate to Cryptography. A quick overview of the logs:
RegOpenKey, HKLM\SOFTWARE\Microsoft\Cryptography\Defaults\Provider Types\Type 001
RegOpenKey, HKLM\SOFTWARE\Microsoft\Cryptography\Defaults\Provider\Microsoft Strong Cryptographic Provider
RegSetInfoKey, HKLM\SOFTWARE\Microsoft\Bryptography\MachineGuid
These are repeated several times, then there are sections like below:
RegQueryValue, HKLM\System\CurrentControlSet\WinSock2\Parameters\Protocol_Catalog9
RegCreateKey, HKLM\System\CurrentControlSet\Services\Tcpip\Parameters
RegCreateKey, HKLM\System\CurrentControlSet\Services\DnsCache\Parameters
RegOpenKey, HKLM\SOFTWARE\Policies\Microsoft\Windows NT\DNSClient
Then there are several reads of the machine.config file followed by multiple TCP connect and receive to the oracle port 1521. Following this is a section reading the time zone from the registry.
My question is, why is the oracle client doing all of this at first open? Is there any way i can predetermine the answer to some of these questions? (like configure the time zone so it doesn't have to 'ask' Oracle for it)?
Only time I have seen something like this was when the address in the tns connect descriptor was not fully qualified, ie host=computername, instead of host=computername.domain.com.
Issue is likely dns resolution as it goes thru suffixes. I imagine you could put in an ip and eliminate dns altogether as a test. Consider posting your tns entry and connection string as well.
FYI, a lot of things are happening when that first connection is created, ie the pool is established and connections are actually opened vs just fetched from the pool, initial parameters for self-tuning are initialized, etc, so i think number of reg reads is probably a red hearing.
Related
I have tiny exe's, I am developing with c# the size is max 100kb - 500kb.
A 3rd party application opens these exes for me, when my application is opened, it reads some data from MySql database, processes it, then writes the result back to MySql database.
they are very normally useful on servers, but sometimes I get this error, it's not clear when, sometimes after 1 week, sometimes twice a day
Authentication to host '192.168.1.200' for user 'root' using method
'mysql_native_password' failed with message: Reading from the stream
has failed.
this error does not happen on every computer, it does not happen all the time, I never encounter it when compiling on my own computer
I tried a lot of things so far I couldn't come to a conclusion by looking at some suggestions on this site
I added or removed MySql password is not a solution , ports defined to firewall and antiviruses
private string connectionstring = "Server={0};Database={1};port={2};username={3};password={4};CharSet=latin1;SslMode=None";
First of all, I have been started to studying the C# from 6 months ago and I'm not good at English.
So I'm sorry if I say something that you cannot understand.
Now, I'm developing the application with C# that ONLY connect to MySQL server(both of remote and local).
And I use a MySql.Data.Client package, it's version 8.0.12.
This ONLY means "I don't send any queries. I want to find that I can connect to server using a UserName and Password.".
Then, I wrote this code. And I have two questions, and I want to get any advises.
sendCommand = string.Format("host={0}; userid={1}; password={2}; SslMode=none;", pIPAddress.ToString(), mOption1, mOption2);
MySqlConnection mysqlConnention = new MySqlConnection(sendCommand);
mysqlConnention.Open();
if (mysqlConnention.State.ToString() == "Open")
{
result = true;
}
mysqlConnention.Close();
Q1
The application should not send unnecessary packets to server.
However I found it from WireShark and it's "show variable" command.
I tried to change some properties about ConnectionString(in this code, it's called "send Command") because I want not to send "show variable" command. (for example, ChacheServerProperties=true, AllowBatch=false, and so on...)
Can I connect without "show variable" command?
Q2
(It's soloved but I cannot find the causion)
When I started developing this application, I used MySql.Data.Client 8.0.11.
But in that time, the MySqlConnection.Open Method was taken a long time for connection to server.(It was about 7~10 second.)
The network problems was none, and I was able to connect using cmd.exe without a long time.(It was not taken a second.)
I tried any patterns, changed server to connect, restarted OS(both of the application side and server side), changed ConnectionString properties).
MySQL server was not dying and It looked like the cause is not in MySQL server and network.(Getting the application "ServerGreeting" packet from server was no time but next action that the application send "LoginRequest" packet to server was taken a long time.)
However I was not able to slove this problem. But I updeted MySql.Data.Client version to 8.0.12, I can slove!
Is this problem caused by MySql.Data.Client 8.0.11?
And have I any solution?
The application should not send unnecessary packets to server. … Can I connect without "show variable" command?
You can, by using an alternative ADO.NET library: MySqlConnector. It sends a lot fewer packets when opening the connection (but it may not send the absolute minimum possible).
But in that time, the MySqlConnection.Open Method was taken a long time for connection to server.(It was about 7~10 second.) … Is this problem caused by MySql.Data.Client 8.0.11?
This sounds very much like MySQL bug 80030, which is a bug in 8.0.11. That case says it's going to be fixed in 8.0.13, not 8.0.12, but perhaps Oracle changed their release plans after making that comment.
This could also be fixed by using MySqlConnector, which never used an inefficient WMI query that was causing the performance problem in MySql.Data.
I have created a windows service using c#.NET, The service will updated oracle tables whenever it receives new files. I have kept timer control and the time limit as 30 seconds. I am using ODP.NET as data access layer.
The very first time I will get error, but subsequently the service will work fine. If service is Idle for a long time if it receives a file, I will get "connection lost error", but after if we receives file it will loaded successfully.
Kindly suggest me do I need add any properties in connection string to fix this error?
Hello Karthik Two issues here it seems.
You are best to open and close a new connection each time your service is called.
Windows services quickly go to a latent state if not called and they will respond slower on the next call. If the caller does not have a sufficient timeout value to accomodate this lag then it will return a time out error. If you address these two points you should be fine.
I'm trying to understand how MS Enterprise Library's data access block manages its connections to SQL. The issue I have is that under a steady load (from a load test), at 10 minute intervals the number of connections to SQL increases quickly - which causes noticeable jump in page response times from the website.
This is the scenario I'm running:
Visual Studio load test tools, running against 3 web servers behind a load balancer
The tools give full visibility over the performance counters to all web + DB boxes
The tests take ~10 seconds each, and perform 4 inserts (form data), and some trivial selects
There are 60 tests running concurrently. There is no increase or decrease in load during the entire test.
The test is run for between 20 minutes and 3 hours, with consistent results.
And this is the issue we see:
Exactly every 10 minutes, the performance counter from SQL for SQL General: User Connections increases - by ~20 connections total
The pages performing the HTTP post / DB insert are the ones most significantly affected. The other pages show moderate, but noticeable rises.
The CPU/memory load on the web servers is unaffected
This increase corresponds with a notable bump in page response times - E.g. from .3 seconds average to up to 5 seconds
After ~5 minutes it releases many of the connections, with no affect on web performance
The following 5 minutes of testing gives the same (normal) web performance
Ultimately, the graph looks like a square wave
Happens all over again, 10 minutes after the first rise
What I've looked at:
Database calls:
All calls in the database start with:
SqlDatabase database = new SqlDatabase([...]);
And execute either proc with no required output:
return database.ExecuteScalar([...], [...]);
Or read wrapped in a using statement:
using (SqlDataReader reader = (SqlDataReader)database.ExecuteReader([...], [...]))
{
[...]
}
There are no direct uses of SqlConnection, no .Open() or .Close() methods, and no exceptions being thrown
Database verification:
We've run SQL profiler over the login / logout events, and taken snapshots with the sp_who2 command, showing who owns the connections. The latter shows that indeed the web site (seen by machine + credential) are holding the connections.
There are no scheduled jobs (DB or web server), and the user connection load is stable when there is no load from the web servers.
Connection pool config
I know the min & max size of the connection pool can be altered with the connection string.
E.g.:
"Data Source=[server];Initial Catalog=[x];Integrated Security=SSPI;Max
Pool Size=75;Min Pool Size=5;"
A fall back measure may be to set the minimum size to ~10
I understand the default max is 100, and the default min is 0 (from here)
I'm a little bit lithe to think of connection pooling (specific to this setting) and the User Connections performance counter from SQL. This article introduces these connection pools as being used to manage connection string, which seems different to what I assume it does (hold a pool of connections generally available, to avoid the cost of re-opening them on SQL)
I still haven't seen any configuration parameters that are handily defaulting to 5 or 10 minutes, to zero in on...
So, any help is appreciated.
I know that 10 minute spikes sounds like a change in load, or new activity is happening - but we've worked quite hard to isolate those & any other factors - and for this question, I am hoping to understand EL scaling its connections up & down.
Thanks.
So, it turns out that SQL user connections are created & added to the pool whenever all other connections are busy. So when long-running queries occur, or the DB is otherwise unresponsive, it will choose to expand to manage the load.
The cause of this in our case happened to be a SQL replication job (unfortunate, but found...) - And the changes in the # of User Connections was just a symptom, not a possible cause.
Although the cause turned out to be elsewhere, I now feel I understand the connection pool management, from this (and assumably other) SQL libraries.
I have run into a frustrating issue which I originally thought was a connection leak but that does not seem to be the case. The secnario is this: the data access for this application is using the Enterprise Libraries (v4) from Microsoft. All data access calls are wrapped in using statements such as
using (DbCommand dbCommand = db.GetStoredProcCommand("sproc"))
{
db.AddInParameter(dbCommand, "MaxReturn", DbType.Int32, MaxReturn);
...more code
}
Now the index of this application makes 8 calls to the database to load everything and I can bring the application to its knees by refreshing the index about 15 times. It seems that when the the database reaches 113 connections is when I recieve this error. Here is what makes this weird:
I have run similar code with the entlib on high traffic sites and have NEVER had this problem ever.
If I kill all the connections to the database and get the production application back up and running everytime I refresh the application I can run this SQL
SELECT DB_NAME(dbid) as 'Database Name',
COUNT(dbid) as 'Total Connections'
FROM sys.sysprocesses WITH (nolock)
WHERE dbid > 0
GROUP BY dbid
I can see the number of connections actively increasing with each page refresh. Running the same code on my local box with the same connection string does not cause this problem. Further if the production website is down I can fire up the site via Visual Studio and run it fine and the only difference between the two is that the production site has Windows authentication turned on and my local copy doesn't. Turning windows authentication off seems to have no effect on the server.
I have absolutely no clue what is causing this or why the connections are not being disposed of in SQL Server. The EntLib objects do no explose .Close() methods for anything so I can't explictily close the object.
Any thoughts?
Thanks!
Edit
Wow I just noticed that I never actually posted the error message. Oy. The actual connection error is: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Check that the stored procedure you are executing is not running into a row or table lock. Also if you can possibly try to deploy in another server and check if the application would crawl again.
Also try to increase the maximum allowed connections for your SQL server.
think the “Timeout Expired” error is a general issue and may have seveal causes. Increasing the TimeOut can solve some of them but not all.
You may also refer to the following links to troubleshoot and fix the error
http://techielion.blogspot.com/2007/01/error-timeout-expired-timeout-period.html
Could it be a configuration issue on the server?
How do you make a connection to the database on the production server?
That might be an area worth looking into.
While I don't know the answer I can suggest that for some reason connections are not being closed by you application when run in production. (Stating the obvious)
You might want examine your network configuration between the web server and sql server. High latency networks can cause connections not being closed in time.
Also it might help looking at the performance counters listed in the end of the following msdn article:
http://msdn.microsoft.com/en-us/library/8xx3tyca%28VS.71%29.aspx
Finally, if nothing else helps, I'd get debugger and Enterprise Library source code on production and debug your code inside the enterprise library to find out why connections are not being closed.
Silly question are you properly closing your DataReader? If not this could be the problem and the difference in behaviour between dev and prod can be caused by different garbage collection patterns.
I would disable connection pooling and try to suppress it (heh). Just add ";Pooling=false" to your connection string.
Or, perhaps you could add something like the following 'cleanup' code to your page (which closes any connection left open when the page unloads) - right in the 'using' clause:
System.Web.UI.Page page = HttpContext.Current.Handler as System.Web.UI.Page;
if (page != null) {
page.Unload += (EventHandler)delegate(object s, EventArgs e) {
try {
dbCommand.Connection.Close();
} catch (Exception) {
} finally {
result = null;
}
};
}
Also, make sure you've enabled the 'shared memory' protocoll if your SQL server and IIS are on the same machine (a real performance booster)!