I have been making a multithreaded server, I cant share much but I will tell in clear theory how it is working.
I have a server that goes in a while(true) loop and it waits for a connection. Everything works, it makes the multithreads. and the server does work when I read one at a time. However when I decide to run 10 threads, it returns and says it fails to read the message (in the cmd window).
Why is this or why could this be? I am also reading in the message by:
int characterNumer;
characterNumber = streamReader.read();
while(CharacterNumber > 0)
{
message += (char)CharacterNumber;
characterNumber = streamreader.read();
}
the second line is in a try catch statement twice so it does read. but for the server it fails. why is this?
What would you recommend or advise? It does work perfectly when it is not multithreading.
I'm using the .NET Connector to access a MySQL database from my C# program. All my queries are done with MySqlCommand.BeginExecuteReader, with the IAsyncResults held in a list so I can check them periodically and invoke appropriate callbacks whenever they finish, fetching the data via MySqlCommand.EndExecuteReader. I am careful never to hold one of these readers open while attempting to read results from something else.
This mostly works fine. But I find that if I start two queries at the same time, then I get the dreaded MySqlException: There is already an open DataReader associated with this Connection which must be closed first exception in EndExecuteReader. And this is happening the first time I invoke EndExecuteReader. So the error message is full of baloney; there is no other open DataReader at that point, unless the connector has somehow opened one behind the scenes without me calling EndExecuteReader. So what's going on?
Here's my update loop, including copious logging:
for (int i=queries.Count-1; i>=0; i--) {
Debug.Log("Checking query: " + queries[i].command.CommandText);
if (!queries[i].operation.IsCompleted) continue;
var q = queries[i];
queries.RemoveAt(i);
Debug.Log("Finished, opening Reader for " + q.command.CommandText);
using (var reader = q.command.EndExecuteReader(q.operation)) {
try {
q.callback(reader, null);
} catch (System.Exception ex) {
Logging.LogError("Exception while processing: " + q.command.CommandText);
Logging.LogError(ex.ToString());
q.callback(null, ex.ToString());
}
}
Debug.Log("And done with callback for: " + q.command.CommandText);
}
And here's the log:
As you can see, I start both queries in rapid succession. (This is the first thing my program does after opening the DB connection, just to pin down what's happening.) Then the first one I check says it's done, so I call EndExecuteReader on it, and boom -- already it claims there's another open one. This happens immediately, before it even gets to my callback method. How can that be?
Is it not valid to have two open queries at once, even if I only call EndExecuteReader on one at a time?
When you run two queries concurrently, you must have two Connection objects. Why? Each Connection can only handle one query at a time. It looks like your code got into some kind of race condition where some of your concurrent queries worked and then a pair of them collided and failed.
At any rate your system will be more resilient in production if you can keep your startup sequences simple. If I were you I'd run one query after another rather than trying to run them all at once. (Obvs if that causes real performance problems you'll have to run them concurrently. But keep it simple until you need it to be complex.)
Application/Code description:
My application is based on c# and uses SQL Server CE and iv'e got this exception only twice at the same code location. the crash with this exception was not introduced till this version. the only change in this version was changing the .net framework to 4.5.2.
I'm getting access violation exception on the dispose of an SqlCeConnection with the following error:
Attempted to read or write protected memory. This is often an
indication that other memory is corrupt.
This exception is not intercepted by the try catch clause of .net- it causes a crash.
In my code I use the following to run
try
{
var connectionString = string.Format("{0}{1}{2}", "Data Source=", _localDB, ";File Mode=Read Write;Max Database Size=4000;Persist Security Info=False;");
using (var sqlCeConnection = new SqlCeConnection(connectionString))
{
using (var sqlCeCommand = new SqlCeCommand())
{
sqlCeCommand.Connection = sqlCeConnection;
sqlCeCommand.CommandText = "SELECT * FROM Application";
sqlCeConnection.Open();
var result = (string)sqlCeCommand.ExecuteScalar();
isValid = !IsValid(result);
}
}
}
catch (Exception ex)
{
_log.Error("exception", ex);
}
call stack for the first crash:
ntdll!ZwWaitForMultipleObjects+a
KERNELBASE!WaitForMultipleObjectsEx+e8
kernel32!WaitForMultipleObjectsExImplementation+b3
kernel32!WerpReportFaultInternal+215
kernel32!WerpReportFault+77
kernel32!BasepReportFault+1f
kernel32!UnhandledExceptionFilter+1fc
ntdll! ?? ::FNODOBFM::`string'+2365
ntdll!_C_specific_handler+8c
ntdll!RtlpExecuteHandlerForException+d
ntdll!RtlDispatchException+45a
ntdll!KiUserExceptionDispatcher+2e
sqlcese35!__SafeRelease+c
sqlcese35!Column::`vector deleting destructor'+5c
sqlcese35!Object::DeleteObjects+39
sqlcese35!Table::`vector deleting destructor'+45
sqlcese35!Table::Release+27
sqlcese35!HashTable::~HashTable+2a
sqlcese35!Store::~Store+12b
sqlcese35!Store::Release+2a
sqlceme35!ME_SafeRelease+17
DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr ByRef)+78
[[InlinedCallFrame] (System.Data.SqlServerCe.NativeMethods.SafeRelease)] System.Data.SqlServerCe.NativeMethods.SafeRelease(IntPtrByRef)
System.Data.SqlServerCe.SqlCeConnection.ReleaseNativeInterfaces()+147
System.Data.SqlServerCe.SqlCeConnection.Dispose(Boolean)+f1
System_ni!System.ComponentModel.Component.Dispose()+18
call stack for the second crash:
ntdll!NtWaitForMultipleObjects+a
KERNELBASE!WaitForMultipleObjectsEx+e8
kernel32!WaitForMultipleObjectsExImplementation+b3
kernel32!WerpReportFaultInternal+215
kernel32!WerpReportFault+77
kernel32!BasepReportFault+1f
kernel32!UnhandledExceptionFilter+1fc
ntdll! ?? ::FNODOBFM::`string'+2335
ntdll!_C_specific_handler+8c
ntdll!RtlpExecuteHandlerForException+d
ntdll!RtlDispatchException+45a
ntdll!KiUserExceptionDispatcher+2e
<Unloaded_sqlcese35.dll>+7c88c
<Unloaded_sqlceqp35.dll>+102790
0x06ccc898
0x06f9efc8
0x1eca8018
0x1f207400
<Unloaded_sqlcese35.dll>+228dc
0x00000004
0x2edff008
0x00000002
0x00000003
0x00000004
<Unloaded_sqlcese35.dll>+3fbd9
0x06ccc898
DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr ByRef)+78
[[InlinedCallFrame] (System.Data.SqlServerCe.NativeMethods.SafeRelease)] System.Data.SqlServerCe.NativeMethods.SafeRelease(IntPtrByRef)
System.Data.SqlServerCe.SqlCeConnection.ReleaseNativeInterfaces()+147
System.Data.SqlServerCe.SqlCeConnection.Dispose(Boolean)+f1
System_ni!System.ComponentModel.Component.Dispose()+1b
I found some references on the internet that suggests some solutions:
Probable solution: check multithreading issue on the same connection (attempted to read write protected memory. this is often an indication that other memory is corrupt)
Rejection:
a. the connection is created in the using brackets and doesn't get reused.
b. the calling method is called every 5 minutes and verified via the dump file that it was not called simultaneously.
Probable solution: sql ce version mismatch (http://blogs.msdn.com/b/sqlservercompact/archive/2009/05/06/troubleshooting-access-violation-exception-while-using-sql-server-compact-database-with-ado-net-provider.aspx)
Probable Rejection: I can see in the version installed is 3.5 SP2 (3.5.8080.0) and from the modules located in the dump I can see the sqlceme35.dll, System.Data.SqlServerCe.dll DLL's are with version 3.05.8080.0
Probable solution which is in question is the following:
https://stackoverflow.com/a/20492181/1447518
Probable Rejection: it doesn't sound right from a statistical perspective- the code crashed twice in the same place although there another place in the application code which writes and read to a different DB and the application didn't crash there.
The last thing I was thinking about, may suggest a unload problem of DLLs (take a look at the second call stack). My guess is that the dll's are unloaded from the application while the application needed them in order to do a dispose, but it seams a bit blurry and a 'long shot'
My question is: what may cause the problem, and what is a probable solution?
Although this solution is not yet verified, the solution is as follows:
From the second call stack i can see there is unload of native DLL's, my guess was that the dispose method of the SQL connection was using one of the methods it currently disposed.
I verified thorough the Process dump that all the SqlCeConnection types were in the process of dispose.
Seeing ErikEj comment made me realize it's will be better if i will take a look in the code differences that was made between SQL-CE 3.5 to 4.0 (System.Data.SqlServerCe.dll).
after viewing the code, i could see that the method of the release was moved to a later position inside the dispose method.
In addition i could see that before calling SafeRelease there was another check that checks if the native DLLs that were needed for safe release was released already- and throw an exception.
bottom line, SQL-CE 4.0 has 2 solutions for the same issue.
my guess is that this issue was caused because of this.
the solution for now was to keep a connection during all the application life-cycle (which has no connection string), this cause the pointer pool to keep the native Dlls in the memory for all the application life-cycle.
the better solution is to move to SQL-CE 4.0 .
I am getting error
80004005 There is a file sharing violation. A different process might be using the file.
when trying to open a SqlCeConnection.
Is there a way to close a SQL Server CE database programmatically, to try to nip that problem in the bud? Something like (pseudocode):
SqlCeDatabase SQLCeDb = "\My Documents\HHSDB003.sdf";
if (SQLCeDb.IsOpen)
{
SQLCeDb.Close();
}
?
Or a way to set the connection so that it doesn't care if the database is open elsewhere/wise, such as:
SqlCeConnection conn = new SqlCeConnection(#"Data Source=\My Documents\HHSDB003.sdf;File Mode = 'shared read'");
...or:
SqlCeConnection conn = new SqlCeConnection(#"Data Source=\My Documents\HHSDB003.sdf;File Mode = 'read write'");
I can't test these at present, because I'm back to getting
Cannot copy HHS.exe The device has either stopped responding or has been disconnected
when I attempt to copy over a new version of the .exe to the handheld.
If there's something more frustrating to program against (and "against" is the correct word here, I think) than the prehistoric versions of Windows CE / Compact Framework / .NET, I'm not at all sure I want to know what it is.
UPDATE
Adding to my frustrusion (haywire combination of confusion and frustration), I found the following at http://www.pocketpcfaq.com/faqs/activesync/exchange_errors.php:
0x80004005 N/A Synchronization failed due to a device software error. Contact your network administrator.
1. Obtain the latest Pocket PC End User Update from your service provider.
UPDATE 2
Is this possibly problematic (than all but the first setting is blank):
UPDATE 3
With this code:
private void menuItemTestSendingXML_Click(object sender, System.EventArgs e)
{
string connStr = "Data Source=My Documents\\HHSDB003.SDF";
SqlCeConnection conn = null;
try
{
try
{
conn = new SqlCeConnection(connStr);
conn.Open();
MessageBox.Show("it must have opened okay");
}
finally
{
conn.Close();
}
}
catch (Exception ex)
{
if (null == ex.InnerException)
{
MessageBox.Show("inner Ex is null");
}
MessageBox.Show(String.Format("msg is {0}", ex.Message));
}
}
...I am now seeing "it must have opened okay" (that's a good thing, but...why it's now working, I have no idea, because the code has not changed since I last ran it and it failed. Something beyond the code must have been at play.
The only thing I can think of that happened that MAY have had a bearing on this change is that, thinking there may have been a rogue instance of either the .exe or its ancillary dll in memory on the handheld device, I wrote an quick-and-dirty utility that looped through the running processes, looking for them and, if finding them, killing them, but they were not there, so the utility really did "nothing" (maybe the Hawthorne effect?).
That is the way working with this combination of tools and technologies seems to go, though: everything is working fine one minute and the next, BAM! It no longer is. Then the reverse can also happen: for no apparent reason it seems to "heal itself".
In the interests of "full disclosure," here is the utility code:
// Got this from http://www.codeproject.com/Articles/36841/Compact-Framework-Process-class-that-supports-full
private void btnKillRogue_Click(object sender, EventArgs e)
{
ProcessInfo[] list = ProcessCE.GetProcesses();
foreach (ProcessInfo item in list)
{
MessageBox.Show("Process item: " + item.FullPath);
if (item.FullPath == #"\Windows\iexplore.exe") item.Kill(); //<= this was the example search; it probably could be a problem, so I'll use it, too
if (item.FullPath.EndsWith("HHS.exe"))
{
MessageBox.Show("about to kill hhs.exe");
item.Kill();
}
if (item.FullPath.EndsWith("HUtilCE.dll"))
{
MessageBox.Show("about to kill hutilce.dll");
item.Kill();
}
}
}
Maybe there was an instance of iexplore.exe resident in memory that was problematic (I'm not showing a messagebox if that is what is found)...?
As an attempt to claim unused bounty ... do not, however, feel obligated to pass around free points on my behalf ...
Aside from force killing of possible tasks, had you rebooted the system amidst your search for an answer? If your tool did not return the message, it is certainly possible that a reboot would have done the very thing that you had attempted with the kill utility - or possible iexplore.exe had something to do with it ... the lack of the additional messagebox may leave you never knowing - unless this issue occurs again.
If no rebooting occurred, then perhaps whatever program/dll was held in memory by some other process concluded its task and released it hold.
There are several scenarios that might have occurred, it is certainly hard to determine with absolution; hence the lack of answers. I would be interested, though, if this problem occurred again.
I'm performing a large number of INSERTS to a SQLite database. I'm using just one thread. I batch the writes to improve performance and have a bit of security in case of a crash. Basically I cache up a bunch of data in memory and then when I deem appropriate, I loop over all of that data and perform the INSERTS. The code for this is shown below:
public void Commit()
{
using (SQLiteConnection conn = new SQLiteConnection(this.connString))
{
conn.Open();
using (SQLiteTransaction trans = conn.BeginTransaction())
{
using (SQLiteCommand command = conn.CreateCommand())
{
command.CommandText = "INSERT OR IGNORE INTO [MY_TABLE] (col1, col2) VALUES (?,?)";
command.Parameters.Add(this.col1Param);
command.Parameters.Add(this.col2Param);
foreach (Data o in this.dataTemp)
{
this.col1Param.Value = o.Col1Prop;
this. col2Param.Value = o.Col2Prop;
command.ExecuteNonQuery();
}
}
this.TryHandleCommit(trans);
}
conn.Close();
}
}
I now employ the following gimmick to get the thing to eventually work:
private void TryHandleCommit(SQLiteTransaction trans)
{
try
{
trans.Commit();
}
catch (Exception e)
{
Console.WriteLine("Trying again...");
this.TryHandleCommit(trans);
}
}
I create my DB like so:
public DataBase(String path)
{
//build connection string
SQLiteConnectionStringBuilder connString = new SQLiteConnectionStringBuilder();
connString.DataSource = path;
connString.Version = 3;
connString.DefaultTimeout = 5;
connString.JournalMode = SQLiteJournalModeEnum.Persist;
connString.UseUTF16Encoding = true;
using (connection = new SQLiteConnection(connString.ToString()))
{
//check for existence of db
FileInfo f = new FileInfo(path);
if (!f.Exists) //build new blank db
{
SQLiteConnection.CreateFile(path);
connection.Open();
using (SQLiteTransaction trans = connection.BeginTransaction())
{
using (SQLiteCommand command = connection.CreateCommand())
{
command.CommandText = DataBase.CREATE_MATCHES;
command.ExecuteNonQuery();
command.CommandText = DataBase.CREATE_STRING_DATA;
command.ExecuteNonQuery();
//TODO add logging
}
trans.Commit();
}
connection.Close();
}
}
}
I then export the connection string and use it to obtain new connections in different parts of the program.
At seemingly random intervals, though at far too great a rate to ignore or otherwise workaround this problem, I get unhandled SQLiteException: Database file is locked. This occurs when I attempt to commit the transaction. No errors seem to occur prior to then. This does not always happen. Sometimes the whole thing runs without a hitch.
No reads are being performed on these files before the commits finish.
I have the very latest SQLite binary.
I'm compiling for .NET 2.0.
I'm using VS 2008.
The db is a local file.
All of this activity is encapsulated within one thread / process.
Virus protection is off (though I think that was only relevant if you were connecting over a network?).
As per Scotsman's post I have implemented the following changes:
Journal Mode set to Persist
DB files stored in C:\Docs + Settings\ApplicationData via System.Windows.Forms.Application.AppData windows call
No inner exception
Witnessed on two distinct machines (albeit very similar hardware and software)
Have been running Process Monitor - no extraneous processes are attaching themselves to the DB files - the problem is definitely in my code...
Does anyone have any idea whats going on here?
I know I just dropped a whole mess of code, but I've been trying to figure this out for way too long. My thanks to anyone who makes it to the end of this question!
brian
UPDATES:
Thanks for the suggestions so far! I've implemented many of the suggested changes. I feel that we are getting closer to the answer...however...
The code above technically works however it is non-deterministic! It is not guaranteed to do anything aside from spin in neutral forever. In practice it seems to work somewhere between the 1st and 10th iteration. If i batch my commits at a reasonable interval damage will be mitigated but I really do not want to leave things in this state...
More suggestions welcome!
It looks like you failed to link the command with the transaction you've created.
Instead of:
using (SQLiteCommand command = conn.CreateCommand())
You should use:
using (SQLiteCommand command = new SQLiteCommand("<INSERT statement here>", conn, trans))
Or you can set its Transaction property after its construction.
While we are at it - your handling of failures is incorrect:
The command's ExecuteNonQuery method can also fail and you are not really protected. You should change the code to something like:
public void Commit()
{
using (SQLiteConnection conn = new SQLiteConnection(this.connString))
{
conn.Open();
SQLiteTransaction trans = conn.BeginTransaction();
try
{
using (SQLiteCommand command = conn.CreateCommand())
{
command.Transaction = trans; // Now the command is linked to the transaction and don't try to create a new one (which is probably why your database gets locked)
command.CommandText = "INSERT OR IGNORE INTO [MY_TABLE] (col1, col2) VALUES (?,?)";
command.Parameters.Add(this.col1Param);
command.Parameters.Add(this.col2Param);
foreach (Data o in this.dataTemp)
{
this.col1Param.Value = o.Col1Prop;
this. col2Param.Value = o.Col2Prop;
command.ExecuteNonQuery();
}
}
trans.Commit();
}
catch (SQLiteException ex)
{
// You need to rollback in case something wrong happened in command.ExecuteNonQuery() ...
trans.Rollback();
throw;
}
}
}
Another thing is that you don't need to cache anything in memory. You can depend on SQLite journaling mechanism for storing incomplete transaction state.
Run Sysinternals Process Monitor and filter on filename while running your program to rule out if any other process does anything to it and to see what exacly your program is doing to the file. Long shot, but might give a clue.
We had a very similar problem using nested Transactions with the TransactionScope class. We thought all database actions occurred on the same thread...however we were caught out by the Transaction mechanism...more specifically the Ambient transaction.
Basically there was a transaction higher up the chain which, by the magic of ado, the connection automatically enlisted in. The result was that, even though we thought we were writing to the database on a single thread, the write didn't really happen until the topmost transaction was committed. At this 'indeterminate' point the database was written to causing it to be locked outside of our control.
The solution was to ensure that the sqlite database did not directly take part in the ambient transaction by ensuring we used something like:
using(TransactionScope scope = new TransactionScope(TransactionScopeOptions.RequiresNew))
{
...
scope.Complete()
}
Things to watch for:
don't use connections across multiple threads/processes.
I've seen it happen when a virus scanner would detect changes to the file and try to scan it. It would lock the file for a short interval and cause havoc.
I started facing this same problem today: I'm studying asp.net mvc, building my first application completely from scratch. Sometimes, when I'd write to the database, I'd get the same exception, saying the database file was locked.
I found it really strange, since I was completely sure that there was just one connection open at that time (based on process explorer's listing of active file handles).
I've also built the whole data access layer from scratch, using System.Data.SQLite .Net provider, and, when I planned it, I took special care with connections and transactions, in order to ensure no connection or transaction was left hanging around.
The tricky part was that setting a breakpoint on ExecuteNonQuery() command and running the application in debug mode would make the error disappear!
Googling, I found something interesting on this site: http://www.softperfect.com/board/read.php?8,5775. There, someone replied the thread suggesting the author to put the database path on the anti-virus ignore list.
I added the database file to the ignore list of my anti-virus (Microsoft Security Essentials) and it solved my problem. No more database locked errors!
Is your database file on the same machine as the app or is it stored on a server?
You should create a new connection in every thread. I would simplefy the creation of a connection, use everywhere: connection = new SQLiteConnection(connString.ToString());
and use a database file on the same machine as the app and test again.
Why the two different ways of creating a connection?
These guys were having similiar problems (mostly, it appears, with the journaling file being locked, maybe TortoiseSVN interactions ... check the referenced articles).
They came up with a set of recommendations (correct directories, changing journaling types from delete to persist, etc). http://sqlite.phxsoftware.com/forums/p/689/5445.aspx#5445
The journal mode options are discussed here: http://www.sqlite.org/pragma.html . You could try TRUNCATE.
Is there a stack trace during the exception into SQL Lite?
You indicate you "batch my commits at a reasonable interval". What is the interval?
I would always use a Connection, Transaction and Command in a using clause. In your first code listing you did, but your third (creating the tables) you didn't. I suggest you do that too, because (who knows?) maybe the commands that create the table somehow continue to lock the file. Long shot... but worth a shot?
Do you have Google Desktop Search (or another file indexer) running? As previously mentioned, Sysinternals Process Monitor can help you track it down.
Also, what is the filename of the database? From PerformanceTuningWindows:
Be VERY, VERY careful what you name your database, especially the extension
For example, if you give all your databases the extension .sdb (SQLite Database, nice name hey? I thought so when I choose it anyway...) you discover that the SDB extension is already associated with APPFIX PACKAGES.
Now, here is the cute part, APPFIX is an executable/package that Windows XP recognizes, and it will, (emphasis mine) ADD THE DATABASE TO THE SYSTEM RESTORE FUNCTIONALITY
This means, stay with me here, every time you write ANYTHING to the database, the Windows XP system thinks a bloody executable has changed and copies your ENTIRE 800 meg database to the system restore directory....
I recommend something like DB or DAT.
While the lock is reported on the COMMIT, the lock is on the INSERT/UPDATE command. Check for record locks not being released earlier in your code.