The following line of code sometimes results in "Specified cast is not valid" exception:
public static object Select(string sql, OleDbTransaction dbt)
{
try
{
OleDbCommand cmd = new OleDbCommand(sql, lib.dbc, dbt);
object obj = cmd.ExecuteScalar(); /* <- this is what fails */
return obj;
}
catch (Exception ex)
{
/* deleted code - error message to the user */
return null;
}
}
This function is executed several times in the program before it fails. If fails olny when it's executed in a new execution thread, and then only sometimes. When I call the part of the program which performs processing in a thread, and it calls this function, either it works all the time (=> I click the button, it executes, no error, I click and execute again and again...), or it never works (=> I click the button and execute, exception, I click and execute again, exception again...).
lib.dbc -> static variable of type OleDbConnection initialized only at program startup and used very often throughout the code, valid
I have no idea how to debug it any further, and above all, what assignment to a variable of type object can fail? ExecuteScalar should return object or null. Database I'm using is Jet (MS Access).
In the exception I find this stack trace, maybe it can help:
at System.Data.OleDb.OleDbConnection.IOpenRowset()
at System.Data.OleDb.OleDbCommand.ExecuteTableDirect(CommandBehavior behavior, Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteCommand(CommandBehavior behavior, Object& executeResult)
at System.Data.OleDb.OleDbCommand.ExecuteReaderInternal(CommandBehavior behavior, String method)
at System.Data.OleDb.OleDbCommand.ExecuteScalar()
at FK.sql.Select(String sql, OleDbTransaction dbt)
If fails olny when it's executed in a new execution thread, and then only sometimes.
This statement, combined with the fact you're passing in the connection as a parameter, suggests you may be trying to use the same database connection, and maybe transaction, on multiple threads.
Don't: instead create a new connection each time you want to access the database: connection pooling means that this will be efficient.
Try this, just as a diagnostic tool:
public static object Select(string sql, OleDbTransaction dbt)
{
try
{
using (OleDbConnection con = new OleDbConnection(lib.dbc.ConnectionString))
using (OleDbCommand cmd = new OleDbCommand(sql, con, dbt))
{
object obj = cmd.ExecuteScalar();
return obj;
}
}
catch (Exception ex)
{
/* deleted code - error message to the user */
return null;
}
}
This may help determine whether you have a threading problem.
Related
Testing environment：
-- Framework version: .NET 6
-- Microsoft.Data.Sqlite, version: 6.0.6
-- Dapper, version: 2.0.123
Q: When only one query is executed, there is no problem. Problems occur when multiple concurrent queries are executed.
(Reference: When I write code in native ADO.NET instead of Dapper(conn.Query), there is no problem!)
Here is a simple demo that can reproduce the problem very easily.
using System.Data;
using Dapper;
using Microsoft.Data.Sqlite;
// SQLite db file
string connStr = $"Data Source={AppDomain.CurrentDomain.BaseDirectory}test.db";
// SQLite connection (Share the singleton connection)
using SqliteConnection conn = new SqliteConnection(connStr);
conn.Open();
bool existError = false;
while (true)
{
if (existError == true)
{
break;
}
// Perform concurrent reads
Parallel.For(0, 100, index =>
{
try
{
// Test SQL
string sql_test = " select * from T_Account where AccountId='ab001' ";
// Perform query data (May throw a null reference exception )
var t = conn.Query(sql_test);
}
catch (Exception ex)
{
existError = true;
// Test output: Object reference not set to an instance of an object.
Console.WriteLine($"Read error ({index}): {ex.Message}");
}
});
Console.WriteLine($"{DateTime.Now}-------- split line --------");
}
The whole project has been using Dapper, I want to know where the problem is, how should I solve it?
You might think that it's a super good idea to reuse the connection, but it isn't. There already is a connection pool built-in to ADO, and it's far more efficient than anything we can come up with. I tried to run your code, obviously with another sqlite database and query, and on the 6th or 7th iteration I got the null reference exception. Then I changed your code like this:
// Perform concurrent reads
Parallel.For(0, 100, index =>
{
try
{
// These two lines moved inside the loop
using SqliteConnection conn = new SqliteConnection(connStr);
conn.Open();
// Test SQL
string sql_test = " select * from T_Account where AccountId='ab001' ";
// Perform query data (May throw a null reference exception )
var t = conn.Query(sql_test);
}
catch (Exception ex)
{
existError = true;
// Test output: Object reference not set to an instance of an object.
Console.WriteLine($"Read error ({index}): {ex.Message}");
}
});
And the code can go on and on as expected. Best practice is to create the database connection in a using statement and have it disposed as soon as you are done with it.
Since I haven't seen your code with raw ADO.NET, I can't tell why that works.
I found that my codebase contains various data access code where I have used using statements in two different ways. Which is the better way if any and are these two methods different? Any problems that could arise from not instantiating the SqlConnection and SqlCommand in the using statement? Ignore the obvious inconsistency problem.
Method 1:
public int SampleScalar(string query, CommandType queryType, SqlParameter[] parameters)
{
int returnValue = 0;
SqlConnection objConn = new SqlConnection(ConnString);
SqlCommand objCmd = new SqlCommand(query, objConn);
objCmd.CommandType = queryType;
if (parameters.Length > 0)
objCmd.Parameters.AddRange(parameters);
using (objConn)
{
using (objCmd)
{
objConn.Open();
try
{
returnValue = (int)objCmd.ExecuteScalar();
}
catch (SqlException e)
{
Errors.handleSqlException(e, objCmd);
throw;
}
}
}
return returnValue;
}
Method 2:
public int SampleScalar2(string query, CommandType queryType, SqlParameter[] parameters)
{
int returnValue = 0;
using (SqlConnection objConn = new SqlConnection(ConnString))
{
using (SqlCommand objCmd = new SqlCommand(query, objConn))
{
objConn.Open();
objCmd.CommandType = queryType;
if (parameters.Length > 0)
objCmd.Parameters.AddRange(parameters);
try
{
returnValue = (int)objCmd.ExecuteScalar();
}
catch (SqlException e)
{
Errors.handleSqlException(e, objCmd);
throw;
}
}
}
return returnValue;
}
In the first snippet, if there are any exceptions that occur after the IDisposable object is created and before the start of the using, then it won't be properly disposed. With the second implementation, there is no such gap that could result in unreleased resources.
Another problem that can occur with the first approach is that you could use an object after it has been disposed, which is not likely to end well.
It's possible that you are ensuring no exception could possibly occur, and maybe you're not. In general, I would never use the first method simply because I don't trust myself (or anyone else) to never ever ever make a mistake in that unprotected space. If nothing else, I'll need to spend time and effort looking very closely to be sure that nothing can ever go wrong. You don't really gain anything from using that less-safe method either.
I always go with the second method. It's easier to read and understand what objects are being disposed of at the end of a given block. It will also prevent you from using an object that has been disposed.
If you not use using you don't dispose your objets no managed, and GC no dispose
GC dispose only objects managed and sql connection is not managed so you must dispose, using use dispose in the end
Second one is better. Please read http://msdn.microsoft.com/en-us/library/yh598w02.aspx last remark.
In first object stays in scope after it's disposal. Using it then is not good practice.
Per MSDN
You can instantiate the resource object and then pass the variable to the using statement, but this is not a best practice. In this case, the object remains in scope after control leaves the using block even though it will probably no longer have access to its unmanaged resources. In other words, it will no longer be fully initialized. If you try to use the object outside the using block, you risk causing an exception to be thrown. For this reason, it is generally better to instantiate the object in the using statement and limit its scope to the using block.
I have the following code
try
{
using (var connection = new SqlConnection(Utils.ConnectionString))
{
connection.Open();
using (var cmd = new SqlCommand("StoredProcedure", connection))
{
cmd.CommandType = System.Data.CommandType.StoredProcedure;
var sqlParam = new SqlParameter("id_document", idDocument);
cmd.Parameters.Add(sqlParam);
int result = cmd.ExecuteNonQuery();
if (result != -1)
return "something";
//do something here
return "something else";
}
}
//do something
}
catch (SqlException ex)
{
return "something AKA didn't work";
}
The question is: Does var connection still get closed if an unexpected error happens between the using brackets ({ })?
The problem is that most of my calls to stored procedures are made this way, and recently I have been getting this error:
System.InvalidOperationException: Timeout expired. The timeout
period elapsed prior to obtaining a connection from the pool. This
may have occurred because all pooled connections were in use and max
pool size was reached.
The other way I access the DB is through nHibernate.
using Statement (C# Reference)
The using statement ensures that Dispose is called even if an
exception occurs while you are calling methods on the object. You can
achieve the same result by putting the object inside a try block and
then calling Dispose in a finally block; in fact, this is how the
using statement is translated by the compiler. The code example
earlier expands to the following code at compile time (note the extra
curly braces to create the limited scope for the object):
Yes, if it gets into the body of the using statement, it will be disposed at the end... whether you reached the end of the block normally, exited via a return statement, or an exception was thrown. Basically the using statement is equivalent to a try/finally block.
Is that the only place you acquire a connection? Has your stored procedure deadlocked somewhere, perhaps, leaving lots of connections genuinely "busy" as far as the client code is concerned?
In terms of your connection pool running out of available connections, if you are in a distributed environment and using many applications to access SQL Server but they all use the same connection string, then they will all be using the same pool on the server. To get around this you can change the connection string for each application by setting the connection WorkstationID to the Environment.MachineName. This will make the server see each connection as different and provide a pool to each machine instead of sharing the pool.
In the below example we even pass in a token to allow an application on the same machine to have multiple pools.
Example:
private string GetConnectionStringWithWorkStationId(string connectionString, string connectionPoolToken)
{
if (string.IsNullOrEmpty(machineName)) machineName = Environment.MachineName;
SqlConnectionStringBuilder cnbdlr;
try
{
cnbdlr = new SqlConnectionStringBuilder(connectionString);
}
catch
{
throw new ArgumentException("connection string was an invalid format");
}
cnbdlr.WorkstationID = machineName + connectionPoolToken;
return cnbdlr.ConnectionString;
}
Replace your above code.. by this.. and check again..
try
{
using (var connection = new SqlConnection(Utils.ConnectionString))
{
connection.Open();
using (var cmd = new SqlCommand("StoredProcedure", connection))
{
cmd.CommandType = System.Data.CommandType.StoredProcedure;
var sqlParam = new SqlParameter("id_document", idDocument);
cmd.Parameters.Add(sqlParam);
int result = cmd.ExecuteNonQuery();
if (result != -1)
return "something";
//do something here
return "something else";
}
connection.Close();
connection.Dispose();
}
//do something
}
catch (SqlException ex)
{
return "something AKA didn't work";
}
Here's a reference:
http://msdn.microsoft.com/en-us/library/yh598w02(v=vs.80).aspx
What I know is that if you use an object within the using {} clause, that object inherits the IDisposable interface (i.e. SqlConnection inherits DbConnection, and DbConnection inherits IDisposable), which means if you get an exception, any object will be closed and disposed properly.
I am using a combination of the Enterprise library and the original Fill method of ADO. This is because I need to open and close the command connection myself as I am capture the event Info Message
Here is my code so far
// Set Up Command
SqlDatabase db = new SqlDatabase(ConfigurationManager.ConnectionStrings[ConnectionName].ConnectionString);
SqlCommand command = db.GetStoredProcCommand(StoredProcName) as SqlCommand;
command.Connection = db.CreateConnection() as SqlConnection;
// Set Up Events for Logging
command.StatementCompleted += new StatementCompletedEventHandler(command_StatementCompleted);
command.Connection.FireInfoMessageEventOnUserErrors = true;
command.Connection.InfoMessage += new SqlInfoMessageEventHandler(Connection_InfoMessage);
// Add Parameters
foreach (Parameter parameter in Parameters)
{
db.AddInParameter(command,
parameter.Name,
(System.Data.DbType)Enum.Parse(typeof(System.Data.DbType), parameter.Type),
parameter.Value);
}
// Use the Old Style fill to keep the connection Open througout the population
// and manage the Statement Complete and InfoMessage events
SqlDataAdapter da = new SqlDataAdapter(command);
DataSet ds = new DataSet();
// Open Connection
command.Connection.Open();
// Populate
da.Fill(ds);
// Dispose of the adapter
if (da != null)
{
da.Dispose();
}
// If you do not explicitly close the connection here, it will leak!
if (command.Connection.State == ConnectionState.Open)
{
command.Connection.Close();
}
...
Now if I pass into the variable StoredProcName = "ThisProcDoesNotExists"
And run this peice of code. The CreateCommand nor da.Fill through an error message. Why is this. The only way I can tell it did not run was that it returns a dataset with 0 tables in it. But when investigating the error it is not appearant that the procedure does not exist.
EDIT
Upon further investigation
command.Connection.FireInfoMessageEventOnUserErrors = true;
is causeing the error to be surpressed into the InfoMessage Event
From BOL
When you set FireInfoMessageEventOnUserErrors to true, errors that were previously treated as exceptions are now handled as InfoMessage events. All events fire immediately and are handled by the event handler. If is FireInfoMessageEventOnUserErrors is set to false, then InfoMessage events are handled at the end of the procedure.
What I want is each print statement from Sql to create a new log record. Setting this property to false combines it as one big string. So if I leave the property set to true, now the question is can I discern a print message from an Error
ANOTHER EDIT
So now I have the code so that the flag is set to true and checking the error number in the method
void Connection_InfoMessage(object sender, SqlInfoMessageEventArgs e)
{
// These are not really errors unless the Number >0
// if Number = 0 that is a print message
foreach (SqlError sql in e.Errors)
{
if (sql.Number == 0)
{
Logger.WriteInfo("Sql Message",sql.Message);
}
else
{
// Whatever this was it was an error
throw new DataException(String.Format("Message={0},Line={1},Number={2},State{3}", sql.Message, sql.LineNumber, sql.Number, sql.State));
}
}
}
The issue now that when I throw the error it does not bubble up to the statement that made the call or even the error handler that is above that. It just bombs out on that line
The populate looks like
// Populate
try
{
da.Fill(ds);
}
catch (Exception e)
{
throw new Exception(e.Message, e);
}
Now even though I see the calling codes and methods still in the Call Stack, this exception does not seem to bubble up?
I spent some time on this and came to the conclusion that the InfoMessageHandler is not raised within the scope of the executing command object. Therefore, exceptions that you throw within the event will not bubble up to command object's method. It must be executing in a different thread.
I assume you are using Visual Studio 2008, because I was able to reproduce your issue exactly in that environment. When I migrated the code to Visual Studio 2010, still using framework 3.5, the new IDE catches the custom exceptions, but I wasn't able to figure out an easy way to catch the exceptions in code. The Visual Studio 2010 debugger is much better at debugging multiple threads.
If you want to catch exceptions from this event, you will have to write code that can track thread exceptions.
Here is the current architecture of my transaction scope source code. The third insert throws an .NET exception (Not a SQL Exception) and it is not rolling back the two previous insert statements. What I am doing wrong?
EDIT: I removed the try/catch from insert2 and insert3. I also removed the exception handling utility from the insert1 try/catch and put "throw ex". It still does not rollback the transaction.
EDIT 2: I added the try/catch back on the Insert3 method and just put a "throw" in the catch statement. It still does not rollback the transaction.
UPDATE:Based on the feedback I received, the "SqlHelper" class is using the SqlConnection object to establish a connection to the database, then creates a SqlCommand object, set the CommandType property to "StoredProcedure" and calls the ExecuteNonQuery method of the SqlCommand.
I also did not add Transaction Binding=Explicit Unbind to the current connection string. I will add that during my next test.
public void InsertStuff()
{
try
{
using(TransactionScope ts = new TransactionScope())
{
//perform insert 1
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /* create parameters for first insert */ };
sh.Insert("MyInsert1", sp);
}
//perform insert 2
this.Insert2();
//perform insert 3 - breaks here!!!!!
this.Insert3();
ts.Complete();
}
}
catch(Exception ex)
{
throw ex;
}
}
public void Insert2()
{
//perform insert 2
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /* create parameters for second insert */ };
sh.Insert("MyInsert2", sp);
}
}
public void Insert3()
{
//perform insert 3
using(SqlHelper sh = new SqlHelper())
{
SqlParameter[] sp = { /*create parameters for third insert */ };
sh.Insert("MyInsert3", sp);
}
}
I have also run into a similar issue. My problem occurred because the SqlConnection I used in my SqlCommands was already open before the TransactionScope was created, so it never got enlisted in the TransactionScope as a transaction.
Is it possible that the SqlHelper class is reusing an instance of SqlConnection that is open before you enter your TransactionScope block?
It looks like you are catching the exception in Insert3() so your code continues after the call. If you want it to rollback you'll need to let the exception bubble up to the try/catch block in the main routine so that the ts.Complete() statement never gets called.
An implicit rollback will only occur if the using is exited without calling ts.complete. Because you are handling the exception in Insert3() the exception never causes an the using statement to exit.
Either rethrow the exception or notify the caller that a rollback is needed (make change the signature of Insert3() to bool Insert3()?)
(based on the edited version that doesn't swallow exceptions)
How long do the operations take? If any of them are very long running, it is possible that the Transaction Binding bug feature has bitten you - i.e. the connection has become detached. Try adding Transaction Binding=Explicit Unbind to the connection string.
I dont see your helper class, but transaction scope rollsback if you don't call complete statement even if you get error from .NET code. I copied one example for you. You may be doing something wrong in debugging. This example has error in .net code and similar catch block as yours.
private static readonly string _connectionString = ConnectionString.GetDbConnection();
private const string inserttStr = #"INSERT INTO dbo.testTable (col1) VALUES(#test);";
/// <summary>
/// Execute command on DBMS.
/// </summary>
/// <param name="command">Command to execute.</param>
private void ExecuteNonQuery(IDbCommand command)
{
if (command == null)
throw new ArgumentNullException("Parameter 'command' can't be null!");
using (IDbConnection connection = new SqlConnection(_connectionString))
{
command.Connection = connection;
connection.Open();
command.ExecuteNonQuery();
}
}
public void FirstMethod()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello1"));
ExecuteNonQuery(command);
}
public void SecondMethod()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello2"));
ExecuteNonQuery(command);
}
public void ThirdMethodCauseNetException()
{
IDbCommand command = new SqlCommand(inserttStr);
command.Parameters.Add(new SqlParameter("#test", "Hello3"));
ExecuteNonQuery(command);
int a = 0;
int b = 1/a;
}
public void MainWrap()
{
TransactionOptions tso = new TransactionOptions();
tso.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
//TransactionScopeOption.Required, tso
try
{
using (TransactionScope sc = new TransactionScope())
{
FirstMethod();
SecondMethod();
ThirdMethodCauseNetException();
sc.Complete();
}
}
catch (Exception ex)
{
logger.ErrorException("eee ",ex);
}
}
If you want to debug your transactions, you can use this script to see locks and waiting status etc.
SELECT
request_session_id AS spid,
CASE transaction_isolation_level
WHEN 0 THEN 'Unspecified'
WHEN 1 THEN 'ReadUncomitted'
WHEN 2 THEN 'Readcomitted'
WHEN 3 THEN 'Repeatable'
WHEN 4 THEN 'Serializable'
WHEN 5 THEN 'Snapshot' END AS TRANSACTION_ISOLATION_LEVEL ,
resource_type AS restype,
resource_database_id AS dbid,
DB_NAME(resource_database_id) as DBNAME,
resource_description AS res,
resource_associated_entity_id AS resid,
CASE
when resource_type = 'OBJECT' then OBJECT_NAME( resource_associated_entity_id)
ELSE 'N/A'
END as ObjectName,
request_mode AS mode,
request_status AS status
FROM sys.dm_tran_locks l
left join sys.dm_exec_sessions s on l.request_session_id = s.session_id
where resource_database_id = 24
order by spid, restype, dbname;
You will see one SPID for two method calls before calling exception method.
Default isolation level is serializable.You can read more about locks and transactions here
I ran into a similar issue when I had a call to a WCF service operation in TransactionScope.
I noticed transaction flow was not allowed due to the 'TransactionFlow' attribute in the service interface. Therefore, the WCF service operation was not using the transaction used by the outer transaction scope. Changing it to allow transaction flow as shown below fixed my problem.
[TransactionFlow(TransactionFlowOption.NotAllowed)]
to
[TransactionFlow(TransactionFlowOption.Allowed)]