C# Multithreading transactions causes exceptions

C# Multithreading transactions causes exceptions - c#

I am trying to simulate a case we're facing where an insert statement is executed in a multi threaded process and causing exceptions,
class Program
{
private readonly static string[] names = { "name1", "name2", "name3", "name4" };
private const string CreateQuery = #"DROP TABLE IF EXISTS name_multi_thread_test CASCADE;
CREATE TABLE name_multi_thread_test (name VARCHAR(20));";
private const string InsertQuery = #"INSERT INTO name_multi_thread_test VALUES('{0}');";
private const string SelectQuery = "SELECT * FROM name_multi_thread_test;";
static void Main(string[] args)
{
var warehouseHelper = new WarehouseHelper();
try
{
System.Threading.Tasks.Parallel.ForEach(names, name =>
{
//foreach(var name in names)
//{
for(int i = 0; i < 10; i++)
{
warehouseHelper.BeginTransaction();
warehouseHelper.ExecuteNonQuery(CreateQuery);
warehouseHelper.ExecuteNonQuery(string.Format(InsertQuery, name));
using (var reader = warehouseHelper.ExecuteReader(SelectQuery))
{
while (reader.Read())
{
Console.WriteLine(reader["name"]);
}
}
warehouseHelper.CommitTranzaction();
}
}//;
);
}
catch(Exception ex)
{
Console.Write(ex.Message);
}
}
}
class WarehouseHelper
{
private IDbConnection _transactionConnection;
private IDbTransaction _transaction;
public void ExecuteNonQuery(string commandText)
{
var connection = GetConnection();
using (var command = connection.CreateCommand())
{
command.Transaction = _transaction;
command.CommandText = commandText;
command.ExecuteNonQuery();
}
}
public IDataReader ExecuteReader(string commandText)
{
var connection = GetConnection();
using (var command = connection.CreateCommand())
{
command.Transaction = _transaction;
command.CommandText = commandText;
return command.ExecuteReader();
}
}
public void BeginTransaction()
{
_transactionConnection = ConnectionManager.CreateConnection();
_transactionConnection.Open();
_transaction = _transactionConnection.BeginTransaction();
}
public void CommitTranzaction()
{
_transaction.Commit();
_transactionConnection.Close();
_transaction = null;
_transactionConnection = null;
}
private IDbConnection GetConnection()
{
if(_transactionConnection != null)
{
return _transactionConnection;
}else
{
var connection = ConnectionManager.CreateConnection();
connection.Open();
return connection;
}
}
}
class ConnectionManager
{
private static string _connectionStringOdbc = "Driver={Vertica};SERVER=x.x.x.x;PORT=5433;DATABASE=mydb;UID=username;PWD=password;";
public static OdbcConnection CreateConnection()
{
return new OdbcConnection(_connectionStringOdbc);
}
}
so basically, for every name in names I begin a transaction, create a table, insert some entries, and read them and this is repeated ten times sequentially in a for loop, what happens is that the first iteration runs fine until the execution reaches CommitTranzation, in this example we have 4 threads, once the first thread sets _transaction to null and the next thread tries to execute _transaction.commit() I get a NullReferenceException
Object reference not set to an instance of an object.
public void CommitTranzaction()
{
_transaction.Commit();
_transactionConnection.Close();
_transaction = null;
_transactionConnection = null;
}
as if it's the same instance of _transaction and _transactionConnection for all the 4 threads, I know we're missing something here in handling Multi threading but what is it,
for now to unblock myself I moved the warehouseHelper.BeginTransaction(); statement outside the Parallel execution, so now it's directly after the try{ and the warehouseHelper.CommitTranzaction(); statement is at the end of the try{ block outside the Parallel execution also, what is the right approach here.

The WarehouseHelper instance is common for all the threads and when the first thread call CommitTranzaction then it sets the _transaction variable to null and when the other threads calls _transaction.Commit() they throw NullRefernceException
I think warehouseHelper.CommitTranzaction() should be outside Parallel.ForEach loop.

Open the database connection inside the thread and start the transaction before the loop and commit after the loop ends.
static void Main(string[] args)
{
try
{
System.Threading.Tasks.Parallel.ForEach(names, name =>
{
var warehouseHelper = new WarehouseHelper();
warehouseHelper.BeginTransaction();
for(int i = 0; i < 10; i++)
{
warehouseHelper.ExecuteNonQuery(CreateQuery);
warehouseHelper.ExecuteNonQuery(string.Format(InsertQuery, name));
}
using (var reader = warehouseHelper.ExecuteReader(SelectQuery))
{
while (reader.Read())
{
Console.WriteLine(reader["name"]);
}
}
warehouseHelper.CommitTranzaction();
}
);
}
catch(Exception ex)
{
Console.Write(ex.Message);
}
}

Related

Get source of an call function, invocation or task execution

In my app, I use an class (MySqlClass) to execute and download data from my database.
public class MySqlClass
{
public void ExecuteQuery(string query) { /* ... */ }
public DataSet GetDataSet(string query) { /* ... */ }
public void Transaction(Action queryToCommit, Action whenRollback) { /* ... */ }
}
For example :
public class MyApp
{
List<MyObjectClass> myList = MyObjectClass.GetMyObjectClass("white");
}
public class MyObjectClass
{
private static MySqlClass sqlConn = new MySqlClass();
public static List<MyObjectClass> GetMyObjectClass(string color)
{
List<MyObjectClass> obj = new List<MyObjectClass>();
using (DataSet ds = sqlConn.GetDataSet(" ... my query ... "))
{
foreach (DataRow dr in ds.Tables[0].Rows)
{
obj.Add(ConvertDataRow(dr));
}
}
return obj;
}
public static MyObjectClass ConvertDataRow(DataRow dr) { /* ... */ }
}
For some criticals utilizations, I want use transaction WITHOUT modify originals functions.
I developed for this "Transaction" function in MySqlClass :
public void Transaction(Action queryToCommit, Action whenRollback)
{
CancellationTokenSource cts = new CancellationTokenSource();
try
{
Task executeAction = new Task(new Action(() =>
{
queryToCommit.Invoke(); // user code executed here
cts.Token.ThrowIfCancellationRequested();
}),
cts.Token
);
executeAction.Start();
executeAction.Wait(cts.Token);
Commit();
}
catch (Exception)
{
Rollback();
whenRollback.Invoke();
/* ... */
}
}
In my app :
public class MyApp
{
MySqlClass sqlConn = new MySqlClass();
List<MyObjectClass> myList = MyObjectClass.GetMyObjectClass("white");
private void WithoutTransaction()
{
MyObjectClass objA = new MyObjectClass();
MyObjectClass objB = new MyObjectClass();
/* ... */
if (...)
{
objA.Insert();
}
else
{
objA.Delete();
objB.Update();
// Exception raised in objB.Update();
// Data correspond to objA in database are lost
}
}
private void WithTransaction()
{
MyObjectClass objA = new MyObjectClass();
MyObjectClass objB = new MyObjectClass();
sqlConn.Transaction(
() => {
if (...)
{
objA.Insert();
}
else
{
objA.Delete();
objB.Update(); // Exception raised in objB.Update()
}
},
CallErrorLogFunction()
);
}
}
My problem :
How i can know, when I use GetDataSet(), ExecuteQuery() or whatever, that I'm in a transaction function ?
StackTrace and StackFrame indicate invocation but not the source function of invocation.
I can't use Thread (for current thread ID) because my colleagues could make operations on the GUI.
I thought to lock instruction to check if in a transaction but I no idea to implement the code
If you have an idea ... :)
Thanks !
EDIT March 19'21 :
public class MySqlClass
{
// They are initialized before use "ExecuteQuery()"
public MyTransactionClass transac;
public SqlConnection conn;
public void ExecuteQuery(string query, Dictionary<string, object> dict = null)
{
SqlDataAdapter da;
try
{
if (conn.State = ConnectionState.Closed)
{
conn.Open();
}
da = new SqlDataAdapter(query, conn);
/* section of code for parameters queries */
if (dict != null) { /* ... */ }
if (IsTransaction())
{
da.SelectCommand.Transaction = transac.TransacSql;
}
da.SelectCommand.ExecuteNonQuery();
}
catch (Exception)
{
throw;
}
finally
{
if (da != null)
{
da.Dispose();
}
}
}
public DataSet GetDataSet(string query) { /* ... */ }
private bool IsTransaction()
{
bool r = false;
if (transac != null)
{
/*
* Check if in StackTrace, "Transaction" of "MyTransactionClass" is in the list
*/
StackTrace st = new StackTrace(true);
StackFrame sf;
for (int i = 0; i < st.FrameCount; i++)
{
sf = st.GetFrame(i);
if (sf.GetMethod().Name == MyTransactionClass.FunctionName_Transaction)
{
r = true;
break;
}
}
}
return r;
}
}
public class MyTransactionClass
{
public static readonly string FunctionName_Transaction = nameof(Transaction);
public SqlTransaction TransacSql;
public void Transaction(Action queryToCommit, Action whenRollback)
{
CancellationTokenSource cts = new CancellationTokenSource();
try
{
Task executeAction = new Task(new Action(() =>
{
queryToCommit.Invoke();
cts.Token.ThrowIfCancellationRequested();
}),
cts.Token
);
executeAction.Start();
executeAction.Wait(cts.Token);
Commit();
}
catch (Exception)
{
Rollback();
whenRollback.Invoke();
/* ... */
}
}
void Commit() { }
void Rollback() { }
}
EDIT March 31'21 :
I make a unit test, with different use cases :
with just a transaction
a transaction and a Action
a transaction with a Action in a Task
Is the last case (similar to I want), System.Transactions.Transaction.Current is null.
[TestMethod]
public void TestTransactionScope()
{
Console.WriteLine($"Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
using (TransactionScope scope = new TransactionScope())
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
MyFunction("direct call");
}
// if I use a invocation
Action<string> fnctTransac = new Action<string>((msg) => MyFunction(msg));
using (TransactionScope scope = new TransactionScope())
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
fnctTransac.Invoke("with invocation");
}
// if I use invocation with a Task, similar to my use
using (TransactionScope scope = new TransactionScope())
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
Task t = new Task(() => fnctTransac.Invoke("with invocation, from a Task"));
t.Start();
Task.WaitAll(t);
}
}
public void MyFunction(string msg)
{
Console.WriteLine($"{msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
}
Logs :
Transaction ID :
(scope) Transaction ID : 1c0127fe-44d5-4954-826c-ece6ad261ee5:1
direct call - Transaction ID : 1c0127fe-44d5-4954-826c-ece6ad261ee5:1
(scope) Transaction ID : 1c0127fe-44d5-4954-826c-ece6ad261ee5:2
with invocation - Transaction ID : 1c0127fe-44d5-4954-826c-ece6ad261ee5:2
(scope) Transaction ID : 1c0127fe-44d5-4954-826c-ece6ad261ee5:3
with invocation, from a Task - Transaction ID :
EDIT April 16'21
With TransactionScope :
Source :
public class MySqlClass
{
public MyTransactionClass transac;
public SqlConnection conn;
public void ExecuteQuery(string query, Dictionary<string, object> dict = null) { /* ... */ }
public DataSet GetDataSet(string query) { /* ... */ }
private bool IsTransaction()
{
bool r = false;
string idTransaction = Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier
if (
transac == null &&
!string.IsNullOrEmpty(idTransaction)
)
{
transac = MyTransactionClass.GetTransaction(idTransaction)
}
return r;
}
}
public class MyTransactionClass
{
public const int MAX_TIME_TRANSAC_SEC = 5
public static readonly string FunctionName_Transaction = nameof(Transaction);
public SqlTransaction TransacSql;
public static void Transaction(Action queryToCommit, Action whenRollback)
{
CancellationTokenSource cts = new CancellationTokenSource();
try
{
Task executeAction = new Task(new Action(() =>
{
queryToCommit.Invoke();
cts.Token.ThrowIfCancellationRequested();
}),
cts.Token
);
using (TransactionScope scope = new TransactionScope(
TransactionScopeOption.Required,
New TimeSpan(0, 0, MAX_TIME_TRANSAC_SEC),
TransactionScopeAsyncFlowOption.Enabled
)
{
executeAction.Start();
executeAction.Wait(cts.Token);
scope.Complete();
Commit();
}
}
catch (Exception)
{
Rollback();
whenRollback.Invoke();
/* ... */
}
}
void Commit() { }
void Rollback() { }
}
Tests :
[TestMethod]
public void E_TransactionScope()
{
Console.WriteLine($"Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
using (TransactionScope scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
MyFunction("direct call");
}
Console.WriteLine();
// if I use a invocation
Action<string> fnctTransac = new Action<string>((msg) => MyFunction(msg));
using (TransactionScope scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
fnctTransac.Invoke("with invocation");
}
Console.WriteLine();
// if I use invocation with a Task, similar to my use
using (TransactionScope scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
Task t = new Task(() => fnctTransac.Invoke("with invocation, from a Task"));
t.Start();
Task.WaitAll(t);
}
Console.WriteLine();
// ultimate use case
Action userCode = () => MyFunction_First("last use case");
Task tk;
using (TransactionScope scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
Console.WriteLine($"(scope) Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
// TransactionID not available when userCode is defined out of using instruction and called directly
// userCode.Start();
tk = new Task(new Action(() => userCode.Invoke()));
tk.Start();
tk.Wait();
}
Console.WriteLine("-------------------------");
}
public void MyFunction(string msg)
{
Console.WriteLine($"{msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
}
public void MyFunction_First(string msg)
{
Console.WriteLine($"MyFunction_First - {msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
Task t = new Task(() => MyFunction_Second(msg));
t.Start();
Task.WaitAll(t);
}
public void MyFunction_Second(string msg)
{
Console.WriteLine($"MyFunction_Second - {msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
Thread th = new Thread(new ThreadStart(() => MyFunction_Third(msg)));
th.Start();
th.Join();
}
public void MyFunction_Third(string msg)
{
using (TransactionScope scope = new TransactionScope())
{
Console.WriteLine($"MyFunction_Third - {msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
Task t = new Task(() => MyFunction_Fourth(msg));
t.Start();
Task.WaitAll(t);
}
}
public void MyFunction_Fourth(string msg)
{
Console.WriteLine($"MyFunction_Fourth - {msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
Task t = new Task(() => MyFunction_Last(msg));
t.Start();
Task.WaitAll(t);
}
public void MyFunction_Last(string msg)
{
Console.WriteLine($"MyFunction_Last - {msg} - Transaction ID : {System.Transactions.Transaction.Current?.TransactionInformation.LocalIdentifier}");
}
Logs :
Transaction ID :
(scope) Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:1
direct call - Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:1
(scope) Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:2
with invocation - Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:2
(scope) Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:3
with invocation, from a Task - Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:3
(scope) Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:4
MyFunction_First - last use case - Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:4
MyFunction_Second - last use case - Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:4
MyFunction_Third - last use case - Transaction ID : 99e1d658-27d8-404d-b57e-d6ead2e2308e:4
MyFunction_Fourth - last use case - Transaction ID :
MyFunction_Last - last use case - Transaction ID :

Have you already considered using the TransactionScope class?
I'm not sure, but with that, you might not need to know whether you are in a transaction or not inside your ExecuteQuery and GetDataSet methods. It might just simply work. ;)
Your Transaction method could look something like this:
public void Transaction(Action queryToCommit, Action whenRollback = null)
{
if (queryToCommit == null)
{
throw new ArgumentNullException(nameof(queryToCommit));
}
CancellationTokenSource cts = new CancellationTokenSource();
Task executeAction = new Task(new Action(() =>
{
queryToCommit.Invoke(); // user code executed here
cts.Token.ThrowIfCancellationRequested();
}),
cts.Token
);
try
{
using (var scope = new System.Transactions.TransactionScope())
{
executeAction.Start();
executeAction.Wait(cts.Token);
scope.Complete();
}
}
catch (Exception)
{
whenRollback.Invoke();
/* ... */
}
}
Note that I haven't tested this. I just based it on your code and the sample in the documentation, so it probably needs some additional work.
Edit:
Based on recent comments, I added a code snippet that shows how I would personally implement your logic:
public class MySqlClass
{
private string connectionString;
public MySqlClass(string connectionString)
{
this.connectionString = connectionString;
}
public int ExecuteQuery(string query, params SqlParameter[] parameters)
{
using (SqlConnection con = new SqlConnection(connectionString))
using (SqlCommand cmd = new SqlCommand(query, con))
{
con.Open();
cmd.Parameters.AddRange(parameters);
return cmd.ExecuteNonQuery();
}
}
public DataSet GetDataSet(string query, params SqlParameter[] parameters)
{
using (SqlConnection con = new SqlConnection(connectionString))
using (SqlCommand cmd = new SqlCommand(query, con))
{
con.Open();
cmd.Parameters.AddRange(parameters);
DataSet ds = new DataSet();
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(ds);
}
return ds;
}
}
public DataTable GetDataTable(string query, params SqlParameter[] parameters)
{
DataSet ds = GetDataSet(query, parameters);
return ds.Tables[0];
}
public static void Transaction(Action queryToCommit, Action whenRollback)
{
if (queryToCommit == null)
{
throw new ArgumentNullException(nameof(queryToCommit));
}
CancellationTokenSource cts = new CancellationTokenSource();
Task executeAction = new Task(new Action(() =>
{
queryToCommit.Invoke(); // user code executed here
cts.Token.ThrowIfCancellationRequested();
}),
cts.Token
);
try
{
using (var scope = new System.Transactions.TransactionScope())
{
executeAction.Start();
executeAction.Wait(cts.Token);
scope.Complete();
}
}
catch
{
whenRollback.Invoke();
/* ... */
}
}
public static SqlParameter CreateParameter(string parameterName, SqlDbType dbType, object value)
{
SqlParameter parameter = new SqlParameter(parameterName, dbType);
parameter.Value = value;
return parameter;
}
public static SqlParameter CreateParameter(string parameterName, SqlDbType dbType, int size, object value)
{
SqlParameter parameter = new SqlParameter(parameterName, dbType, size);
parameter.Value = value;
return parameter;
}
}
/*
CREATE TABLE [MyObjects] (
[Id] INT IDENTITY NOT NULL,
[Color] VARCHAR(20) NOT NULL,
--...
CONSTRAINT [PK_MyObjects] PRIMARY KEY ([Id])
);
*/
public class MyObjectModel
{
// Sample private fields
public int? Id { get; set; }
public string Color { get; set; }
/* ... */
}
public class MyObjectRepository
{
private MySqlClass sqlConn;
public MyObjectRepository(MySqlClass sqlConn)
{
this.sqlConn = sqlConn;
}
public List<MyObjectModel> GetMyObjects(string colorFilter)
{
List<MyObjectModel> obj = new List<MyObjectModel>();
string query = (string.IsNullOrWhiteSpace(colorFilter))
? "SELECT [Id], [Color] FROM [MyObjects];"
: "SELECT [Id], [Color] FROM [MyObjects] WHERE [Color] LIKE '%' + #Color + '%';";
using (DataTable dt = sqlConn.GetDataTable(query,
MySqlClass.CreateParameter("#Color", SqlDbType.VarChar, 20, colorFilter)))
{
foreach (DataRow dr in dt.Rows)
{
obj.Add(ConvertDataRow(dr));
}
}
return obj;
}
private static MyObjectModel ConvertDataRow(DataRow dr)
{
return new MyObjectModel
{
/* set class properties here... */
Id = dr.Field<int>("Id"),
Color = dr.Field<string>("Color")
/* ... */
};
}
public void Insert(MyObjectModel model)
{
if (model.Id.HasValue)
{
return;
}
using (DataTable dt = sqlConn.GetDataTable("INSERT INTO [MyObjects] ([Color]) OUTPUT INSERTED.[Id] VALUES (#Color);",
MySqlClass.CreateParameter("#Color", SqlDbType.VarChar, 20, model.Color)))
{
model.Id = (int)dt.Rows[0]["Id"];
}
}
public int? Update(MyObjectModel model)
{
if (!model.Id.HasValue)
{
return null;
}
return sqlConn.ExecuteQuery("UPDATE [MyObjects] SET [Color] = #Color WHERE [Id] = #Id;",
MySqlClass.CreateParameter("#Id", SqlDbType.Int, model.Id),
MySqlClass.CreateParameter("#Color", SqlDbType.VarChar, 20, model.Color));
}
public int Delete(int id)
{
return sqlConn.ExecuteQuery("DELETE FROM [MyObjects] WHERE [Id] = #Id;",
MySqlClass.CreateParameter("#Id", SqlDbType.Int, id));
}
}
public class MyApp
{
private MyObjectRepository repo;
public MyApp()
{
MySqlClass sqlConn = new MySqlClass(" ... connection string here ... ");
repo = new MyObjectRepository(sqlConn);
}
List<MyObjectModel> GetMyObjects(string colorFilter = "white")
{
return repo.GetMyObjects(colorFilter);
}
private void WithoutTransaction()
{
MyObjectModel objA = new MyObjectModel();
MyObjectModel objB = new MyObjectModel();
/* ... */
if (true)
{
repo.Insert(objA);
}
else
{
repo.Delete(objA.Id.Value);
repo.Update(objB);
// Exception raised in objB.Update();
// Data correspond to objA in database are lost
}
}
private void WithTransaction()
{
MyObjectModel objA = new MyObjectModel();
MyObjectModel objB = new MyObjectModel();
MySqlClass.Transaction(
() =>
{
if (true)
{
repo.Insert(objA);
}
else
{
repo.Delete(objA.Id.Value);
repo.Update(objB); // Exception still raised now???
}
},
CallErrorLogFunction
);
}
private void CallErrorLogFunction()
{
/* ... */
}
}
From recent edits to your question, I am not sure if this would be a valid solution, however. This is the best I could come up with so far. Perhaps you can extract something useful from it anyway. (I haven't tested it, so despite me being careful, the code might contain some stupid mistakes.)

Visual Studio - C# Console App - How to write a unit test and testable code

I'm new to unit testing and would like to unit test part of my code.
The code attempts to connect to a database N times to get a list of tasks, if it exhausts the attempts it throws an exception to the caller.
I've done by best to write it in a testable way trying to keep AttemptRecursive referentially transparent.
I'd like to write tests that for example fail on the 1st/2nd attempt to check my code behaves correctly but I'm struggling to write the code in a way that will allow that as I can't stop in the middle of the recursive call in my Attempter class.
There seems to be a lot of differing opinions around the how & whys of unit tests so I'm not sure if I've just completely missed the point!
Essentially my thought is the database could go down and I'd like to test that when it happens my code behaves correctly.
I've read articles on unit testing - a lot of them are on what to test for and whether it's useful but not on how to write code that's open to testing. The examples are of functions like factorials where the tests are just if the returned list contains the expected values.
Any help or just pointing in the right direction would be greatly appreciated!
Thanks
Attempter.cs - Handles attempting an action N times then throwing exception to bubble up to the caller
public class Attempter<T>
{
private int _defaultMaximumAttempts;
private int _defaultStartingMillisecondsTillNextTry;
public Attempter(int defaultMaximumAttempts, int defaultStartingMillisecondsTillNextTry)
{
_defaultMaximumAttempts = defaultMaximumAttempts;
_defaultStartingMillisecondsTillNextTry = defaultStartingMillisecondsTillNextTry;
}
public void Attempt(T sender, Action action)
{
AttemptRecursive(
sender: sender,
action: action,
attemptNumber: 1,
maximumAttempts: _defaultMaximumAttempts,
millisecondsTillNextTry: _defaultStartingMillisecondsTillNextTry);
}
public void AttemptRecursive(T sender, Action action, int attemptNumber,
int maximumAttempts, int millisecondsTillNextTry)
{
try
{
action.Invoke();
return;
}
catch (Exception ex)
{
if(attemptNumber >= maximumAttempts)
{
throw;
}
else
{
Thread.Sleep(millisecondsTillNextTry);
AttemptRecursive(sender, action, attemptNumber + 1,
maximumAttempts, millisecondsTillNextTry * 2);
}
}
}
DatabaseService.cs - Uses the Attempter to attempt to connect to the database and run queries
public class DatabaseService
{
private Attempter<DatabaseService> _attempter;
public SqlConnectionStringBuilder ConnectionStringBuilder { get; set; }
public DatabaseService(SqlConnectionStringBuilder connectionStringBuilder)
{
ConnectionStringBuilder = connectionStringBuilder;
_attempter = new Attempter<DatabaseService>(
maximumAttempts: 1,
startingMillisecondsTillNextTry: 1000);
}
public DatabaseService(SqlConnectionStringBuilder connectionStringBuilder,
Attempter<DatabaseService> attempter)
{
ConnectionStringBuilder = connectionStringBuilder;
_attempter = attempter;
}
public void AttemptExecuteNonQuery(string cmdText)
{
_attempter.Attempt(
sender: this,
action: () => ExecuteNonQuery(cmdText));
}
private void ExecuteNonQuery(string cmdText)
{
using (var con = new SqlConnection(ConnectionStringBuilder.ConnectionString))
{
con.Open();
//con.State == System.Data.ConnectionState.Open
using (var cmd = new SqlCommand(cmdText, con))
{
cmd.ExecuteNonQuery();
}
}
}
public List<T> AttemptExecuteReader<T>(string cmdText,
Func<SqlDataReader, T> constructorFunction)
{
var returnList = new List<T>();
_attempter.Attempt(
sender: this,
action: () => returnList = ExecuteReader<T>(cmdText, constructorFunction));
return returnList;
}
private List<T> ExecuteReader<T>(string cmdText, Func<SqlDataReader,T> constructorFunction)
{
var returnList = new List<T>();
using (var con = new SqlConnection(ConnectionStringBuilder.ConnectionString))
{
con.Open();
using (var cmd = new SqlCommand(cmdText, con))
{
var reader = cmd.ExecuteReader();
while (reader.Read())
{
returnList.Add(constructorFunction(reader));
}
}
}
return returnList;
}
}

C#: How to Wrap my Unit of Work code in my repository classes

See my repository code
public abstract class AdoRepository<T> where T : class
{
private static SqlConnection _connection;
public AdoRepository(string connectionString)
{
_connection = new SqlConnection(connectionString);
}
public virtual T PopulateRecord(SqlDataReader reader)
{
return null;
}
public virtual void GetDataCount(int count)
{
}
protected IEnumerable<T> GetRecords(SqlCommand command)
{
var list = new List<T>();
command.Connection = _connection;
_connection.Open();
try
{
var reader = command.ExecuteReader();
try
{
while (reader.Read())
{
list.Add(PopulateRecord(reader));
}
reader.NextResult();
if (reader.HasRows)
{
while (reader.Read())
{
GetDataCount(Convert.ToInt32(reader["Count"].ToString()));
}
}
}
finally
{
// Always call Close when done reading.
reader.Close();
}
}
catch(Exception ex)
{
string Msg = ex.Message;
}
finally
{
_connection.Close();
}
return list;
}
protected T GetRecord(SqlCommand command)
{
T record = null;
command.Connection = _connection;
_connection.Open();
try
{
var reader = command.ExecuteReader();
try
{
while (reader.Read())
{
record = PopulateRecord(reader);
break;
}
}
finally
{
// Always call Close when done reading.
reader.Close();
}
}
finally
{
_connection.Close();
}
return record;
}
protected IEnumerable<T> ExecuteStoredProc(SqlCommand command)
{
var list = new List<T>();
command.Connection = _connection;
command.CommandType = CommandType.StoredProcedure;
_connection.Open();
try
{
var reader = command.ExecuteReader();
try
{
while (reader.Read())
{
var record = PopulateRecord(reader);
if (record != null) list.Add(record);
}
}
finally
{
// Always call Close when done reading.
reader.Close();
}
}
finally
{
_connection.Close();
}
return list;
}
}
public class StudentRepository : AdoRepository<Student>
{
public int DataCounter { get; set; }
public StudentRepository(string connectionString)
: base(connectionString)
{
}
public IEnumerable<Student> GetAll()
{
// DBAs across the country are having strokes
// over this next command!
using (var command = new SqlCommand("SELECT ID, FirstName,LastName,IsActive,StateName,CityName FROM vwListStudents"))
{
return GetRecords(command);
}
}
public Student GetById(string id)
{
// PARAMETERIZED QUERIES!
using (var command = new SqlCommand("SELECT ID, FirstName,LastName,IsActive,StateName,CityName FROM vwListStudents WHERE Id = #id"))
{
command.Parameters.Add(new ObjectParameter("id", id));
return GetRecord(command);
}
}
public IEnumerable<Student> GetStudents(int StartIndex, int EndIndex, string sortCol, string sortOrder)
{
string strSQL = "SELECT * FROM vwListStudents WHERE ID >=" + StartIndex + " AND ID <=" + EndIndex;
strSQL += " ORDER BY " + sortCol + " " + sortOrder;
strSQL += ";SELECT COUNT(*) AS Count FROM vwListStudents";
var command = new SqlCommand(strSQL);
return GetRecords(command);
}
public override Student PopulateRecord(SqlDataReader reader)
{
return new Student
{
ID = Convert.ToInt32(reader["ID"].ToString()),
FirstName = reader["FirstName"].ToString(),
LastName = reader["LastName"].ToString(),
IsActive = Convert.ToBoolean(reader["IsActive"]),
StateName = reader["StateName"].ToString(),
CityName = reader["CityName"].ToString()
};
}
public override void GetDataCount(int count)
{
DataCounter = count;
}
}
this way unit of work code added
public class UnitOfWorkFactory
{
public static IUnitOfWork Create()
{
var connection = new SqlConnection(ConfigurationManager.ConnectionStrings("MyDb").ConnectionString);
connection.Open();
return new AdoNetUnitOfWork(connection, true);
}
}
public class AdoNetUnitOfWork : IUnitOfWork
{
public AdoNetUnitOfWork(IDbConnection connection, bool ownsConnection)
{
_connection = connection;
_ownsConnection=ownsConnection;
_transaction = connection.BeginTransaction();
}
public IDbCommand CreateCommand()
{
var command = _connection.CreateCommand();
command.Transaction = _transaction;
return command;
}
public void SaveChanges()
{
if (_transaction == null)
throw new InvalidOperationException("Transaction have already been commited. Check your transaction handling.");
_transaction.Commit();
_transaction = null;
}
public void Dispose()
{
if (_transaction != null)
{
_transaction.Rollback();
_transaction = null;
}
if (_connection != null && _ownsConnection)
{
_connection.Close();
_connection = null;
}
}
}
using (var uow = UnitOfWorkFactory.Create())
{
var repos = new UserRepository(uow);
uow.SaveChanges();
}
public class UserRepository
{
private AdoNetUnitOfWork _unitOfWork;
public UserRepository(IUnitOfWork uow)
{
if (uow == null)
throw new ArgumentNullException("uow");
_unitOfWork = uow as AdoNetUnitOfWork;
if (_unitOfWork == null)
throw new NotSupportedException("Ohh my, change that UnitOfWorkFactory, will you?");
}
public User Get(Guid id)
{
using (var cmd = _unitOfWork.CreateCommand())
{
cmd.CommandText = "SELECT * FROM Users WHERE Id = #id");
cmd.AddParameter("id", id);
// uses an extension method which I will demonstrate in a
// blog post in a couple of days
return cmd.FirstOrDefault<User>();
}
}
}
but i want to add unit of work code in my repository class. may be in AdoRepository class or StudentRepository class as a result i do not have to write the below code again and again.
using (var uow = UnitOfWorkFactory.Create())
{
var repos = new UserRepository(uow);
uow.SaveChanges();
}
so when i will be working with any repository like UserRepository then i have to repeat again and again this lines of code UnitOfWorkFactory.Create() which i do not want rather i am looking for a way to embed unit of work code in some where centralize as a result i could reduce the repeated code.
so looking for idea and suggestion. if possible give me modified version of code.
thanks

I will be straightforward about this:
A UoW that contains repositories is an anti pattern at least if DDD is involved. If it isn't, then you're dealing with a very simple app so make you life easier and use a ORM which implements what you want.
As a thumb rule, a repository may contain a UoW as an implementation detail and you should be certain that you really need a repository. A repository inside a UoW is a very specific case which is valid ONLY with CRUD apps and you'll just be reinventing a very small part of an ORM.
Using ado.net directly is wasting time (and money). Either use an ORM or a data mapper aka micro-ORM. With the latter option the UoW is the Db transaction.
You don't gain anything significant using Ado.Net,the performance gain is so small it's not even funny (I know that because I've benchmarked it). The only thing you'll get is longer devel time and human errors.
Actually, if I take a better look at your code, you're basically building a micro ORM, without knowing it. It's still not a repository though, because the abstraction doesn't have business semantics and it's too tightly coupled to an implementation detail.

Encountering ObjectDisposedException when trying to read from SQLiteDataReader

I am trying to read through a stored SQLiteDataReader object. In theory, it "should" work because the object is stored in a variable before it is referenced (and doesn't hit an error until the reference line is reached), but maybe I have the wrong idea.
I'm trying to keep my application in a neatly layered architecture. So, each database table having its own C# class with its own methods for select, insert, update, and delete; only the data layer knows how to communicate with the database, etc.
I was running into connection issues earlier when I tried to make one static SQLiteConnection object that all the data layer classes could reference (so as to keep it open and minimize overhead, if any). So I'm trying to go with the using block to make sure the connection is properly disposed each time I need to access the database, and hoping that this won't cause performance issues.
So basically, here is the method in my DatabaseConnection class that handles basic query execution:
public SQLiteDataReader ExecuteQuery(string sql)
{
SQLiteDataReader rdr = null;
using(SQLiteConnection conn = new SQLiteConnection(ConnectionString))
{
conn.Open();
SQLiteCommand cmd = conn.CreateCommand();
cmd.CommandText = sql;
rdr = cmd.ExecuteReader();
}
return rdr;
}
And here is the code that calls that method. I'll use an object/record of the Associate table as an example.
public class Associate
{
public int RowId { get; private set; }
public int Id { get; set; }
public string Name { get; set; }
private string password;
public string Password
{
get
{
return password;
}
set
{
password = Hash(value); // external password hashing method
}
}
public Associate() { } // default constructor with default values
public Associate(int id)
{
this.Id = id;
Select();
}
// select, insert, update, delete methods
private void Select() { ... }
// these are non-queries and return true/false based on success
public bool Insert() { ... }
public bool Update() { ... }
public bool Delete() { ... }
/* Method that causes the error */
public static Associate[] GetAll()
{
DatabaseConnection con = new DatabaseConnection();
SQLiteDataReader rdr = con.ExecuteQuery("SELECT id FROM Associate");
List<Associate> list = new List<Associate>();
if (rdr != null)
{
while (rdr.Read()) /* this line throws the exception */
{
int next = rdr.GetInt32(0);
list.Add(new Associate(next));
}
}
return list.ToArray();
}
}
The idea here is that using the rdr object, I can access the column names directly so that if the database ever changes, I won't have to rewrite a bunch of code to adjust for the column indices (rdr["id"], rdr["name"], etc.)
So what I don't understand is why rdr in the calling method is having "object disposed" issues because it's stored in a variable before I reference it. I know the connection is disposed at the end of the called method, but since the returned result is stored, shouldn't it technically be able to "survive" outside the using block?

It is the connection that got disposed. The data reader can only read data while the connection still exists.
public SQLiteDataReader ExecuteQuery(string sql)
{
SQLiteDataReader rdr = null;
using(SQLiteConnection conn = new SQLiteConnection(ConnectionString))
{
conn.Open();
SQLiteCommand cmd = conn.CreateCommand();
cmd.CommandText = sql;
rdr = cmd.ExecuteReader();
}
// *** Connection gone at this stage ***
return rdr;
}
Your options are to either return a DataTable, e.g.
public DataTable ExecuteQuery(string sql)
{
SQLiteDataReader rdr = null;
using(SQLiteConnection conn = new SQLiteConnection(ConnectionString))
{
conn.Open();
SQLiteCommand cmd = conn.CreateCommand();
cmd.CommandText = sql;
rdr = cmd.ExecuteReader();
var dataTable = new DataTable();
dataTable.Load(rdr);
return dataTable;
}
}
otherwise, you could keep the connection alive inside the DatabaseConnection class:
class DatabaseConnection : IDisposable
{
private readonly IDbConnection _conn;
public DatabaseConnection()
{
_conn = new SQLiteConnection(ConnectionString);
}
public void Dispose()
{
_conn.Dispose();
}
public SQLDataReader ExecuteQuery(string sql)
{
...
}
}
// sample usage
using (var conn = new DatabaseConnection())
{
using (var reader = conn.ExecuteQuery("SELECT ...")
{
// do your work in here
}
}

How to start DB transaction, run multiple queries on MySQL then commit or rollback using c#?

I have a generic class that is allowing me to run queries on MySQL database.
The only issue that I am seeing with this class is that it open a connections run a query and close the connection.
Now, I need to be able to
START TRANSACTION;
SELECT ...
INSERT INTO table1...
INSERT INTO table2...
INSERT INTO table3...
if all the above queries worked with no error COMMIT; otherwise ROLLBACK;
Yes, I need a transaction and I need to make sure all queries run 100% or I need to rollback and correct the error prior tarting again.
How can I modify my class to allow me to handle the steps above?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using MySql.Data.MySqlClient;
using System.Windows.Forms;
namespace RM
{
public class dbConnetion
{
//private OdbcConnection conn;
private static readonly string mServer = "localhost";
private static readonly string mDatabase = "my_db_name";
private static readonly string mUid = "my_db_user";
private static readonly string mPassword = "my_user_password";
private static readonly string mPort = "3306";
private string conn_string = String.Format("server={0};user={1};database={2};port={3};password={4};", mServer, mUid, mDatabase, mPort, mPassword);
public string SYSTEM_NAME { get; set; }
public dbConnetion()
{
Initilize_System_Settings();
}
// query the data base
public IEnumerable<T> getData<T>(string query, List<MySqlParameter> pars, Func<IDataRecord, T> transform)
{
using (var conn = new MySqlConnection(conn_string))
using (var cmd = new MySqlCommand(query, conn))
{
if (pars != null)
{
foreach (MySqlParameter p in pars)
{
cmd.Parameters.Add(p);
}
}
conn.Open();
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
yield return transform(rdr);
}
}
conn.Close();
}
}
// query the data base
public T getValue<T>(string query, List<MySqlParameter> pars)
{
T value;
using (var conn = new MySqlConnection(conn_string))
using (var cmd = new MySqlCommand(query, conn))
{
if (pars != null)
{
foreach (MySqlParameter p in pars)
{
cmd.Parameters.Add(p);
}
}
try
{
conn.Open();
object rawValue = cmd.ExecuteScalar();
if (rawValue != null)
{
value = (T)Convert.ChangeType(rawValue, typeof(T));
}
else
{
value = default(T);
}
}
catch (Exception ex)
{
Common.Alert(ex.ToString(), "SQL Error");
value = default(T);
}
finally
{
conn.Close();
}
}
return value;
}
public bool processQuery(string strSQL, List<MySqlParameter> pars)
{
bool toReturn = true;
using (var conn = new MySqlConnection(this.conn_string))
using (var cmd = new MySqlCommand(strSQL, conn))
{
foreach (MySqlParameter param in pars)
{
cmd.Parameters.Add(param);
}
try
{
conn.Open();
cmd.ExecuteNonQuery();
}
catch (MySqlException ex)
{
Common.Alert(ex.ToString(), "SQL Error");
toReturn = false;
}
finally
{
conn.Close();
}
}
return toReturn;
}
}
}
My first thoughts was to add a new parameter to the getData method to allow me to not open/close connection if there is an open transaction, like this
// query the data base
public IEnumerable<T> getData<T>(string query, List<MySqlParameter> pars, Func<IDataRecord, T> transform, bool inTransaction = false)
{
using (var conn = new MySqlConnection(conn_string))
using (var cmd = new MySqlCommand(query, conn))
{
if (pars != null)
{
foreach (MySqlParameter p in pars)
{
cmd.Parameters.Add(p);
}
}
if(! inTransaction){
conn.Open();
}
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
yield return transform(rdr);
}
}
if(! inTransaction){
conn.Close();
}
}
}
but I think this is not going to work because of the using statement

trans = Conn.BeginTransaction();
trans.Commit();
trans.Rollback();
Cannot access SqlTransaction object to rollback in catch block

You have to determine where to start and end the transaction then,
using (var conn = new MySqlConnection(conn_string))
{
var tx = conn.BeginTransaction();
try
{
using (var cmd1 = new MySqlCommand(query1, conn))
{
cmd1.Transaction = tx;
//do cmd 1 stuff here
}
using (var cmd2 = new MySqlCommand(query2, conn))
{
cmd2.Transaction = tx;
//do cmd 1 stuff here
}
//do other commands....
tx.Commit(); //or Rollback() based on results
}
catch (Exception ex)
{
tx.Rollback();
throw ex;
}
}

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

C# Multithreading transactions causes exceptions - c#

Related

Get source of an call function, invocation or task execution

Visual Studio - C# Console App - How to write a unit test and testable code

C#: How to Wrap my Unit of Work code in my repository classes

Encountering ObjectDisposedException when trying to read from SQLiteDataReader

How to start DB transaction, run multiple queries on MySQL then commit or rollback using c#?

Categories

Resources