Trying to implement a retry logic in a correct way, but couldn't find any good sample on how to properly leverage EnterpriseLibrary.TransientFaultHandling.
So far I found two samples:
First - using ReliableSqlConnection & conn.Open(retryPolicy)
var retryStrategy = new Incremental(3, TimeSpan.FromMilliseconds(500), TimeSpan.FromSeconds(1));
var retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(retryStrategy);
retryPolicy.ExecuteAction(() =>
{
using (var conn = new ReliableSqlConnection(datasetConnectionString))
{
conn.Open(retryPolicy);
using (var command = conn.CreateCommand())
{
command.CommandText = insertToParameters;
command.CommandTimeout = 0;
conn.ExecuteCommand(command);
}
}
});
and second - without the ReliableSqlConnection:
var retryStrategy = new Incremental(3, TimeSpan.FromMilliseconds(500), TimeSpan.FromSeconds(1));
var retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(retryStrategy);
retryPolicy.ExecuteAction(() =>
{
using (var conn = new SqlConnection(datasetConnectionString))
{
conn.Open();
using (var command = conn.CreateCommand())
{
command.CommandText = insertToParameters;
command.CommandTimeout = 0;
conn.ExecuteCommand(command);
}
}
});
So few questions:
which one is better and why?
Is the external retryPolicy.ExecuteAction really needed - in older samples I see people retrying only individual actions, like OpenConnectionWithRetries, ExecuteCommandWithRetries, etc, but not the whole thing - I wonder if that is possible that the connection could potentially be closed between those retries.
Answering my own question:
With newer transient error handling block, use SqlConnection with the provided extension methods, such as OpenWithRetry, etc.
Use retryPolicy.ExecuteAction(() => {...}) whenever there is no support for the retry in the API, such as SqlBulkCopy, filling the dataset table, async methods etc. Make sure to re-open connection in the retry block. You still can use the SqlConnection with retry-able extension methods inside the retry block.
UPDATE: edited to cause less confusion
The best pattern to use is RetryPolicy.ExecuteAction(() => . Wrap your database interactions into an ExecuteAction() lambda and treat that as a unit of work. This maintains the highest level of compatibility with ADO.Net's APIs. It also allows you to work with the new Async methods in ADO.Net.
The ReliableSQLConnection is primarily offered for backward compatibility.
Related
I was using Microsoft.Practice.TransientFaultHandling block for retry logic.
Now I switched my application to .Net 4.8 and use the new build in retry logic for SqlConnection.
I was wondering if I need a special retry logic for my SqlCommand (I used Polly before) or if this is also build in. There is no possibility to log a retry when relying on the build in functions which makes it really hard to test.
Microsoft states here :
"There is a subtlety. If a transient error occurs while your query is
being executed, your SqlConnection object doesn't retry the connect
operation. It certainly doesn't retry your query. However,
SqlConnection very quickly checks the connection before sending your
query for execution. If the quick check detects a connection problem,
SqlConnection retries the connect operation. If the retry succeeds,
your query is sent for execution."
I tested this by just disconnecting and reconnecting the internet within the retry time range and my command got executed after a while.
So it seems to work for this simple scenario. But is it really safe to rely on this or do I still have to implement a retry logic for my SqlCommand?
Here is my code:
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder(ConnectionString);
builder.ConnectRetryCount = 5;
builder.ConnectRetryInterval = 3;
MyDataSet m_myDataSet = new MyDataSet();
using (SqlConnection sqlConnection = new SqlConnection(builder.ConnectionString))
{
try
{
sqlConnection.Open();
}
catch (SqlException sqlEx)
{
// do some logging
return false;
}
try
{
using (SqlCommand cmd = new SqlCommand(selectCmd, sqlConnection))
{
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(m_myDataSet, tableName);
}
}
}
}
The answer to your question is to analyze why your connection to the database is open so long that it is going idle and timing out. The ConnectRetryCount and ConnectRetryInterval properties allow you to adjust reconnection attempts after the server identifies an idle connection failure. I would follow the Microsoft recommendations on this one:
Connection Pooling Recommendation
We strongly recommend that you always close the connection when you
are finished using it so that the connection will be returned to the
pool. You can do this using either the Close or Dispose methods of the
Connection object, or by opening all connections inside a using
statement in C#, or a Using statement in Visual Basic. Connections
that are not explicitly closed might not be added or returned to the
pool. For more information, see using Statement or How to: Dispose of
a System Resource for Visual Basic.
Open your connections and close them when no longer needed like this:
MyDataSet m_myDataSet = new MyDataSet();
try
{
using (SqlConnection sqlConnection = new SqlConnection(ConnectionString))
{
sqlConnection.Open();
using (SqlCommand cmd = new SqlCommand(selectCmd, sqlConnection))
{
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(m_myDataSet, tableName);
}
}
}
}
catch (SqlException sqlEx)
{
// do some logging
return false;
}
Hope that helps.
Happy coding!!!
I have the following code:
public void Execute(string Query, params SqlParameter[] Parameters)
{
using (var Connection = new SqlConnection(Configuration.ConnectionString))
{
Connection.Open();
using (var Command = new SqlCommand(Query, Connection))
{
if (Parameters.Length > 0)
{
Command.Parameters.Clear();
Command.Parameters.AddRange(Parameters);
}
Command.ExecuteNonQuery();
}
}
}
The method may be called 2 or 3 times for different queries but in same manner.
For example:
Insert an Employee
Insert Employee Certificates
Update Degree of Employee on another table [ Fail can cause here. for example ]
If Point [3] fails, all already committed commands shouldn't execute and must be rolled back.
I know I can put SqlTransaction above and use Commit() method. But what about 3rd point if failed? I think point 3 only will rollback and other point 1,2 will not? How to solve this and what approach should I do??
Should I use SqlCommand[] arrays? What I should I do?
I only find similar question but in CodeProject:
See Here
Without changing your Execute method you can do this
var tranOpts = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = TransactionManager.MaximumTimeout
};
using (var tran = new TransactionScope(TransactionScopeOption.Required, tranOpts)
{
Execute("INSERT ...");
Execute("INSERT ...");
Execute("UPDATE ...");
tran.Complete();
}
SqlClient will cache the internal SqlConnection that is enlisted in the Transaction and reuse it for each call to Execute. So you even end up with a local (not distributed) transaction.
This is all explained in the docs here: System.Transactions Integration with SQL Server
There are a few ways to do it.
The way that probably involves changing the least code and involves the least complexity is to chain multiple SQL statements into a single query. It's perfectly fine to build a string for the Query argument that runs more than one statement, including BEGIN TRANSACTION, COMMIT, and (if needed) ROLLBACK. Basically, keep a whole stored procedure in your C# code. This also has the nice benefit of making it easier to use version control with your procedures.
But it still feels kind of hackish.
One way to reduce that effect is marking the Execute() method private. Then, have an additional method in the class for each query. In this way, the long SQL strings are isolated, and when you're using the database it feels more like using a local API. For more complicated applications, this might instead be a whole separate assembly with a few types managing logical functional areas, where the core methods like Exectue() are internal. This is a good idea anyway, regardless of how you end up supporting transactions.
And speaking of procedures, stored procedures are also a perfectly fine way to handle this. Have one stored procedure to do all the work, and call it when ready.
Another option is overloading the method to accept multiple queries and parameter collections:
public void Execute(string TransactionName, string[] Queries, params SqlParameter[][] Parameters)
{
using (var Connection = new SqlConnection(Configuration.ConnectionString))
using (var Transaction = new SqlTransaction(TransactionName))
{
connection.Transaction = Transaction;
Connection.Open();
try
{
for (int i = 0; i < Queries.Length; i++)
{
using (var Command = new SqlCommand(Queries[i], Connection))
{
command.Transaction = Transaction;
if (Parameters[i].Length > 0)
{
Command.Parameters.Clear();
Command.Parameters.AddRange(Parameters);
}
Command.ExecuteNonQuery();
}
}
Transaction.Commit();
}
catch(Exception ex)
{
Transaction.Rollback();
throw; //I'm assuming you're handling exceptions at a higher level in the code
}
}
}
Though I'm not sure how the params keyword works with an array of arrays... I've just not tried that option, but something along these lines would work. The weakness here is also that it's not trivial to have a later query depend on a result from an earlier query, and even queries with no parameter would still need a Parameters array as a placeholder.
A final option is extending the type holding your Execute() method to support transactions. The trick here is it's common (and desirable) to have this type be static, but supporting transactions requires re-using common connection and transaction objects. Given the implied long-running nature of a transaction, you have to support more than one at a time, which means both instances and implementing IDisposable.
using (var connection = new SqlConnection(Configuration.ConnectionString))
{
SqlCommand command = connection.CreateCommand();
SqlTransaction transaction;
connection.Open();
transaction = connection.BeginTransaction("Transaction");
command.Connection = connection;
command.Transaction = transaction;
try
{
if (Parameters.Length > 0)
{
command.Parameters.Clear();
command.Parameters.AddRange(Parameters);
}
command.ExecuteNonQuery();
transaction.Commit();
}
catch (Exception e)
{
try
{
transaction.Rollback();
}
catch (Exception ex2)
{
//trace
}
}
}
I'm using a simple method for connecting to a MySql database but connecting to this database takes a while & this causes the app to be in "not responding" mode.
now, can I use async for solving this?
Script is:
Private void button_clicked()
{
MysqlConnection connection = new MysqlConnection(constring);
connection.open();
}
MySQL Connector/NET (i.e., MySql.Data) exposes the async ADO.NET methods, e.g., MySqlConnection.OpenAsync, MySqlCommand.ExecuteNonQueryAsync, but these methods all execute synchronously. This is a longstanding bug in Connector/NET.
You can get asynchronous database operations by switching to MySqlConnector (NuGet, GitHub), an OSS alternative that provides asynchronous I/O and higher performance.
MySql.Data.MySqlClient.MySqlConnection.OpenAsync() performs synchronous whereas MySqlConnector.MySqlConnection.OpenAsync() performs asynchronous
I am sure that’s possible. In addition to structuring your code like the following (source):
public Task<DataSet> GetDataSetAsync(string sConnectionString, string sSQL, params SqlParameter[] parameters)
{
return Task.Run(() =>
{
using (var newConnection = new SqlConnection(sConnectionString))
using (var mySQLAdapter = new SqlDataAdapter(sSQL, newConnection))
{
mySQLAdapter.SelectCommand.CommandType = CommandType.Text;
if (parameters != null) mySQLAdapter.SelectCommand.Parameters.AddRange(parameters);
DataSet myDataSet = new DataSet();
mySQLAdapter.Fill(myDataSet);
return myDataSet;
}
});
}
This combined with the use of “await” keyword, will get you the results you need.
//Use Async method to get data
DataSet results = await GetDataSetAsync(sConnectionString, sSQL, sqlParams);
Also update the connection string by adding “Asynchronous Processing=true” connection property (source)
I would also recommend you to have a look at the “OpenAsync” method. You can read more about it in the docs.
At the end, I found the exact answer according to #MikaalAnwar's answer!
We don't need to add any new options (like Asynchronous Processing=true) to connection string; That's for SQL connections & doesn't work for MySql.
So, what should we do now?
we make any void that is respected to have async option, async. Then we add an await task & run it (Task.Run). inside that task, we do what ever we want with our connection.
for example: (We don't want to use any datasets)
private async void DBConnect(String connectionString)
{
await Task.Run(() =>
{
MySqlConnection dbConnection = new MySqlConnection(connectionString);
dbConnection.Open();
}
);
}
& We don't use DBConnection.OpenAsync()because the void is async & we've used await for the task.
Finished:)
Given the following SQLCLR function:
[Microsoft.SqlServer.Server.SqlProcedure]
public static void ExecSQL(string sql, string connectionString)
{
WindowsIdentity clientId = null;
WindowsImpersonationContext impersonatedUser = null;
clientId = SqlContext.WindowsIdentity;
try
{
try
{
impersonatedUser = clientId.Impersonate();
if (impersonatedUser != null)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
//** HERE I WILL HAVE MULTIPLE VARIATIONS OF FETCHING THE SQLDATAREADER
connection.Open();
SqlCommand command = new SqlCommand(sql, connection);
SqlDataReader r = command.ExecuteReader();
impersonatedUser.Undo();
SqlContext.Pipe.Send(r);
}
}
else
{ throw new Exception("Impersonation failed."); }
}
finally
{
if (impersonatedUser != null) { impersonatedUser.Undo(); }
}
}
catch
{
throw;
}
}
Are there any performance or other ramifications of moving this section:
connection.Open();
SqlCommand command = new SqlCommand(sql, connection);
SqlDataReader r = command.ExecuteReader();
out into a separate GetDataReader() method?
I ask because I know I am going to want to load data readers for many different scenarios (Sql text, Stored Procedure, Table Valued Function, Scalar Function, etc), so I would like to encapsulate each of those different implementations into their own function rather than having a big switch statement in the middle of this function.
Are there any performance or other ramifications
Not that I can think of or have run into. Of course, when it comes to performance-related questions of this nature, I always recommend testing it out to see which is better, because anyone answering questions can always be wrong, and there are definitely cases of "accepted" answers being incorrect. Your software running on your system, however, is the definitive answer :-).
Beyond that, there are a few notes about the code fragment shown in the question.
You could move the impersonatedUser.Undo(); to just after the connection.Open();. It was only needed to establish the connection. But, then again, perhaps it would be cleaner to keep the Impersonation handling in the main method and not move just the Undo() to the new method.
OR, you could also keep the connection.Open(); in the main method here instead. You are going to have to pass along the connection object in either case.
You could wrap the impersonatedUser = clientId.Impersonate(); in an if condition, testing the connectionString to see if it is "Context Connection = true;". This would allow you to use the Context Connection, which otherwise won't work since it can't be used with Impersonation. I suppose you can just re-work the current if (impersonatedUser != null) since that test would no longer be valid (the new test would only care if impersonatedUser was null if the Context Connection wasn't being use.
WindowsImpersonationContext is Disposable, so it would be better to move your finally block to the outer try, and add impersonatedUser.Dispose(); after the Undo().
I have N process to run over SQL Server 2008. If any of the processes fails I need to rollback all the others.
I was thinking to use the TPL creating a parent task and N child task. All of this enclosed with a transactionScope (IsolationLevel.ReadCommitted) but in my example below, the child2 throws an error(customers2 is not a valid table) and the child1 doesn't rolled back.
Am I assuming something wrong here? is there other way to manage this scenario?
Here is my test code:
edit
I modified the code as below using the DependClone on the current transaction. I think is working.
try
{
using (TransactionScope mainTransaction = TransactionUtils.CreateTransactionScope())
{
var parentTransactionClone1 = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var parentTransactionClone2 = Transaction.Current.DependentClone(DependentCloneOption.BlockCommitUntilComplete);
var parentTask = Task.Factory.StartNew(() =>
{
var childTask1 = Task.Factory.StartNew(() =>
{
using (TransactionScope childScope1 = new TransactionScope(parentTransactionClone1))
{
SqlConnection cnn = new SqlConnection("Server=.\\sqlexpress;Database=northwind;Trusted_Connection=True;");
cnn.Open();
SqlCommand cmd = new SqlCommand("update customers set city ='valXXX' where customerID= 'ALFKI'", cnn);
cmd.ExecuteNonQuery();
cnn.Close();
childScope1.Complete();
}
parentTransactionClone1.Complete();
}, TaskCreationOptions.AttachedToParent);
var childTask2 = Task.Factory.StartNew(() =>
{
using (TransactionScope childScope2 = new TransactionScope(parentTransactionClone2))
{
SqlConnection cnn = new SqlConnection("Server=.\\sqlexpress;Database=northwind;Trusted_Connection=True;");
cnn.Open();
SqlCommand cmd = new SqlCommand("update customers2 set city ='valyyy' where customerID= 'ANATR'", cnn);
cmd.ExecuteNonQuery();
cnn.Close();
childScope2.Complete();
}
parentTransactionClone2.Complete();
}, TaskCreationOptions.AttachedToParent);
});
parentTask.Wait();
mainTransaction.Complete();
}
}
catch (Exception ex)
{
// manage ex
}
public static TransactionScope CreateTransactionScope()
{
var transactionOptions = new TransactionOptions();
transactionOptions.IsolationLevel = IsolationLevel.ReadCommitted;
transactionOptions.Timeout = TransactionManager.MaximumTimeout;
return new TransactionScope(TransactionScopeOption.Required, transactionOptions);
}
The TransactionScope class sets the ambient transaction for the current thread (see also Transaction.Current only.
You should at least assume that each task runs in a separate thread (although that is not a necessity with the TPL).
Review the "important" box in the remarks section of the relevant article - if you want to share a transaction between threads, you need to use the DependentTransaction class.
Personally, I am sure that the whole facility to share a transaction amongst multiple threads works technically, however, I have always found it easier to write up a design that uses a seperate transaction per thread.
Task Parallel Library cannot figure out on its own the details of the task, and it won't be rolling back automatically, the closest you can do is from the parent task you define another task child1-rollback that gets executed only if child1 fails, and you can define this very nicely by specifying a TaskContinuationOption set to OnlyOnFailure so the task will execute only if child1 fails, same can be said about child2.