I'm using a simple method for connecting to a MySql database but connecting to this database takes a while & this causes the app to be in "not responding" mode.
now, can I use async for solving this?
Script is:
Private void button_clicked()
{
MysqlConnection connection = new MysqlConnection(constring);
connection.open();
}
MySQL Connector/NET (i.e., MySql.Data) exposes the async ADO.NET methods, e.g., MySqlConnection.OpenAsync, MySqlCommand.ExecuteNonQueryAsync, but these methods all execute synchronously. This is a longstanding bug in Connector/NET.
You can get asynchronous database operations by switching to MySqlConnector (NuGet, GitHub), an OSS alternative that provides asynchronous I/O and higher performance.
MySql.Data.MySqlClient.MySqlConnection.OpenAsync() performs synchronous whereas MySqlConnector.MySqlConnection.OpenAsync() performs asynchronous
I am sure that’s possible. In addition to structuring your code like the following (source):
public Task<DataSet> GetDataSetAsync(string sConnectionString, string sSQL, params SqlParameter[] parameters)
{
return Task.Run(() =>
{
using (var newConnection = new SqlConnection(sConnectionString))
using (var mySQLAdapter = new SqlDataAdapter(sSQL, newConnection))
{
mySQLAdapter.SelectCommand.CommandType = CommandType.Text;
if (parameters != null) mySQLAdapter.SelectCommand.Parameters.AddRange(parameters);
DataSet myDataSet = new DataSet();
mySQLAdapter.Fill(myDataSet);
return myDataSet;
}
});
}
This combined with the use of “await” keyword, will get you the results you need.
//Use Async method to get data
DataSet results = await GetDataSetAsync(sConnectionString, sSQL, sqlParams);
Also update the connection string by adding “Asynchronous Processing=true” connection property (source)
I would also recommend you to have a look at the “OpenAsync” method. You can read more about it in the docs.
At the end, I found the exact answer according to #MikaalAnwar's answer!
We don't need to add any new options (like Asynchronous Processing=true) to connection string; That's for SQL connections & doesn't work for MySql.
So, what should we do now?
we make any void that is respected to have async option, async. Then we add an await task & run it (Task.Run). inside that task, we do what ever we want with our connection.
for example: (We don't want to use any datasets)
private async void DBConnect(String connectionString)
{
await Task.Run(() =>
{
MySqlConnection dbConnection = new MySqlConnection(connectionString);
dbConnection.Open();
}
);
}
& We don't use DBConnection.OpenAsync()because the void is async & we've used await for the task.
Finished:)
Related
I'm having this weird issue (or I think my brain is burned..). While I run this code from outside a Task, it works fine, but when I run within a Task.Run(() => ...), I get a SqlException timeout:
public static Item GetItemById(int id)
{
Item result;
using (var conn = App.DbFactory.CreateConnection())
{
result = _repository.GetById(id, conn) ?? throw new ElementNotFoundException();
}
return result;
}
// _repository.GetById
public Item GetById(int id, IDatabaseConnection conn)
{
Item result;
var cmd = conn.CreateCommand();
cmd.CommandText = "QUERY COMMAND";
using (var dr = cmd.ExecuteReader()) <-- EXCEPTION
{
result = dr.Read() ? Create(dr) : null;
}
return result;
}
// Method that works.
public static Item GetItemTest()
{
return GetItemById(12);
}
// Method that doesn't work.
public static async Item GetItemAsyncTest()
{
return await Task.Run(() => GetItemById(12));
}
App.DbFactory.CreateConnection() returns IDbConnection.
Update: The exception is thrown when executing the DbCommand (ExecuteReader).
The GeyById method only calls a repository method (raw TSQL query). The caller method is supposed to read each item from the database from a foreach (using Task/async/await, no more than 5 concurrent connections). At this point, I don't know if I'm doing something wrong or if I'm missing some concept.
I'm using .NET Framework 4.5 and SQL Server 2012.
Temp solution:
Ok, I'd wasted almost 6hs trying to figure out what was wrong...and it was NUnit. When running GetItemAsyncTest() from a Test, it throws the timeout exception. Running the same async method from a Controller works like a charm.
I'm googling about this thing. If anyone had the same issue, I'll be glad to know what it is :)
Thanks!!
Well, finally solved! It was a workaround between NUnit and TransactionScope. Testing an async method that hits the database using a TransactionScope (to keep the database clean) causes a timeout exception when executing the command.
The way to solve this issue was: upgrade to NET Framework 4.5.1 and add TransactionScopeAsyncFlowOption.Enabled to the TransactionScope constructor.
I'm just starting out with async and Task's and my code has stopped processing. It happens when I have an incoming network packet and I try and communicate with the database inside the packet handler.
public class ClientConnectedPacket : IClientPacket
{
private readonly EntityFactory _entityFactory;
public ClientConnectedPacket(EntityFactory entityFactory)
{
_entityFactory= entityFactory;
}
public async Task Handle(NetworkClient client, ClientPacketReader reader)
{
client.Entity = await _entityFactory.CreateInstanceAsync( reader.GetValueByKey("unique_device_id"));
// this Console.WriteLine never gets reached
Console.WriteLine($"Client [{reader.GetValueByKey("unique_device_id")}] has connected");
}
}
The Handle method gets called from an async task
if (_packetRepository.TryGetPacketByName(packetName, out var packet))
{
await packet.Handle(this, new ClientPacketReader(packetName, packetData));
}
else
{
Console.WriteLine("Unknown packet: " + packetName);
}
Here is the method which I think is causing the issue
public async Task<Entity> CreateInstanceAsync(string uniqueId)
{
await using (var dbConnection = _databaseProvider.GetConnection())
{
dbConnection.SetQuery("SELECT COUNT(NULL) FROM `entities` WHERE `unique_id` = #uniqueId");
dbConnection.AddParameter("uniqueId", uniqueId);
var row = await dbConnection.ExecuteRowAsync();
if (row != null)
{
return new Entity(uniqueId, false);
}
}
return new Entity(uniqueId,true);
}
DatabaseProvider's GetConnection method:
public DatabaseConnection GetConnection()
{
var connection = new MySqlConnection(_connectionString);
var command = connection.CreateCommand();
return new DatabaseConnection(_logFactory.GetLogger(), connection, command);
}
DatabaseConnection's constructor:
public DatabaseConnection(ILogger logger, MySqlConnection connection, MySqlCommand command)
{
_logger = logger;
_connection = connection;
_command = command;
_connection.Open();
}
When I comment out this line, it reaches the Console.WriteLine
_connection.Open();
I ran a POC project spinning 100 parallel tasks both with MySql.Data 8.0.19 and MySqlConnector 0.63.2 on .NET Core 3.1 console application. I create, open and dispose the connection into the context of every single task. Both providers runs to completion without errors.
The specifics are that MySql.Data queries run synchronously although the library provide async methods signature e.g. ExecuteReaderAsync() or ExecuteScalarAsync(), while MySqlConnector run truly asynchronously.
You may be running into:
a deadlock situation not specifically related to the mysql provider
not properly handling exceptions inside your tasks (you may inspect the task associated aggregate exception and also monitor mysql db logs)
you execution be still blocked (not returning result) when you assume it’s not working, if you running a high number of parallel tasks with MySql.Data as it executes synchronously
Multi-threading with MySQL must use independent connections. Given that, multithreading is not a MySQL question but an issue for the client language, C# in your question.
That is, build your threads without regard to MySQL, then create a connection in each thread that needs to do queries. It will be on your shoulders if you need to pass data between the threads.
I usually find that optimizing queries eliminates the temptation to multi-thread my applications.
I have an odd situation with TransactionScope and async/synchronous SQL calls that I'm having difficulty understanding. I hope that someone with a deeper understanding of the ins and outs of these kinds of operations can shed some light on the issue.
The situation:
I have a NUnit testfixture which creates a TransactionScope during [SetUp] and Disposes it at [TearDown] to let each test run on the same data. I have a series of tests which kick off an asynchronous operation on the database and then execute a synchronous operation on the database. The first such test completes successfully. The second such test fails with "There is already an open DataReader associated with this Command which must be closed first.".
If I comment out the TransactionScope entirely, all the tests pass.
I tried various different TransactionScope options, and Complete / Dispose, but the same issue occurs.
I am using the Resharper test runner on an NUnit test, .NET 4.5.1.
I realize the "correct" answer may be "make everything async await". That's not an option for me, unfortunately.
I don't want to enable MARS, as this issue only occurs in tests.
I don't want to use GetAwaiter().GetResult() due to the potential deadlocks.
What it looks like to me is that once a TransactionScope.Dispose/Complete is called, the automatic SQLConnection pooling loses track of which connections have open DataReaders. It hands out the same SqlConnection to two simultaneously running operations, and the second dies.
My primary question is "what is causing this behavior (specifically)?"
My secondary question is "is there anything that can be done to safely resolve the issue?"
The replicating code below prints out the client connection Ids. On my machine, the ClientConnectionId for the ASYNC and SYNC calls in the Second test case are always the same.
Replicating Code:
[TestFixture]
public class DataReaderTests
{
private TransactionScope _scope;
private string _connString = #"my connection string";
[SetUp]
public void Setup()
{
var options = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = TimeSpan.FromMinutes(1)
};
_scope = new TransactionScope(TransactionScopeOption.RequiresNew, options, TransactionScopeAsyncFlowOption.Enabled);
}
[Test]
[TestCase("First")]
[TestCase("Second")]
public void Test(string name)
{
DoAsyncThing().ConfigureAwait(false);
using (var conn = new SqlConnection(_connString))
{
try
{
conn.Open();
Console.WriteLine("SYNC: " + conn.ClientConnectionId);
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT 1";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
int id = reader.GetInt32(0);
}
}
}
}
catch (TransactionAbortedException tax)
{
Console.WriteLine("ERROR: " + ((SqlException)tax.InnerException.InnerException).ClientConnectionId);
throw;
}
}
}
private async Task DoAsyncThing()
{
using (var connection = new SqlConnection(_connString))
{
await connection.OpenAsync();
Console.WriteLine("ASYNC: " + connection.ClientConnectionId);
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = "WAITFOR DELAY '00:02';";
await cmd.ExecuteNonQueryAsync();
Console.WriteLine("ASYNC COMPLETE");
}
}
}
[TearDown]
public void Teardown()
{
_scope.Dispose();
}
}`
Check out this answer
I think the gist is that you cannot have two active sql commands executing over the same connection at the same time without a special connection string property. When you are operating under the transaction scope, you should find that both SqlConnection objects have the same client ID. However, if you remove the transaction scope they are different, which I believe implies that they are operating on separate connections.
Adding "MultipleActiveResultSets=true" to the connection string fixed the issue for me. Another alternative is to replace
DoAsyncThing().ConfigureAwait(false);
with
DoAsyncThing().ConfigureAwait(false).GetAwaiter().GetResult();
which will terminate the first command before starting the second command.
I'm designing a small desktop app that fetches data from SQL server. I used BackgroundWorker to make the query execute in background. The code that fetches data generally comes down to this:
public static DataTable GetData(string sqlQuery)
{
DataTable t = new DataTable();
using (SqlConnection c = new SqlConnection(GetConnectionString()))
{
c.Open();
using (SqlCommand cmd = new SqlCommand(sqlQuery))
{
cmd.Connection = c;
using (SqlDataReader r = cmd.ExecuteReader())
{
t.Load(r);
}
}
}
return t;
}
Since query can take up 10-15 minutes I want to implement cancellation request and pass it from GUI layer to DAL. Cancellation procedure of BackroundWorker won't let me cancel SqlCommand.ExecuteReader() beacuse it only stops when data is fetched from server or an exception is thrown by Data Provider.
I tried to use Task and async/await with SqlCommand.ExecuteReaderAsync(CancellationToken) but I am confused where to use it in multi-layer app (GUI -> BLL -> DAL).
Have you tried using the SqlCommand.Cancel() method ?
Aproach: encapsulate that GetData method in a Thread/Worker and then when you cancel/stop that thread call the Cancel() method on the SqlCommand that is being executed.
Here is an example on how to use it on a thread
using System;
using System.Data;
using System.Data.SqlClient;
using System.Threading;
class Program
{
private static SqlCommand m_rCommand;
public static SqlCommand Command
{
get { return m_rCommand; }
set { m_rCommand = value; }
}
public static void Thread_Cancel()
{
Command.Cancel();
}
static void Main()
{
string connectionString = GetConnectionString();
try
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
Command = connection.CreateCommand();
Command.CommandText = "DROP TABLE TestCancel";
try
{
Command.ExecuteNonQuery();
}
catch { }
Command.CommandText = "CREATE TABLE TestCancel(co1 int, co2 char(10))";
Command.ExecuteNonQuery();
Command.CommandText = "INSERT INTO TestCancel VALUES (1, '1')";
Command.ExecuteNonQuery();
Command.CommandText = "SELECT * FROM TestCancel";
SqlDataReader reader = Command.ExecuteReader();
Thread rThread2 = new Thread(new ThreadStart(Thread_Cancel));
rThread2.Start();
rThread2.Join();
reader.Read();
System.Console.WriteLine(reader.FieldCount);
reader.Close();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
static private string GetConnectionString()
{
// To avoid storing the connection string in your code,
// you can retrieve it from a configuration file.
return "Data Source=(local);Initial Catalog=AdventureWorks;"
+ "Integrated Security=SSPI";
}
}
You can only do Cancelation checking and Progress Reporting between Distinct lines of code. Usually both require that you disect the code down to the lowest loop level, so you can do both these things between/in the loop itterations. When I wrote my first step into BGW, I had the advantage that I needed to do the loop anyway so it was no extra work. You have one of the worse cases - pre-existing code that you can only replicate or use as is.
Ideal case:
This operation should not take nearly as long is it does. 5-10 minutes indicates that there is something rather wrong with your design.
If the bulk of the time is transmission of data, then you are propably retreiving way to much data. Retrieving everything to do filtering in the GUI is a very common mistake. Do as much filtering in the query as possible. Usign a Distributed Database might also help with transmission performance.
If the bulk of the time is processing as part of the query operation (complex Conditions), something in your general approach might have to change. There are various ways to trade off complex calculation with a bit of memory on the DBMS side. Views afaik can cache the results of operations, while still maintaining transactional consistency.
But it really depends what your backend DB/DBMS and use case are. A lot of the use SQL as Query Language. So it does not allow us to predict wich options you have.
Second best case:
The second best thing if you can not cut it down, would be if you had the actually DB access code down to the lowest loop and would do progress reporting/cancelation checking on it. That way you could actually use the existing Cancelation Token System inherent in BGW.
Everything else
Using any other approach to Cancelation is really a fallback. I wrote a lot on why it is bad, but felt that this might work better if I focus on the core issue - likely something wrong in design of he DB and/or Query. Because those might well eliminate the issue altogether.
Trying to implement a retry logic in a correct way, but couldn't find any good sample on how to properly leverage EnterpriseLibrary.TransientFaultHandling.
So far I found two samples:
First - using ReliableSqlConnection & conn.Open(retryPolicy)
var retryStrategy = new Incremental(3, TimeSpan.FromMilliseconds(500), TimeSpan.FromSeconds(1));
var retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(retryStrategy);
retryPolicy.ExecuteAction(() =>
{
using (var conn = new ReliableSqlConnection(datasetConnectionString))
{
conn.Open(retryPolicy);
using (var command = conn.CreateCommand())
{
command.CommandText = insertToParameters;
command.CommandTimeout = 0;
conn.ExecuteCommand(command);
}
}
});
and second - without the ReliableSqlConnection:
var retryStrategy = new Incremental(3, TimeSpan.FromMilliseconds(500), TimeSpan.FromSeconds(1));
var retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(retryStrategy);
retryPolicy.ExecuteAction(() =>
{
using (var conn = new SqlConnection(datasetConnectionString))
{
conn.Open();
using (var command = conn.CreateCommand())
{
command.CommandText = insertToParameters;
command.CommandTimeout = 0;
conn.ExecuteCommand(command);
}
}
});
So few questions:
which one is better and why?
Is the external retryPolicy.ExecuteAction really needed - in older samples I see people retrying only individual actions, like OpenConnectionWithRetries, ExecuteCommandWithRetries, etc, but not the whole thing - I wonder if that is possible that the connection could potentially be closed between those retries.
Answering my own question:
With newer transient error handling block, use SqlConnection with the provided extension methods, such as OpenWithRetry, etc.
Use retryPolicy.ExecuteAction(() => {...}) whenever there is no support for the retry in the API, such as SqlBulkCopy, filling the dataset table, async methods etc. Make sure to re-open connection in the retry block. You still can use the SqlConnection with retry-able extension methods inside the retry block.
UPDATE: edited to cause less confusion
The best pattern to use is RetryPolicy.ExecuteAction(() => . Wrap your database interactions into an ExecuteAction() lambda and treat that as a unit of work. This maintains the highest level of compatibility with ADO.Net's APIs. It also allows you to work with the new Async methods in ADO.Net.
The ReliableSQLConnection is primarily offered for backward compatibility.