I'm having this weird issue (or I think my brain is burned..). While I run this code from outside a Task, it works fine, but when I run within a Task.Run(() => ...), I get a SqlException timeout:
public static Item GetItemById(int id)
{
Item result;
using (var conn = App.DbFactory.CreateConnection())
{
result = _repository.GetById(id, conn) ?? throw new ElementNotFoundException();
}
return result;
}
// _repository.GetById
public Item GetById(int id, IDatabaseConnection conn)
{
Item result;
var cmd = conn.CreateCommand();
cmd.CommandText = "QUERY COMMAND";
using (var dr = cmd.ExecuteReader()) <-- EXCEPTION
{
result = dr.Read() ? Create(dr) : null;
}
return result;
}
// Method that works.
public static Item GetItemTest()
{
return GetItemById(12);
}
// Method that doesn't work.
public static async Item GetItemAsyncTest()
{
return await Task.Run(() => GetItemById(12));
}
App.DbFactory.CreateConnection() returns IDbConnection.
Update: The exception is thrown when executing the DbCommand (ExecuteReader).
The GeyById method only calls a repository method (raw TSQL query). The caller method is supposed to read each item from the database from a foreach (using Task/async/await, no more than 5 concurrent connections). At this point, I don't know if I'm doing something wrong or if I'm missing some concept.
I'm using .NET Framework 4.5 and SQL Server 2012.
Temp solution:
Ok, I'd wasted almost 6hs trying to figure out what was wrong...and it was NUnit. When running GetItemAsyncTest() from a Test, it throws the timeout exception. Running the same async method from a Controller works like a charm.
I'm googling about this thing. If anyone had the same issue, I'll be glad to know what it is :)
Thanks!!
Well, finally solved! It was a workaround between NUnit and TransactionScope. Testing an async method that hits the database using a TransactionScope (to keep the database clean) causes a timeout exception when executing the command.
The way to solve this issue was: upgrade to NET Framework 4.5.1 and add TransactionScopeAsyncFlowOption.Enabled to the TransactionScope constructor.
Related
I have a program, which is related to database. I need to unit-test some methods from class, which represents repository. I've decided to do it without localdb, but using Rollback attribute implementation:
public class Rollback : Attribute, ITestAction
{
private TransactionScope transaction;
public void BeforeTest(ITest test)
{
transaction = new TransactionScope();
}
public void AfterTest(ITest test)
{
transaction.Dispose();
}
public ActionTargets Targets => ActionTargets.Test;
}
I've got that from the Internet. This attribute implicitly begins transaction before method's code, and rolls it back after. That really works nice.
But at one moment I've wanted to debug this test:
[Test, Rollback]
public async Task AddingExistingDictionaryTypeThrowsExceptionTest()
{
await _repository.AddDictionaryType(tempDictionaryTypeName, tempDictionaryTypeDescription);
Assert.ThrowsAsync<Exception>(async () => { await _repository.AddDictionaryType(tempDictionaryTypeName, tempDictionaryTypeDescription); });
}
AddDictionaryType - is a method from repository. It adds new entity to database after checking, if such record doesn't exist already:
public async Task AddDictionaryType(string name, string description)
{
try
{
var sameNameCollection = _dbContext.DictionaryTypes.FromSqlRaw(#$"select * from dictionary_type where name = '{name}'");
var sameDescriptionCollection = _dbContext.DictionaryTypes.FromSqlRaw(#$"select * from dictionary_type where description = '{description}'");
if (sameNameCollection.Any() || sameDescriptionCollection.Any())
{
throw new AddingExistingDictionaryException();
}
_dbContext.Add(new DictionaryType(name, description));
await _dbContext.SaveChangesAsync();
}
catch (Exception ex)
{
throw ex;
}
}
Don't pay attention to two SELECTS, I know, that I may make one check, but that's for my needs.
So, I've set the point after first call of AddDictionaryType method to check new record in database, after that I've done SELECT in SSMS(I know, that was silly, too, because method worked in transaction) from same table, in which I've tried to insert record. So, I've got an error. And now I will tell you about most interesting:
After that I can't normally execute the test, I always get error: "This connection was used with the ambient transaction. The original ambient transaction needs to be completed before this connection can be used outside of it".
So I think, that there is that transaction, which hasn't been closed.
The problem is that I cannot find it and kill.
I've operated with such queries to find it:
SELECT * FROM sys.sysprocesses;
SELECT * from sys.dm_tran_current_transaction with(nolock);
SELECT * from sys.dm_tran_active_transactions;
EXEC SP_who2;
SELECT * FROM sys. dm_exec_sessions;
SELECT * FROM fn_dblog(NULL, NULL);
I've seen processes, tried to kill them, didn't help.
I've reloaded server, didn't help.
There is no any information about such transaction in transaction log.
No I see, that testing like this is a big problem, because I don't even know, how that transaction could be named to rollback it by my hands.
May be it's not because of transaction? But I don't have any ideas. What about you?
I've refused using Rollback attribute, but follow
Charlieface's advice. Now I use using statement in each test, and everything works nice:
[Test]
public async Task Test()
{
using (var tran = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
// test logic
}
}
Also, I've understodd, that there is know any stuck transaction in database.
I have this code in a method:
Task<int> linksCount = Task<int>.Factory.StartNew(() => { return DatabaseIn.GetUrlsCount(); });
Task<int> imagesCount = Task<int>.Factory.StartNew(() => { return DatabaseIn.GetImagesCount(); });
Later the linksCount and imagesCount.Result is displayed. However, when I run this code the first time. I get the following exception:
There is already an open DataReader associated with this Connection which must be closed first.
The code where the exception is thrown is:
MySqlCommand comm = null;
string cmdString = "SELECT COUNT(*) FROM damocles.imagestoassess;";
comm = new MySqlCommand(cmdString, conn);
if (comm.ExecuteScalar() == null)
{
comm.Dispose();
conn.Close();
return 0;
}
int res = Convert.ToInt32(comm.ExecuteScalar());
comm.Dispose();
conn.Close();
return res;
The exception is thrown on the first call to ExecuteScalar. If I enclose the code in a try..catch clause the code continues but it throws another error in another part of the code.
Question:
Why does it throw the exception? It doesn't throw any exception when run synchronously.
How do I fix the code to work Asynchronously?
Thanks.
A MySqlConnection can only be used by one thread at a time. Your exception is happening because both Tasks are using the same conn object. For more details, see https://mysqlconnector.net/troubleshooting/connection-reuse/.
Your code will work if you create and use a new MySqlConnection inside each Task.
Additionally, it's a longstanding bug that async operations in MySql.Data aren't actually asynchronous. You'll need to switch to MySqlConnector to get asynchronous I/O for MySQL operations.
I'm just starting out with async and Task's and my code has stopped processing. It happens when I have an incoming network packet and I try and communicate with the database inside the packet handler.
public class ClientConnectedPacket : IClientPacket
{
private readonly EntityFactory _entityFactory;
public ClientConnectedPacket(EntityFactory entityFactory)
{
_entityFactory= entityFactory;
}
public async Task Handle(NetworkClient client, ClientPacketReader reader)
{
client.Entity = await _entityFactory.CreateInstanceAsync( reader.GetValueByKey("unique_device_id"));
// this Console.WriteLine never gets reached
Console.WriteLine($"Client [{reader.GetValueByKey("unique_device_id")}] has connected");
}
}
The Handle method gets called from an async task
if (_packetRepository.TryGetPacketByName(packetName, out var packet))
{
await packet.Handle(this, new ClientPacketReader(packetName, packetData));
}
else
{
Console.WriteLine("Unknown packet: " + packetName);
}
Here is the method which I think is causing the issue
public async Task<Entity> CreateInstanceAsync(string uniqueId)
{
await using (var dbConnection = _databaseProvider.GetConnection())
{
dbConnection.SetQuery("SELECT COUNT(NULL) FROM `entities` WHERE `unique_id` = #uniqueId");
dbConnection.AddParameter("uniqueId", uniqueId);
var row = await dbConnection.ExecuteRowAsync();
if (row != null)
{
return new Entity(uniqueId, false);
}
}
return new Entity(uniqueId,true);
}
DatabaseProvider's GetConnection method:
public DatabaseConnection GetConnection()
{
var connection = new MySqlConnection(_connectionString);
var command = connection.CreateCommand();
return new DatabaseConnection(_logFactory.GetLogger(), connection, command);
}
DatabaseConnection's constructor:
public DatabaseConnection(ILogger logger, MySqlConnection connection, MySqlCommand command)
{
_logger = logger;
_connection = connection;
_command = command;
_connection.Open();
}
When I comment out this line, it reaches the Console.WriteLine
_connection.Open();
I ran a POC project spinning 100 parallel tasks both with MySql.Data 8.0.19 and MySqlConnector 0.63.2 on .NET Core 3.1 console application. I create, open and dispose the connection into the context of every single task. Both providers runs to completion without errors.
The specifics are that MySql.Data queries run synchronously although the library provide async methods signature e.g. ExecuteReaderAsync() or ExecuteScalarAsync(), while MySqlConnector run truly asynchronously.
You may be running into:
a deadlock situation not specifically related to the mysql provider
not properly handling exceptions inside your tasks (you may inspect the task associated aggregate exception and also monitor mysql db logs)
you execution be still blocked (not returning result) when you assume it’s not working, if you running a high number of parallel tasks with MySql.Data as it executes synchronously
Multi-threading with MySQL must use independent connections. Given that, multithreading is not a MySQL question but an issue for the client language, C# in your question.
That is, build your threads without regard to MySQL, then create a connection in each thread that needs to do queries. It will be on your shoulders if you need to pass data between the threads.
I usually find that optimizing queries eliminates the temptation to multi-thread my applications.
I have an odd situation with TransactionScope and async/synchronous SQL calls that I'm having difficulty understanding. I hope that someone with a deeper understanding of the ins and outs of these kinds of operations can shed some light on the issue.
The situation:
I have a NUnit testfixture which creates a TransactionScope during [SetUp] and Disposes it at [TearDown] to let each test run on the same data. I have a series of tests which kick off an asynchronous operation on the database and then execute a synchronous operation on the database. The first such test completes successfully. The second such test fails with "There is already an open DataReader associated with this Command which must be closed first.".
If I comment out the TransactionScope entirely, all the tests pass.
I tried various different TransactionScope options, and Complete / Dispose, but the same issue occurs.
I am using the Resharper test runner on an NUnit test, .NET 4.5.1.
I realize the "correct" answer may be "make everything async await". That's not an option for me, unfortunately.
I don't want to enable MARS, as this issue only occurs in tests.
I don't want to use GetAwaiter().GetResult() due to the potential deadlocks.
What it looks like to me is that once a TransactionScope.Dispose/Complete is called, the automatic SQLConnection pooling loses track of which connections have open DataReaders. It hands out the same SqlConnection to two simultaneously running operations, and the second dies.
My primary question is "what is causing this behavior (specifically)?"
My secondary question is "is there anything that can be done to safely resolve the issue?"
The replicating code below prints out the client connection Ids. On my machine, the ClientConnectionId for the ASYNC and SYNC calls in the Second test case are always the same.
Replicating Code:
[TestFixture]
public class DataReaderTests
{
private TransactionScope _scope;
private string _connString = #"my connection string";
[SetUp]
public void Setup()
{
var options = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = TimeSpan.FromMinutes(1)
};
_scope = new TransactionScope(TransactionScopeOption.RequiresNew, options, TransactionScopeAsyncFlowOption.Enabled);
}
[Test]
[TestCase("First")]
[TestCase("Second")]
public void Test(string name)
{
DoAsyncThing().ConfigureAwait(false);
using (var conn = new SqlConnection(_connString))
{
try
{
conn.Open();
Console.WriteLine("SYNC: " + conn.ClientConnectionId);
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT 1";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
int id = reader.GetInt32(0);
}
}
}
}
catch (TransactionAbortedException tax)
{
Console.WriteLine("ERROR: " + ((SqlException)tax.InnerException.InnerException).ClientConnectionId);
throw;
}
}
}
private async Task DoAsyncThing()
{
using (var connection = new SqlConnection(_connString))
{
await connection.OpenAsync();
Console.WriteLine("ASYNC: " + connection.ClientConnectionId);
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = "WAITFOR DELAY '00:02';";
await cmd.ExecuteNonQueryAsync();
Console.WriteLine("ASYNC COMPLETE");
}
}
}
[TearDown]
public void Teardown()
{
_scope.Dispose();
}
}`
Check out this answer
I think the gist is that you cannot have two active sql commands executing over the same connection at the same time without a special connection string property. When you are operating under the transaction scope, you should find that both SqlConnection objects have the same client ID. However, if you remove the transaction scope they are different, which I believe implies that they are operating on separate connections.
Adding "MultipleActiveResultSets=true" to the connection string fixed the issue for me. Another alternative is to replace
DoAsyncThing().ConfigureAwait(false);
with
DoAsyncThing().ConfigureAwait(false).GetAwaiter().GetResult();
which will terminate the first command before starting the second command.
I have an application leveraging Entity Framework 6. For queries that are relatively fast (e.g. taking less than a minute to execute) it is working fine.
But I have a stored procedure that queries a table which doesn't have appropriate indices and so the time taken to execute the query has been clocked to take anywhere between 55 and 63 seconds. Obviously, indexing the table would bring that time down but unfortunately I don't have the luxury of controlling the situation and have to deal the hand I was dealt.
What I am seeing is when EF6 is used to call the stored procedure it continues through the code in less than 3 seconds total time and returns a result of 0 records; when I know there are 6 records the SPROC will return when executed directly in the database.
There are no errors whatsoever, so the code is executing fine.
Performing a test; I constructed some code using the SqlClient library and made the same call and it returned 6 records. Also noted that unlike the EF6 execution, that it actually took a few more seconds as if it were actually waiting to receive a response.
Setting the CommandTimeout on the context doesn't appear to make any difference either and I suspect possibly because it isn't timing out but rather not waiting for the result before it continues through the code?
I don't recall seeing this behavior in prior versions but then again maybe the time required to execute my prior queries were within the expected range of EF???
Is there a way to set the actual time that EF will wait for a response before continuing through the code? Or is there a way that I can enforce an asynchronous operation since it seems to be a default synchronous task by default?? Or is there a potential flaw in the code?
Sample of Code exhibiting (synchronous) execution: No errors but no records returned
public static List<Orphan> GetOrphanItems()
{
try
{
using (var ctx = new DBEntities(_defaultConnection))
{
var orphanage = from orp in ctx.GetQueueOrphans(null)
select orp;
var orphans = orphanage.Select(o => new Orphan
{
ServiceQueueId = o.ServiceQueueID,
QueueStatus = o.QueueStatus,
OrphanCode = o.OrphanCode,
Status = o.Status,
EmailAddress = o.EmailAddress,
TemplateId = o.TemplateId
}).ToList();
return orphans;
}
}
catch(Exception exc)
{
// Handle the error
}
}
Sample Code using SqlClient Library (asynchronous) takes slightly longer to execute but returns 6 records
public static List<Orphan> GetOrphanItems()
{
long ServiceQueueId = 0;
bool QueueStatus;
var OrphanCode = String.Empty;
DateTime Status;
var EmailAddress = String.Empty;
int TemplateId = 0;
var orphans = new List<Orphan> ();
SqlConnection conn = new SqlConnection(_defaultConnection);
try
{
var cmdText = "EXEC dbo.GetQueueOrphans";
SqlCommand cmd = new SqlCommand(cmdText, conn);
conn.Open();
SqlDataReader reader;
reader = cmd.ExecuteReader();
while(reader.Read())
{
long.TryParse(reader["ServiceQueueId"].ToString(), out ServiceQueueId);
bool.TryParse(reader["QueueStatus"].ToString(), out QueueStatus);
OrphanCode = reader["OrphanCode"].ToString();
DateTime.TryParse(reader["Status"].ToString(), out Status);
EmailAddress = reader["EmailAddress"].ToString();
int.TryParse(reader["TemplateId"].ToString(), out TemplateId);
orphans.Add(new Orphan { ServiceQueueId = ServiceQueueId, QueueStatus=QueueStatus, OrphanCode=OrphanCode,
EmailAddress=EmailAddress, TemplateId=TemplateId});
}
conn.Close();
catch(Exception exc)
{
// Handle the error
}
finally
{
conn.Close();
}
}
Check the type of executing method.
private async void MyMethod()
{
db.executeProdecudeAsync();
}
Forgetting to await task in async void method can cause described behavior without any InteliSense warning.
Fix:
private async Task MyMethod()
{
await db.executeProdecudeAsync();
}
Or just use db.executeProdecudeAsync().Wait() if you want to run in synchronous mode.