Develop Reference

Canceling query with while loop hangs forever

I am trying to use query cancellation (via cancellation tokens) to cancel a long-running complex query. I have found that in some cases not only does cancellation fail to halt the query but also the call to CancellationToken.Cancel() hangs indefinitely. Here is a simple repro that replicates this behavior (can be run in LinqPad):
void Main()
{
var cancellationTokenSource = new CancellationTokenSource();
var blocked = RunSqlAsync(cancellationTokenSource.Token);
blocked.Wait(TimeSpan.FromSeconds(1)).Dump(); // false (blocked in SQL as expected)
cancellationTokenSource.Cancel(); // hangs forever?!
Console.WriteLine("Finished calling Cancel()");
blocked.Wait();
}
public async Task RunSqlAsync(CancellationToken cancellationToken)
{
var connectionString = new SqlConnectionStringBuilder { DataSource = #".\sqlexpress", IntegratedSecurity = true, Pooling = false }.ConnectionString;
using (var connection = new SqlConnection(connectionString))
{
await connection.OpenAsync().ConfigureAwait(false);
using (var command = connection.CreateCommand())
{
command.CommandText = #"
WHILE 1 = 1
BEGIN
DECLARE #x INT = 1
END
";
command.CommandTimeout = 0;
Console.WriteLine("Running query");
await command.ExecuteNonQueryAsync(cancellationToken).ConfigureAwait(false);
}
}
}
Interestingly, the same query run in SqlServer Management Studio cancels instantly via the "Cancel Executing Query" button.
Is there some caveat to query cancellation where it cannot cancel tight WHILE loops?
My version of SqlServer:
Microsoft SQL Server 2012 - 11.0.2100.60 (X64)
Feb 10 2012 19:39:15
Copyright (c) Microsoft Corporation
Express Edition (64-bit) on Windows NT 6.2 (Build 9200: )
I am running on Windows 10, and .NET's Environment.Version is 4.0.30319.42000.
EDIT
Some additional information:
Here is the stack trace pulled from Visual Studio when cancellationToken.Cancel() hangs:
Another thread is stuck here:
Additionally, I tried updating to SqlServer Express 2017 and I am seeing the same behavior.
EDIT
I've filed this as a bug with corefx: https://github.com/dotnet/corefx/issues/26623

I can reproduce the issue in a console application. (The code in the question was code from LINQPad.)
I'm going to make this an answer and say that this is a bug in ADO.NET. ADO.NET should send a query cancellation signal to SQL Server. I can see from the CPU usage, that SQL Server continues executing the loop. Therefore, it did not receive cancellation from the client. We also know that SSMS is able to cancel this loop.
While the loop is running I can see that the console app is using 50% of one CPU core and receiving data from SQL Server at 70MB/sec. I do not know what data this is. It might be ROWCOUNT information or something related.
I think the bug is related to the fact that the loop is continuously sending data so that ADO.NET never has an opportunity to send the cancellation. It's still a bug and it would be a community service if you reported it. You can link to this question.
If the loop is throttled using ...
WHILE 1 = 1
BEGIN
DECLARE #x INT = 1
WAITFOR DELAY '00:00:01' --new
END
... then cancellation is quick.
Also, you can generally not rely on cancellation being quick. If the network dropped it might take 30sec for the client to notice this and throw.
Therefore you need to code your program so that it continues executing and not wait for the query to finish. It could look like this:
var queryTask = ...;
var cancellationToken = ...;
await Task.WhenAll(queryTask, cancellationToken);
That way cancellation always looks instantaneous. Make sure that resources are still disposed. All SQL interaction should be encapsulated in queryTask so that it simply continues in the background and eventually cleans up.

C#, Dapper, SQL Server and Connection Pooling

I've written some code to write some data to SQL Server, using Dapper. I don't need to wait for this write to complete before continuing other work, so want to use Task.Run() to make this asynchronous.
I have (using) statements for calling this in the rest of my system:
using (IDataAccess ida = new DAL())
{
ida.WriteMessageToDB(id, routingKey, msgBody);
}
My DAL will automatically check the dbConnection.State when the using statement is ran, and attempt a simple fix if it's closed. This works just fine for any non-async/TPL select calls.
However, when I throw a load of writes at the same time, the Task.Run() code was falling over as the connection was closed for some of them - essentially I think the parallel nature of the code meant the state was being closed by other tasks.
I 'fixed' this by doing a check to open the Connection.State within the Task.Run() code, and this appears to have 'solved' the problem. Like so:
Task.Run(() =>
{
if (dbConnection.State == ConnectionState.Closed)
{
dbConnection.Open();
}
if (dbConnection.State == ConnectionState.Open)
{
*Dapper SQL String and Execute Commands*
}
});
When I run SELECT * FROM sys.dm_exec_connections from SSMS after this, I see a lot more connections. To be expected?
Now as I understand it:
Dapper doesn't deal with connection pooling
SQL Server should automatically deal with connection pooling?
Is there anything wrong with this solution? Or a better way of doing it? I'd like to use connection pooling for obvious reasons, and as painlessly as possible.
Thanks in advance.

Thanks Juharr - I've upvoted your reply.
For reference to others, I changed write function to await and Dapper async:
private async Task WriteMessageToDB(Guid id, string tableName, string jsonString)
{
string sql = *Redacted*
await dbConnection.ExecuteScalarAsync<int>(sql, new { ID = id, Body = jsonString });
}
And then created a new task in the caller that monitors the outcome.
This is working consistently under load, and not seeing excessive new connections being created either.

Deadlock when parallel DB call [duplicate]

When creating a report I have to execute 3 queries that involve separated entities of the same context. Because they are quite heavy ones I decided to use the .ToListAsync(); in order to have them run in parallel, but, to my surprise, I get a exception out of it...
What is the correct way to perform queries in parallel using EF 6? Should I manually start new Tasks?
Edit 1
The code is basically
using(var MyCtx = new MyCtx())
{
var r1 = MyCtx.E1.Where(bla bla bla).ToListAsync();
var r2 = MyCtx.E2.Where(ble ble ble).ToListAsync();
var r3 = MyCtx.E3.Where(ble ble ble).ToListAsync();
Task.WhenAll(r1,r2,r3);
DoSomething(r1.Result, r2.Result, r3.Result);
}

The problem is this:
EF doesn't support processing multiple requests through the same DbContext object. If your second asynchronous request on the same DbContext instance starts before the first request finishes (and that's the whole point), you'll get an error message that your request is processing against an open DataReader.
Source: https://visualstudiomagazine.com/articles/2014/04/01/async-processing.aspx
You will need to modify your code to something like this:
async Task<List<E1Entity>> GetE1Data()
{
using(var MyCtx = new MyCtx())
{
return await MyCtx.E1.Where(bla bla bla).ToListAsync();
}
}
async Task<List<E2Entity>> GetE2Data()
{
using(var MyCtx = new MyCtx())
{
return await MyCtx.E2.Where(bla bla bla).ToListAsync();
}
}
async Task DoSomething()
{
var t1 = GetE1Data();
var t2 = GetE2Data();
await Task.WhenAll(t1,t2);
DoSomething(t1.Result, t2.Result);
}

As a matter of interest, when using EF Core with Oracle, multiple parallel operations like the post here using a single DB context work without issue (despite Microsoft's documentation). The limitation is in the Microsoft.EntityFrameworkCore.SqlServer.dll driver, and is not a generalized EF issue. The corresponding Oracle.EntityFrameworkCore.dll driver doesn't have this limitation.

Check out https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/enabling-multiple-active-result-sets
From the documentation:
Statement interleaving of SELECT and BULK INSERT statements is
allowed. However, data manipulation language (DML) and data definition
language (DDL) statements execute atomically.
Then your above code works and you get the performance benefits for reading data.

ExecuteScalarAsync hangs but ExecuteScalar returns instantly

So I've run into a little issue that puzzles me and I've not been able to find a good explanation for this - I imagine I'm probably mis-using the async/await feature somehow but I really don't know what I'm doing wrong.
So I have some sql code that queries my database and returns a single value. I was therefore using ExecuteScalarAsync to get that value out into c#.
The code is as follows:
public void CheckOldTransactionsSync()
{
CheckOldTransactions().Wait();
}
public async Task CheckOldTransactions()
{
DateTimeOffset beforeThis = DateTime.UtcNow.Subtract(TimeSpan.FromHours(6));
using (SqlConnection connection = new SqlConnection(SqlConnectionString))
{
await connection.OpenAsync(cts.Token);
using (SqlCommand command = new SqlCommand(#"SELECT TOP 1 1 AS value FROM SyncLog WHERE [TimeStamp] < #BeforeThis;", connection))
{
command.Parameters.Add("#BeforeThis", System.Data.SqlDbType.DateTimeOffset, 7);
command.Prepare();
command.Parameters["#BeforeThis"].Value = beforeThis;
Int32 oldTransactions = (Int32)await command.ExecuteScalarAsync(cts.Token);
// do stuff with oldTransactions
}
}
}
So elsewhere in my code the CancellationTokenSource called cts is created and set to expire after 2 minutes using the CancelAfter method.
Now I've stepped through this code with the debugger and I reach the line where I await the call to ExecuteScalarAsync without a problem. However I seem to have two issues with the execution of that line which are that it doesn't seem to return and 2 it ignores my cancellation token and is still running some time after my two minute cancellation token has expired.
Now I've run the sql query in Sql Studio and it returns very quickly - the table has only around 4000 rows at this time.
I've resolved the problem for now by changing that line to:
Int32 oldTransactions = (Int32) command.ExecuteScalar();
Which returns almost instantaneously.
That is the only line of code I've changed and I changed it back just to make sure and the same issue occurred. So my question is, what did I do wrong with the asynchronous call?

You are calling Wait.
That's a classic ASP.NET deadlock. Don't block, or use synchronous IO.

Implementing thread pool for stored procedure calls

First, let me start with saying that I am a C# beginner. My background has mostly been with databases. I am working on a project where there will be frequent calls to a C# server which then calls various stored procedures (about 20 or so) to retrieve data from a SQL Server DB. Right now, the C# server was set up doing synchronous calls. While the SP calls are quick and small, we would still like to implement a thread pool to handle a large pool of users and simultaneous requests.
My questions:
How do I implement a thread pool? Most likely, the thread pool will start around 500, but could grow depending on use of the application.
How do I add the SP calls to the thread pool. Right now my SP call looks like this:
int SPCall(string param1, string param2)
{
string MyConnString = "...";
SqlConnection MyConn = new SqlConnection(MyConnString);
MyConn.Open();
SqlCommand SPCommand = new SqlCommand("wh_SP");
SPCommand.Connection = MyConn;
SPCommand.Parameters.Add(...) = param1;
SPCommand.Parameters.Add(...) = param2;
SPCommand.CommandType = System.Data.CommandType.StoredProcedure;
SPCommand.ExecuteNonQuery();
int outPut = (int)SPCommand.Parameters["#OUTPUT"].Value;
return outPut;
}

As mentioned in the comments, you should use the .NET ThreadPool instead of implementing your own. Even better, use the newer .NET Parallel library and chunk each of these out into a task. You will have far better control over how your concurrency is handled with relatively little code.
public void PerformWork()
{
// setup your inputs
IEnumerable<string> inputs = CreateYourInputList();
// Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
Parallel.ForEach(inputs, input =>
{
// call your code that issues the stored procedure here
this.SPCall(input);
} //close lambda expression
); //close method invocation
// Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.");
Console.ReadKey();
}