I have a set of async methods I wrote to handle a large amount of database pulls and compiles in a speedy manner. For the most part these work fantastic and have really worked wonders for my software. However, recently I have run into a small problem with the methods: Every now and then the user messes up and the time frames the software is pulling the data between become enormous and the data adapter times out before obtaining the information. Normally in a sync method you would use a try/catch to handle such issues but I have tried this to no avail. Is there a way to asynchronously handle exceptions to simply throw as a sync method would so my catch all try/catch can work properly?
This is an example of the data adapter async method i use:
private async Task<DataTable> WWQuery2Run
(string A, string B, string C, string D)
{
using ( var conn = new System.Data.SqlClient.SqlConnection(ReportResources.ConnString) )
{
var temp = new DataTable();
var DA = new SqlDataAdapter(string.Format(ReportResources.Instance.CureInfoQueries["WWQuery2"], A, B, C, D), conn);
await Task.Run(() => DA.Fill(temp));
return temp;
}
}
EDIT:
I realized after all the trouble of trying to handle the exceptions from the timeout that it is not a good practice to work that way. I went ahead and added a method to calculate the duration before entering the shown async method and warning the user of the length and gave them an option to abort the compile. With that in place I increased the timeout of the query to an amount that should cover all but the worst of the worst scenarios for if the user wishes to continue. I also added to the description on the items within the program a calculated duration so they know that it went long before trying to query and compile.
Thank you #Murad Garibzada for your assistance.
Along with increasing the command timeout, you can use a try / catch block. Since you are awaiting, control will not return to your calling code until WWQuery2Run has completed. If WWQuery2Run throws SomeException, it will be caught and handled by the code.
private async Task<DataTable> WWQuery2Run(string A, string B, string C, string D)
{
try
{
using ( var conn = new System.Data.SqlClient.SqlConnection(ReportResources.ConnString) )
{
var temp = new DataTable();
var DA = new SqlDataAdapter(string.Format(ReportResources.Instance.CureInfoQueries["WWQuery2"], A, B, C, D), conn);
await Task.Run(() => DA.Fill(temp));
return temp;
}
}
catch (SomeException ex)
{
// handle exception
}
}
Because your connection conn is wrapped in a using statement, there is a possibility that it is disposed before your call to Fill is executed.
Probably something like the following would work:
private async Task<DataTable> WWQuery2Run (string A, string B, string C, string D)
{
var temp = new DataTable();
await Task.Run(() => {
using ( var conn = new System.Data.SqlClient.SqlConnection(ReportResources.ConnString) )
{
var DA = new SqlDataAdapter(string.Format(ReportResources.Instance.CureInfoQueries["WWQuery2"], A, B, C, D), conn);
DA.Fill(temp)
}
});
return temp;
}
You can increase adapter command timeout like below:
SqlDataAdapter adapter= new SqlDataAdapter(strSQLString, conUS);
adapter.SelectCommand.CommandTimeout=120;
Handling exception:
private async Task<DataTable> WWQuery2Run
(string A, string B, string C, string D)
{
using ( var conn = new System.Data.SqlClient.SqlConnection(ReportResources.ConnString) )
{
var temp = new DataTable();
var DA = new SqlDataAdapter(string.Format(ReportResources.Instance.CureInfoQueries["WWQuery2"], A, B, C, D), conn);
Task task = new Task(() => DA.Fill(temp));
try
{
task.Wait();
}
catch (Exception ae)
{
}
return temp;
}
}
Related
So I know this question has been asked here before, but the situation here is a little different.
I have a service Application that spawns worker threads. The main service thread is organized as so:
public void PollCrunchFilesTask()
{
try
{
var stuckDeletedServTableFiles = MaintenanceDbContext.stuckDeletedServTableFiles;
var stuckErrorStatusFiles = MaintenanceDbContext.stuckErrorStatusFiles;
while (_signalPollAutoEvent.WaitOne())
{
try
{
Poll();
lock (stuckDelLock)
{
if(stuckDeletedServTableFiles.Count > 0)
MaintenanceDbContext.DeleteFilesToBeDeletedInServiceTable(stuckDeletedServTableFiles);
}
lock (errorStatusLock)
{
if (stuckErrorStatusFiles.Count > 0)
MaintenanceDbContext.UpdateStuckErrorServiceLogEntries(stuckErrorStatusFiles);
}
}
catch (Exception ex)
{
}
}
}
catch (Exception ex)
{
}
}
Inside Poll you have this logic:
public void Poll()
{
try
{
if (ProducerConsumerQueue.Count() == 0 && ThreadCount_Diff_ActiveTasks > 0)
{
var dequeuedItems = MetadataDbContext.UpdateOdfsServiceEntriesForProcessingOnPollInterval(ThreadCount_Diff_ActiveTasks);
var handlers = Producer.GetParserHandlers(dequeuedItems);
foreach (var handler in handlers)
{
ProducerConsumerQueue.EnqueueTask(handler.Execute, CancellationTokenSource.Token);
}
}
}
catch (Exception ex)
{
}
}
That ProducerConsumerQueue.EnqueueTask(handler.Execute, CancellationTokenSource.Token); launches 4 worker threads and inside any one of these threads, the following function is called at any time:
public static int DeleteServiceEntry(string logFileName)
{
int rowsAffected = 0;
var stuckDeletedServTableFiles = MaintenanceDbContext.stuckDeletedServTableFiles;
try
{
string connectionString = GetConnectionString();
throw new Exception($"Testing Del HashSet");
using (SqlConnection connection = new SqlConnection())
{
//Attempt some query
}
}
catch (Exception ex)
{
lock (stuckDelLock)
{
stuckDeletedServTableFiles.Add(logFileName);
}
}
return rowsAffected;
}
Now I am testing the stuckDeletedServTableFiles hashset which is only called when there is an exception during a query. this is why I purposefully throw an exception. That hashset is the one that is operated on on the main service thread in the function DeleteFilesToBeDeletedInServiceTable(); who's excerpt is defined below:
public static int DeleteFilesToBeDeletedInServiceTable(HashSet<string> stuckDeletedServTableFiles)
{
int rowsAffected = 0;
string logname = String.Empty; //used to collect error log
var removedHashset = new HashSet<string>();
try
{
var dbConnString = MetadataDbContext.GetConnectionString();
string serviceTable = Constants.SERVICE_LOG_TBL;
using (SqlConnection connection = new SqlConnection(dbConnString))
{
SqlCommand cmd = new SqlCommand();
cmd.CommandType = CommandType.Text;
cmd.CommandText = $"DELETE FROM {serviceTable} WHERE LOGNAME = #LOGNAME";
cmd.Parameters.Add("#LOGNAME", SqlDbType.NVarChar);
cmd.Connection = connection;
connection.Open();
foreach (var logFname in stuckDeletedServTableFiles)
{
cmd.Parameters["#LOGNAME"].Value = logFname;
logname = logFname;
int currRowsAffected = cmd.ExecuteNonQuery();
rowsAffected += currRowsAffected;
if (currRowsAffected == 1)
{
removedHashset.Add(logFname);
Logger.Info($"Removed Stuck {logFname} Marked for Deletion from {serviceTable}");
}
}
Logger.Info($"Removed {rowsAffected} stuck files Marked for Deletion from {serviceTable}");
}
stuckDeletedServTableFiles.ExceptWith(removedHashset);
}
catch (Exception ex)
{
}
return rowsAffected;
}
Given that the hashset stuckDeletedServTableFiles is able to be accessed by multiple threads at the same time including the main service thread, I put a lock on the mainservice thread before it is operated on by DeleteFilesToBeDeletedInServiceTable() and in the function DeleteServiceEntry(). I am new to C# but I assumed this would be enough? I assume since a lock is called on the main service thread for the function DeleteFilesToBeDeletedInServiceTable(), that lock would prevent anything from using the hashset since its being operated on by the function. Why am I getting this error?
Note I am not modifying the Hashset in the forloop. I only do it after the loop is done. I am getting this error while I loop through the Hashset. I suppose because another thread is attempting to modify it. The question then is, why is a thread able to modify the hashSet when I have a lock on the function that calls it on the service level?
For Now, I fixed it by surrounding the for loop in DeleteFilesToBeDeletedInServiceTable() with a lock and also calling the same lock on the statement stuckDeletedServTableFiles.ExceptWith(removedHashset); inside that function. I am not sure if there are costs to this but it seems it will work. And giving how in practice how infrequent this issue will occur, I suppose it wont' cost much. Especially since the times that function is called, we are not doing intensive things. The file crunching would have finished by then and the main caller is utilizing another thread
Is it possible to run the following method 3 times (specifically the code within the try) before throwing an error message (i.e. fail-retry, fail-retry, fail-retry, throw error message), with a break of 1 seconds between each attempt?
public static int CheckBidDuplicate(int plotId, int operatorId)
{
const string query = "SELECT Count(*) FROM bid WHERE plot_id=#plot_id AND operator_id=#operator_id GROUP BY plot_id;";
using (var cmd = new MySqlCommand(query, DbConnect.Connection))
{
cmd.Parameters.AddWithValue(("#plot_id"), plotId);
cmd.Parameters.AddWithValue(("#operator_id"), operatorId);
try
{
var da = new MySqlDataAdapter(cmd);
var dtCounts = new DataTable();
da.Fill(dtCounts);
var count = dtCounts.Rows.Count;
return count;
}
catch(Exception ex)
{
ErrorHandlingComponent.LogError(ex.ToString());
throw;
}
}
}
This pattern of execution is reasonably common. I would say in most cases, people implement a helper type to encapsulate the redundant part. For example:
static class Retry
{
public static T Invoke<T>(Func<T> func, int tryCount, TimeSpan tryInterval)
{
if (tryCount < 1)
{
throw new ArgumentOutOfRangeException("tryCount");
}
while (true)
{
try
{
return func();
}
catch (Exception e)
{
if (--tryCount > 0)
{
Thread.Sleep(tryInterval);
continue;
}
ErrorHandlingComponent.LogError(ex.ToString());
throw;
}
}
}
}
Used like this:
int result = Retry.Invoke(
() => CheckBidDuplicate(plotId, operatorId), 3, TimeSpan.FromSeconds(1));
In my experience, you may wind up wanting additional features. Customization of logging, customizing of exception handling (e.g. don't bother retrying on certain exceptions known to be terminally fatal), default try-count and try-interval values, that sort of thing.
But the above should be a good starting point.
We are pretty new to this, but we're trying to make use of the new async datareader
we have been following the sample from http://blogs.msdn.com/b/adonet/archive/2012/07/15/using-sqldatareader-s-new-async-methods-in-net-4-5-beta-part-2-examples.aspx
but experiencing problems.
What we are trying to achieve is :-
The stored proc returns 2 tables, using a datareader, we are trying to populate 2 models asynchronously, ie the 2 models will be created at the same time and not have to wait for the first model to be built before the second one can begin
(If that makes sense?)
Below is what we have at the moment :-
public async Task<UsertResultsModel> GetResults(string user_ID)
{
UsertResultsModel rm = new UsertResultsModel();
List<TableColumn> ltc = new List<TableColumn>();
List<List<TableColumn>> lltc = new List<List<TableColumn>>();
var col = 0;
try
{
using (SqlConnection con = new SqlConnection(Config.DatabaseStringCDA))
{
await con.OpenAsync();
SqlCommand com = new SqlCommand(#"USP_GET_USER_RESULTS", con);
com.CommandType = CommandType.StoredProcedure;
using (SqlDataReader rdr = await com.ExecuteReaderAsync())
{
col = rdr.FieldCount;
while (await rdr.ReadAsync())
{
for (int i = 0; i < rdr.FieldCount; i++)
{
string colName = rdr.GetName(i).ToSafeString();
ltc.Add(new TableColumn(colName, rdr[i].ToSafeString(), 50, EnumColumnType.String, colName));
}
lltc.Add(ltc);
}
rm.Summary = new TableViewModel { Grid = lltc };
await rdr.NextResultAsync();
lltc = new List<List<TableColumn>>();
while (rdr.Read())
{
for (int i = 0; i < rdr.FieldCount; i++)
{
string colName = rdr.GetName(i).ToSafeString();
ltc.Add(new TableColumn(colName, rdr[i].ToSafeString(), 50, EnumColumnType.String, colName));
}
lltc.Add(ltc);
}
rm.Trend = new TableViewModel { Grid = lltc };
}
}
}
catch (Exception ex)
{
log.Error(MethodBase.GetCurrentMethod(), ex);
return null;
}
await Task.WhenAll();
return rm;
// return lltc;
}
however, stepping through the code, as soon as we hit any await, The application simply loads, and no further code is hit..
Any advice or recommendations for this would be greatly appreciated.
The async part is the DB I/O, meaning that while you are awaiting an open, or a read, your app is free to do other things rather than blocking on the DB.
await/async allows you to write code in a sequential manner even though parts may take a while. The code flows exactly as you would expect in sequence, the line after each await doesn't run until the awaited item has finished, but the current thread is unwound back up the stack so it can get on with other things.
If you want to do two things at once using await/async, then you need to wrap each one in a function that returns a Task just like traditional TPL. That way you can keep hold of the tasks WITHOUT awaiting their results until you want to.
I have dilemma on how to actually create proper awaitable function. I don't get the entire concept fully, probably due the language barrier :)
A
public async Task<int> InsertASync(string tableName, Dictionary<string, string> data)
{
int r = 0;
using (SQLiteConnection con = NewConnection())
{
await con.OpenAsync();
using (SQLiteCommand com = new SQLiteCommand(Query_Insert(tableName, data), con))
{
r = await com.ExecuteNonQueryAsync();
}
con.Close();
}
return r;
}
or
B
public Task<int> InsertASync(string tableName, Dictionary<string, string> data)
{
return Task<int>.Run(async () =>
{
int r = 0;
using (SQLiteConnection con = NewConnection())
{
await con.OpenAsync();
using (SQLiteCommand com = new SQLiteCommand(Query_Insert(tableName, data), con))
{
r = await com.ExecuteNonQueryAsync();
}
con.Close();
}
return r;
});
}
Reason i'm wondering about it is because the way i create awaitable methods via cheap way.
Example
public void DoHeavyWork() //this will block UI
{
//Do work
}
public Task DoHeavyWorkASync() //this won't
{
return Task.Run(() =>
{
DoHeavyWork();
});
}
You want to go the A route. The compiler will take care of creating the returned task and returning the actual integer, when available.
You can read Asynchronous Programming - Pause and Play with ... for some good in-depth information on what the compiler does internally (how your method is converted into a state machine)
I have this:
private void BtnCheckClick(object sender, EventArgs e)
{
var a = txtLot.Text;
var b = cmbMcu.SelectedItem.ToString();
var c = cmbLocn.SelectedItem.ToString();
btnCheck.BackColor = Color.Red;
var task = Task.Factory.StartNew(() =>
Dal.GetLotAvailabilityF41021(a, b, c));
task.ContinueWith(t =>
{
btnCheck.BackColor = Color.Transparent;
lblDescriptionValue.Text = t.Result.Description;
lblItemCodeValue.Text = t.Result.Code;
lblQuantityValue.Text = t.Result.AvailableQuantity.ToString();
},TaskScheduler .FromCurrentSynchronizationContext() );
LotFocus(true);
}
and i followed J. Skeet's advice to move into async,await in my .NET 4.0 app. I converted into this:
private async void BtnCheckClick(object sender, EventArgs e)
{
var a = txtLot.Text;
var b = cmbMcu.SelectedItem.ToString();
var c = cmbLocn.SelectedItem.ToString();
btnCheck.BackColor = Color.Red;
JDEItemLotAvailability itm = await Task.Factory.StartNew(() => Dal.GetLotAvailabilityF41021(a, b, c));
btnCheck.BackColor = Color.Transparent;
lblDescriptionValue.Text = itm.Description;
lblItemCodeValue.Text = itm.Code;
lblQuantityValue.Text = itm.AvailableQuantity.ToString();
LotFocus(true);
}
It works fine. What confuses me is that i could do it without using Task but just the method of my Dal. But that means that i must have modified my Dal method, which is something i dont want?
I would appreciate if someone would explain to me in "plain" words if what i did is optimal or not and why.
Thanks
P.s. My dal method
public bool CheckLotExistF41021(string _lot, string _mcu, string _locn)
{
using (OleDbConnection con = new OleDbConnection(this.conString))
{
OleDbCommand cmd = new OleDbCommand();
cmd.CommandText = "select lilotn from proddta.f41021 " +
"where lilotn = ? and trim(limcu) = ? and lilocn= ?";
cmd.Parameters.AddWithValue("#lotn", _lot);
cmd.Parameters.AddWithValue("#mcu", _mcu);
cmd.Parameters.AddWithValue("#locn", _locn);
cmd.Connection = con;
con.Open();
OleDbDataReader rdr = cmd.ExecuteReader();
bool _retval = rdr.HasRows;
rdr.Close();
con.Close();
return _retval;
}
}
No, that's not optimal at all. If you cannot change your DAL layer to be asynchronous you are not gaining much by using async/await. You are simply running your blocking DAL method inside a seprate background thread. If you want real gain, you should modify your DAL method to use asynchronous ADO.NET, a.k.a BeginXXX and EndXXX methods. Once you do that you will get real benefit from I/O Completion Ports. No threads will ever be jeopardized during the execution of the database call.
If you cannot modify your DAL method, whether you are using JDEItemLotAvailability itm = await Task.Factory.StartNew(() => Dal.GetLotAvailabilityF41021(a, b, c)); or manual thread creation, really, you gain nothing.