I'm trying to improve the execution time for a operation.
I get a list of users, and by the user id I'm executing a method from a service which calculates some data. After i get the data by the user id, I set it to every entity, and than just save the context.
Currently with this simple code it take about 1 hour. Approximately 21 - 24 seconds per 100 users. But it depends on each and every user.
using (var ctx = new MyContext())
{
var users = ctx.Users.Where(u => u.Status != -1).ToList();
foreach (var user in users)
{
user.ProfilePercentage = await UserAlgorithm.CalculatePercentage(user.Id);
}
await ctx.SaveChangesAsync();
}
I also tried like this, but takes even longer:
Parallel.ForEach(
users,
new ParallelOptions { MaxDegreeOfParallelism = -1 },
user => {
user.ProfilePercentage = await UserAlgorithm.CalculatePercentage(user.Id); }
);
**CalculatePercentage **
public async Task<decimal> CalculatePercentage (int userId)
{
decimal completeness = 0;
bool? hasEducationsTags = null; //--> nullable boolea will handle 3 cases on SP side
bool? hasPosition = null; //--> nullable boolea will handle 3 cases on SP side
///-- check if user has any education
if(await SchoolService.HasEducations(userId).ConfigureAwait(false))
{
hasEducationsTags = await SchoolService.HasAnyFieldsOfStudy(user=>user.Id == userId);
}
///-- check if user can set position
if (await UserInfoService.CanSetLocation(userId).ConfigureAwait(false))
{
///-- check if position was set
string accountName = await UserInfoService.GetAccountName(userId).ConfigureAwait(false);
hasPosition = await HasUserSetPosition(accountName).ConfigureAwait(false);
}
///-- call store procedure
using (SqlConnection sqlConnection = ConnectionFactory.Create())
{
await sqlConnection.OpenAsync().ConfigureAwait(false);
using (SqlCommand sqlCommand = sqlConnection.CreateCommand())
{
sqlCommand.CommandType = CommandType.StoredProcedure;
sqlCommand.CommandText = "ProfileCompletionAlgorithm";
sqlCommand.Parameters.AddWithValue("#USER_ID", userId);
sqlCommand.Parameters.AddWithValue("#HAS_POSITION", hasPosition);
sqlCommand.Parameters.AddWithValue("#HAS_SCHOOL_TAGS", hasEducationsTags);
///-- not using OUTPUT parameter but by getting the return result
SqlParameter sqlParameter = sqlCommand.Parameters.Add("#RESULT", SqlDbType.Decimal);
sqlParameter.Direction = ParameterDirection.ReturnValue;
sqlCommand.ExecuteNonQuery();
completeness = Convert.ToDecimal(sqlParameter.Value);
}
}
return completeness;
}
Some of the links that I've checked:
https://msdn.microsoft.com/en-us/library/system.threading.tasks.paralleloptions.maxdegreeofparallelism.aspx
https://msdn.microsoft.com/en-us/library/system.threading.tasks.parallel.foreach(v=vs.110).aspx
How can I optimize this logic in order to get the best time?
If you chose to downvote please explain
Thanks.
Since your code is I/O-bound (not CPU-bound), throwing more threads (or parallelism) at the problem isn't going to help. What you want is asynchronous concurrency (Task.WhenAll), not parallel concurrency (Parallel):
using (var ctx = new MyContext())
{
var users = ctx.Users.Where(u => u.Status != -1);
var tasks = users.Select(UpdateUserProfilePercentageAsync);
await Task.WhenAll(tasks);
await ctx.SaveChangesAsync();
}
private async Task UpdateUserProfilePercentageAsync(User user)
{
user.ProfilePercentage = await UserAlgorithm.CalculatePercentage(user.Id);
}
You may find out that your database or other services don't appreciate being hammered so hard, in which case you could do some throttling. E.g., for 20 at a time:
private SemaphoreSlim _throttle = new SemaphoreSlim(20);
private async Task UpdateUserProfilePercentageAsync(User user)
{
await _throttle.WaitAsync();
try
{
user.ProfilePercentage = await UserAlgorithm.CalculatePercentage(user.Id);
}
finally { _throttle.Release(); }
}
Related
I am working on a worker service. In this worker service, I am calling a web API of mine that calls an external API. My web API is an asp.net core web API saves requests and responses into database. This external web API returns online game codes.(I am sending how many game codes and for which game.) In a single request there are maximum of 10 codes. The user enters the total amount of online codes and the game number from a blazor server web application. Behind the scenes, this total amount of codes are divided into 20 codes each and are saved into database. The reason why it is saved as 20 is because I am sending 2 threads to the web API in Parallel.ForEachAsync in my worker service. This worker service checks the database if there is a record which status = 0, it gets this record and starts the process.
For example;
Below are 20 coupon requests waiting to be processed. When the worker service receives this records from the database and sends it to my web API, then the external API is called 10 by 10. (Because I want the maximum game code to come in 1 request.) Therefore, in the best scenario, 2 requests will be sent and 2 responses will be returned. The reason why I make 2 threads with Parallel.ForEachAsync in the worker is to speed up the process a little more because sometimes a total of 20000 game codes are requested.
Here is a portion of worker service where I am calling my API. If the response is a success, I am updating the status = 1 to the table above saying that this job is completed. In this particular scenario, if both threads returns success there is no problem. There will be 2 requests and responses in the database. (status = 1) If both threads are failed again no problem there won't be any requests and responses in the database which is fine, errors are logged. (status = 0, worker service will try it again after a random time interval) But what if one thread is a success and the other one is failed. I mocked my API so that it returns a success and a error. In this case, 1 response was received from the external API and saved in the database, but the update method in the worker service updated the record as status=1. Only 10 game codes have been received. Since the update in the Worker service is running, the relevant record in the above table has been updated as status = 1. 20 requested game codes were not met. How can I resolve this situation?
I'm thinking of sending to the external API 10 at a time instead of 20 by 20, but is there a better way?
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text;
using Newtonsoft.Json;
using PurchaseRazerCodesMultipleRequestService;
using PurchaseRazerCodesMultipleRequestService.Modal;
using TrendyolGameCodeService.Modal;
namespace PurchaseRazerCodesMultipleRequestService
{
public class Worker : BackgroundService
{
private readonly ILogger<Worker> _logger;
private readonly IHttpClientFactory _httpClientFactory;
private readonly IConfiguration _configuration;
public Worker(ILogger<Worker> logger, IHttpClientFactory httpClientFactory, IConfiguration configuration)
{
_logger = logger;
_httpClientFactory = httpClientFactory;
_configuration = configuration;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
//Random Timer
var timer = new PeriodicTimer(TimeSpan.FromMinutes(new Random().Next(3, 7)));
_logger.LogInformation("Timer: {timer}", DateTimeOffset.Now);
while (await timer.WaitForNextTickAsync(stoppingToken))
{
await GetData();
_logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
}
}
private async Task MakeRequestsToRemoteService(string productCode, long id, int amount, int bulkId)
{
if (id <= 0) throw new ArgumentOutOfRangeException(nameof(id));
try
{
var httpClient = _httpClientFactory.CreateClient("RazerClient");
//Token
httpClient.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/x-www-form-urlencoded"));
var tokenContent = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("username", "Test"),
new KeyValuePair<string, string>("password", "+REyN-#V5!_DgUn+y%hVj7VmyhN^+?%y+Qxkc-bLZR6$uqsYV")
});
using var tokenResponse = await httpClient.PostAsync(_configuration["Token:Production"], tokenContent);
if ((int)tokenResponse.StatusCode == 200)
{
var token = await tokenResponse.Content.ReadAsStringAsync();
//Call Razer Multi Requests
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer",
token);
httpClient.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/x-www-form-urlencoded"));
#region Calculate quantity
var num = amount;
var firstNum = 10;
var secondNum = 10;
if (num < 20)
{
firstNum = (num + 1) / 2;
secondNum = num - firstNum;
}
#endregion
var quantities = new List<int> { firstNum, secondNum};
var cts = new CancellationTokenSource();
ParallelOptions parallelOptions = new()
{
MaxDegreeOfParallelism = 2,
CancellationToken = cts.Token
};
try
{
await Parallel.ForEachAsync(quantities, parallelOptions, async (quantity, ct) =>
{
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("productCode", productCode),
new KeyValuePair<string, string>("quantity", quantity.ToString()),
new KeyValuePair<string, string>("clientTrxRef", bulkId.ToString())
});
using var response =
await httpClient.PostAsync(_configuration["Razer:ProductionMock"], content, ct);
if ((int)response.StatusCode == 200)
{
var coupon = await response.Content.ReadFromJsonAsync<Root>(cancellationToken: ct);
_logger.LogInformation("REFERENCE ID: {referenceId}", coupon.ReferenceId);
await UpdateData(id);
}
else
{
_logger.LogError("Purchase ServiceError: {statusCode}",
(int)response.StatusCode);
}
});
}
catch (OperationCanceledException ex)
{
_logger.LogError("Operation canceled: {Message}",
ex.Message);
}
}
else
{
_logger.LogError("Token ServiceError: {statusCode}",
(int)tokenResponse.StatusCode);
}
}
catch (Exception e)
{
_logger.LogError("Error: {Error} ", e.Message);
}
}
private async Task GetData()
{
var sw = Stopwatch.StartNew();
var connString = _configuration["ConnectionStrings:Default"];
await using var sqlConnection = new SqlConnection(connString);
sqlConnection.Open();
await using var command = new SqlCommand { Connection = sqlConnection };
const string sql = #"Select TOP 1 Id, BulkPurchaseRequestId, Amount, ProductCode from BulkPurchases where status = 0 ORDER BY NEWID()";
command.CommandText = sql;
try
{
await using var reader = await command.ExecuteReaderAsync();
while (reader.Read())
{
_logger.LogInformation(
"Order {Id}, {BulkId}, {Amount}, {ProductCode}",
reader.GetInt32(0), reader.GetInt32(1), reader.GetInt32(2), reader.GetString(3));
await MakeRequestsToRemoteService(reader.GetString(3).Trim(), reader.GetInt32(0), reader.GetInt32(2),reader.GetInt32(1));
}
}
catch (SqlException exception)
{
_logger.LogError("Error: {Error} ", exception.Message);
}
sw.Stop();
_logger.LogInformation($"******** ELAPSED TIME: {sw.Elapsed.TotalSeconds} seconds ********");
}
private async Task UpdateData(long id)
{
var connString = _configuration["ConnectionStrings:Default"];
await using var sqlConnection = new SqlConnection(connString);
sqlConnection.Open();
await using var command = new SqlCommand { Connection = sqlConnection };
const string sql = #"Update BulkPurchases set status = 1 where Id = #id";
command.CommandText = sql;
command.Parameters.Add(new SqlParameter("id", id));
try
{
await command.ExecuteNonQueryAsync();
}
catch (SqlException exception)
{
_logger.LogError("Error: {Error} ", exception.Message);
}
}
}
}
I think you might resolve the problem by doing all the work in a transaction of sorts. Basically, retreive all the data first and only if all the requests succeed, save all the data to the database. Otherwise, exit and save nothing.
Does this help?
private async Task RetreiveAndSaveCouponsAsync()
{
// code was ommited here because you haven't shown all of your solution
var quantities = new List<int> { 10, 10 };
// create a task that retreives coupones for each quantity
var tasks = quantities
.Select(quantity => GetCodesAsync("someProductCode", quantity, 123, ct))
.ToList();
try
{
// await all the tasks
await Task.WhenAll(tasks);
}
catch
{
// extract the exceptions from the tasks
var exceptions = tasks
.Where(x => x.Exception is not null)
.Select(x => x.Exception);
// log exceptions
foreach (var ex in exceptions)
{
_logger.LogError(ex);
}
return;
}
// check if any of them failed
if (tasks.Any(x => x.Result is null))
{
// one or more tasks failed, log error and don't save anything into db
_logger.LogError("one or more tasks failed");
return;
}
// extract the coupons from the tasks
IEnumerable<Root> coupons = tasks.Select(x => x.Result).Cast<Root>();
// save all coupons to the database
foreach (var coupon in coupons)
{
await SaveCouponToDatabaseAsync(coupon);
}
}
private async Task<Root?> GetCodesAsync(string productCode, int quantity, int bulkId, CancellationToken ct)
{
var httpClient = new HttpClient(); // realistically, you'd get the HttpClient from IHttpClientFactory here
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("productCode", productCode),
new KeyValuePair<string, string>("quantity", quantity.ToString()),
new KeyValuePair<string, string>("clientTrxRef", bulkId.ToString())
});
using var response = await httpClient.PostAsync(_configuration["Razer:ProductionMock"], content, ct);
if ((int)response.StatusCode == 200)
{
var coupon = await response.Content.ReadFromJsonAsync<Root>(cancellationToken: ct);
_logger.LogInformation("REFERENCE ID: {referenceId}", coupon.ReferenceId);
return coupon;
}
_logger.LogError("Purchase ServiceError: {statusCode}", (int)response.StatusCode);
return null; // I'm returning null in case the request fail, you should decide how to handle this problem in your app
}
I have created a supposed to be asynchronous method. And inside of that it has several await statements.
I am calling this method (in another class) from the main method.
But it doesn't return the control back to the caller (main), to execute the next statement (to get more data),
even though the operation takes long time to complete.
What is wrong here?
Could it be something with the return statement? Or do I need to wrap the logic inside a Task?
internal async static Task<List<Cars>> GetCarsAsync()
{
var cars = new List<Cars>();
try
{
using (var connection = new OracleConnection(ConfigurationManager.ConnectionStrings["KWC"].ConnectionString))
{
await connection.OpenAsync(); //first await
logger.Info("Opened database connection.");
StringBuilder selectStatement = new StringBuilder(Properties.Settings.Default.SelectCarsView);
using (var command = connection.CreateCommand())
{
command.CommandText = selectStatement.ToString();
logger.Info("About to execute following query on database: {0}", command.CommandText);
using (var reader = await command.ExecuteReaderAsync()) //second await
{
logger.Info("Executed query: {0}", command.CommandText);
while (await reader.ReadAsync()) //third await
{
var car = new Car { model = reader.GetString(0) };
//more code
cars.Add(car);
}
}
}
}
return cars;
}
catch (OracleException ex)
{
logger.Fatal(ex.Message);
throw;
}
}
static async Task Main(string[] args)
{
var getCarsTask = CarsManager.GetCarsAsync();
var getSomethingElseTask = DummyManager.GetMoreDataAsync();
// more similar tasks
var cars = await getCarsTask;
var moreData = await getSomethingElseTask;
// more code
}
I'm using ASP.NET Core MVC 2.0 and I've got an API that calls a method which opens a DB connection and inserts values into a model.
I'm hitting the same method 7 times and I'm just passing different information into it the parameters to build out my model. I HAVE to hit it seven different times, no rewrite of code will prevent this.
Instead of creating a loop and calling that method I wish to make parallel calls to the db so the response is faster. I've found multiple articles explaining how to do this e.g. How to properly make asynchronous / parallel database calls but there's enough difference that I can't seem to get it to work.
Following is the my method hitting the DB:
public async Task<RS_Model> Get_Results(RequestS_Model values)
{
//Deleted a bunch of code to keep it simple
string strQS = #"EXEC Get param1,param2,etc";
using (SqlConnection conSQL = new SqlConnection(connectionString))
{
using (SqlCommand cmdSQL = new SqlCommand(strQS, conSQL))
{
conSQL.Open();
using (SqlDataReader dtrSQL = cmdSQL.ExecuteReader())
{
while (dtrSQL.Read())
{
Double.TryParse(dtrSQL["Value1"].ToString(), out dblVal1);
} //Ends While
} //end SQLDataReader
} //Ends cmdSQL
} //ends using
results.Price = dblVal1;
return results;
} //Ends Get Results
My IActionResult for the api is:
[HttpGet]
public async Task<IActionResult> Get([FromQuery] RequestS_Model values)
{
SV_Results Results = new SV_Results();
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
RS_Model model = new RS_Model();
model.dblr[0] = await Results.Get_Results(values);
values.Parm1 = 2;
model.dblr[1] = await Results.Get_Results(values);
values.Parm1 = 3;
model.dblr[2] = await Results.Get_Results(values);
values.Parm1 = 4;
model.dblr[3] = await Results.Get_Results(values);
values.Parm1 = 5;
model.dblr[4] = await Results.Get_Results(values);
values.Parm1 = 6;
model.dblr[5] = await Results.Get_Results(values);
values.Parm1 = 7;
model.dblr[6] = await Results.Get_Results(values);
//int[] results = await Task.WhenAll(new Task<int>[] { task1, task2 });
return new OkObjectResult(model);
} //IActionResults
I know that I've forced them into a synchronous call be doing what I am, but I can't seem to make the 7 calls asynchronous and then wait for them to be all done before I build my final model. The final response needs to be in Json, but I haven't even gotten that far yet.
Instead of this:
model.dblr[0] = await Results.Get_Results(values);
model.dblr[1] = await Results.Get_Results(values);
model.dblr[2] = await Results.Get_Results(values);
model.dblr[3] = await Results.Get_Results(values);
model.dblr[4] = await Results.Get_Results(values);
model.dblr[5] = await Results.Get_Results(values);
model.dblr[6] = await Results.Get_Results(values);
Create a list of tasks and await them as a group:
var tasks = Enumerable.Range(0,7).Select( i => Results.Get_Results(values) ).ToList();
await Task.WhenAll(tasks);
for (int i=0; i<7; i++) model.dblr[i] = tasks[i].Result;
I migrated an Access database to SQL using Microsoft SQL Server Migration Assistant.
Now, I am having trouble reading data.
return reader.GetInt32(0); Throws Invalid attempt to read when no data is present exception after 32 rows retrieved. If I add CommandBehavior.SequentialAccess to my command, I am able to read 265 rows, every time.
The data
The query (in SQL Management Studio):
SELECT *
FROM Products2
WHERE Products2.PgId = 337;
There is nothing special about row 32, and if I reverse the order, it is still row 32 that kills it.
Row 265, just for good measure.
The code
The query:
SELECT *
FROM Products2
WHERE Products2.PgId = #productGroupId;
The parameter:
Name = "productGroupId"
Value = 337
The execution:
public async Task ExecuteAsync(string query, Action<IDataReader> onExecute, params DataParameter[] parameters)
{
using(var command = builder.BuildCommand(query))
{
foreach(var parameter in parameters)
command.AddParameter(parameter);
if(!connection.IsOpen)
connection.Open();
await Task.Run(async () =>
{
using(var reader = await command.ExecuteAsync())
if(await reader.ReadAsync())
onExecute(reader);
});
}
}
And reading:
await executor.ExecuteAsync(query, async reader =>
{
do
{
products.Add(GetProduct(reader));
columnValues.Enqueue(GetColumnValues(reader).ToList());
} while(await reader.ReadAsync());
}, parameters.ToArray());
await reader.ReadAsync() returns true, but when GetProduct(reader) calls reader.GetInt32(0); for the 32nd time, it throws the exception.
It works fine if the data is less than 32 rows, or 265 in case of SequentialAccess.
I tried increasing the CommandTimeout, but it didn't help. When I swap the connection to OleDb again, it works just fine.
Thanks in advance.
EDIT
If I replace * in the query with just a few specific columns, it works. When I read ~12 columns, it fails, but later than row 32.
As per request, GetProduct:
private Product GetProduct(IDataReader productReader)
{
return new Product
{
Id = productReader.ReadLong(0),
Number = productReader.ReadString(2),
EanNo = productReader.ReadString(3),
Frequency = productReader.ReadInt(4),
Status = productReader.ReadInt(5),
NameId = productReader.ReadLong(6),
KMPI = productReader.ReadByte(7),
OEM = productReader.ReadString(8),
CurvesetId = productReader.ReadInt(9),
HasServiceInfo = Convert.ToBoolean(productReader.ReadByte(10)),
ColumnData = new List<ColumnData>()
};
}
GetColumnData:
private IEnumerable<long> GetColumnValues(IDataReader productReader)
{
var columnValues = new List<long>();
int columnIndex = 11;
while(!productReader.IsNull(columnIndex))
columnValues.Add(productReader.ReadLong(columnIndex++));
return columnValues;
}
And the adapter:
public long ReadLong(int columnIndex)
{
return reader.GetInt32(columnIndex);
}
Alright, it is getting long. :)
Thanks to #Igor, I tried creating a small working example. This seem to work fine:
private static async Task Run()
{
var result = new List<Product>();
string conString = #" ... ";
var con = new SqlConnection(conString);
con.Open();
using(var command = new SqlCommand("SELECT * FROM Products2 WHERE Products2.PgId = #p;", con))
{
command.Parameters.Add(new SqlParameter("p", 337));
await Task.Run(async () =>
{
using(var productReader = await command.ExecuteReaderAsync())
while(await productReader.ReadAsync())
{
result.Add(new Product
{
Id = productReader.GetInt32(0),
Number = productReader.GetString(2),
EanNo = productReader.GetString(3),
Frequency = productReader.GetInt16(4),
Status = productReader.GetInt16(5),
NameId = productReader.GetInt32(6),
KMPI = productReader.GetByte(7),
OEM = productReader.GetString(8),
CurvesetId = productReader.GetInt16(9),
HasServiceInfo = Convert.ToBoolean(productReader.GetByte(10))
});
GetColumnValues(productReader);
}
});
}
Console.WriteLine("End");
}
private static IEnumerable<long> GetColumnValues(SqlDataReader productReader)
{
var columnValues = new List<long>();
int columnIndex = 11;
while(!productReader.IsDBNull(columnIndex))
columnValues.Add(productReader.GetInt32(columnIndex++));
return columnValues;
}
}
Here's the data in Access, just in case:
I managed to fix the problem.
I still have questions though, since I don't understand why this didn't affect Access.
I changed:
public async Task ExecuteAsync(string query, Action<IDataReader> onExecute, params DataParameter[] parameters)
{
using(var command = builder.BuildCommand(query))
{
foreach(var parameter in parameters)
command.AddParameter(parameter);
if(!connection.IsOpen)
connection.Open();
await Task.Run(async () =>
{
using(var reader = await command.ExecuteAsync())
if(await reader.ReadAsync())
onExecute(reader);
});
}
}
To:
public async Task ExecuteAsync(string query, Func<IDataReader, Task> onExecute, params DataParameter[] parameters)
{
using(var command = builder.BuildCommand(query))
{
foreach(var parameter in parameters)
command.AddParameter(parameter);
if(!connection.IsOpen)
connection.Open();
using(var reader = await command.ExecuteAsync())
await onExecute(reader);
}
}
If someone can explain why this helped, I would really appreciate it.
I am refactoring an application and trying to add an asynchronous version of an existing function to improve performance times in an ASP.NET MVC application. I understand that there is an overhead involved with asynchronous functions, but I expected that with enough iterations, the I/O intensive nature of loading the data from the database would more than compensate for the overhead penalty and that I would receive significant performance gains.
The TermusRepository.LoadByTermusId function loads data by retrieving a bunch of datatables from the database (using ADO.NET and the Oracle Managed Client), populates a model, and returns it. TermusRepository.LoadByTermusIdAsync is similar, except it does so asynchronously, with a slightly different method of loading up datatable download tasks when there's multiple datatables to retrieve.
public async Task<ActionResult> AsyncPerformanceTest()
{
var vm = new AsyncPerformanceTestViewModel();
Stopwatch watch = new Stopwatch();
watch.Start();
for (int i = 0; i < 60; i++)
{
TermusRepository.LoadByTermusId<Termus2011_2012EndYear>("1");
TermusRepository.LoadByTermusId<Termus2011_2012EndYear>("5");
TermusRepository.LoadByTermusId<Termus2011_2012EndYear>("6");
TermusRepository.LoadByTermusId<Termus2011_2012EndYear>("7");
}
watch.Stop();
vm.NonAsyncElapsedTime = watch.Elapsed;
watch.Reset();
watch.Start();
var tasks = new List<Task<Termus2011_2012EndYear>>();
for (int i = 0; i < 60; i++)
{
tasks.Add(TermusRepository.LoadByTermusIdAsync<Termus2011_2012EndYear>("1"));
tasks.Add(TermusRepository.LoadByTermusIdAsync<Termus2011_2012EndYear>("5"));
tasks.Add(TermusRepository.LoadByTermusIdAsync<Termus2011_2012EndYear>("6"));
tasks.Add(TermusRepository.LoadByTermusIdAsync<Termus2011_2012EndYear>("7"));
}
await Task.WhenAll(tasks.ToArray());
watch.Stop();
vm.AsyncElapsedTime = watch.Elapsed;
return View(vm);
}
public static async Task<T> LoadByTermusIdAsync<T>(string termusId) where T : Appraisal
{
var AppraisalHeader = new OracleCommand("select tu.termus_id, tu.manager_username, tu.evaluee_name, tu.evaluee_username, tu.termus_complete_date, termus_start_date, tu.termus_status, tu.termus_version, tn.managername from tercons.termus_users tu left outer join tercons.termus_names tn on tu.termus_id=tn.termus_id where tu.termus_id=:termusid");
AppraisalHeader.BindByName = true;
AppraisalHeader.Parameters.Add("termusid", termusId);
var dt = await Database.GetDataTableAsync(AppraisalHeader);
T Termus = Activator.CreateInstance<T>();
var row = dt.AsEnumerable().Single();
Termus.TermusId = row.Field<decimal>("termus_id").ToString();
Termus.ManagerUsername = row.Field<string>("manager_username");
Termus.EvalueeUsername = row.Field<string>("evaluee_username");
Termus.EvalueeName = row.Field<string>("evaluee_name");
Termus.ManagerName = row.Field<string>("managername");
Termus.TERMUSCompleteDate = row.Field<DateTime?>("termus_complete_date");
Termus.TERMUSStartDate = row.Field<DateTime>("termus_start_date");
Termus.Status = row.Field<string>("termus_status");
Termus.TERMUSVersion = row.Field<string>("termus_version");
Termus.QuestionsAndAnswers = new Dictionary<string, string>();
var RetrieveQuestionIdsCommand = new OracleCommand("select termus_question_id from tercons.termus_questions where termus_version=:termus_version");
RetrieveQuestionIdsCommand.BindByName = true;
RetrieveQuestionIdsCommand.Parameters.Add("termus_version", Termus.TERMUSVersion);
var QuestionIdsDt = await Database.GetDataTableAsync(RetrieveQuestionIdsCommand);
var QuestionIds = QuestionIdsDt.AsEnumerable().Select(r => r.Field<string>("termus_question_id"));
//There's about 60 questions/answers, so this should result in 60 calls to the database. It'd be a good spot to combine to a single DB call, but left it this way so I could see if async would speed it up for learning purposes.
var DownloadAnswersTasks = new List<Task<DataTable>>();
foreach (var QuestionId in QuestionIds)
{
var RetrieveAnswerCommand = new OracleCommand("select termus_response, termus_question_id from tercons.termus_responses where termus_id=:termus_id and termus_question_id=:questionid");
RetrieveAnswerCommand.BindByName = true;
RetrieveAnswerCommand.Parameters.Add("termus_id", termusId);
RetrieveAnswerCommand.Parameters.Add("questionid", QuestionId);
DownloadAnswersTasks.Add(Database.GetDataTableAsync(RetrieveAnswerCommand));
}
while (DownloadAnswersTasks.Count > 0)
{
var FinishedDownloadAnswerTask = await Task.WhenAny(DownloadAnswersTasks);
DownloadAnswersTasks.Remove(FinishedDownloadAnswerTask);
var AnswerDt = await FinishedDownloadAnswerTask;
var Answer = AnswerDt.AsEnumerable().Select(r => r.Field<string>("termus_response")).SingleOrDefault();
var QuestionId = AnswerDt.AsEnumerable().Select(r => r.Field<string>("termus_question_id")).SingleOrDefault();
if (!String.IsNullOrEmpty(Answer))
{
Termus.QuestionsAndAnswers.Add(QuestionId, System.Net.WebUtility.HtmlDecode(Answer));
}
}
return Termus;
}
public static async Task<DataTable> GetDataTableAsync(OracleCommand command)
{
DataTable dt = new DataTable();
using (var connection = GetDefaultOracleConnection())
{
command.Connection = connection;
await connection.OpenAsync();
dt.Load(await command.ExecuteReaderAsync());
}
return dt;
}
public static T LoadByTermusId<T>(string TermusId) where T : Appraisal
{
var RetrieveAppraisalHeaderCommand = new OracleCommand("select tu.termus_id, tu.manager_username, tu.evaluee_name, tu.evaluee_username, tu.termus_complete_date, termus_start_date, tu.termus_status, tu.termus_version, tn.managername from tercons.termus_users tu left outer join tercons.termus_names tn on tu.termus_id=tn.termus_id where tu.termus_id=:termusid");
RetrieveAppraisalHeaderCommand.BindByName = true;
RetrieveAppraisalHeaderCommand.Parameters.Add("termusid", TermusId);
var AppraisalHeaderDt = Database.GetDataTable(RetrieveAppraisalHeaderCommand);
T Termus = Activator.CreateInstance<T>();
var AppraisalHeaderRow = AppraisalHeaderDt.AsEnumerable().Single();
Termus.TermusId = AppraisalHeaderRow.Field<decimal>("termus_id").ToString();
Termus.ManagerUsername = AppraisalHeaderRow.Field<string>("manager_username");
Termus.EvalueeUsername = AppraisalHeaderRow.Field<string>("evaluee_username");
Termus.EvalueeName = AppraisalHeaderRow.Field<string>("evaluee_name");
Termus.ManagerName = AppraisalHeaderRow.Field<string>("managername");
Termus.TERMUSCompleteDate = AppraisalHeaderRow.Field<DateTime?>("termus_complete_date");
Termus.TERMUSStartDate = AppraisalHeaderRow.Field<DateTime>("termus_start_date");
Termus.Status = AppraisalHeaderRow.Field<string>("termus_status");
Termus.TERMUSVersion = AppraisalHeaderRow.Field<string>("termus_version");
Termus.QuestionsAndAnswers = new Dictionary<string, string>();
var RetrieveQuestionIdsCommand = new OracleCommand("select termus_question_id from tercons.termus_questions where termus_version=:termus_version");
RetrieveQuestionIdsCommand.BindByName = true;
RetrieveQuestionIdsCommand.Parameters.Add("termus_version", Termus.TERMUSVersion);
var QuestionIdsDt = Database.GetDataTable(RetrieveQuestionIdsCommand);
var QuestionIds = QuestionIdsDt.AsEnumerable().Select(r => r.Field<string>("termus_question_id"));
//There's about 60 questions/answers, so this should result in 60 calls to the database. It'd be a good spot to combine to a single DB call, but left it this way so I could see if async would speed it up for learning purposes.
foreach (var QuestionId in QuestionIds)
{
var RetrieveAnswersCommand = new OracleCommand("select termus_response from tercons.termus_responses where termus_id=:termus_id and termus_question_id=:questionid");
RetrieveAnswersCommand.BindByName = true;
RetrieveAnswersCommand.Parameters.Add("termus_id", TermusId);
RetrieveAnswersCommand.Parameters.Add("questionid", QuestionId);
var AnswersDt = Database.GetDataTable(RetrieveAnswersCommand);
var Answer = AnswersDt.AsEnumerable().Select(r => r.Field<string>("termus_response")).SingleOrDefault();
if (!String.IsNullOrEmpty(Answer))
{
Termus.QuestionsAndAnswers.Add(QuestionId, System.Net.WebUtility.HtmlDecode(Answer));
}
}
return Termus;
}
public static DataTable GetDataTable(OracleCommand command)
{
DataTable dt = new DataTable();
using (var connection = GetDefaultOracleConnection())
{
command.Connection = connection;
connection.Open();
dt.Load(command.ExecuteReader());
}
return dt;
}
public static OracleConnection GetDefaultOracleConnection()
{
return new OracleConnection(ConfigurationManager.ConnectionStrings[connectionstringname].ConnectionString);
}
Results for 60 iterations are:
Non Async 18.4375460 seconds
Async 19.8092854 seconds
The results of this test are consistent. No matter how many iterations I go through of the for loop in AsyncPerformanceTest() action method, the async stuff runs about 1 second slower than the non-async. (I run the test multiple times in a row to account for the JITter warming up.) What am I doing wrong that's causing the async to be slower than the non-async? Am I misunderstanding something fundamental about writing asynchronous code?
The asynchronous version will always be slower than the synchronous version when there is no concurrency. It's doing all of the same work as the non-async version, but with a small amount of overhead added to manage the asynchrony.
Asynchrony is advantageous, with respect to performance, by allowing improved availability. Each individual request will be slower, but if you make 1000 requests at the same time, the asynchronous implementation will be able to handle them all more quickly (at least in certain circumstances).
This happens because the asynchronous solution allows the thread that was allocated to handle the request to go back to the pool and handle other requests, whereas the synchronous solution forces the thread to sit there and do nothing while it waits for the asynchronous operation to complete. There is overhead in structuring the program in a way that allows the thread to be freed up to do other work, but the advantage is the ability of that thread to go do other work. In your program there is no other work for the thread to go do, so it ends up being a net loss.
Turns out the Oracle Managed Driver is "fake async", which would partially explain why my async code is running slower.