I am using the AsyncEnumerator library by #Serge Semenov, and after getting the orders, creating the ConcurrentBag and setting count to 0, nothing happens - can anyone see what's missing here? It never hits the processedOrders.ToList() line when debugging.
private async Task<List<OrderDto>> ProcessOrderEvents(DateTime cob)
{
var orders = await _client.GetOrderEvents(cob);
var processedOrders = new ConcurrentBag<OrderDto>();
var count = 0;
await orders.ParallelForEachAsync(async order =>
{
var orderDto = new FidessaOrderDto {Cob = cob};
var orderId = order.Ids.PrimaryId;
Debug.WriteLine($"Processing OrderId: {orderId} #= {++count}");
var instrumentDetails = await GetInstrumentDetails(order);
if (instrumentDetails.Results.Any())
{
//...
}
var tradeDetails = await _client.GetTrade(tradeId: fill, cob);
processedOrders.Add(orderDto);
}, maxDegreeOfParallelism:2);
return processedOrders.ToList();
}
Update: Adding Test root.
[TestMethod]
public async Task Test_20_07_2022()
{
var service = new DataService(new ApiClient());
var data = await service.ProcessData(new DateTime(2022, 07, 20), ReportDataType.Orders);
Assert.AreEqual(920, data.Count);
}
public async Task<List<OrderDto>> ProcessData(DateTime cob,
ReportDataType dataType)
{
await PopulateRouteEvents(cob);
var reportData = new List<OrderDto>();
if (dataType == ReportDataType.Orders)
reportData = await ProcessOrderEventsTpl(cob);
return reportData;
}
I need to wait for the whole task to finish then return a value. My code gives me an error of "Not all code paths return a value". Could you show me how to fix this? I am confused really with Tasks.
public async override Task<TagInfoReply> GetInfo(TagInfoRequest request, ServerCallContext context)
{
var tagInfo = JsonSerializer.Deserialize<TagInfo>(request.TagInfo);
var parameter = JsonSerializer.Deserialize<ParameterGetWhiteListWithEntryInfo>(request.DetectedTag);
var task = Task.Run(() =>
{
return filters.FilterTags(tagInfo);
});
await task.ContinueWith(
async antecedent =>
{
if (antecedent.Result)
{
var reply = await pcs.GetTagInfo(parameter, tagInfo, TagFilterTime);
return new TagInfoReply
{
FetchedTagInfo = reply
};
}
else
{
return new TagInfoReply
{
FetchedTagInfo = string.Empty
};
}
}, TaskContinuationOptions.OnlyOnRanToCompletion);
}
If I put in a
return new TagInfoReply
{
FetchedTagInfo = string.Empty
};
at the very end and outside the task.ContinueWith, well the error goes away but the problem is when it runs, it doesn't wait for the task to complete. It goes directly to the last return.
At the moment, your return code only returns from the anonymous method you've passed to ContinueWith, and they return to the calling code within ContinueWith. You're not actually returning anything from GetInfo.
The result of ContinueWith is made available to you as its result, so you should be able to rewrite your code like this:
public async override Task<TagInfoReply> GetInfo(TagInfoRequest request, ServerCallContext context)
{
var tagInfo = JsonSerializer.Deserialize<TagInfo>(request.TagInfo);
var parameter = JsonSerializer.Deserialize<ParameterGetWhiteListWithEntryInfo>(request.DetectedTag);
var task = Task.Run(() =>
{
return filters.FilterTags(tagInfo);
});
return await task.ContinueWith(
async antecedent =>
{
if (antecedent.Result)
{
var reply = await pcs.GetTagInfo(parameter, tagInfo, TagFilterTime);
return new TagInfoReply
{
FetchedTagInfo = reply
};
}
else
{
return new TagInfoReply
{
FetchedTagInfo = string.Empty
};
}
}, TaskContinuationOptions.OnlyOnRanToCompletion);
}
The core of your problem is that you need to return a value. A side problem is that you shouldn't be using ContinueWith at all. If you replace ContinueWith with await, then your code becomes much clearer:
public async override Task<TagInfoReply> GetInfo(TagInfoRequest request, ServerCallContext context)
{
var tagInfo = JsonSerializer.Deserialize<TagInfo>(request.TagInfo);
var parameter = JsonSerializer.Deserialize<ParameterGetWhiteListWithEntryInfo>(request.DetectedTag);
var result = await Task.Run(() =>
{
var filteredTags = filters.FilterTags(tagInfo);
if (antecedent.Result)
{
var reply = await pcs.GetTagInfo(parameter, tagInfo, TagFilterTime);
return new TagInfoReply
{
FetchedTagInfo = reply
};
}
else
{
return new TagInfoReply
{
FetchedTagInfo = string.Empty
};
}
});
return result;
}
It's also likely that you don't need the Task.Run in there:
public async override Task<TagInfoReply> GetInfo(TagInfoRequest request, ServerCallContext context)
{
var tagInfo = JsonSerializer.Deserialize<TagInfo>(request.TagInfo);
var parameter = JsonSerializer.Deserialize<ParameterGetWhiteListWithEntryInfo>(request.DetectedTag);
var filteredTags = filters.FilterTags(tagInfo);
if (antecedent.Result)
{
var reply = await pcs.GetTagInfo(parameter, tagInfo, TagFilterTime);
return new TagInfoReply
{
FetchedTagInfo = reply
};
}
else
{
return new TagInfoReply
{
FetchedTagInfo = string.Empty
};
}
}
I'm using Google Analytics and that service has limit of 10 concurrent requests. I had to limit my API somehow, so I decided to use a semaphore, but it seems it doesn't work. All requests are triggered simultaneously. I can't find the problem in my code.
public async Task<SiteAnalyticsDTO> Handle(GetSiteAnalyticsParameter query)
{
var todayVisits = _googleAnalyticsService.GetTodayVisitsNumber();
var todayTraffic = _googleAnalyticsService.GetTodayTraffic();
var newAndReturningUsers = _googleAnalyticsService.GetNewAndReturningUsersNumber();
var averageSessionDuration = _googleAnalyticsService.GetAverageSessionDuration();
var deviceCategory = _googleAnalyticsService.GetSessionNumberByDeviceCategory();
var topPages = _googleAnalyticsService.GetTodaysTopPages();
var guestsAndRegisteredUsers = _googleAnalyticsService.GetGuestsVsRegisteredUsers();
var averageNumberOfSessionsPerDay = _googleAnalyticsService.GetAverageSessionsNumber();
var visitsPerWeekday = _googleAnalyticsService.GetTrafficByWeekday();
var visitsByHours = _googleAnalyticsService.GetTrafficByTimeOfDay();
var usersByPrefectures = _googleAnalyticsService.GetUsersByPrefectures();
var usersByCountry = _googleAnalyticsService.GetUsersByCountry();
var tasks = new List<Task>()
{
todayVisits, todayTraffic, newAndReturningUsers,
averageSessionDuration, deviceCategory, topPages,
guestsAndRegisteredUsers, averageNumberOfSessionsPerDay, visitsPerWeekday,
visitsByHours, usersByPrefectures, usersByCountry
};
var throttler = new SemaphoreSlim(MaxRequests, MaxRequests);
foreach(var task in tasks)
{
await throttler.WaitAsync();
try
{
await task;
await Task.Delay(1000); // It's important due to limits of Google Analytics requests (10 queries per second per IP address)
}
finally
{
throttler.Release();
}
}
await Task.WhenAll(tasks);
return new SiteAnalyticsDTO()
{
TodayVisits = await todayVisits,
TodayTraffic = await todayTraffic,
NewAndReturningUsers = await newAndReturningUsers,
AverageSessionDuration = await averageSessionDuration,
DeviceCategory = await deviceCategory,
TopPages = await topPages,
GuestsAndRegisteredUsers = await guestsAndRegisteredUsers,
AverageNumberOfSessionsPerDay = await averageNumberOfSessionsPerDay,
VisitsPerWeekday = await visitsPerWeekday,
VisitsByHours = await visitsByHours,
UsersByPrefectures = await usersByPrefectures,
UsersByCountry = await usersByCountry
};
}
And here is some example method for Google Analytics call:
public async Task<int> GetTodayVisitsNumber(List<long> listingIds = null)
{
string filter = GetFilter(listingIds);
var getReportsRequest = GetReportsRequestModel(GetTodayDateRange(), "ga:sessionCount", "ga:sessions", _configuration.MainViewId, filter);
var response = await _service.Reports.BatchGet(getReportsRequest).ExecuteAsync();
Console.WriteLine(response);
var data = response.Reports.FirstOrDefault();
return Convert.ToInt32(data?.Data.Totals[0].Values[0]);
}
All requests are triggered simultaneously.
Let's take a look here
var todayVisits = _googleAnalyticsService.GetTodayVisitsNumber();
var todayTraffic = _googleAnalyticsService.GetTodayTraffic();
var newAndReturningUsers = _googleAnalyticsService.GetNewAndReturningUsersNumber();
var averageSessionDuration = _googleAnalyticsService.GetAverageSessionDuration();
var deviceCategory = _googleAnalyticsService.GetSessionNumberByDeviceCategory();
var topPages = _googleAnalyticsService.GetTodaysTopPages();
var guestsAndRegisteredUsers = _googleAnalyticsService.GetGuestsVsRegisteredUsers();
var averageNumberOfSessionsPerDay = _googleAnalyticsService.GetAverageSessionsNumber();
var visitsPerWeekday = _googleAnalyticsService.GetTrafficByWeekday();
var visitsByHours = _googleAnalyticsService.GetTrafficByTimeOfDay();
var usersByPrefectures = _googleAnalyticsService.GetUsersByPrefectures();
var usersByCountry = _googleAnalyticsService.GetUsersByCountry();
You are storing the result of each of these methods. When you use the parenthesis tokens such as "methodName();" you invoke the method and store the result in var.
You then store the results of these methods in a list, then await each of them with a Semaphore to limit the number of tasks that can be awaited at once.
The issue is: each await finishes immediately because you already waited(synchronously) them when you initially invoked them above.
This leads you to believe that the SemaphoreSlim isn't working, because if each Task immediately returns when awaited(because they've already been invoked) then there is no time between them.
Store the async methods for later instead of invoking them all at once.
You can't store a delegate like those in a var you have to store them in an explicitly typed variable Func<TResult>.
For example:
Func<Task<object>> todayVisits = _googleAnalyticsService.GetTodayVisitsNumber;
editors note, I have no clue what these methods return I substituted object to be as general as possible
Now - it would be pretty cumbersome if we stored each one in a variable, so instead of storing them in individual variables let's just put them directly in a list like this:
var awaitableTasks = new List<Func<Task<object>>>()
{
_googleAnalyticsService.GetTodayVisitsNumber,
_googleAnalyticsService.GetTodayTraffic,
_googleAnalyticsService.GetNewAndReturningUsersNumber,
_googleAnalyticsService.GetAverageSessionDuration,
_googleAnalyticsService.GetSessionNumberByDeviceCategory,
_googleAnalyticsService.GetTodaysTopPages,
_googleAnalyticsService.GetGuestsVsRegisteredUsers,
_googleAnalyticsService.GetAverageSessionsNumber,
_googleAnalyticsService.GetTrafficByWeekday,
_googleAnalyticsService.GetTrafficByTimeOfDay,
_googleAnalyticsService.GetUsersByPrefectures,
_googleAnalyticsService.GetUsersByCountry
};
Because these new objects themselves aren't tasks but rather methods that return a Task we have to change how we store and invoke them, for this we'll use a local method so I'll go over each of the changes I made.
Let's create that Semaphore and create somewhere we can put tasks to keep track of them.
Let's also create somewhere we can store the results of each of those tasks when we await them.
var throttler = new SemaphoreSlim(MaxRequests, MaxRequests);
var tasks = new List<Task>();
ConcurrentDictionary<string, object> results = new();
Let's create a local method with a couple of responsibilities
Accept a Func<Task<object>> as a param
Await the method
Put the result of that method somewhere we can get it later
Release the Semphore even if it encounters an error
async Task Worker(Func<Task<object>> awaitableFunc)
{
try
{
resultDict.TryAdd(awaitableFunc.GetMethodInfo().Name, await awaitableFunc());
}
finally
{
throttler.Release();
}
}
Editors note: You can pull the same thing off with a lambda expression, but I prefer to use a local method for clarity and formatting.
Start the workers and store the tasks they return.
That way.. if they aren't done by the time the last couple are created, then we can wait for them to finish before creating the final object(since we are going to need all of the results they provide to create the final object).
foreach (var task in awaitableTasks)
{
await throttler.WaitAsync();
tasks.Add(Task.Run(() => Worker(task)));
}
// wait for the tasks to finish
await Task.WhenAll(tasks);
Create the final object then return it.
return new SiteAnalyticsDTO()
{
TodayVisits = resultDict[nameof(_googleAnalyticsService.GetTodayVisitsNumber)],
TodayTraffic = resultDict[nameof(_googleAnalyticsService.GetTodayTraffic)],
NewAndReturningUsers = resultDict[nameof(_googleAnalyticsService.GetNewAndReturningUsersNumber)],
AverageSessionDuration = resultDict[nameof(_googleAnalyticsService.GetAverageSessionDuration)],
DeviceCategory = resultDict[nameof(_googleAnalyticsService.GetSessionNumberByDeviceCategory)],
TopPages = resultDict[nameof(_googleAnalyticsService.GetTodaysTopPages)],
GuestsAndRegisteredUsers = resultDict[nameof(_googleAnalyticsService.GetGuestsVsRegisteredUsers)],
AverageNumberOfSessionsPerDay = resultDict[nameof(_googleAnalyticsService.GetAverageSessionsNumber)],
VisitsPerWeekday = resultDict[nameof(_googleAnalyticsService.GetTrafficByWeekday)],
VisitsByHours = resultDict[nameof(_googleAnalyticsService.GetTrafficByTimeOfDay)],
UsersByPrefectures = resultDict[nameof(_googleAnalyticsService.GetUsersByPrefectures)],
UsersByCountry = resultDict[nameof(_googleAnalyticsService.GetUsersByCountry)]
};
Wrap it all together and I think we have something that might work, or at least be easy to modify to meet your needs.
public static async Task<SiteAnalyticsDTO> Handle(GetSiteAnalyticsParameter query)
{
// store these methods so we can iterate and execute them later
var awaitableTasks = new List<Func<Task<object>>>()
{
_googleAnalyticsService.GetTodayVisitsNumber,
_googleAnalyticsService.GetTodayTraffic,
_googleAnalyticsService.GetNewAndReturningUsersNumber,
_googleAnalyticsService.GetAverageSessionDuration,
_googleAnalyticsService.GetSessionNumberByDeviceCategory,
_googleAnalyticsService.GetTodaysTopPages,
_googleAnalyticsService.GetGuestsVsRegisteredUsers,
_googleAnalyticsService.GetAverageSessionsNumber,
_googleAnalyticsService.GetTrafficByWeekday,
_googleAnalyticsService.GetTrafficByTimeOfDay,
_googleAnalyticsService.GetUsersByPrefectures,
_googleAnalyticsService.GetUsersByCountry
};
// create a way to limit the number of concurrent requests
var throttler = new SemaphoreSlim(MaxRequests, MaxRequests);
// create a place to store the tasks we create
var finalTasks = new List<Task>();
// make sure we have some where to put our results
ConcurrentDictionary<string, object> resultDict = new();
// make a worker that accepts one of those methods, invokes it
// then adds the result to the dict
async Task Worker(Func<Task<object>> awaitableFunc)
{
try
{
resultDict.TryAdd(awaitableFunc.GetMethodInfo().Name, await awaitableFunc());
}
finally
{
// make sure even if we encounter an error we still release the semphore
throttler.Release();
}
}
// iterate over the tasks, wait for the sempahore
// when we get a slot, create a worker and send it to the background
foreach (var task in awaitableTasks)
{
await throttler.WaitAsync();
finalTasks.Add(Task.Run(() => Worker(task)));
}
// wait for any remaining tasks to finish up in the background if they are still running
await Task.WhenAll(finalTasks);
// create the return object from the results of the dictionary
return new SiteAnalyticsDTO()
{
TodayVisits = resultDict[nameof(_googleAnalyticsService.GetTodayVisitsNumber)],
TodayTraffic = resultDict[nameof(_googleAnalyticsService.GetTodayTraffic)],
NewAndReturningUsers = resultDict[nameof(_googleAnalyticsService.GetNewAndReturningUsersNumber)],
AverageSessionDuration = resultDict[nameof(_googleAnalyticsService.GetAverageSessionDuration)],
DeviceCategory = resultDict[nameof(_googleAnalyticsService.GetSessionNumberByDeviceCategory)],
TopPages = resultDict[nameof(_googleAnalyticsService.GetTodaysTopPages)],
GuestsAndRegisteredUsers = resultDict[nameof(_googleAnalyticsService.GetGuestsVsRegisteredUsers)],
AverageNumberOfSessionsPerDay = resultDict[nameof(_googleAnalyticsService.GetAverageSessionsNumber)],
VisitsPerWeekday = resultDict[nameof(_googleAnalyticsService.GetTrafficByWeekday)],
VisitsByHours = resultDict[nameof(_googleAnalyticsService.GetTrafficByTimeOfDay)],
UsersByPrefectures = resultDict[nameof(_googleAnalyticsService.GetUsersByPrefectures)],
UsersByCountry = resultDict[nameof(_googleAnalyticsService.GetUsersByCountry)]
};
}
The problem of your setup is that all tasks are started at the same time, and only their awaiting is throttled. Throttling the awaiting has no useful effect. Only your continuations are delayed. The target service receives all the requests in bulk.
My suggestion is to use a dedicated class to encapsulate the throttling logic. It seems that you need to limit both the concurrency and the rate of sending the requests, and each one of these limitations can be achieved by using a separate SemaphoreSlim. Here is a simple implementation:
public class ThrottledExecution
{
private readonly SemaphoreSlim _concurrencySemaphore;
private readonly SemaphoreSlim _delaySemaphore;
private readonly TimeSpan _delay;
public ThrottledExecution(int concurrencyLimit, TimeSpan rateLimitTime,
int rateLimitCount)
{
// Arguments validation omitted
_concurrencySemaphore = new SemaphoreSlim(concurrencyLimit, concurrencyLimit);
_delaySemaphore = new SemaphoreSlim(rateLimitCount, rateLimitCount);
_delay = rateLimitTime;
}
public async Task<TResult> Run<TResult>(Func<Task<TResult>> action)
{
await _delaySemaphore.WaitAsync();
ScheduleDelaySemaphoreRelease();
await _concurrencySemaphore.WaitAsync();
try { return await action().ConfigureAwait(false); }
finally { _concurrencySemaphore.Release(); }
}
private async void ScheduleDelaySemaphoreRelease()
{
await Task.Delay(_delay).ConfigureAwait(false);
_delaySemaphore.Release();
}
}
And here is how you could use it:
public async Task<SiteAnalyticsDTO> Handle(GetSiteAnalyticsParameter query)
{
var throttler = new ThrottledExecution(MaxRequests, TimeSpan.FromSeconds(1), 1);
var todayVisits = throttler.Run(() => _service.GetTodayVisitsNumber());
var todayTraffic = throttler.Run(() => _service.GetTodayTraffic());
var newAndReturningUsers = throttler.Run(() => _service.GetNewAndReturningUsersNumber());
var averageSessionDuration = throttler.Run(() => _service.GetAverageSessionDuration());
var deviceCategory = throttler.Run(() => _service.GetSessionNumberByDeviceCategory());
var topPages = throttler.Run(() => _service.GetTodaysTopPages());
var guestsAndRegisteredUsers = throttler.Run(() => _service.GetGuestsVsRegisteredUsers());
var averageNumberOfSessionsPerDay = throttler.Run(() => _service.GetAverageSessionsNumber());
var visitsPerWeekday = throttler.Run(() => _service.GetTrafficByWeekday());
var visitsByHours = throttler.Run(() => _service.GetTrafficByTimeOfDay());
var usersByPrefectures = throttler.Run(() => _service.GetUsersByPrefectures());
var usersByCountry = throttler.Run(() => _service.GetUsersByCountry());
var tasks = new List<Task>()
{
todayVisits, todayTraffic, newAndReturningUsers,
averageSessionDuration, deviceCategory, topPages,
guestsAndRegisteredUsers, averageNumberOfSessionsPerDay, visitsPerWeekday,
visitsByHours, usersByPrefectures, usersByCountry
};
await Task.WhenAll(tasks);
return new SiteAnalyticsDTO()
{
TodayVisits = await todayVisits,
TodayTraffic = await todayTraffic,
NewAndReturningUsers = await newAndReturningUsers,
AverageSessionDuration = await averageSessionDuration,
DeviceCategory = await deviceCategory,
TopPages = await topPages,
GuestsAndRegisteredUsers = await guestsAndRegisteredUsers,
AverageNumberOfSessionsPerDay = await averageNumberOfSessionsPerDay,
VisitsPerWeekday = await visitsPerWeekday,
VisitsByHours = await visitsByHours,
UsersByPrefectures = await usersByPrefectures,
UsersByCountry = await usersByCountry,
};
}
It seems that a partially successful result is not useful to you, so you could consider adding some auto-cancellation logic inside the ThrottledExecution class. In case a task fails, all pending and subsequent asynchronous operations should be canceled.
I have a scenario where I need to make multiple api calls (same api with different parameters) parallely in c# (Xamarin iOS and Xamarin Android). And I don't want to wait for all tasks to complete, instead whenever a response comes I should process it and update the UI accordingly.
Method that needs to be called multiple times
public async Task<Response> GetProductsAsync(int categoryId, int pageNo = -1, int pageSize = -1)
{
try
{
string url = "";
if (pageNo == -1 || pageSize == -1)
url = $"catalog/v1/categories/{categoryId}/products";
else
url = $"catalog/v1/categories/{categoryId}/products?page-number={pageNo}&page-size={pageSize}";
var response = await client.GetAsync(url);
string responseString = await response.Content.ReadAsStringAsync();
GetParsedData(response.IsSuccessStatusCode, responseString);
}
catch (Exception e)
{
apiResponse.status = "internalError";
apiResponse.data = e.Message;
}
return apiResponse;
}
From calling function you may write the code as below
public void CallingFunctionToGetProductsAsync() {
Task.Run(async () =>
{
var response = await GetProductsAsync(1);
ProcessResponse(response);
});
Task.Run(async () =>
{
var response = await GetProductsAsync(2);
ProcessResponse(response);
});
}
This is how you can wait for multiple tasks asynchron and update the UI whenever any of them completes.
async Task GetSomeProductsAsync( IEnumerable<int> categoryIds )
{
List<Task<Response>> tasks = categoryIds
.Select( catId => GetProductsAsync( catId ) )
.ToList();
while ( tasks.Any() )
{
var completed = await Task.WhenAny( tasks );
tasks.Remove( completed );
var response = completed.Result;
// update the ui from this response
}
}
As a side note:
You should add ConfigureAwait(false) to your awaiting code in GetProducsAsync to avoid uneccessary sync with the caller thread (wich would be the UI here)
public async Task<Response> GetProductsAsync(int categoryId, int pageNo = -1, int pageSize = -1)
{
try
{
string url = "";
if (pageNo == -1 || pageSize == -1)
url = $"catalog/v1/categories/{categoryId}/products";
else
url = $"catalog/v1/categories/{categoryId}/products?page-number={pageNo}&page-size={pageSize}";
var response = await client.GetAsync(url).ConfigureAwait(false);
string responseString = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
GetParsedData(response.IsSuccessStatusCode, responseString);
}
catch (Exception e)
{
apiResponse.status = "internalError";
apiResponse.data = e.Message;
}
return apiResponse;
}
You can read more about it in Stephen Cleary's blog article: Don't block on Async Code
I have a list of items, and for each item, I need to execute series of tasks.
For accessing data layer, I am using the following code:
public async Task<ExampleResult> GetExampleResultAsync(Parameter parameter, CancellationToken cancellationToken = default(CancellationToken))
{
GetCustomerResult result = null;
OracleConnection connection = this.Database.GetOracleConnection();
bool needClose = false;
if (connection.State != ConnectionState.Open)
{
await connection.OpenAsync(cancellationToken);
needClose = true;
}
try
{
using (OracleCommand cmd = connection.CreateCommand())
{
.... do the work
}
}
finally
{
if (needClose)
connection.Close();
}
return result;
}
however, this leads to concurrency and thus I am getting
the connection was not closed error.
The possible workaround for this that I ended up is to change the way of connecting to the database. I am thinking to use new connection instance for every request and surround these instances with using statement.
This would lead me lots of work and I would like to get an idea about best practices in the field for handling concurrency issues.
EDIT:
my caller function is below for your reference
public async Task<DomainResult<IList<MbRiskDto>>> QueryAsync(Action<MbrAccountAutoMatcherQueryParameter> parameter,
CancellationToken cancellationToken = default(CancellationToken))
{
parameter(_parameter);
var nonDeclaredMbrAccounts = await _nonDeclaredMbrAccountsQuery.QueryAsync(param => param.TransactionDate = _parameter.TransactionDate, cancellationToken);
if (nonDeclaredMbrAccounts.IsFailed)
nonDeclaredMbrAccounts.Errors.ForEach(error => _errors.Add(error));
var taskList = new List<Task<MbRiskDto>>();
var throttler = new SemaphoreSlim(initialCount: 10);
foreach (var nonDeclaredAccount in nonDeclaredMbrAccounts.Result)
{
await throttler.WaitAsync(cancellationToken);
var account = (AccountDto) nonDeclaredAccount.Clone();
var firstAccountHolder = Convert.ToInt32(account.AccountHolders.FirstOrDefault());
var task = Task.Run(async () =>
{
MbRiskDto result;
try
{
result = new MbRiskDto
{
KimNo = await GetKimNo(firstAccountHolder),
HesNo = account.AccountNo,
FinCode = await GetFinanceCode(account, firstAccountHolder, cancellationToken),
Unvan = account.SMA.Substring(0, Math.Min(account.SMA.Length, 54))
};
}
finally
{
throttler.Release();
}
return result;
}, cancellationToken);
taskList.Add(task);
}
var taskResult = await Task.WhenAll(taskList);
return DomainResult<IList<MbRiskDto>>.Success(taskResult);
}
According to the comments I removed Task.Run() clause and replaced with the code below. Right now, I believe that my concurrency problems are solved.
Func<Task<MbRiskDto>> mbRiskTask = async () => new MbRiskDto
{
KimNo = await GetKimNo(firstAccountHolder, cancellationToken),
HesNo = account.AccountNo,
FinCode = await GetFinanceCode(account, firstAccountHolder, cancellationToken),
Unvan = account.SMA.Substring(0, Math.Min(account.SMA.Length, 54))
};
taskList.Add(mbRiskTask.Invoke());
Thus, Resulting code can be seen below
public async Task<DomainResult<IList<MbRiskDto>>> QueryAsync(Action<MbrAccountAutoMatcherQueryParameter> parameter,
CancellationToken cancellationToken = default(CancellationToken))
{
parameter(_parameter);
var nonDeclaredMbrAccounts = await _nonDeclaredMbrAccountsQuery.QueryAsync(param => param.TransactionDate = _parameter.TransactionDate, cancellationToken);
if (nonDeclaredMbrAccounts.IsFailed)
nonDeclaredMbrAccounts.Errors.ForEach(error => _errors.Add(error));
var taskList = new List<Task<MbRiskDto>>();
foreach (var nonDeclaredAccount in nonDeclaredMbrAccounts.Result)
{
var account = (AccountDto) nonDeclaredAccount.Clone();
var firstAccountHolder = Convert.ToInt32(account.AccountHolders.FirstOrDefault());
Func<Task<MbRiskDto>> mbRiskTask = async () => new MbRiskDto
{
KimNo = await GetKimNo(firstAccountHolder, cancellationToken),
HesNo = account.AccountNo,
FinCode = await GetFinanceCode(account, firstAccountHolder, cancellationToken),
Unvan = account.SMA.Substring(0, Math.Min(account.SMA.Length, 54))
};
//var task = Task.Run();
taskList.Add(mbRiskTask.Invoke());
}
var taskResult = await Task.WhenAll(taskList);
return DomainResult<IList<MbRiskDto>>.Success(taskResult);
}
EDIT:
With this refactoring, the operation takes approximately 4minutes to finish. Do you have any suggestion in order to make it work faster?