One question about parallel programming.
I need to POST 2 APIs (one after another) to get a Order list. First one is for getting token..
After I got the list, I need to POST 3 APIs (one after another) to integrate these Orders.
These 3 APIs don't accept arrays, so I need to send one by one. I cannot send batch data. With 1 thread it only integrates 10 orders in 1 minute. I need more performance. How can I run foreach part in parallel?
using System.Net.Http;
using System.Text;
using System.Text.Json;
namespace Order_Integrator
{
public class Program
{
static readonly HttpClient client = new HttpClient();
static async Task Main()
{
//Auth
var connectResponse = await client.PostAsync(connectUrl, connectContent);
var connectResponseString = await connectResponse.Content.ReadAsStringAsync();
var connect = JsonSerializer.Deserialize<connectResponse>(connectResponseString);
var token = connect.Token;
//Get Order List
var orderResponse = await client.PostAsync(orderUrl, orderContent);
var orderResponseString = await orderResponse.Content.ReadAsStringAsync();
var orders = JsonSerializer.Deserialize<orderResponse>(orderResponseString);
foreach (var order in orders)
{
//Get Order Details
//Generate getOrderDetailsContent with order
var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
var getOrderDetailsResponseString = await getOrderDetailsResponse.Content.ReadAsStringAsync();
var getOrderDetails = JsonSerializer.Deserialize<getOrderDetailsResponse>(getOrderDetailsResponseString);
//Create Order
//Generate createOrderContent with GetOrderDetails
var createOrderResponse = await client.PostAsync(createOrderUrl, createOrderContent);
var createOrderResponseString = await createOrderResponse.Content.ReadAsStringAsync();
var createOrder = JsonSerializer.Deserialize<createOrderResponse>(createOrderResponseString);
//Create Log
//Generate createLogContent with CreateOrderResponse
var createLogResponse = await client.PostAsync(createLogUrl, createLogContent);
var createLogResponseString = await createLogResponse.Content.ReadAsStringAsync();
var createLog = JsonSerializer.Deserialize<createLogResponse>(createLogResponseString);
}
}
}
}
You can use Parallel.ForEachAsync
var options = new ParallelOptions()
{
MaxDegreeOfParallelism = 20
};
await Parallel.ForEachAsync(orders, options, async (OrderNumber, ct) => {
var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
});
Since you're using asynchronous code, you should be using asynchronous concurrency, not (multithreaded) parallelism. Parallel.ForEachAsync will do both, but it's not available on older runtimes.
You can do just asynchronous concurrency by using Select and then Task.WhenAll:
var tasks = orders.Select(async order =>
{
//Get Order Details
//Generate getOrderDetailsContent with order
var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
var getOrderDetailsResponseString = await getOrderDetailsResponse.Content.ReadAsStringAsync();
var getOrderDetails = JsonSerializer.Deserialize<getOrderDetailsResponse>(getOrderDetailsResponseString);
//Create Order
//Generate createOrderContent with GetOrderDetails
var createOrderResponse = await client.PostAsync(createOrderUrl, createOrderContent);
var createOrderResponseString = await createOrderResponse.Content.ReadAsStringAsync();
var createOrder = JsonSerializer.Deserialize<createOrderResponse>(createOrderResponseString);
//Create Log
//Generate createLogContent with CreateOrderResponse
var createLogResponse = await client.PostAsync(createLogUrl, createLogContent);
var createLogResponseString = await createLogResponse.Content.ReadAsStringAsync();
var createLog = JsonSerializer.Deserialize<createLogResponse>(createLogResponseString);
return createLog;
}).ToList();
await Task.WhenAll(tasks);
Related
I'm using Google Analytics and that service has limit of 10 concurrent requests. I had to limit my API somehow, so I decided to use a semaphore, but it seems it doesn't work. All requests are triggered simultaneously. I can't find the problem in my code.
public async Task<SiteAnalyticsDTO> Handle(GetSiteAnalyticsParameter query)
{
var todayVisits = _googleAnalyticsService.GetTodayVisitsNumber();
var todayTraffic = _googleAnalyticsService.GetTodayTraffic();
var newAndReturningUsers = _googleAnalyticsService.GetNewAndReturningUsersNumber();
var averageSessionDuration = _googleAnalyticsService.GetAverageSessionDuration();
var deviceCategory = _googleAnalyticsService.GetSessionNumberByDeviceCategory();
var topPages = _googleAnalyticsService.GetTodaysTopPages();
var guestsAndRegisteredUsers = _googleAnalyticsService.GetGuestsVsRegisteredUsers();
var averageNumberOfSessionsPerDay = _googleAnalyticsService.GetAverageSessionsNumber();
var visitsPerWeekday = _googleAnalyticsService.GetTrafficByWeekday();
var visitsByHours = _googleAnalyticsService.GetTrafficByTimeOfDay();
var usersByPrefectures = _googleAnalyticsService.GetUsersByPrefectures();
var usersByCountry = _googleAnalyticsService.GetUsersByCountry();
var tasks = new List<Task>()
{
todayVisits, todayTraffic, newAndReturningUsers,
averageSessionDuration, deviceCategory, topPages,
guestsAndRegisteredUsers, averageNumberOfSessionsPerDay, visitsPerWeekday,
visitsByHours, usersByPrefectures, usersByCountry
};
var throttler = new SemaphoreSlim(MaxRequests, MaxRequests);
foreach(var task in tasks)
{
await throttler.WaitAsync();
try
{
await task;
await Task.Delay(1000); // It's important due to limits of Google Analytics requests (10 queries per second per IP address)
}
finally
{
throttler.Release();
}
}
await Task.WhenAll(tasks);
return new SiteAnalyticsDTO()
{
TodayVisits = await todayVisits,
TodayTraffic = await todayTraffic,
NewAndReturningUsers = await newAndReturningUsers,
AverageSessionDuration = await averageSessionDuration,
DeviceCategory = await deviceCategory,
TopPages = await topPages,
GuestsAndRegisteredUsers = await guestsAndRegisteredUsers,
AverageNumberOfSessionsPerDay = await averageNumberOfSessionsPerDay,
VisitsPerWeekday = await visitsPerWeekday,
VisitsByHours = await visitsByHours,
UsersByPrefectures = await usersByPrefectures,
UsersByCountry = await usersByCountry
};
}
And here is some example method for Google Analytics call:
public async Task<int> GetTodayVisitsNumber(List<long> listingIds = null)
{
string filter = GetFilter(listingIds);
var getReportsRequest = GetReportsRequestModel(GetTodayDateRange(), "ga:sessionCount", "ga:sessions", _configuration.MainViewId, filter);
var response = await _service.Reports.BatchGet(getReportsRequest).ExecuteAsync();
Console.WriteLine(response);
var data = response.Reports.FirstOrDefault();
return Convert.ToInt32(data?.Data.Totals[0].Values[0]);
}
All requests are triggered simultaneously.
Let's take a look here
var todayVisits = _googleAnalyticsService.GetTodayVisitsNumber();
var todayTraffic = _googleAnalyticsService.GetTodayTraffic();
var newAndReturningUsers = _googleAnalyticsService.GetNewAndReturningUsersNumber();
var averageSessionDuration = _googleAnalyticsService.GetAverageSessionDuration();
var deviceCategory = _googleAnalyticsService.GetSessionNumberByDeviceCategory();
var topPages = _googleAnalyticsService.GetTodaysTopPages();
var guestsAndRegisteredUsers = _googleAnalyticsService.GetGuestsVsRegisteredUsers();
var averageNumberOfSessionsPerDay = _googleAnalyticsService.GetAverageSessionsNumber();
var visitsPerWeekday = _googleAnalyticsService.GetTrafficByWeekday();
var visitsByHours = _googleAnalyticsService.GetTrafficByTimeOfDay();
var usersByPrefectures = _googleAnalyticsService.GetUsersByPrefectures();
var usersByCountry = _googleAnalyticsService.GetUsersByCountry();
You are storing the result of each of these methods. When you use the parenthesis tokens such as "methodName();" you invoke the method and store the result in var.
You then store the results of these methods in a list, then await each of them with a Semaphore to limit the number of tasks that can be awaited at once.
The issue is: each await finishes immediately because you already waited(synchronously) them when you initially invoked them above.
This leads you to believe that the SemaphoreSlim isn't working, because if each Task immediately returns when awaited(because they've already been invoked) then there is no time between them.
Store the async methods for later instead of invoking them all at once.
You can't store a delegate like those in a var you have to store them in an explicitly typed variable Func<TResult>.
For example:
Func<Task<object>> todayVisits = _googleAnalyticsService.GetTodayVisitsNumber;
editors note, I have no clue what these methods return I substituted object to be as general as possible
Now - it would be pretty cumbersome if we stored each one in a variable, so instead of storing them in individual variables let's just put them directly in a list like this:
var awaitableTasks = new List<Func<Task<object>>>()
{
_googleAnalyticsService.GetTodayVisitsNumber,
_googleAnalyticsService.GetTodayTraffic,
_googleAnalyticsService.GetNewAndReturningUsersNumber,
_googleAnalyticsService.GetAverageSessionDuration,
_googleAnalyticsService.GetSessionNumberByDeviceCategory,
_googleAnalyticsService.GetTodaysTopPages,
_googleAnalyticsService.GetGuestsVsRegisteredUsers,
_googleAnalyticsService.GetAverageSessionsNumber,
_googleAnalyticsService.GetTrafficByWeekday,
_googleAnalyticsService.GetTrafficByTimeOfDay,
_googleAnalyticsService.GetUsersByPrefectures,
_googleAnalyticsService.GetUsersByCountry
};
Because these new objects themselves aren't tasks but rather methods that return a Task we have to change how we store and invoke them, for this we'll use a local method so I'll go over each of the changes I made.
Let's create that Semaphore and create somewhere we can put tasks to keep track of them.
Let's also create somewhere we can store the results of each of those tasks when we await them.
var throttler = new SemaphoreSlim(MaxRequests, MaxRequests);
var tasks = new List<Task>();
ConcurrentDictionary<string, object> results = new();
Let's create a local method with a couple of responsibilities
Accept a Func<Task<object>> as a param
Await the method
Put the result of that method somewhere we can get it later
Release the Semphore even if it encounters an error
async Task Worker(Func<Task<object>> awaitableFunc)
{
try
{
resultDict.TryAdd(awaitableFunc.GetMethodInfo().Name, await awaitableFunc());
}
finally
{
throttler.Release();
}
}
Editors note: You can pull the same thing off with a lambda expression, but I prefer to use a local method for clarity and formatting.
Start the workers and store the tasks they return.
That way.. if they aren't done by the time the last couple are created, then we can wait for them to finish before creating the final object(since we are going to need all of the results they provide to create the final object).
foreach (var task in awaitableTasks)
{
await throttler.WaitAsync();
tasks.Add(Task.Run(() => Worker(task)));
}
// wait for the tasks to finish
await Task.WhenAll(tasks);
Create the final object then return it.
return new SiteAnalyticsDTO()
{
TodayVisits = resultDict[nameof(_googleAnalyticsService.GetTodayVisitsNumber)],
TodayTraffic = resultDict[nameof(_googleAnalyticsService.GetTodayTraffic)],
NewAndReturningUsers = resultDict[nameof(_googleAnalyticsService.GetNewAndReturningUsersNumber)],
AverageSessionDuration = resultDict[nameof(_googleAnalyticsService.GetAverageSessionDuration)],
DeviceCategory = resultDict[nameof(_googleAnalyticsService.GetSessionNumberByDeviceCategory)],
TopPages = resultDict[nameof(_googleAnalyticsService.GetTodaysTopPages)],
GuestsAndRegisteredUsers = resultDict[nameof(_googleAnalyticsService.GetGuestsVsRegisteredUsers)],
AverageNumberOfSessionsPerDay = resultDict[nameof(_googleAnalyticsService.GetAverageSessionsNumber)],
VisitsPerWeekday = resultDict[nameof(_googleAnalyticsService.GetTrafficByWeekday)],
VisitsByHours = resultDict[nameof(_googleAnalyticsService.GetTrafficByTimeOfDay)],
UsersByPrefectures = resultDict[nameof(_googleAnalyticsService.GetUsersByPrefectures)],
UsersByCountry = resultDict[nameof(_googleAnalyticsService.GetUsersByCountry)]
};
Wrap it all together and I think we have something that might work, or at least be easy to modify to meet your needs.
public static async Task<SiteAnalyticsDTO> Handle(GetSiteAnalyticsParameter query)
{
// store these methods so we can iterate and execute them later
var awaitableTasks = new List<Func<Task<object>>>()
{
_googleAnalyticsService.GetTodayVisitsNumber,
_googleAnalyticsService.GetTodayTraffic,
_googleAnalyticsService.GetNewAndReturningUsersNumber,
_googleAnalyticsService.GetAverageSessionDuration,
_googleAnalyticsService.GetSessionNumberByDeviceCategory,
_googleAnalyticsService.GetTodaysTopPages,
_googleAnalyticsService.GetGuestsVsRegisteredUsers,
_googleAnalyticsService.GetAverageSessionsNumber,
_googleAnalyticsService.GetTrafficByWeekday,
_googleAnalyticsService.GetTrafficByTimeOfDay,
_googleAnalyticsService.GetUsersByPrefectures,
_googleAnalyticsService.GetUsersByCountry
};
// create a way to limit the number of concurrent requests
var throttler = new SemaphoreSlim(MaxRequests, MaxRequests);
// create a place to store the tasks we create
var finalTasks = new List<Task>();
// make sure we have some where to put our results
ConcurrentDictionary<string, object> resultDict = new();
// make a worker that accepts one of those methods, invokes it
// then adds the result to the dict
async Task Worker(Func<Task<object>> awaitableFunc)
{
try
{
resultDict.TryAdd(awaitableFunc.GetMethodInfo().Name, await awaitableFunc());
}
finally
{
// make sure even if we encounter an error we still release the semphore
throttler.Release();
}
}
// iterate over the tasks, wait for the sempahore
// when we get a slot, create a worker and send it to the background
foreach (var task in awaitableTasks)
{
await throttler.WaitAsync();
finalTasks.Add(Task.Run(() => Worker(task)));
}
// wait for any remaining tasks to finish up in the background if they are still running
await Task.WhenAll(finalTasks);
// create the return object from the results of the dictionary
return new SiteAnalyticsDTO()
{
TodayVisits = resultDict[nameof(_googleAnalyticsService.GetTodayVisitsNumber)],
TodayTraffic = resultDict[nameof(_googleAnalyticsService.GetTodayTraffic)],
NewAndReturningUsers = resultDict[nameof(_googleAnalyticsService.GetNewAndReturningUsersNumber)],
AverageSessionDuration = resultDict[nameof(_googleAnalyticsService.GetAverageSessionDuration)],
DeviceCategory = resultDict[nameof(_googleAnalyticsService.GetSessionNumberByDeviceCategory)],
TopPages = resultDict[nameof(_googleAnalyticsService.GetTodaysTopPages)],
GuestsAndRegisteredUsers = resultDict[nameof(_googleAnalyticsService.GetGuestsVsRegisteredUsers)],
AverageNumberOfSessionsPerDay = resultDict[nameof(_googleAnalyticsService.GetAverageSessionsNumber)],
VisitsPerWeekday = resultDict[nameof(_googleAnalyticsService.GetTrafficByWeekday)],
VisitsByHours = resultDict[nameof(_googleAnalyticsService.GetTrafficByTimeOfDay)],
UsersByPrefectures = resultDict[nameof(_googleAnalyticsService.GetUsersByPrefectures)],
UsersByCountry = resultDict[nameof(_googleAnalyticsService.GetUsersByCountry)]
};
}
The problem of your setup is that all tasks are started at the same time, and only their awaiting is throttled. Throttling the awaiting has no useful effect. Only your continuations are delayed. The target service receives all the requests in bulk.
My suggestion is to use a dedicated class to encapsulate the throttling logic. It seems that you need to limit both the concurrency and the rate of sending the requests, and each one of these limitations can be achieved by using a separate SemaphoreSlim. Here is a simple implementation:
public class ThrottledExecution
{
private readonly SemaphoreSlim _concurrencySemaphore;
private readonly SemaphoreSlim _delaySemaphore;
private readonly TimeSpan _delay;
public ThrottledExecution(int concurrencyLimit, TimeSpan rateLimitTime,
int rateLimitCount)
{
// Arguments validation omitted
_concurrencySemaphore = new SemaphoreSlim(concurrencyLimit, concurrencyLimit);
_delaySemaphore = new SemaphoreSlim(rateLimitCount, rateLimitCount);
_delay = rateLimitTime;
}
public async Task<TResult> Run<TResult>(Func<Task<TResult>> action)
{
await _delaySemaphore.WaitAsync();
ScheduleDelaySemaphoreRelease();
await _concurrencySemaphore.WaitAsync();
try { return await action().ConfigureAwait(false); }
finally { _concurrencySemaphore.Release(); }
}
private async void ScheduleDelaySemaphoreRelease()
{
await Task.Delay(_delay).ConfigureAwait(false);
_delaySemaphore.Release();
}
}
And here is how you could use it:
public async Task<SiteAnalyticsDTO> Handle(GetSiteAnalyticsParameter query)
{
var throttler = new ThrottledExecution(MaxRequests, TimeSpan.FromSeconds(1), 1);
var todayVisits = throttler.Run(() => _service.GetTodayVisitsNumber());
var todayTraffic = throttler.Run(() => _service.GetTodayTraffic());
var newAndReturningUsers = throttler.Run(() => _service.GetNewAndReturningUsersNumber());
var averageSessionDuration = throttler.Run(() => _service.GetAverageSessionDuration());
var deviceCategory = throttler.Run(() => _service.GetSessionNumberByDeviceCategory());
var topPages = throttler.Run(() => _service.GetTodaysTopPages());
var guestsAndRegisteredUsers = throttler.Run(() => _service.GetGuestsVsRegisteredUsers());
var averageNumberOfSessionsPerDay = throttler.Run(() => _service.GetAverageSessionsNumber());
var visitsPerWeekday = throttler.Run(() => _service.GetTrafficByWeekday());
var visitsByHours = throttler.Run(() => _service.GetTrafficByTimeOfDay());
var usersByPrefectures = throttler.Run(() => _service.GetUsersByPrefectures());
var usersByCountry = throttler.Run(() => _service.GetUsersByCountry());
var tasks = new List<Task>()
{
todayVisits, todayTraffic, newAndReturningUsers,
averageSessionDuration, deviceCategory, topPages,
guestsAndRegisteredUsers, averageNumberOfSessionsPerDay, visitsPerWeekday,
visitsByHours, usersByPrefectures, usersByCountry
};
await Task.WhenAll(tasks);
return new SiteAnalyticsDTO()
{
TodayVisits = await todayVisits,
TodayTraffic = await todayTraffic,
NewAndReturningUsers = await newAndReturningUsers,
AverageSessionDuration = await averageSessionDuration,
DeviceCategory = await deviceCategory,
TopPages = await topPages,
GuestsAndRegisteredUsers = await guestsAndRegisteredUsers,
AverageNumberOfSessionsPerDay = await averageNumberOfSessionsPerDay,
VisitsPerWeekday = await visitsPerWeekday,
VisitsByHours = await visitsByHours,
UsersByPrefectures = await usersByPrefectures,
UsersByCountry = await usersByCountry,
};
}
It seems that a partially successful result is not useful to you, so you could consider adding some auto-cancellation logic inside the ThrottledExecution class. In case a task fails, all pending and subsequent asynchronous operations should be canceled.
I'm using ASP.NET Core MVC 2.0 and I've got an API that calls a method which opens a DB connection and inserts values into a model.
I'm hitting the same method 7 times and I'm just passing different information into it the parameters to build out my model. I HAVE to hit it seven different times, no rewrite of code will prevent this.
Instead of creating a loop and calling that method I wish to make parallel calls to the db so the response is faster. I've found multiple articles explaining how to do this e.g. How to properly make asynchronous / parallel database calls but there's enough difference that I can't seem to get it to work.
Following is the my method hitting the DB:
public async Task<RS_Model> Get_Results(RequestS_Model values)
{
//Deleted a bunch of code to keep it simple
string strQS = #"EXEC Get param1,param2,etc";
using (SqlConnection conSQL = new SqlConnection(connectionString))
{
using (SqlCommand cmdSQL = new SqlCommand(strQS, conSQL))
{
conSQL.Open();
using (SqlDataReader dtrSQL = cmdSQL.ExecuteReader())
{
while (dtrSQL.Read())
{
Double.TryParse(dtrSQL["Value1"].ToString(), out dblVal1);
} //Ends While
} //end SQLDataReader
} //Ends cmdSQL
} //ends using
results.Price = dblVal1;
return results;
} //Ends Get Results
My IActionResult for the api is:
[HttpGet]
public async Task<IActionResult> Get([FromQuery] RequestS_Model values)
{
SV_Results Results = new SV_Results();
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
RS_Model model = new RS_Model();
model.dblr[0] = await Results.Get_Results(values);
values.Parm1 = 2;
model.dblr[1] = await Results.Get_Results(values);
values.Parm1 = 3;
model.dblr[2] = await Results.Get_Results(values);
values.Parm1 = 4;
model.dblr[3] = await Results.Get_Results(values);
values.Parm1 = 5;
model.dblr[4] = await Results.Get_Results(values);
values.Parm1 = 6;
model.dblr[5] = await Results.Get_Results(values);
values.Parm1 = 7;
model.dblr[6] = await Results.Get_Results(values);
//int[] results = await Task.WhenAll(new Task<int>[] { task1, task2 });
return new OkObjectResult(model);
} //IActionResults
I know that I've forced them into a synchronous call be doing what I am, but I can't seem to make the 7 calls asynchronous and then wait for them to be all done before I build my final model. The final response needs to be in Json, but I haven't even gotten that far yet.
Instead of this:
model.dblr[0] = await Results.Get_Results(values);
model.dblr[1] = await Results.Get_Results(values);
model.dblr[2] = await Results.Get_Results(values);
model.dblr[3] = await Results.Get_Results(values);
model.dblr[4] = await Results.Get_Results(values);
model.dblr[5] = await Results.Get_Results(values);
model.dblr[6] = await Results.Get_Results(values);
Create a list of tasks and await them as a group:
var tasks = Enumerable.Range(0,7).Select( i => Results.Get_Results(values) ).ToList();
await Task.WhenAll(tasks);
for (int i=0; i<7; i++) model.dblr[i] = tasks[i].Result;
How to get the individual API call status success response in C#.
I am creating a mobile application using Xamarin Forms,
In my application, I need to prefetch certain information when app launches to use the mobile application.
Right now, I am calling the details like this,
public async Task<Response> GetAllVasInformationAsync()
{
var userDetails = GetUserDetailsAsync();
var getWageInfo = GetUserWageInfoAsync();
var getSalaryInfo = GetSalaryInfoAsync();
await Task.WhenAll(userDetails,
getWageInfo,
getSalaryInfo,
);
var resultToReturn = new Response
{
IsuserDetailsSucceeded = userDetails.Result,
IsgetWageInfoSucceeded = getWageInfo.Result,
IsgetSalaryInfoSucceeded = getSalaryInfo.Result,
};
return resultToReturn;
}
In my app I need to update details based on the success response. Something like this (2/5) completed. And the text should be updated whenever we get a new response.
What is the best way to implement this feature? Is it possible to use along with Task.WhenAll. Because I am trying to wrap everything in one method call.
In my app I need to update details based on the success response.
The proper way to do this is IProgress<string>. The calling code should supply a Progress<string> that updates the UI accordingly.
public async Task<Response> GetAllVasInformationAsync(IProgress<string> progress)
{
var userDetails = UpdateWhenComplete(GetUserDetailsAsync(), "user details");
var getWageInfo = UpdateWhenComplete(GetUserWageInfoAsync(), "wage information");
var getSalaryInfo = UpdateWhenComplete(GetSalaryInfoAsync(), "salary information");
await Task.WhenAll(userDetails, getWageInfo, getSalaryInfo);
return new Response
{
IsuserDetailsSucceeded = await userDetails,
IsgetWageInfoSucceeded = await getWageInfo,
IsgetSalaryInfoSucceeded = await getSalaryInfo,
};
async Task<T> UpdateWhenComplete<T>(Task<T> task, string taskName)
{
try { return await task; }
finally { progress?.Report($"Completed {taskName}"); }
}
}
If you also need a count, you can either use IProgress<(int, string)> or change how the report progress string is built to include the count.
So here's what I would do in C# 8 and .NET Standard 2.1:
First, I create the method which will produce the async enumerable:
static async IAsyncEnumerable<bool> TasksToPerform() {
Task[] tasks = new Task[3] { userDetails, getWageInfo, getSalaryInfo };
for (i = 0; i < tasks.Length; i++) {
await tasks[i];
yield return true;
}
}
So now you need to await foreach on this task enumerable. Every time you get a return, you know that a task has been finished.
int numberOfFinishedTasks = 0;
await foreach (var b in TasksToPerform()) {
numberOfFinishedTasks++;
//Update UI here to reflect the finished task number
}
No need to over-complicate this. This code will show how many of your tasks had exceptions. Your await task.whenall just triggers them and waits for them to finish. So after that you can do whatever you want with the tasks :)
var task = Task.Delay(300);
var tasks = new List<Task> { task };
var faultedTasks = 0;
tasks.ForEach(t =>
{
t.ContinueWith(t2 =>
{
//do something with a field / property holding ViewModel state
//that your view is listening to
});
});
await Task.WhenAll(tasks);
//use this to respond with a finished count
tasks.ForEach(_ => { if (_.IsFaulted) faultedTasks++; });
Console.WriteLine($"{tasks.Count() - faultedTasks} / {tasks.Count()} completed.");
.WhenAll() will allow you to determine if /any/ of the tasks failed, they you just count the tasks that have failed.
public async Task<Response> GetAllVasInformationAsync()
{
var userDetails = GetUserDetailsAsync();
var getWageInfo = GetUserWageInfoAsync();
var getSalaryInfo = GetSalaryInfoAsync();
await Task.WhenAll(userDetails, getWaitInfo, getSalaryInfo)
.ContinueWith((task) =>
{
if(task.IsFaulted)
{
int failedCount = 0;
if(userDetails.IsFaulted) failedCount++;
if(getWaitInfo.IsFaulted) failedCount++;
if(getSalaryInfo.IsFaulted) failedCount++;
return $"{failedCount} tasks failed";
}
});
var resultToReturn = new Response
{
IsuserDetailsSucceeded = userDetails.Result,
IsgetWageInfoSucceeded = getWageInfo.Result,
IsgetSalaryInfoSucceeded = getSalaryInfo.Result,
};
return resultToReturn;
}
I used to use this command FindOne to query the result from mongoDB with C# in the past and it can be used at that time. But now I use the same code but it doesn't work. What should I use instead of FindOne?
My code is like this:
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var database = client.GetDatabase("extend");
var collection = database.GetCollection<Entity>("user");
var query = Query<Entity>.EQ(e => e.user_id, int.Parse(targetUser.CurrentUser));
var entity_TargetUser = collection.FindOne(query);
When I try to run it, I got this error
Error CS1061: 'IMongoCollection<Entity>' does not contain a definition for 'FindOne' and no extension method 'FindOne' accepting a first argument of type 'IMongoCollection<Entity>' could be found (are you missing a using directive or an assembly reference?) (CS1061)
What command that I can use instead of FindOne?
You are dealing with an id, so I assume each one is unique - if they aren't, they should be. Assuming user_id is unique then I would do the following
public static class MongoDataService
{
public static async Task<List<BsonDocument>> GetDocumentCollectionAsync(
MongoClient client, FilterDefinition<BsonDocument> filter,
string databaseName, string collectionName, CancellationToken token,
int? limit = null)
{
return await Task.Run(async () =>
{
long i = 1;
List<BsonDocument> items = new List<BsonDocument>();
var collection = GetCollection<BsonDocument>(client, databaseName, collectionName);
using (var cursor = await collection.FindAsync(filter))
{
while (await cursor.MoveNextAsync())
{
var batch = cursor.Current;
foreach (var doc in batch)
{
items.Add(doc);
if (token.IsCancellationRequested || i == limit)
return items;
i++;
}
}
}
return items;
}, token);
}
}
This method with the correct filter will return single documents, or can be used to return batches of documents again according to the imposed filter. Calling this method for your case, you can do
var filterBuilder = Builders<BsonDocument>.Filter;
var filter = filterBuilder.Eq("user_id", int.Parse(targetUser.CurrentUser));
var documents = await MongoDataService.GetDocumentCollectionAsync(client, filter, "extend", "user", token, null);
There are other methods to do what you want, but this should do what you want.
Note, I am assuming you are using the offical MongoDB.Driver.
you can do something like this:
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var database = client.GetDatabase("extend");
var collection = database.GetCollection<Entity>("user");
var query = Query<Entity>.EQ(e.user_id,int.Parse(targetUser.CurrentUser));
var entity_TargetUser = collection.AsQueryable().where(query).single();
or
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var database = client.GetDatabase("extend");
var collection = database.GetCollection<Entity>("user");
var entity_TargetUser = collection.AsQueryable().where(e=>e.user_id ==
int.Parse(targetUser.CurrentUser)).single();
I'm asynchronously retrieving some rss articles with my Portable Class Library that uses the Microsoft.Bcl library (which doesn't have Task.WhenAll). Each article has a url to rss comments that I need to asynchronously retrieve as well.
The code below is my library. I call GetArticles() but it does not return any of the which creates a list of tasks that call GetComments() to asynchronously get the comments.
I've tried using Task.WaitAll in GetArticles to wait for the comments but it does not block the thread. Any help would be appreciated.
private const string ArticlesUri = "";
public async Task<List<ArticleBrief>> GetArticles()
{
var results = new List<ArticleBrief>();
try
{
var wfw = XNamespace.Get("http://wellformedweb.org/CommentAPI/");
var media = XNamespace.Get("http://search.yahoo.com/mrss/");
var dc = XNamespace.Get("http://purl.org/dc/elements/1.1/");
var t = await WebHttpRequestAsync(ArticlesUri);
StringReader stringReader = new StringReader(t);
using (var xmlReader = System.Xml.XmlReader.Create(stringReader))
{
var doc = System.Xml.Linq.XDocument.Load(xmlReader);
results = (from e in doc.Element("rss").Element("channel").Elements("item")
select
new ArticleBrief()
{
Title = e.Element("title").Value,
Description = e.Element("description").Value,
Published = Convert.ToDateTime(e.Element("pubDate").Value),
Url = e.Element("link").Value,
CommentUri = e.Element(wfw + "commentRss").Value,
ThumbnailUri = e.Element(media + "thumbnail").FirstAttribute.Value,
Categories = GetCategoryElements(e.Elements("category")),
Creator = e.Element(dc + "creator").Value
}).ToList();
}
var tasks = new Queue<Task>();
foreach (var result in results)
{
tasks.Enqueue(
Task.Factory.StartNew(async ()=>
{
result.Comments = await GetComments(result.CommentUri);
}
));
}
Task.WaitAll(tasks.ToArray());
}
catch (Exception ex)
{
// should do some other
// logging here. for now pass off
// exception to callback on UI
throw ex;
}
return results;
}
public async Task<List<Comment>> GetComments(string uri)
{
var results = new List<Comment>();
try
{
var wfw = XNamespace.Get("http://wellformedweb.org/CommentAPI/");
var media = XNamespace.Get("http://search.yahoo.com/mrss/");
var dc = XNamespace.Get("http://purl.org/dc/elements/1.1/");
var t = await WebHttpRequestAsync(uri);
StringReader stringReader = new StringReader(t);
using (var xmlReader = System.Xml.XmlReader.Create(stringReader))
{
var doc = System.Xml.Linq.XDocument.Load(xmlReader);
results = (from e in doc.Element("rss").Element("channel").Elements("item")
select
new Comment()
{
Description = e.Element("description").Value,
Published = Convert.ToDateTime(e.Element("pubDate").Value),
Url = e.Element("link").Value,
Creator = e.Element(dc + "creator").Value
}).ToList();
}
}
catch (Exception ex)
{
// should do some other
// logging here. for now pass off
// exception to callback on UI
throw ex;
}
return results;
}
private static async Task<string> WebHttpRequestAsync(string url)
{
//TODO: look into getting
var request = WebRequest.Create(url);
request.Method = "GET";
var response = await request.GetResponseAsync();
return ReadStreamFromResponse(response);
}
private static string ReadStreamFromResponse(WebResponse response)
{
using (Stream responseStream = response.GetResponseStream())
using (StreamReader sr = new StreamReader(responseStream))
{
string strContent = sr.ReadToEnd();
return strContent;
}
}
private List<string> GetCategoryElements(IEnumerable<XElement> categories)
{
var listOfCategories = new List<string>();
foreach (var category in categories)
{
listOfCategories.Add(category.Value);
}
return listOfCategories;
}
Updated Code from Solution, just added .UnWrap() on the Enqueue method:
var tasks = new Queue<Task>();
foreach (var result in results)
{
tasks.Enqueue(
Task.Factory.StartNew(async ()=>
{
result.Comments = await GetComments(result.CommentUri);
}
).Unwrap());
}
Task.WaitAll(tasks.ToArray());
It is waiting appropriately. The problem is that you are creating a Task which creates another task (i.e. StartNew is returning a Task<Task> and you are only waiting on the outer Task which completes rather quickly (it completes before the inner Task is complete)).
The questions will be:
Do you really want that inner task?
If yes, then you can use Task.Unwrap to get a proxy task that represents the completion of both the inner and outer Task and use that to Wait on.
If no, then you could remove the use of async/await in StartNew so that there is not an inner task (I think this would be prefered, it's not clear why you need the inner task).
Do you really need to do a synchronous Wait on an asynchronous Task? Read some of Stephen Cleary's blog: http://blog.stephencleary.com/2012/02/async-unit-tests-part-1-wrong-way.html
As an aside, if you are not using C# 5, then watch out for closing over the foreach variable result See
Has foreach's use of variables been changed in C# 5?, and
http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closing-over-the-loop-variable-considered-harmful.aspx)
In Microsoft.Bcl.Async we couldn't add any static methods to Task. However, you can find most of the methods on TaskEx, for example, TaskEx.WhenAll() does exist.