I'm trying to get all entries in my table asynchronously but am unable to figure out how to work with the continuation token. I suspect I need to take my anonymous method and convert it to a delegate, then recursively call it with the continuation token.
How do I take the following code and perform an Async call and fetch all entries in the new API?
Task<string[]> GetAllTableEntries(CloudTable tbl, string[] urls, string name, CancellationToken token)
{
TableRequestOptions reqOptions = new TableRequestOptions() { };
OperationContext ctx = new OperationContext() { ClientRequestID = "" };
object state = null;
// Register Cancelation Token
ICancellableAsyncResult result = null;
TableQuery qry = new TableQuery();
TableContinuationToken tok = null;
result = tbl.BeginExecuteQuerySegmented(qry, tok, reqOptions, ctx, (o) =>
{
var response = (o.AsyncState as CloudTable).EndExecuteQuerySegmented(o);
Console.WriteLine("Found " + response.Results.Count + " records");
// The following code was used in the previous version of the SDK
//
//26: // add first segment of data
//27: pageData.CompletedList.AddRange(
//28: from wu in response.Results
//29: select new CompletedWorkUnit(wu));
//30:
//31: // continue fetching segments to complete page
//32: while (response.HasMoreResults)
//33: {
//34: response = response.GetNext();
//35: pageData.CompletedList.AddRange(
//36: from wu in response.Results
//37: select new CompletedWorkUnit(wu));
//38: }
//39:
//40: // set continuation token for next page request
//41: pageData.ContinuationToken = response.ContinuationToken;
//42: evt.Set();
}, state);
// Add cancellation token according to guidance from Table Client 2.0 Breaking Changes blog entry
token.Register((o) => result.Cancel(), state);
Please try this:
static void ExecuteQuery()
{
TableContinuationToken token = null;
TableRequestOptions reqOptions = new TableRequestOptions() { };
OperationContext ctx = new OperationContext() { ClientRequestID = "" };
long totalEntitiesRetrieved = 0;
while (true)
{
CloudTable table = cloudTableClient.GetTableReference("MyTable");
TableQuery<TempEntity> query = (new TableQuery<TempEntity>()).Take(100);
System.Threading.ManualResetEvent evt = new System.Threading.ManualResetEvent(false);
var result = table.BeginExecuteQuerySegmented<TempEntity>(query, token, reqOptions, ctx, (o) =>
{
var response = (o.AsyncState as CloudTable).EndExecuteQuerySegmented<TempEntity>(o);
token = response.ContinuationToken;
int recordsRetrieved = response.Count();
totalEntitiesRetrieved += recordsRetrieved;
Console.WriteLine("Records retrieved in this attempt = " + recordsRetrieved + " | Total records retrieved = " + totalEntitiesRetrieved);
evt.Set();
}, table);
evt.WaitOne();
if (token == null)
{
break;
}
}
}
One thing I noticed is that if I execute a query which returns a dynamic table entity, I'm getting an error related to DateTimeOffset. That's why I ended up creating a temporary entity.
Hope this helps.
Here is a another alternative, but this time in TPL / Task Parallel Library. Full source code is available here
/*
The following overloads of ExecuteQuerySegmentedAsync is executed like this)
*/
CloudTableClient client = acct.CreateCloudTableClient();
CloudTable tableSymmetricKeys = client.GetTableReference("SymmetricKeys5");
TableContinuationToken token = new TableContinuationToken() { };
TableRequestOptions opt = new TableRequestOptions() { };
OperationContext ctx = new OperationContext() { ClientRequestID = "ID" };
CancellationToken cancelToken = new CancellationToken();
List<Task> taskList = new List<Task>();
while (true)
{
Task<TableQuerySegment<DynamicTableEntity>> task3 = tableSymmetricKeys.ExecuteQuerySegmentedAsync(query, token, opt, ctx, cancelToken);
// Run the method
task3.Wait();
token = task3.Result.ContinuationToken;
Console.WriteLine("Records retrieved in this attempt = " + task3.Result.Count ());
if (token == null)
{
break;
}
else
{
// persist token
// token.WriteXml()
}
}
*/
// Overload #4
public static Task<TableQuerySegment<DynamicTableEntity>> ExecuteQuerySegmentedAsync(this CloudTable tbl, TableQuery query, TableContinuationToken continuationToken, TableRequestOptions opt, OperationContext ctx ,CancellationToken token )
{
ICancellableAsyncResult result = null;
if (opt == null && ctx == null)
result = tbl.BeginExecuteQuerySegmented(query, continuationToken, null, tbl);
else
result = tbl.BeginExecuteQuerySegmented(query, continuationToken, opt, ctx, null, tbl);
// Add cancellation token according to guidance from Table Client 2.0 Breaking Changes blog entry
var cancellationRegistration = token.Register(result.Cancel);
return Task.Factory.FromAsync(result, iAsyncResult =>
{
CloudTable currentTable = iAsyncResult.AsyncState as CloudTable;
//cancellationRegistration.Dispose();
return currentTable.EndExecuteQuerySegmented(result);
});
}
This person made some extensions methods for ExecuteQueryAsync that wrap the segmented methods
https://github.com/glueckkanja/tasync/blob/master/StorageExtensions.cs
Here is an alternate implementation of #Gaurav Mantri's code.
public class AzureTableQueryState
{
public CloudTable CloudTable { get; set; }
public TableContinuationToken Token { get; set; }
public ManualResetEvent Evt { get; set; } //i'm guessing that's what this is
//any other variables you were using
public int TotalEntitiesRetrieved { get; set; }
}
// snip....
while (true)
{
// Initialize variables
TableQuery query = (new TableQuery()).Take(100);
AzureTableQueryState queryState = new AzureTableQueryState();
queryState.Evt = new System.Threading.ManualResetEvent(false);
queryState.TotalEntitiesRetrieved = 0;
AsyncCallback asyncCallback = (iAsyncResult) =>
{
AzureTableQueryState state = iAsyncResult.AsyncState as AzureTableQueryState;
var response = state.CloudTable.EndExecuteQuerySegmented(iAsyncResult);
token = response.ContinuationToken;
int recordsRetrieved = response.Results.Count;
state.TotalEntitiesRetrieved += recordsRetrieved;
Console.WriteLine("Records retrieved in this attempt = " + recordsRetrieved + " | Total records retrieved = " + state.TotalEntitiesRetrieved);
state.Evt.Set();
};
// Run the method
var result = tableSymmetricKeys.BeginExecuteQuerySegmented(query, token, opt, ctx, asyncCallback, tableSymmetricKeys);
// Add cancellation token according to guidance from Table Client 2.0 Breaking Changes blog entry
cancelToken.Register((o) => result.Cancel(), null);
queryState.Evt.WaitOne();
if (token == null)
{
break;
}
else
{
// persist token
// token.WriteXml()
}
}
Related
A bit of background, I am looking to replace existing code in a C# App from the existing Microsoft.Azure.Management.Fluent (now deprecated) to the newer Azure.ResourceManager components.
Existing code to copy a database:
public async Task<bool> CopyDb(string? server, string? fromName, string? toName)
{
_log.LogInformation("Connecting to Azure");
var azure = GetAzureObject();
var servers = await azure.SqlServers.ListAsync();
var fromServer = servers.FirstOrDefault(f => server != null && server.Contains(f.Name));
if (fromServer == null)
{
throw new InvalidOperationException("Unable to find original database server");
}
var toNameBackup = $"{toName}-Old";
var existingDbs = await fromServer.Databases.ListAsync();
var fromDB = existingDbs.FirstOrDefault(f => f.Name.Equals(fromName));
if (fromDB == null)
{
throw new InvalidOperationException("Unable to find original database");
}
if (existingDbs.Any(a => a.Name.Equals(toNameBackup, StringComparison.OrdinalIgnoreCase))
&& existingDbs.Any(a => a.Name.Equals(toName, StringComparison.OrdinalIgnoreCase)))
{
_log.LogInformation("Deleting any existing backup called {0}", toNameBackup);
await fromServer.Databases.DeleteAsync(toNameBackup);
}
if (existingDbs.Any(a => a.Name.Equals(toName, StringComparison.OrdinalIgnoreCase)))
{
_log.LogInformation("Renaming target database from {0} to {1} (if exists)", toName, toNameBackup);
await (await fromServer.Databases.GetAsync(toName)).RenameAsync(toNameBackup);
}
_log.LogInformation("Copying database from from {0} to {1}", fromName, toName);
var result = await fromServer.Databases.
Define(toName).
WithSourceDatabase(fromDB).
WithMode(Microsoft.Azure.Management.Sql.Fluent.Models.CreateMode.Copy).CreateAsync();
return result != null;
}
private Microsoft.Azure.Management.Fluent.IAzure GetAzureObject()
{
var clientId = _configuration["AzureClientId"];
var clientSecret = _configuration["AzureClientSecret"];
var tenantId = _configuration["AzureTenantId"];
var subscriptionId = _configuration["AzureSubscriptionId"];
var credentials = Microsoft.Azure.Management.ResourceManager.Fluent.SdkContext.AzureCredentialsFactory.FromServicePrincipal(
clientId: clientId,
clientSecret: clientSecret,
tenantId: tenantId,
environment: Microsoft.Azure.Management.ResourceManager.Fluent.AzureEnvironment.AzureGlobalCloud);
return Microsoft.Azure.Management.Fluent.Azure.Configure().Authenticate(credentials).WithSubscription(subscriptionId);
}
The newer components all work with resources and I've been struggling how to do a couple operations with the newer Azure.ArmClient. I've been able to query with it finding my SQL server and databases. I can even delete some DBs, but I'm unable to work out how to rename or copy databases like the above code. I know there are alternative ways to do this directly in SQL, but I'd prefer to see how to do it in code.
I have had a look around MS docs, I can only find information on the object definitions but no examples.
I have managed to get down to the point of renaming:-
var backupDb = fromServer.GetSqlDatabase(toName);
if (backupDb != null && backupDb.Value != null)
{
// What do I pass in to the definition?
var moveDefinition = new SqlResourceMoveDefinition()
{
// What to set here?
};
await (await backupDb.Value.GetAsync()).Value.RenameAsync(moveDefinition);
}
I'm not sure on how to define the SqlResourceMoveDefinition. I also can't work out at all how to perform the copy like in the older SDK.
Anyone have any guides on how to achieve these operations in C#?
Managed to work it out after eventually working from https://learn.microsoft.com/en-us/dotnet/azure/sdk/resource-management?tabs=PowerShell. There may be better ways to do this, and I'll edit the answer when I find them if others don't by then!
public async Task<bool> CopyDb(string? server, string? fromName, string? toName)
{
_log.LogInformation("Connecting to Azure");
var azure = GetAzureSubscription();
var servers = azure.GetSqlServers().ToList();
var fromServer = servers.SingleOrDefault(f => server != null && f.Data != null && server.Contains(f.Data.Name));
if (fromServer == null)
{
throw new InvalidOperationException("Unable to find original database server");
}
var oldName = $"{toName}-Old";
var databases = fromServer.GetSqlDatabases();
_log.LogInformation("Check for any existing backup called {0}", oldName);
if (await databases.ExistsAsync(oldName))
{
_log.LogInformation("Deleting for any existing backup called {0}", oldName);
var oldBackup = await databases.GetAsync(oldName);
await oldBackup.Value.DeleteAsync(WaitUntil.Completed);
}
_log.LogInformation("Check target database {0} exists", toName, oldName);
if (await databases.ExistsAsync(toName))
{
_log.LogInformation("Renaming target database from {0} to {1}", toName, oldName);
var toDbBackup = await databases.GetAsync(toName);
var resourceIdString = toDbBackup.Value.Data.Id.Parent?.ToString();
var newResourceId = new ResourceIdentifier($"{resourceIdString}/databases/{oldName}");
var moveDefinition = new SqlResourceMoveDefinition(newResourceId);
var toDb = await toDbBackup.Value.GetAsync();
await toDb.Value.RenameAsync(moveDefinition);
}
_log.LogInformation("Copying database from from {0} to {1}", fromName, toName);
var fromDb = await databases.GetAsync(fromName);
var result = await databases.CreateOrUpdateAsync(WaitUntil.Completed, toName, fromDb.Value.Data);
_log.LogInformation("Operation completed!");
return result.HasValue;
}
private SubscriptionResource GetAzureSubscription()
{
var configValue = _configuration["AzureSubscriptionId"];
var subscriptionId = new ResourceIdentifier($"/subscriptions/{configValue}");
return GetAzureArmClient().GetSubscriptionResource(subscriptionId);
}
private ArmClient GetAzureArmClient()
{
var clientId = _configuration["AzureClientId"];
var clientSecret = _configuration["AzureClientSecret"];
var tenantId = _configuration["AzureTenantId"];
var credentials = new ClientSecretCredential(
clientId: clientId,
clientSecret: clientSecret,
tenantId: tenantId);
return new ArmClient(credentials);
}
My code returns Task<List<...>>.
There is a type conversion error from Task<List<...>> to List<...>.
Tell me, please, where I did not finish await ?
public async Task<List<DepartamentsResponse>> Handle(GetDepartmentsRequest token, CancellationToken cancellationToken)
{
var departments = await _departmentApiClient.GetReception(token.accessToken, OdpgDepartmentType.Reception);
var result = departments.ConvertAll(async d => new DepartamentsResponse
{
FederalDistrict = GetFederalDistrictCode(d.FederalRegion.DistrictCode),
SubjectName = d.Name,
Supervisor = GetDirector(d.Users.Where(u => u.InOdpgRole(OdpgUserRole.Director)).FirstOrDefault()),
ContactsSupervisor = GetContacts(d.Users.Where(u => u.InOdpgRole(OdpgUserRole.Director)).FirstOrDefault()),
Schedule = "C 9:00 18:00",
ReceptionContacts = await GetAddressAsync(d.Addresses.FirstOrDefault(d => d.AddressType == DepartmentAddressType.Reception), token)
});
return result;
}
private async Task<string> GetAddressAsync(DepartmentAddressDto? address, GetDepartmentsRequest token)
{
if (address != null)
{
var fullAddress = await _fiasApiClient.GetFullAddress(token.accessToken,
new ESOG.Fias.Api.Model.GetFullAddressRequest
{ BaseAddressId = address.FiasId, Building = address.Building, Room = address.Room });
//var res = JsonConvert.DeserializeObject<DepartmentAddress>(fullAddress);
return fullAddress;
}
return "";
}
GetFederalDistrictCode, GetDirector, GetContacts - these methods are not asynchronous
It should just return a List<>, not Task<List<>>
Your call to departments.ConvertAll is returning a List<Task<DepartamentsResponse>>, but you want a Task<List<DepartamentsResponse>>.
Look at the Task.WhenAll method for how you would convert a collection of Tasks to a single Task that returns a collection. Then await that single Task and build your final list.
I have written an app that goes through our own properties and scraps the data. To make sure I don't run through the same URLs, I am using a MySQL database to store the URL, flag it once its processed. All this was being done in a single thread and it's fine if I had only few thousand entries. But I have few hundred thousand entries that I need to parse so I need to make changes in the code (I am newbie in multithreading in general). I found an example and was trying to copy the style but doesn't seem to work. Anyone know what the issue is with the following code?
EDIT: Sorry didn't mean to make people guess the issue but was stupid of me to include the exception. Here is the exception
"System.InValidCastException: 'Specified cast is not valid.'"
When I start the process it collects the URLs from the database and then never hits DoWork method
//This will get the entries from the database
List<Mappings> items = bot.GetUrlsToProcess(100);
if (items != null)
{
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
Worker.Done = new Worker.DoneDelegate(WorkerDone);
foreach (var item in items)
{
urls.Add(item.Url);
WaitingTasks.Enqueue(new Task(id => new Worker().DoWork((int)id, item.Url, token), item.Url, token));
}
LaunchTasks();
}
static async void LaunchTasks()
{
// keep checking until we're done
while ((WaitingTasks.Count > 0) || (RunningTasks.Count > 0))
{
// launch tasks when there's room
while ((WaitingTasks.Count > 0) && (RunningTasks.Count < MaxRunningTasks))
{
Task task = WaitingTasks.Dequeue();
lock (RunningTasks) RunningTasks.Add((int)task.AsyncState, task);
task.Start();
}
UpdateConsole();
await Task.Delay(300); // wait before checking again
}
UpdateConsole(); // all done
}
static void UpdateConsole()
{
Console.Write(string.Format("\rwaiting: {0,3:##0} running: {1,3:##0} ", WaitingTasks.Count, RunningTasks.Count));
}
static void WorkerDone(int id)
{
lock (RunningTasks) RunningTasks.Remove(id);
}
public class Worker
{
public delegate void DoneDelegate(int taskId);
public static DoneDelegate Done { private get; set; }
public async void DoWork(object id, string url, CancellationToken token)
{
if (token.IsCancellationRequested) return;
Content obj;
try
{
int tries = 0;
bool IsUrlProcessed = true;
DateTime dtStart = DateTime.Now;
string articleDate = string.Empty;
try
{
ScrapeWeb bot = new ScrapeWeb();
SearchApi searchApi = new SearchApi();
SearchHits searchHits = searchApi.Url(url, 5, 0);
if (searchHits.Hits.Count() == 0)
{
obj = await bot.ReturnArticleObject(url);
if (obj.Code != HttpStatusCode.OK)
{
Console.WriteLine(string.Format("\r Status is {0}", obj.Code));
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.HttpCode = obj.Code;
}
else
{
string title = obj.Title;
string content = obj.Contents;
string description = obj.Description;
Articles article = new Articles();
article.Site = url.GetSite();
article.Content = content;
article.Title = title;
article.Url = url.ToLower();
article.Description = description;
string strThumbNail = HtmlHelper.GetImageUrl(url, obj.RawResponse);
article.Author = HtmlHelper.GetAuthor(url, obj.RawResponse);
if (!string.IsNullOrEmpty(strThumbNail))
{
//This condition needs to be added to remove ?n=<number> from EP thumbnails
if (strThumbNail.Contains("?"))
{
article.ImageUrl = strThumbNail.Substring(0, strThumbNail.IndexOf("?")).Replace("http:", "https:");
}
else
article.ImageUrl = strThumbNail.Replace("http:", "https:");
}
else
{
article.ImageUrl = string.IsNullOrEmpty(strThumbNail) ? article.Url.GetDefaultImageUrls() : strThumbNail.Replace("http:", "https:");
}
articleDate = HtmlHelper.GetPublishDate(url, obj.RawResponse);
if (string.IsNullOrEmpty(articleDate))
article.Pubdate = DateTime.Now;
else
article.Pubdate = DateTime.Parse(articleDate);
var client = new Index(searchApi);
var result = client.Upsert(article);
itemfound.HttpCode = obj.Code;
if (result)
{
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate);
UpdateItem(itemfound);
}
else
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
UpdateItem(itemfound, tries, IsUrlProcessed);
}
}
}
else
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = true;
itemfound.HttpCode = HttpStatusCode.OK;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
}
}
catch (Exception e)
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
}
finally
{
DateTime dtEnd = DateTime.Now;
Console.WriteLine(string.Format("\r Total time taken to process items is {0}", (dtEnd - dtStart).TotalSeconds));
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
Done((int)id);
}
}
All this code is based from Best multi-thread approach for multiple web requests this link. Can someone tell me how to get this approach running?
I think the problem is in the way you're creating your tasks:
new Task(id => new Worker().DoWork((int)id, item.Url, token), item.Url, token)
This Task constructor overload expected Action<object> delegate. That means id will be typed as object and you need to cast it back to something useful first.
Parameters
action
Type: System.Action<Object>
The delegate that represents the code to execute in the task.
state
Type: System.Object
An object representing data to be used by the action.
cancellationToken
Type: System.Threading.CancellationToken
-The CancellationToken that that the new task will observe.
You decided to cast it to int by calling (int)id, but you're passing item.Url as the object itself. I can't tell you 100% what the type of Url is but I don't expect Url-named property to be of type int.
Based on what #MarcinJuraszek said I just went back to my code and added an int as I couldn't find another way to resolve it. Here is the change I made
int i=0
foreach (var item in items)
{
urls.Add(item.Url);
WaitingTasks.Enqueue(new Task(id => new Worker().DoWork((string)id, item.Url, token), item.Url, token));
i++;
}
I am writing below code to retrieve all entities from Azure table. But I am kind of stuck up in passing entity resolver delegate. I could not find much reference on MSDN.
Can some one please point out, how to use EntityResover in below code?
public class ATSHelper<T> where T : ITableEntity, new()
{
CloudStorageAccount storageAccount;
public ATSHelper(CloudStorageAccount storageAccount)
{
this.storageAccount = storageAccount;
}
public async Task<IEnumerable<T>> FetchAllEntities(string tableName)
{
List<T> allEntities = new List<T>();
CloudTable table = storageAccount.CreateCloudTableClient().GetTableReference(tableName);
TableContinuationToken contToken = new TableContinuationToken();
TableQuery query = new TableQuery();
CancellationToken cancelToken = new CancellationToken();
do
{
var qryResp = await table.ExecuteQuerySegmentedAsync<T>(query, ???? EntityResolver ???? ,contToken, cancelToken);
contToken = qryResp.ContinuationToken;
allEntities.AddRange(qryResp.Results);
}
while (contToken != null);
return allEntities;
}
}
Here is a nice article describing the Table Storage in deep. It also includes couple of samples for EntityResolver.
Ideal would be to have one Generic Resolver, that produces the desired result. Then you can include it in your call. I will just quote here one example from the provided article:
EntityResolver<ShapeEntity> shapeResolver = (pk, rk, ts, props, etag) =>
{
ShapeEntity resolvedEntity = null;
string shapeType = props["ShapeType"].StringValue;
if (shapeType == "Rectangle") { resolvedEntity = new RectangleEntity(); }
else if (shapeType == "Ellipse") { resolvedEntity = new EllipseEntity(); }
else if (shapeType == "Line") { resolvedEntity = new LineEntity(); }
// Potentially throw here if an unknown shape is detected
resolvedEntity.PartitionKey = pk;
resolvedEntity.RowKey = rk;
resolvedEntity.Timestamp = ts;
resolvedEntity.ETag = etag;
resolvedEntity.ReadEntity(props, null);
return resolvedEntity;
};
currentSegment = await drawingTable.ExecuteQuerySegmentedAsync(drawingQuery, shapeResolver, currentSegment != null ? currentSegment.ContinuationToken : null);
Read the full article to better understand the deal with resolvers.
Yesterday I've found out how to create several async http requests without async/await. But today I need to do it in a loop: if some of responses don't satisfy some condition - I need to change a request for them and send these requests again. It may be repeated several times.
I've tried this code:
do
{
var loadingCoordinatesTasks = new List<Task<Terminal>>();
var totalCountOfTerminals = terminalPresetNode.ChildNodes.Count;
var uiTaskScheduler = TaskScheduler.FromCurrentSynchronizationContext();
foreach (var terminal in terminals.Except(_terminalsWithCoordinates))
{
var address = terminal.GetNextAddress();
var webRequest = (HttpWebRequest)WebRequest.Create(GeoCoder.GeoCodeUrl + address);
var webRequestTask = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse,
webRequest.EndGetResponse,
terminal);
var parsingTask = webRequestTask.ContinueWith(antecedent =>
{
// Parse the response
});
loadingCoordinatesTasks.Add(parsingTask);
}
Task.Factory.ContinueWhenAll(loadingCoordinatesTasks.ToArray(), antecedents =>
{
foreach (var antecedent in antecedents)
{
var terminalWithCoordinates = antecedent.Result;
if (antecedent.Status == TaskStatus.RanToCompletion &&
!terminalWithCoordinates.Coordinates.AreUnknown)
{
_terminalsWithCoordinates.Add(terminalWithCoordinates);
_countOfProcessedTerminals++;
}
}
});
} while (_countOfProcessedTerminals < totalCountOfTerminals);
but is it possible to check the condition in while just after every single set of requests executed?
You can perform the check after increasing the count:
_countOfProcessedTerminals++;
if (_countOfProcessedTerminals >= totalCountOfTerminals)
{
break;
}
Is _countOfProcessedTerminals thread-safe though?
I manage to do it using recursion:
public void RunAgainFailedTasks(IEnumerable<Task<Terminal>> tasks)
{
Task.Factory.ContinueWhenAll(tasks.ToArray(), antecedents =>
{
var failedTasks = new List<Task<Terminal>>();
foreach (var antecedent in antecedents)
{
var terminal = antecedent.Result;
// Previous request was failed
if (terminal.Coordinates.AreUnknown)
{
string address;
try
{
address = terminal.GetNextAddress();
}
catch (FormatException) // No versions more
{
continue;
}
var getCoordinatesTask = CreateGetCoordinatesTask(terminal, address);
failedTasks.Add(getCoordinatesTask);
}
else
{
_terminalsWithCoordinates.Add(terminal);
}
}
if (failedTasks.Any())
{
RunAgainFailedTasks(failedTasks);
}
else
{
// Display a map
}
}, CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
private Task<Terminal> CreateGetCoordinatesTask(Terminal terminal, string address)
{
var webRequest = (HttpWebRequest)WebRequest.Create(GeoCoder.GeoCodeUrl + address);
webRequest.KeepAlive = false;
webRequest.ProtocolVersion = HttpVersion.Version10;
var webRequestTask = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse,
webRequest.EndGetResponse,
terminal);
var parsingTask = webRequestTask.ContinueWith(webReqTask =>
{
// Parse the response
});
return parsingTask;
}