Develop Reference

How can I cancel an asynchronous task after a given time and how can I restart a failed task?

How can I cancel an asynchronous task when it takes very long to complete it or if it will probably never complete? Is it possible to use a given time(for example 10 seconds) for each task and when it doesn't complete in this given time, then the task will automatically be cancelled?
Is it possible to restart a task or create the same task again after it failed? What can I do if one of the tasks in a task list fails? Is it possible to only restart the failed task?
In my code, playerCountryDataUpdate should only be executed after each task in TasksList1 completed without error or exception. I want to restart a task when it fails. When the same task fails again, then don't restart it and display an error message on the screen. How can I do that?
bool AllMethods1Completed = false;
bool AllMethods2Completed = false;
public async Task PlayerAccountDetails()
{
var playerCountryDataGet = GetPlayerCountryData();
var playerTagsData = GetPlayerTagsData();
var TasksList1 = new List<Task> { playerCountryDataGet, playerTagsData };
try
{
await Task.WhenAll(TasksList1);
AllMethods1Completed = true;
}
catch
{
AllMethods1Completed = false;
}
if (AllMethods1Completed == true)
{
var playerCountryDataUpdate = UpdatePlayerCountryData("Germany", "Berlin");
var TasksList2 = new List<Task> { playerCountryDataUpdate };
try
{
await Task.WhenAll(TasksList2);
AllMethods2Completed = true;
}
catch
{
AllMethods2Completed = false;
}
}
}
private async Task GetPlayerTagsData()
{
var resultprofile = await PlayFabServerAPI.GetPlayerTagsAsync(new PlayFab.ServerModels.GetPlayerTagsRequest()
{
PlayFabId = PlayerPlayFabID
});
if (resultprofile.Error != null)
Console.WriteLine(resultprofile.Error.GenerateErrorReport());
else
{
if ((resultprofile.Result != null) && (resultprofile.Result.Tags.Count() > 0))
PlayerTag = resultprofile.Result.Tags[0].ToString();
}
}
private async Task GetPlayerCountryData()
{
var resultprofile = await PlayFabClientAPI.GetUserDataAsync(new PlayFab.ClientModels.GetUserDataRequest()
{
PlayFabId = PlayerPlayFabID,
Keys = null
});
if (resultprofile.Error != null)
Console.WriteLine(resultprofile.Error.GenerateErrorReport());
else
{
if (resultprofile.Result.Data == null || !resultprofile.Result.Data.ContainsKey("Country") || !resultprofile.Result.Data.ContainsKey("City"))
Console.WriteLine("No Country/City");
else
{
PlayerCountry = resultprofile.Result.Data["Country"].Value;
PlayerCity = resultprofile.Result.Data["City"].Value;
}
}
}
private async Task UpdatePlayerCountryData(string country, string city)
{
var resultprofile = await PlayFabClientAPI.UpdateUserDataAsync(new PlayFab.ClientModels.UpdateUserDataRequest()
{
Data = new Dictionary<string, string>() {
{"Country", country},
{"City", city}
},
Permission = PlayFab.ClientModels.UserDataPermission.Public
});
if (resultprofile.Error != null)
Console.WriteLine(resultprofile.Error.GenerateErrorReport());
else
Console.WriteLine("Successfully updated user data");
}

You need to build a cancellation mechanism directly into the task itself. C# provides a CancellationTokenSource and CancellationToken classes to assist with this. https://learn.microsoft.com/en-us/dotnet/api/system.threading.cancellationtoken?view=netcore-3.1
Add an (optional) CancellationToken to your task's parameters. Then check the token at appropriate intervals to determine if the task needs to abort before it completes.
In the case of a long running query, it would be best to figure out how to break the query into chunks and then check the CancellationToken between queries.
private async Task GetPlayerXXXData(CancellationToken ct = null) {
int limit = 100;
int total = Server.GetPlayerXXXCount();
List<PlayerXXXData> results = new List<PlayerXXXData>();
while((ct == null || ct.IsCancellationRequested) && result.Count < total) {
result.AddRange(Server.GetPlayerXXXData(result.Count, limit));
}
return results;
}
Mind the above has no error handling in it; but you get the idea. You might consider making it faster (to start using the data) by implementing Deferred Execution with your own custom IEnumerable implementation. Then you can query one chunk and iterate over that chunk before querying for the next chunk. This could also help prevent you from loading too much into RAM - depending upon the number of records you are intending to process.

set timeout in your logic to suspend the task:
int timeout = 1000;
var task = SomeOperationAsync();
if (await Task.WhenAny(task, Task.Delay(timeout)) == task) {
// task completed within timeout
} else {
// timeout logic
}
Asynchronously wait for Task<T> to complete with timeout
and also put try catch blocks in a while loop with a flag until you want to retry
var retry=0;
while (retry<=3)
{
try{
await with timeout
raise timeout exception
}
catch(catch timeout exception here )
{
retry++;
if(retry ==3)
{
throw the catched exception here
}
}
}

Multithreaded c# console app to scrape data from sites

I have written an app that goes through our own properties and scraps the data. To make sure I don't run through the same URLs, I am using a MySQL database to store the URL, flag it once its processed. All this was being done in a single thread and it's fine if I had only few thousand entries. But I have few hundred thousand entries that I need to parse so I need to make changes in the code (I am newbie in multithreading in general). I found an example and was trying to copy the style but doesn't seem to work. Anyone know what the issue is with the following code?
EDIT: Sorry didn't mean to make people guess the issue but was stupid of me to include the exception. Here is the exception
"System.InValidCastException: 'Specified cast is not valid.'"
When I start the process it collects the URLs from the database and then never hits DoWork method
//This will get the entries from the database
List<Mappings> items = bot.GetUrlsToProcess(100);
if (items != null)
{
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
Worker.Done = new Worker.DoneDelegate(WorkerDone);
foreach (var item in items)
{
urls.Add(item.Url);
WaitingTasks.Enqueue(new Task(id => new Worker().DoWork((int)id, item.Url, token), item.Url, token));
}
LaunchTasks();
}
static async void LaunchTasks()
{
// keep checking until we're done
while ((WaitingTasks.Count > 0) || (RunningTasks.Count > 0))
{
// launch tasks when there's room
while ((WaitingTasks.Count > 0) && (RunningTasks.Count < MaxRunningTasks))
{
Task task = WaitingTasks.Dequeue();
lock (RunningTasks) RunningTasks.Add((int)task.AsyncState, task);
task.Start();
}
UpdateConsole();
await Task.Delay(300); // wait before checking again
}
UpdateConsole(); // all done
}
static void UpdateConsole()
{
Console.Write(string.Format("\rwaiting: {0,3:##0} running: {1,3:##0} ", WaitingTasks.Count, RunningTasks.Count));
}
static void WorkerDone(int id)
{
lock (RunningTasks) RunningTasks.Remove(id);
}
public class Worker
{
public delegate void DoneDelegate(int taskId);
public static DoneDelegate Done { private get; set; }
public async void DoWork(object id, string url, CancellationToken token)
{
if (token.IsCancellationRequested) return;
Content obj;
try
{
int tries = 0;
bool IsUrlProcessed = true;
DateTime dtStart = DateTime.Now;
string articleDate = string.Empty;
try
{
ScrapeWeb bot = new ScrapeWeb();
SearchApi searchApi = new SearchApi();
SearchHits searchHits = searchApi.Url(url, 5, 0);
if (searchHits.Hits.Count() == 0)
{
obj = await bot.ReturnArticleObject(url);
if (obj.Code != HttpStatusCode.OK)
{
Console.WriteLine(string.Format("\r Status is {0}", obj.Code));
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.HttpCode = obj.Code;
}
else
{
string title = obj.Title;
string content = obj.Contents;
string description = obj.Description;
Articles article = new Articles();
article.Site = url.GetSite();
article.Content = content;
article.Title = title;
article.Url = url.ToLower();
article.Description = description;
string strThumbNail = HtmlHelper.GetImageUrl(url, obj.RawResponse);
article.Author = HtmlHelper.GetAuthor(url, obj.RawResponse);
if (!string.IsNullOrEmpty(strThumbNail))
{
//This condition needs to be added to remove ?n=<number> from EP thumbnails
if (strThumbNail.Contains("?"))
{
article.ImageUrl = strThumbNail.Substring(0, strThumbNail.IndexOf("?")).Replace("http:", "https:");
}
else
article.ImageUrl = strThumbNail.Replace("http:", "https:");
}
else
{
article.ImageUrl = string.IsNullOrEmpty(strThumbNail) ? article.Url.GetDefaultImageUrls() : strThumbNail.Replace("http:", "https:");
}
articleDate = HtmlHelper.GetPublishDate(url, obj.RawResponse);
if (string.IsNullOrEmpty(articleDate))
article.Pubdate = DateTime.Now;
else
article.Pubdate = DateTime.Parse(articleDate);
var client = new Index(searchApi);
var result = client.Upsert(article);
itemfound.HttpCode = obj.Code;
if (result)
{
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate);
UpdateItem(itemfound);
}
else
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
UpdateItem(itemfound, tries, IsUrlProcessed);
}
}
}
else
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = true;
itemfound.HttpCode = HttpStatusCode.OK;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
}
}
catch (Exception e)
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
}
finally
{
DateTime dtEnd = DateTime.Now;
Console.WriteLine(string.Format("\r Total time taken to process items is {0}", (dtEnd - dtStart).TotalSeconds));
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
Done((int)id);
}
}
All this code is based from Best multi-thread approach for multiple web requests this link. Can someone tell me how to get this approach running?

I think the problem is in the way you're creating your tasks:
new Task(id => new Worker().DoWork((int)id, item.Url, token), item.Url, token)
This Task constructor overload expected Action<object> delegate. That means id will be typed as object and you need to cast it back to something useful first.
Parameters
action
Type: System.Action<Object>
The delegate that represents the code to execute in the task.
state
Type: System.Object
An object representing data to be used by the action.
cancellationToken
Type: System.Threading.CancellationToken
-The CancellationToken that that the new task will observe.
You decided to cast it to int by calling (int)id, but you're passing item.Url as the object itself. I can't tell you 100% what the type of Url is but I don't expect Url-named property to be of type int.

Based on what #MarcinJuraszek said I just went back to my code and added an int as I couldn't find another way to resolve it. Here is the change I made
int i=0
foreach (var item in items)
{
urls.Add(item.Url);
WaitingTasks.Enqueue(new Task(id => new Worker().DoWork((string)id, item.Url, token), item.Url, token));
i++;
}

How do I wait for a delegate to return a certain value?

I am currently using Selenium WebDriverWait to wait for things to happend where I don't need the IWebDriver functionality. My code looks like this:
public static T WaitForNotNull<T>(this IWebDriver driver, Func<T> func)
{
var result = default(T);
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
wait.Until(d => (result = func()) != null);
return result;
}
public static void WaitForNull<T>(this IWebDriver driver, Func<T> func)
{
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
wait.Until(d => func() == null);
}
Is there a similar construct in .Net which I can use instead of WebDriverWait?

The answer is
No
There is no such thing in the .NET Framework, you will have to write such a method yourself.

This is the original Until implementation (source). You can improve it IMO.
And if you don't want to block the calling thread (in case of UI thread), you can easily use the async\await pattern
public TResult Until<TResult>(Func<T, TResult> condition)
{
if (condition == null)
{
throw new ArgumentNullException("condition", "condition cannot be null");
}
var resultType = typeof(TResult);
if ((resultType.IsValueType && resultType != typeof(bool)) || !typeof(object).IsAssignableFrom(resultType))
{
throw new ArgumentException("Can only wait on an object or boolean response, tried to use type: " + resultType.ToString(), "condition");
}
Exception lastException = null;
var endTime = this.clock.LaterBy(this.timeout);
while (true)
{
try
{
var result = condition(this.input);
if (resultType == typeof(bool))
{
var boolResult = result as bool?;
if (boolResult.HasValue && boolResult.Value)
{
return result;
}
}
else
{
if (result != null)
{
return result;
}
}
}
catch (Exception ex)
{
if (!this.IsIgnoredException(ex))
{
throw;
}
lastException = ex;
}
// Check the timeout after evaluating the function to ensure conditions
// with a zero timeout can succeed.
if (!this.clock.IsNowBefore(endTime))
{
string timeoutMessage = string.Format(CultureInfo.InvariantCulture, "Timed out after {0} seconds", this.timeout.TotalSeconds);
if (!string.IsNullOrEmpty(this.message))
{
timeoutMessage += ": " + this.message;
}
this.ThrowTimeoutException(timeoutMessage, lastException);
}
Thread.Sleep(this.sleepInterval);
}
}

Generic method throws COM exception on return

I have an application using the office interop libraries (Word and Excel 2007) library version 12.0 and I have a generic method for executing tasks with a timeout in case Office stops responding for some reason but when the method returns its value it throws an exception stating the Com object can't be used when separated from its rcw.
Method:
public static R ExecuteOfficeTimoutAction<R>(Func<R> ToExecute, TimeSpan TimeOut)
{
object returnVal = new OfficeEmptyReturn(); // Empty Class Used With Reflection To Tell If There Was Really A Value Returned
try
{
Exception threadError = null;
System.Threading.Thread timedThread = new System.Threading.Thread(() =>
{
try
{
returnVal = ToExecute();
}
catch (Exception ex)
{
threadError = ex;
}
});
timedThread.Priority = System.Threading.ThreadPriority.Normal;
timedThread.SetApartmentState(System.Threading.ApartmentState.STA);
timedThread.Start();
timedThread.Join(TimeOut);
if (threadError != null)
{
throw threadError;
}
if (timedThread.ThreadState != System.Threading.ThreadState.Stopped)
{
bool abortCalled = false;
bool processKilled = false;
int waitTime = 0;
do
{
if (!abortCalled)
{
timedThread.Abort();
abortCalled = true;
}
if (waitTime < 2500)
{
waitTime += 250;
System.Threading.Thread.Sleep(250);
}
else if (!processKilled)
{
processKilled = true;
foreach (System.Diagnostics.Process process in System.Diagnostics.Process.GetProcesses())
{
if (process.ProcessName.ToLower().Contains("winword.exe") || process.ProcessName.ToLower().Contains("excel.exe"))
{
process.Close();
}
}
}
else
{
throw new TimeoutException("The Timeout Action Could Not Be Terminated");
}
}
while (!(timedThread.ThreadState == System.Threading.ThreadState.Aborted || timedThread.ThreadState == System.Threading.ThreadState.Stopped));
}
}
catch (Exception)
{
throw;
}
return (R)returnVal; // Throws Exception Here
}
Method Where It Throws An Error:
public Document GetDocument(string FilePath)
{
Document newDoc = null;
if (!String.IsNullOrWhiteSpace(FilePath))
{
if (File.Exists(FilePath) && IsWordDoc(new FileInfo(FilePath)))
{
if (IsActive)
{
if (FilePath.Length < 255)
{
newDoc = ExecuteOfficeTimoutAction<Application, Document>((Application app) =>
{
Document tempDoc = app.Documents.Open(FilePath, Type.Missing, true);
return tempDoc;
}, App, new TimeSpan(0, 0, 10));
if (newDoc == null || newDoc is OfficeEmptyReturn)
{
newDoc = null;
}
}
else
{
throw new ArgumentOutOfRangeException("The Word Document's Full Path Must Be Shorter Than 255 Characters To Be Reached");
}
}
else
{
throw new InvalidOperationException("The Word Application Must Be Open To Process Word Data");
}
}
else
{
throw new FileNotFoundException("The Word File Did Not Exist At The Specified Path");
}
}
else
{
throw new ArgumentNullException("The File Path Was Null");
}
return newDoc;
}
The App object being passed in is a static Word.Application object stored in a field within the containing word class so that it is visible to all applicable methods and disposed of appropriately when no longer needed.
I don't see why it would be giving this exception when returning a value, even if it is a com object.
My desired outcome is for the method to return the value generated by the passed Func, throw the exception generated by said delegate or return OfficeEmptyReturn.
Any help or pointers with this would be greatly appreciated.
Side note, I do have fallback error handling for when the office application is terminated outside the application automating it, I would open the task manager and manually terminate word when it froze my code then it would continue but that is not an acceptable outcome.

Selenium Webdriver wait on element click?

I have been searching for a solution for this, but to no avail. I have a button I'm clicking, that is sometimes taking a long while to return data, and the driver is timing out and just killing the app I guess.
I am trying to use the WebDriverWait class to accomplish this, but the Click() method is not available in the way I'm using it.
WebDriverWait wait = new WebDriverWait(browser, new TimeSpan(0, 5, 0));
bool clicked = wait.Until<bool>((elem) =>
{
elem.Click(); //Doesn't Work
return true;
});
The ImplicitlyWait() method is only for waiting for elements to load, but this times out on Click(), so it can't even look for an element.
The SetScriptTimeout() method just works with executing javascript, which I'm not doing.
Does anyone know of a way to do this?

try this :
WebDriverWait wait = new WebDriverWait(driver , 1000) ;
wait.until(ExcepctedConditions.elementToBeClickable(ById("element"));
Element can be ID of any element present on the next page you are redirected to .
Once Page loads fully then it will start executing your code .

Instead of Click you could try to use SendKeys. Unlike Click, SendKeys does not wait for the page to finish loading before resuming code execution. So you can do something like this:
WebDriverWait wait = new WebDriverWait(browser, new TimeSpan(0, 5, 0));
elem.SendKeys(Keys.Enter);
wait.Until<bool>((_driver) =>
{
//Check here if results have loaded yet
return true;
});
As a side note, I'm pretty sure Until takes in a IWebBrowser as an input, not an element, which is why you can't click on elem.

In addition to prestomanifesto's solution I can offer a less than ideal solution to that I implemented to solve this issue. It turns out it is throwing an exception - No Response etc... - so I just surrounded it in a try catch then waited for the popup to close, which seems to work fine.
You can substitute whatever you want in your loop, just make sure to put a counter in so it won't loop forever.
try
{
element.Click();
}
catch
{
cnt++;
do
{
//wait for whatever
cnt++;
Thread.Sleep(1000);
// Wait for 30 seconds for popup to close
} while (!string.IsNullOrEmpty(browser.CurrentWindowHandle) && cnt < 30);
}

I use this script :
private static void waitUntilScriptFoundAndExecute(String script) {
int tries = 0;
boolean found = false;
do {
tries++;
try {
driver.executeScript(script);
found = true;
} catch (NoSuchElementException nse) {
System.out.println("Wait for script NSE (" + tries + ")");
} catch (WebDriverException wde) {
System.out.println("Wait for script WDE (" + tries + ")");
} catch (Exception e) {
System.out.println("Wait for script E (" + tries + ")");
}
// Waiting
if (!found) {
System.out.println("Wait for script Not found (" + tries + ")");
waiting(SCRIPT_WAITING_INTERVAL);
}
} while (!found && tries < MAX_SCRIPT_WAIT_TRIES);
if (!found) {
System.out.println("Script aborted: " + script);
}
}

RepeatUntil Extension Method using LINQ Lambda expressions
Copy this code to your project:
public static class SeleniumExtensionMethods
{
public static IWebElement RepeatUntil<T>(this T obj,
Func<T, IEnumerable<IWebElement>> func,
Func<IWebElement, bool> compare,
int MaxRetry = 20)
{
//call function to get elements
var eles = func(obj);
IWebElement element = null;
while (element == null && MaxRetry > 0)
{
MaxRetry-=1;
//call the iterator
element = IterateCollection(compare, eles);
if (element == null)
{
Thread.Sleep(500);
//get new collection of elements
eles = func(obj);
}
};
return element;
}
private static IWebElement IterateCollection(
Func<IWebElement, bool> compare,
IEnumerable<IWebElement> eles){
IWebElement element = null;
eles.ToList().ForEach(
ele =>
{
//call the comparator
var found = compare(ele);
if (found) element = ele;
});
return element;
}
}
Call it using this syntax:
// You can change PageObjectType to IWebDriver or IWebElement so that
// cb is of any type.
var element = cb.RepeatUntil<MyPageObjectType>(
//This is the first function to provide the elements
p => p.FindElements(By.ClassName("TreeNode")),
//This is the comparator
ele => ele.Text == nodeText && ele.Location.Y>YLocation);
Note: In the example above we are passing in a PageObjectType but you could change this to be of type IWebDriver or event IWebElement. All the type parameter does it allow you to use this as an extension method for the type you specify.
Note the flexibility of the extension method in that the caller can determine the collection as well as the comparator.

don't use thread sleep
public class(IWebDriver driver)
{
this.driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromMinutes(1);
wait = new OpenQA.Selenium.Support.UI.WebDriverWait(driver,TimeSpan.FromMinutes(1));
}
public void class1()
{
wait.Until(ExpectedConditions.ElementToBeClickable(elem)).Click();
}