I have some nested async methods calling each other and it is confusing. I am trying to convert a project which downloads the files in an async download.
On the click of the download button this is the method triggered:
private async void enableOfflineModeToolStripMenuItem_Click(object sender, EventArgs e)
{
for(int i = 0; i < _playlists.Count; i++)
{
DoubleDimList.Add(new List<String>());
for(int j = 0; j < 5; j++)
{
string sMp3 = IniReadValue(_playlists[i], "Track " + j);
DoubleDimList[i].Add(sMp3);
}
await Task.Run(() => _InetGetHTMLSearchAsyncs(DoubleDimList[i]));
}
}
It creates a 2d List which at the end looks like this DoubleDimList[3][20].
At the end of each sublist I am doing an async download as you can see. The method looks like this
private async Task _InetGetHTMLSearchAsyncs(List<string> urlList)
{
foreach (var url in urlList)
{
await Task.Run(() => _InetGetHTMLSearchAsync(url));
}
}
the _InetGetHTMLSearchAsync method looks like this and here is where it gets tricky
private async Task _InetGetHTMLSearchAsync(string sTitle)
{
Runs++;
if (AudioDumpQuery == string.Empty)
{
//return string.Empty;
}
string sResearchURL = "http://www.audiodump.biz/music.html?" + AudioDumpQuery + sTitle.Replace(" ", "+");
try
{
using (var client = new WebClient())
{
client.Headers.Add("Referer", #"http://www.audiodump.com/");
client.Headers.Add("user-agent", "Mozilla / 5.0(Macintosh; Intel Mac OS X 10_9_3) AppleWebKit / 537.75.14(KHTML, like Gecko) Version / 7.0.3 Safari / 7046A194A");
client.DownloadStringCompleted += Client_DownloadStringCompleted;
await Task.Run(() => client.DownloadStringAsync(new Uri(sResearchURL)));
}
}
catch (Exception ex)
{
Console.WriteLine("Debug message: " + ex.Message + "InnerEx: " + ex.StackTrace);
Console.WriteLine("Runs: " + Runs);
return;
}
}
On Client_DownloadStringCompleted there is another async method called. Here it is
private async void Client_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
string[] sStringArray;
string aRet = e.Result;
string[] aTable = _StringBetween(aRet, "<BR><table", "table><BR>", RegexOptions.Singleline);
if (aTable != null)
{
string[] aInfos = _StringBetween(aTable[0], ". <a href=\"", "<a href=\"");
if (aInfos != null)
{
for (int i = 0; i < 1; i++)
{
sStringArray = aInfos[i].Split('*');
sStringArray[0] = sStringArray[0].Replace("'", "'");
aLinks.Add(sStringArray[0]);
}
await Task.Run(() => DownloadFile(aLinks[FilesDownloaded]));
}
}
}
From there, surprise! Another async call.
private async Task DownloadFile(string url)
{
try
{
using (var client = new WebClient())
{
client.Headers.Add("Referer", #"http://www.audiodump.biz/");
client.Headers.Add("user-agent", "Mozilla / 5.0(Macintosh; Intel Mac OS X 10_9_3) AppleWebKit / 537.75.14(KHTML, like Gecko) Version / 7.0.3 Safari / 7046A194A");
client.DownloadFileCompleted += Client_DownloadFileCompleted;
await Task.Run(() => client.DownloadFileTaskAsync(url, mp3Path + "\\" + count + ".mp3"));
}
}
catch (Exception Ex)
{
Console.WriteLine("File download error: " + Ex.StackTrace);
}
}
Now the first part after the creation of the 2d List is to retrieve the download links of the mp3s. The second part is to download the mp3 as soon as a valid URL was provided. It works but in a bizarre way. Instead of downloading the file normally(1st, 2nd, 3rd...), it will download randomly the files(1st, 5th, 8th...).
It is my first go for async download and boy, I am already far from my limits.
Where am I messing this up? And the main question, will this ever work the way it is supposed to work?
Your code looks pretty good, except for two things:
You shouldn't be using Task.Run. The primary use case for Task.Run is for moving CPU-bound work off a GUI thread so it doesn't block the UI. I have a series on the proper use of Task.Run that goes into this in detail.
You should use a consistent asynchronous pattern, ideally TAP. Your code is currently using TAP everywhere except in _InetGetHTMLSearchAsync, which is using EAP. This is what is causing the odd behavior you're seeing.
A fixed _InetGetHTMLSearchAsync would look something like this:
private async Task _InetGetHTMLSearchAsync(string sTitle)
{
Runs++;
string sResearchURL = "http://www.audiodump.biz/music.html?" + AudioDumpQuery + sTitle.Replace(" ", "+");
try
{
using (var client = new WebClient())
{
client.Headers.Add("Referer", #"http://www.audiodump.com/");
client.Headers.Add("user-agent", "Mozilla / 5.0(Macintosh; Intel Mac OS X 10_9_3) AppleWebKit / 537.75.14(KHTML, like Gecko) Version / 7.0.3 Safari / 7046A194A");
string[] sStringArray;
string aRet = await client.DownloadStringTaskAsync(new Uri(sResearchURL));
string[] aTable = _StringBetween(aRet, "<BR><table", "table><BR>", RegexOptions.Singleline);
if (aTable != null)
{
string[] aInfos = _StringBetween(aTable[0], ". <a href=\"", "<a href=\"");
if (aInfos != null)
{
for (int i = 0; i < 1; i++)
{
sStringArray = aInfos[i].Split('*');
sStringArray[0] = sStringArray[0].Replace("'", "'");
aLinks.Add(sStringArray[0]);
}
await DownloadFile(aLinks[FilesDownloaded]); // Should really be called "DownloadFileAsync"
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("Debug message: " + ex.Message + "InnerEx: " + ex.StackTrace);
Console.WriteLine("Runs: " + Runs);
return;
}
}
Related
I'm writing a project, and the part I'm doing now is getting arrow shaped real fast. How can I remove the nested if statements, but still have the same behaviour?
The code below might not look so bad now, but I'm planning on refactoring to include more methods.
public async Task FirstDiffTestAsync()
{
string folderDir = "../../../";
string correctReportDir = folderDir + "Reports To Compare/Testing - Copy.pdf";
string OptyNumber = "122906";
//Making a POST call to generate report
string result = ReportGeneration(OptyNumber).Result;
Response reportResponse = JsonConvert.DeserializeObject<Response>(result);
string newURL = reportResponse.documentUrl;
//Logging the Response to a text file for tracking purposes
await File.WriteAllTextAsync(Context.TestRunDirectory + "/REST_Response.txt", result);
using (StreamWriter w = File.AppendText(Context.TestDir + "/../log.txt"))
{
//Checking if the Integration failed
if (reportResponse.Error == null)
{
//now we have the url, reading in the pdf reports
List<string> Files = new List<string> { correctReportDir, newURL };
List<string> parsedText = PdfToParsedText(Files);
DiffPaneModel diff = InlineDiffBuilder.Diff(parsedText[0], parsedText[1]);
// DiffReport is a customised object
DiffReport diffReport = new DiffReport(correctReportDir, newURL);
diffReport.RunDiffReport(diff);
//In-test Logging
string indent = "\n - ";
string logMsg = $"{indent}Opty Number: {OptyNumber}{indent}Activity Number: {reportResponse.ActivityNumber}{indent}File Name: {reportResponse.FileName}";
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
logMsg += $"{indent}Different lines: {diffReport.insertCounter} Inserted, {diffReport.deleteCounter} Deleted";
}
LogTesting(logMsg, w);
//Writing HTML report conditionally
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
}
Assert.IsTrue(diffReport.insertCounter + diffReport.deleteCounter == 0);
}
else
{
LogTesting($" Integration Failed: {reportResponse.Error}", w);
Assert.IsNull(reportResponse.Error);
}
}
}
As mentioned in the comment, the indentation level is fine for now, but its always better to minimize when possible, especially when you are repeating same blocks of code.
The best way to do this is to write a separate function that contains that block of code and then call that function instead of the nested if statements.
In your case it would be something like this:
private async void checkTotalDiff(diffReport) {
...
}
You could pass anything you might need in the parameters. This way in your main code, you could replace the if statements with checkTotalDiff(diffReport) and save the return (if any) to a variable.
Also note I used void for return but you could change the type depending on what the function returns.
I wouldn't consider this as having an excessive amount of nested if-statements. It is fine as is. Otherwise you could do the following (also suggested by #Caius Jard):
public async Task FirstDiffTestAsync()
{
string folderDir = "../../../";
string correctReportDir = folderDir + "Reports To Compare/Testing - Copy.pdf";
string OptyNumber = "122906";
//Making a POST call to generate report
string result = ReportGeneration(OptyNumber).Result;
Response reportResponse = JsonConvert.DeserializeObject<Response>(result);
//Checking if the Integration failed
if (reportResponse.Error != null)
{
LogTesting($" Integration Failed: {reportResponse.Error}", w);
Assert.IsNull(reportResponse.Error);
return;
}
string newURL = reportResponse.documentUrl;
//Logging the Response to a text file for tracking purposes
await File.WriteAllTextAsync(Context.TestRunDirectory + "/REST_Response.txt", result);
using (StreamWriter w = File.AppendText(Context.TestDir + "/../log.txt"))
{
//now we have the url, reading in the pdf reports
List<string> Files = new List<string> { correctReportDir, newURL };
List<string> parsedText = PdfToParsedText(Files);
DiffPaneModel diff = InlineDiffBuilder.Diff(parsedText[0], parsedText[1]);
// DiffReport is a customised object
DiffReport diffReport = new DiffReport(correctReportDir, newURL);
diffReport.RunDiffReport(diff);
//In-test Logging
string indent = "\n - ";
string logMsg = $"{indent}Opty Number: {OptyNumber}{indent}Activity Number: {reportResponse.ActivityNumber}{indent}File Name: {reportResponse.FileName}";
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
logMsg += $"{indent}Different lines: {diffReport.insertCounter} Inserted, {diffReport.deleteCounter} Deleted";
}
LogTesting(logMsg, w);
//Writing HTML report conditionally
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
}
Assert.IsTrue(diffReport.insertCounter + diffReport.deleteCounter == 0);
}
}
I am using upload function in my desktop application. i want if upload function become unsuccessful then this method retry to upload it. Currently when upload is unsuccessful then it be me error of unhandled exception .i need solution of this
**This is function**
async Task Uploaded()
{
using (var dbx = new DropboxClient(token))
{
bmp = new Bitmap(picboxcapture.Image);
string folder = "/folder/"+Login.recuser+"";
string filename = DateTime.Now.ToString() + " " + " " + MyTodo_Project.rectsk + ".JPG";
string URL = picboxcapture.Image.ToString();
ImageConverter converter = new ImageConverter();
MemoryStream(File.ReadAllBytes(#"C:\Users\home\Downloads\FazalNEwTEst.JPG"));
byte[] bytes = (byte[])converter.ConvertTo(picboxcapture.Image, typeof(byte[]));
var mem = new MemoryStream(bytes);
var updated = dbx.Files.UploadAsync(folder + "/" + filename,
WriteMode.Overwrite.Instance, body: mem);
updated.Wait();
var tx = dbx.Sharing.CreateSharedLinkWithSettingsAsync(folder + "/" + filename);
tx.Wait();
URL = tx.Result.Url;
}
}
**The function call**
private async void save_Load(object sender, EventArgs e)
{
await Uploaded();
}
I want that when upload unsuccessful then it will retry to upload it in page load event . how can
i do this in C#
Just write your own retry handler. Or use Polly (recommended, as it's a mature and very successful library).
However, this is an example of how you might build your own.
Given
public class RetriesExceededException : Exception
{
public RetriesExceededException() { }
public RetriesExceededException(string message) : base(message) { }
public RetriesExceededException(string message, Exception innerException) : base(message, innerException) { }
}
public static async Task RetryOnExceptionAsync(int retryCount, int msDelay, Func<Task> func)
{
for (var i = 0; i < retryCount; i++)
{
try
{
await func.Invoke();
return;
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
if (i == retryCount - 1)
throw new RetriesExceededException($"{retryCount} retries failed", ex);
}
await Task.Delay(msDelay)
.ConfigureAwait(false);
}
}
Usage
await RetryOnExceptionAsync(3, 1000, Uploaded);
I have an odd problem with setting time range for oneshot search. I set time range for my oneshot search, but results are just first found on server matching query. It seems like oneshot is just ignoring time range.
I've read:
How to run searches and jobs using the Splunk SDK for C# -
http://dev.splunk.com/view/csharp-sdk/SP-CAAAEQG
Service.SearchOneShotAsync Method -
http://docs.splunk.com/DocumentationStatic/CshrpSDK/2.1.1/Splunk.Client/html/a5323948-7506-ad15-6f04-7a95b70e616d.htm
JobArgs Class -
http://docs.splunk.com/DocumentationStatic/CshrpSDK/2.1.1/Splunk.Client/html/7dc4e71d-1ed7-4eb1-5a10-183d7663da26.htm
Time modifiers for search -
http://docs.splunk.com/Documentation/Splunk/6.0.3/SearchReference/SearchTimeModifiers
but after hours of tests and experiments - nothing.
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
var connectArgs = new ServiceArgs
{
Host = "myip",
Port = 8089,
Scheme = "https"
};
Splunk.Service service = new Splunk.Service(connectArgs);
service.Login("login", "password");
var oneshotSearchArgs = new Splunk.Client.JobArgs();
oneshotSearchArgs.EarliestTime = "2015-08-23 13:00";//textBoxOD.Text + "T" + textBoxODG.Text + ":00.000";
oneshotSearchArgs.LatestTime = "2015-08-23 14:00";//textBoxDO.Text + "T" + textBoxDOG.Text + ":00.000";
String oneshotSearchQuery = "search query *" + textBox1.text + "* | head 500";
var outArgs = new JobResultsArgs
{
OutputMode = JobResultsArgs.OutputModeEnum.Xml,
Count = 0,
};
try
{
using (var stream = service.Oneshot(oneshotSearchQuery, outArgs))
{
using (var rr = new ResultsReaderXml(stream))
{
string raw = "_raw";
foreach (var #event in rr)
{
wynik += "EVENT:" + Environment.NewLine;
foreach (string key in #event.Keys)
{
if (key.Contains(raw))
{
wynik += " " + key + " -> " + #event[key] + Environment.NewLine + Environment.NewLine;
}
}
}
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
Sorry for being so late with an answer. I landed into the same issue.
The reason that your OneShot search ignores the time range is because it does not take one. (I did not come across any documentation to do so)
To overcome this issue, I tried the 2.X SDK for C#. It fixed the issue.
You can use the 2.X version in 3 steps:
Setting up Service parameters. (host, port etc.)
Authenticating the Service.
Setting up parameters for the OneShot job and executing it.
You can find the NuGet package "Splunk PCL Client for .Net" in NuGet library here.
Here is a sample code:
// Setting up Service parameters
Service _splunkService = new Service(Scheme.Https, "your-api-or-ip", 8089);
// Authenticating
await _splunkService.LogOnAsync("username", "password");
// Setting up parameters for the OneShot job and executing it
var query = "your search query";
var oneShot = new JobArgs();
oneShot.EarliestTime = DateTime.Now.AddMinutes(-2).Date.ToString("yyyy-MM-dd") + "T" + DateTime.Now.AddMinutes(-2).TimeOfDay; //"2015-09-12T12:00:00.000-07:00";
oneShot.LatestTime = "your latest time";
using (var stream = await _splunkService.SearchOneShotAsync(query, 0, oneShot))
{
try
{
foreach (var result in stream)
{
var rawValue = Convert.ToString(result.GetValue("_raw"));
if (rawValue != null)
{
// do something.
}
}
}
}
Make sure the 'await' parts go inside an async method.
What I need to do is be able to cancel a task that is running async.
I have been searching and cannot seem to wrap my head around it. I just cant seem to discern how it would be implemented into my current setup.
Here is my code that fires my task off. Any help on where or how to implement a cancellation token would be greatly appreciated.
private async void startThread()
{
//do ui stuff before starting
ProgressLabel.Text = String.Format("0 / {0} Runs Completed", index.Count());
ProgressBar.Maximum = index.Count();
await ExecuteProcesses();
//sort list of output lines
outputList = outputList.OrderBy(o => o.RunNumber).ToList();
foreach (Output o in outputList)
{
string outStr = o.RunNumber + "," + o.Index;
foreach (double oV in o.Values)
{
outStr += String.Format(",{0}", oV);
}
outputStrings.Add(outStr);
}
string[] csvOut = outputStrings.ToArray();
File.WriteAllLines(settings.OutputFile, csvOut);
//do ui stuff after completing.
ProgressLabel.Text = index.Count() + " runs completed. Output written to file test.csv";
}
private async Task ExecuteProcesses()
{
await Task.Factory.StartNew(() =>
{
int myCount = 0;
int maxRuns = index.Count();
List<string> myStrings = index;
Parallel.ForEach(myStrings,
new ParallelOptions()
{
MaxDegreeOfParallelism = settings.ConcurrentRuns
}, (s) =>
{
//This line gives us our run count.
int myIndex = myStrings.IndexOf(s) + 1;
string newInputFile = Path.Combine(settings.ProjectPath + "files/", Path.GetFileNameWithoutExtension(settings.InputFile) + "." + s + ".inp");
string newRptFile = Path.Combine(settings.ProjectPath + "files/", Path.GetFileNameWithoutExtension(settings.InputFile) + "." + s + ".rpt");
try
{
//load in contents of input file
string[] allLines = File.ReadAllLines(Path.Combine(settings.ProjectPath, settings.InputFile));
string[] indexSplit = s.Split('.');
//change parameters here
int count = 0;
foreach (OptiFile oF in Files)
{
int i = Int32.Parse(indexSplit[count]);
foreach (OptiParam oP in oF.Parameters)
{
string line = allLines[oP.LineNum - 1];
if (oP.DecimalPts == 0)
{
string sExpression = oP.Value;
sExpression = sExpression.Replace("%i", i.ToString());
EqCompiler oCompiler = new EqCompiler(sExpression, true);
oCompiler.Compile();
int iValue = (int)oCompiler.Calculate();
allLines[oP.LineNum - 1] = line.Substring(0, oP.ColumnNum - 1) + iValue.ToString() + line.Substring(oP.ColumnNum + oP.Length);
}
else
{
string sExpression = oP.Value;
sExpression = sExpression.Replace("%i", i.ToString());
EqCompiler oCompiler = new EqCompiler(sExpression, true);
oCompiler.Compile();
double dValue = oCompiler.Calculate();
dValue = Math.Round(dValue, oP.DecimalPts);
allLines[oP.LineNum - 1] = line.Substring(0, oP.ColumnNum - 1) + dValue.ToString() + line.Substring(oP.ColumnNum + oP.Length);
}
}
count++;
}
//write new input file here
File.WriteAllLines(newInputFile, allLines);
}
catch (IOException ex)
{
MessageBox.Show(ex.ToString());
}
var process = new Process();
process.StartInfo = new ProcessStartInfo("swmm5.exe", newInputFile + " " + newRptFile);
process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
process.Start();
process.WaitForExit();
Output output = new Output();
output.RunNumber = myIndex;
output.Index = s;
output.Values = new List<double>();
foreach(OutputValue oV in OutputValues) {
output.Values.Add(oV.getValue(newRptFile));
}
outputList.Add(output);
//get rid of files after run
File.Delete(newInputFile);
File.Delete(newRptFile);
myCount++;
ProgressBar.BeginInvoke(
new Action(() =>
{
ProgressBar.Value = myCount;
}
));
ProgressLabel.BeginInvoke(
new Action(() =>
{
ProgressLabel.Text = String.Format("{0} / {1} Runs Completed", myCount, maxRuns);
}
));
});
});
}
The best way to support cancellation is to pass a CancellationToken to the async method. The button press can then be tied to cancelling the token
class TheClass
{
CancellationTokenSource m_source;
void StartThread() {
m_source = new CancellationTokenSource;
StartThread(m_source.Token);
}
private async void StartThread(CancellationToken token) {
...
}
private void OnCancelClicked(object sender, EventArgs e) {
m_source.Cancel();
}
}
This isn't quite enough though. Both the startThread and StartProcess methods will need to be updated to cooperatively cancel the task once the CancellationToken registers as cancelled
I'm currently working with the Async CTP and need to convert this code into code where I can use Task.WhenAll().
What I did until now was using the UserState object and put my identifier (AID) into it and then use it in the completed event.
However the wc.DownloadFileTaskAsync methode doesn't have an overload with UserState. What can I do?
for (int i = 0; i < SortedRecommendations.Count; i++)
{
string tempfilepath = filepath + SortedRecommendations[i].Aid + ".jpg";
if (File.Exists(tempfilepath))
continue;
WebClient wc = new WebClient();
wc.DownloadFileCompleted += (s, e) =>
{
var q = SortedRecommendations.Where(x => x.Aid == (int)e.UserState);
if (q.Count() > 0)
q.First().Image = tempfilepath;
};
wc.DownloadFileAsync(new Uri(SortedRecommendations[i].Image.Replace("t.jpg", ".jpg")), tempfilepath, SortedRecommendations[i].Aid);
}
This is basically with what I came up with, however I'm getting a out ouf bounds exception at y.Aid == SortedRecommendations[i].Aid because i is now obvioulsy something else then it was when the download started. Only other possibility I see is using something like TaskEx.Run( () => { // download data synchronously }; but I don't like this approach.
for (int i = 0; i < SortedRecommendations.Count; i++)
{
string tempfilepath = filepath + SortedRecommendations[i].Aid + ".jpg";
if (File.Exists(tempfilepath))
continue;
WebClient wc = new WebClient();
wc.DownloadFileCompleted += (s, e) =>
{
var q = SortedRecommendations.Where(x => x.Aid == SortedRecommendations[i].Aid);
if (q.Count() > 0)
q.First().Image = tempfilepath;
};
tasks.Add(wc.DownloadFileTaskAsync(new Uri(SortedRecommendations[i].Image.Replace("t.jpg", ".jpg")), tempfilepath));
}
await TaskEx.WhenAll(tasks);
//Everything finished
First, I think you shouldn't base your logic on ids (unless you really have to). You should use references to the objects in the SortedRecommendations collection.
Now, if you wanted to download only one file at a time, you could simply use await:
for (int i = 0; i < SortedRecommendations.Count; i++)
{
string tempfilepath = filepath + SortedRecommendations[i].Aid + ".jpg";
if (File.Exists(tempfilepath))
continue;
WebClient wc = new WebClient();
var recommendation = SortedRecommendations[i];
await wc.DownloadFileTaskAsync(new Uri(recommendation.Image.Replace("t.jpg", ".jpg")), tempfilepath);
recommendation.Image = tempfilepath;
}
But, if you wanted to start all of the downloads at the same time, like your DownloadFileAsync() code does, you could use ContinueWith() instead. And you don't need user state, that's what closures are for:
for (int i = 0; i < SortedRecommendations.Count; i++)
{
string tempfilepath = filepath + SortedRecommendations[i].Aid + ".jpg";
if (File.Exists(tempfilepath))
continue;
WebClient wc = new WebClient();
var recommendation = SortedRecommendations[i];
var downloadTask = wc.DownloadFileTaskAsync(new Uri(recommendation.Image.Replace("t.jpg", ".jpg")), tempfilepath);
var continuation = downloadTask.ContinueWith(t => recommendation.Image = tempfilepath);
tasks.Add(continuation);
}
await Task.WhenAll(tasks);
The best solution would probably be to download a limited number of files at once, not one or all of them. Doing that is more complicated and one solution would be to use ActionBlock from TPL Dataflow with MaxDegreeOfParallelism set.