Can I speed up my code for downloading XML pages? - c#

I have a program that needs to download ~200 XML pages of ~250 lines. It does this by grabbing a list of ID's from a main page, and then it then iterates through each ID, inserting it into a URL and using a StreamWriter to write the XML page to a file. The methods look like this:
private static void XMLUpdate(string path)
{
try
{
//create client for downloading XMLs
var client = new WebClient();
foreach (var node in GetId())
{
//stores xml text into a string
var s = client.DownloadString("https://example" + node + "/xml");
// assign to the output
var file = new StreamWriter(path + "\\" + node + ".xml");
file.WriteLine(s);
file.Close();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
private static string[] GetId()
{
var idList = new List<string>();
var request = WebRequest.Create("https://example/xml");
var i = 0;
using (var response = request.GetResponse())
using (var xmlReader = XmlReader.Create(response.GetResponseStream()))
{
while (xmlReader.Read())
{
xmlReader.ReadToFollowing("n" + i);
//go through each of the n nodes in the xmldocument and get the name and id
if (xmlReader.NodeType != XmlNodeType.Element || xmlReader.Name != "n" + i) continue;
xmlReader.ReadToFollowing("id");
if (xmlReader.NodeType == XmlNodeType.Element && xmlReader.Name == "id")
{
var id = xmlReader.ReadElementContentAsString();
idList.Add(id);
}
i += 1;
}
}
var IDs = idList.ToArray();
return IDs;
}
At the moment the program takes a long time to download everything I need. Is there something I can do to speed up the process?

Yes, you can run things in parallel, by using Parallel.ForEach:
Parallel.ForEach
( GetId()
, new ParallelOptions() { MaxDegreeOfParallelism = 32 } // run 32 in parallel
, node =>
{
//stores xml text into a string
var client = new WebClient();
var s = client.DownloadString("https://example" + node + "/xml");
// assign to the output
var file = new StreamWriter(path + "\\" + node + ".xml");
file.WriteLine(s);
file.Close();
}
);
You can tweak the MaxDegreeOfParallelism as you want and your service can handle.

Related

Cleaning up nested if statements to be more readable?

I'm writing a project, and the part I'm doing now is getting arrow shaped real fast. How can I remove the nested if statements, but still have the same behaviour?
The code below might not look so bad now, but I'm planning on refactoring to include more methods.
public async Task FirstDiffTestAsync()
{
string folderDir = "../../../";
string correctReportDir = folderDir + "Reports To Compare/Testing - Copy.pdf";
string OptyNumber = "122906";
//Making a POST call to generate report
string result = ReportGeneration(OptyNumber).Result;
Response reportResponse = JsonConvert.DeserializeObject<Response>(result);
string newURL = reportResponse.documentUrl;
//Logging the Response to a text file for tracking purposes
await File.WriteAllTextAsync(Context.TestRunDirectory + "/REST_Response.txt", result);
using (StreamWriter w = File.AppendText(Context.TestDir + "/../log.txt"))
{
//Checking if the Integration failed
if (reportResponse.Error == null)
{
//now we have the url, reading in the pdf reports
List<string> Files = new List<string> { correctReportDir, newURL };
List<string> parsedText = PdfToParsedText(Files);
DiffPaneModel diff = InlineDiffBuilder.Diff(parsedText[0], parsedText[1]);
// DiffReport is a customised object
DiffReport diffReport = new DiffReport(correctReportDir, newURL);
diffReport.RunDiffReport(diff);
//In-test Logging
string indent = "\n - ";
string logMsg = $"{indent}Opty Number: {OptyNumber}{indent}Activity Number: {reportResponse.ActivityNumber}{indent}File Name: {reportResponse.FileName}";
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
logMsg += $"{indent}Different lines: {diffReport.insertCounter} Inserted, {diffReport.deleteCounter} Deleted";
}
LogTesting(logMsg, w);
//Writing HTML report conditionally
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
}
Assert.IsTrue(diffReport.insertCounter + diffReport.deleteCounter == 0);
}
else
{
LogTesting($" Integration Failed: {reportResponse.Error}", w);
Assert.IsNull(reportResponse.Error);
}
}
}
As mentioned in the comment, the indentation level is fine for now, but its always better to minimize when possible, especially when you are repeating same blocks of code.
The best way to do this is to write a separate function that contains that block of code and then call that function instead of the nested if statements.
In your case it would be something like this:
private async void checkTotalDiff(diffReport) {
...
}
You could pass anything you might need in the parameters. This way in your main code, you could replace the if statements with checkTotalDiff(diffReport) and save the return (if any) to a variable.
Also note I used void for return but you could change the type depending on what the function returns.
I wouldn't consider this as having an excessive amount of nested if-statements. It is fine as is. Otherwise you could do the following (also suggested by #Caius Jard):
public async Task FirstDiffTestAsync()
{
string folderDir = "../../../";
string correctReportDir = folderDir + "Reports To Compare/Testing - Copy.pdf";
string OptyNumber = "122906";
//Making a POST call to generate report
string result = ReportGeneration(OptyNumber).Result;
Response reportResponse = JsonConvert.DeserializeObject<Response>(result);
//Checking if the Integration failed
if (reportResponse.Error != null)
{
LogTesting($" Integration Failed: {reportResponse.Error}", w);
Assert.IsNull(reportResponse.Error);
return;
}
string newURL = reportResponse.documentUrl;
//Logging the Response to a text file for tracking purposes
await File.WriteAllTextAsync(Context.TestRunDirectory + "/REST_Response.txt", result);
using (StreamWriter w = File.AppendText(Context.TestDir + "/../log.txt"))
{
//now we have the url, reading in the pdf reports
List<string> Files = new List<string> { correctReportDir, newURL };
List<string> parsedText = PdfToParsedText(Files);
DiffPaneModel diff = InlineDiffBuilder.Diff(parsedText[0], parsedText[1]);
// DiffReport is a customised object
DiffReport diffReport = new DiffReport(correctReportDir, newURL);
diffReport.RunDiffReport(diff);
//In-test Logging
string indent = "\n - ";
string logMsg = $"{indent}Opty Number: {OptyNumber}{indent}Activity Number: {reportResponse.ActivityNumber}{indent}File Name: {reportResponse.FileName}";
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
logMsg += $"{indent}Different lines: {diffReport.insertCounter} Inserted, {diffReport.deleteCounter} Deleted";
}
LogTesting(logMsg, w);
//Writing HTML report conditionally
if (diffReport.totalDiff != 0)
{
await File.WriteAllTextAsync(Context.TestRunDirectory + "/DiffReport.html", diffReport.htmlDiffHeader + diffReport.htmlDiffBody);
}
Assert.IsTrue(diffReport.insertCounter + diffReport.deleteCounter == 0);
}
}

What is the fastest way to go through a XML file in C#?

I have a program that goes through thousands of files and has to check if they have the correct xml-format.
The problem is that it takes ages to complete, and I think that's because of the type of xml reader I use.
In the Method below are 3 different versions which I tried, the first one is the fastest, but only by 5%. (the method does not need to check if the file is a xml)
private bool HasCorrectXmlFormat(string filePath)
{
try
{
//-Version 1----------------------------------------------------------------------------------------
XmlReader reader = XmlReader.Create(filePath, new XmlReaderSettings() { IgnoreComments = true, IgnoreWhitespace = true });
string[] elementNames = new string[] { "DocumentElement", "Protocol", "DateTime", "Item", "Value" };
int i = 0;
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Element)
{
if (reader.Name != elementNames.ElementAt(i))
{
return false;
}
if (i >= 4)
{
return true;
}
i++;
}
}
return false;
//--------------------------------------------------------------------------------------------------
//- Version 2 ------------------------------------------------------------------------------------
IEnumerable<XElement> xmlElements = XDocument.Load(filePath).Descendants();
string[] elementNames = new string[] { "DocumentElement", "Protocol", "DateTime", "Item", "Value" };
for (int i = 0; i < 5; i++)
{
if (xmlElements.ElementAt(i).Name != elementNames.ElementAt(i))
{
return false;
}
}
return true;
//--------------------------------------------------------------------------------------------------
//- Version 3 ------------------------------------------------------------------------------------
XDocument doc = XDocument.Load(filePath);
if (doc.Root.Name != "DocumentElement")
{
return false;
}
if (doc.Root.Elements().First().Name != "Protocol")
{
return false;
}
if (doc.Root.Elements().First().Elements().ElementAt(0).Name != "DateTime")
{
return false;
}
if (doc.Root.Elements().First().Elements().ElementAt(1).Name != "Item")
{
return false;
}
if (doc.Root.Elements().First().Elements().ElementAt(2).Name != "Value")
{
return false;
}
return true;
//--------------------------------------------------------------------------------------------------
}
catch (Exception)
{
return false;
}
}
What I need is a faster way to do this. Is there a faster way to go through a xml file? I only have to check if the first 5 Elements have the correct names.
UPDATE
The Xml-Files are only 2-5 KBs in size, rarely more than that. Files are located on a local server. I am on a laptop which has a ssd.
Here are some test results:
I should also add that the files are filtered before, so only xml files are given to the method. I get the files with the following Method:
public List<FileInfo> GetCompatibleFiles()
{
return new DirectoryInfo(folderPath)
.EnumerateFiles("*", searchOption)
.AsParallel()
.Where(file => file.Extension == ".xml" ? HasCorrectXmlFormat(file.FullName) : false)
.ToList();
}
This Method is not in my code like this (it put two methods together), this is just to show how the HasCorrectXmlFormat Method is called. You dont have to correct this Method, I know it can be improved.
UDPATE 2
Here are the two full methods mentioned at the end of update 1:
private void WriteAllFilesInList()
{
allFiles = new DirectoryInfo(folderPath)
.EnumerateFiles("*", searchOption)
.AsParallel()
.ToList();
}
private void WriteCompatibleFilesInList()
{
compatibleFiles = allFiles
.Where(file => file.Extension == ".xml" ? HasCorrectXmlFormat(file.FullName) : false)
.ToList();
}
Both methods are only called once in the entire program (if either the allFiles or compatibleFiles List is null).
UPDATE 3
It seems like the WriteAllFilesInList Method is the real problem here, shown here:
FINAL UPDATE
As it seems, my method doesn't need any improvement as the bottleneck is something else.
Here is the example, which reads sample XML and shows comparison between Linq/XMlReader and XmlDocument
Linq is fastest.
Sample Code
using System;
using System.Diagnostics;
using System.Linq;
using System.Xml;
using System.Xml.Linq;
namespace ReadXMLInCsharp
{
class Program
{
static void Main(string[] args)
{
//returns url of main directory which contains "/bin/Debug"
var url=System.IO.Path.GetDirectoryName(
System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase);
//correction in path to point it in Root directory
var mainpath = url.Replace("\\bin\\Debug", "") + "\\books.xml";
var stopwatch = new Stopwatch();
stopwatch.Start();
//create XMLDocument object
XmlDocument xmlDoc = new XmlDocument();
//load xml file
xmlDoc.Load(mainpath);
//save all nodes in XMLnodelist
XmlNodeList nodeList = xmlDoc.DocumentElement.SelectNodes("/catalog/book");
//loop through each node and save it value in NodeStr
var NodeStr = "";
foreach (XmlNode node in nodeList)
{
NodeStr = NodeStr + "\nAuthor " + node.SelectSingleNode("author").InnerText;
NodeStr = NodeStr + "\nTitle " + node.SelectSingleNode("title").InnerText;
NodeStr = NodeStr + "\nGenre " + node.SelectSingleNode("genre").InnerText;
NodeStr = NodeStr + "\nPrice " + node.SelectSingleNode("price").InnerText;
NodeStr = NodeStr + "\nDescription -" + node.SelectSingleNode("description").InnerText;
}
//print all Authors details
Console.WriteLine(NodeStr);
stopwatch.Stop();
Console.WriteLine();
Console.WriteLine("Time elapsed using XmlDocument (ms)= " + stopwatch.ElapsedMilliseconds);
Console.WriteLine();
stopwatch.Reset();
stopwatch.Start();
NodeStr = "";
//linq method
//get all elements inside book
foreach (XElement level1Element in XElement.Load(mainpath).Elements("book"))
{
//print each element value
//you can also print XML attribute value, instead of .Element use .Attribute
NodeStr = NodeStr + "\nAuthor " + level1Element.Element("author").Value;
NodeStr = NodeStr + "\nTitle " + level1Element.Element("title").Value;
NodeStr = NodeStr + "\nGenre " + level1Element.Element("genre").Value;
NodeStr = NodeStr + "\nPrice " + level1Element.Element("price").Value;
NodeStr = NodeStr + "\nDescription -" + level1Element.Element("description").Value;
}
//print all Authors details
Console.WriteLine(NodeStr);
stopwatch.Stop();
Console.WriteLine();
Console.WriteLine("Time elapsed using linq(ms)= " + stopwatch.ElapsedMilliseconds);
Console.WriteLine();
stopwatch.Reset();
stopwatch.Start();
//method 3
//XMLReader
XmlReader xReader = XmlReader.Create(mainpath);
xReader.ReadToFollowing("book");
NodeStr = "";
while (xReader.Read())
{
switch (xReader.NodeType)
{
case XmlNodeType.Element:
NodeStr = NodeStr + "\nElement name:" + xReader.Name;
break;
case XmlNodeType.Text:
NodeStr = NodeStr + "\nElement value:" + xReader.Value;
break;
case XmlNodeType.None:
//do nothing
break;
}
}
//print all Authors details
Console.WriteLine(NodeStr);
stopwatch.Stop();
Console.WriteLine();
Console.WriteLine("Time elapsed using XMLReader (ms)= " + stopwatch.ElapsedMilliseconds);
Console.WriteLine();
stopwatch.Reset();
Console.ReadKey();
}
}
}
Output:
-- First Run
Time elapsed using XmlDocument (ms)= 15
Time elapsed using linq(ms)= 7
Time elapsed using XMLReader (ms)= 12
-- Second Run
Time elapsed using XmlDocument (ms)= 18
Time elapsed using linq(ms)= 3
Time elapsed using XMLReader (ms)= 15
I have removed some output to show only comparison data.
Source: Open and Read XML in C# (Examples using Linq, XMLReader, XMLDocument)
Edit: If i comment 'Console.WriteLine(NodeStr)' from all methods and prints only time comparison.
This is what I get
Time elapsed using XmlDocument (ms)= 11
Time elapsed using linq(ms)= 0
Time elapsed using XMLReader (ms)= 0
Basically it depends on how you are processing the data and how you are reading XML.
Linq/XML reader once look more promising in terms of speed.
I would write code like this using Xml Linq which is a little faster than your code. You code is looping through the xml file multiple times while mine is going through file only once.
try
{
XDocument doc = XDocument.Load(filePath);
XElement root = doc.Root;
if (doc.Root.Name != "DocumentElement")
{
return false;
}
else
{
XElement protocol = root.Elements().First();
if (protocol.Name != "Protocol")
{
return false;
}
else
{
XElement dateTime = protocol.Elements().First();
if (dateTime.Name != "DateTime")
{
return false;
}
XElement item = protocol.Elements().Skip(1).First();
if (item.Name != "Item")
{
return false;
}
XElement value = protocol.Elements().Skip(2).First();
if (doc.Root.Elements().First().Elements().ElementAt(2).Name != "Value")
{
return false;
}
}
}
}
catch (Exception)
{
return false;
}
return true;
}

StreamWriter: Starting and ending on a specific line number

I would like to ask some tips and help on a reading/writing part of my C#.
Situation:
I have to read a CSV file; - OK
If the CSV file name starts with "Load_", I want to write on another CSV the data from line 2 to the last one;
If the CSV file name starts with "RO_", I want to write on 2 different CSVs, 1 with the line 1 to 4 and the other 4 to the last one;
What I have so far is:
public static void ProcessFile(string[] ProcessFile)
{
// Keeps track of your current position within a record
int wCurrLine = 0;
// Number of rows in the file that constitute a record
const int LINES_PER_ROW = 1;
int ctr = 0;
foreach (string filename in ProcessFile)
{
var sbText = new System.Text.StringBuilder(100000);
int stop_line = 0;
int start_line = 0;
// Used for the output name of the file
var dir = Path.GetDirectoryName(filename);
var fileName = Path.GetFileNameWithoutExtension(filename);
var ext = Path.GetExtension(filename);
var folderbefore = Path.GetFullPath(Path.Combine(dir, #"..\"));
var lineCount = File.ReadAllLines(#filename).Length;
string outputname = folderbefore + "output\\" + fileName;
using (StreamReader Reader = new StreamReader(#filename))
{
if (filename.Contains("RO_"))
{
start_line = 1;
stop_line = 5;
}
else
{
start_line = 2;
stop_line = lineCount;
}
ctr = 0;
while (!Reader.EndOfStream && ctr < stop_line)
{
// Add the text
sbText.Append(Reader.ReadLine());
// Increment our current record row counter
wCurrLine++;
// If we have read all of the rows for this record
if (wCurrLine == LINES_PER_ROW)
{
// Add a line to our buffer
sbText.AppendLine();
// And reset our record row count
wCurrLine = 0;
}
ctr++;
} // end of the while
}
int total_lenght = sbText.Length
// When all of the data has been loaded, write it to the text box in one fell swoop
using (StreamWriter Writer = new StreamWriter(dir + "\\" + "output\\" + fileName + "_out" + ext))
{
Writer.Write.(sbText.);
}
} // end of the foreach
} // end of ProcessFile
I was thinking about using the IF/ELSE: "using (StreamWriter Writer = new StreamWriter(dir + "\" + "output\" + fileName + "_out" + ext))" part. However, I am not sure how to pass, to StreamWriter, to only write from/to a specific line number.
Any Help is welcome! If I am missing some information, please, let me know (I am pretty new on stackoverflow).
Thank you.
Code is way too complicated
using System.Collections.ObjectModel;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace ConsoleApplication57
{
class Program
{
static void Main(string[] args)
{
}
public static void ProcessFile(string[] ProcessFile)
{
foreach (string filename in ProcessFile)
{
// Used for the output name of the file
var dir = Path.GetDirectoryName(filename);
var fileName = Path.GetFileNameWithoutExtension(filename);
var ext = Path.GetExtension(filename);
var folderbefore = Path.GetFullPath(Path.Combine(dir, #"..\"));
var lineCount = File.ReadAllLines(#filename).Length;
string outputname = folderbefore + "output\\" + fileName;
using (StreamWriter Writer = new StreamWriter(dir + "\\" + "output\\" + fileName + "_out" + ext))
{
int rowCount = 0;
using (StreamReader Reader = new StreamReader(#filename))
{
rowCount++;
string inputLine = "";
while ((inputLine = Reader.ReadLine()) != null)
{
if (filename.Contains("RO_"))
{
if (rowCount <= 4)
{
Writer.WriteLine(inputLine);
}
if (rowCount == 4) break;
}
else
{
if (rowCount >= 2)
{
Writer.WriteLine(inputLine);
}
}
} // end of the while
Writer.Flush();
}
}
} // end of the foreach
} // end of ProcessFile
}
}
You can use LINQ to Take and Skip lines.
public abstract class CsvProcessor
{
private readonly IEnumerable<string> processFiles;
public CsvProcessor(IEnumerable<string> processFiles)
{
this.processFiles = processFiles;
}
protected virtual IEnumerable<string> GetAllLinesFromFile(string fileName)
{
using(var stream = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read))
using(var reader = new StreamReader(stream))
{
var line = String.Empty;
while((line = reader.ReadLine()) != null)
{
yield return line;
}
}
}
protected virtual void ProcessFiles()
{
var sb1 = new StringBuilder();
var sb2 = new StringBuilder();
foreach(var file in this.processFiles)
{
var fileName = Path.GetFileNameWithoutExtension(file);
var lines = GetAllLinesFromFile(file);
if(fileName.StartsWith("RO_", StringComparison.InvariantCultureIgnoreCase))
{
sb1.AppendLine(lines.Take(4)); //take only the first four lines
sb2.AppendLine(lines.Skip(4).TakeWhile(s => !String.IsNullOrEmpty(s))); //skip the first four lines, take everything else
}
else if(fileName.StartsWith("Load_", StringComparison.InvariantCultureIgnoreCase)
{
sb2.AppendLine(lines.Skip(1).TakeWhile(s => !String.IsNullOrEmpty(s)));
}
}
// now write your StringBuilder objects to file...
}
protected virtual void WriteFile(StringBuilder sb1, StringBuilder sb2)
{
// ... etc..
}
}

How to edit a file, appending data at the beginning, without rewriting the whole file?

I have a csv file, and I need to add a unique ID based on the first two characters of the file. I have the following code:
using (StreamReader sr = new StreamReader(f))
{
string currentLine;
int id = 0;
while ((currentLine = sr.ReadLine()) != null)
{
string row = currentLine.ToString();
string FirstTwoCharacters = currentLine.Substring(0, 2);
if (FirstTwoCharacters == "01")
{
id = id + 1;
row += "*" + id.ToString();
using (StreamWriter files = File.AppendText(dir + newfilename))
{
files.WriteLine(row);
}
}
else
{
row += "*" + id.ToString();
using (StreamWriter files = File.AppendText(dir + newfilename))
{
files.WriteLine(row);
}
}
}
}
The csv files can be huge, 1Gb in size, around 6 million rows. Just wanted advice, if there is a quicker way to handling this, as it currently can take 3+ hours to process a file, and multiple files can be received in one go.
Instead of opening new file for appending line for each line of input file you can keep stream writer opened:
using (StreamReader sr = new StreamReader(f))
using (StreamWriter files = File.AppendText(dir + newfilename))
{
string currentLine;
int id = 0;
while ((currentLine = sr.ReadLine()) != null)
{
string firstTwoCharacters = currentLine.Substring(0, 2);
if (firstTwoCharacters == "01")
id++;
files.WriteLine(currentLine + "*" + id);
}
}
You can also use File.ReadLines to enumerate source lines:
using (StreamWriter writer = File.AppendText(dir + newfilename))
{
int id = 0;
foreach(var line in File.ReadLines(f))
{
if (line.Substring(0,2) == "01")
id++;
writer.WriteLine($"{line}*{id}");
}
}
Or even LINQ approach
int id = 0;
var newLines = from line in File.ReadLines(f)
let incrementId = line.Substring(0,2) == "01"
select $"{line}*{incrementId ? (++id) : id}";
File.WriteAllLines(dir + newfilename, newLines);
opening the (File.AppendText) inside the while loop is costly, move this to outside the while
using (StreamReader sr = new StreamReader(f))
{
string currentLine;
int id = 0;
using (StreamWriter files = File.AppendText(dir + newfilename))
{
while ((currentLine = sr.ReadLine()) != null)
{
string row = currentLine.ToString();
string FirstTwoCharacters = currentLine.Substring(0, 2);
if (FirstTwoCharacters == "01")
{
id = id + 1;
row += "*" + id.ToString();
files.WriteLine(row);
}
else
{
row += "*" + id.ToString();
files.WriteLine(row);
}
}
}
}

C# System.IO.IOException

I have following code:
using System;
using System.Collections.Generic;
using System.IO;
using VirusTotalNET;
using VirusTotalNET.Objects;
using System.Linq;
using System.Security.Permissions;
namespace VirusTotalNETClient
{
class Program
{
private const string ScanUrl = "http://www.google.com/";
static void Main(string[] args)
{
VirusTotal virusTotal = new VirusTotal("5d8684f50946c2bdeaf5c4fd966f61f3661de808e9d7324b99788d6f4fb7ad57");
//Use HTTPS instead of HTTP
virusTotal.UseTLS = true;
//creating folder for programs reliqies and output log
string folderName = "C:\\OnlineScanner";
System.IO.Directory.CreateDirectory(folderName);
//get list of files to analyse
var paths = Traverse("C:\test");
File.WriteAllLines("C:\\OnlineScanner\\test.txt", paths);
foreach (string line in File.ReadLines("C:\\test.txt"))
{
//Define what file you want to analyse
FileInfo fileInfo = new FileInfo(line);
//Check if the file has been scanned before.
FileReport fileReport = virusTotal.GetFileReport(fileInfo);
bool hasFileBeenScannedBefore = fileReport.ResponseCode == ReportResponseCode.Present;
//If the file has been scanned before, the results are embedded inside the report.
if (hasFileBeenScannedBefore)
{
int detekce = fileReport.Positives;
if (detekce >= 1)
{
using (var writer = new StreamWriter("C:\\OnlineScanner\\OnlineScannerLog.txt"))
{
writer.WriteLine(line);
writer.WriteLine("URL to test: " + fileReport.Permalink);
writer.WriteLine("Detect ratio: " + fileReport.Positives + "/54");
writer.WriteLine("Message: " + fileReport.VerboseMsg);
writer.WriteLine();
writer.WriteLine();
}
}
System.Threading.Thread.Sleep(16000);
}
else
{
ScanResult fileResult = virusTotal.ScanFile(fileInfo);
int detekce = fileReport.Positives;
if (detekce >= 1)
{
using (var writer = new StreamWriter("C:\\OnlineScanner\\OnlineScannerLog.txt"))
{
writer.WriteLine(line);
writer.WriteLine("URL to test: " + fileReport.Permalink);
writer.WriteLine("Detect ratio: " + fileReport.Positives + "/54");
writer.WriteLine("Message: " + fileReport.VerboseMsg);
writer.WriteLine();
writer.WriteLine();
}
}
System.Threading.Thread.Sleep(16000);
}
}
}
private static IEnumerable<string> Traverse(string rootDirectory)
{
IEnumerable<string> files = Enumerable.Empty<string>();
IEnumerable<string> directories = Enumerable.Empty<string>();
try
{
// The test for UnauthorizedAccessException.
var permission = new FileIOPermission(FileIOPermissionAccess.PathDiscovery, rootDirectory);
permission.Demand();
files = Directory.GetFiles(rootDirectory);
directories = Directory.GetDirectories(rootDirectory);
}
catch
{
// Ignore folder (access denied).
rootDirectory = null;
}
foreach (var file in files)
{
yield return file;
}
// Recursive call for SelectMany.
var subdirectoryItems = directories.SelectMany(Traverse);
foreach (var result in subdirectoryItems)
{
yield return result;
}
}
}
}
This code run some time (arround 15secs) but then program crashs.
The error is
System.IO.IOException, process can't access to file C:\hiberfil.sys.
http://upnisito.cz/images/2016_12/319crasherrror.png
Do you have any idea how to solve it?

Categories

Resources