I am writing a C# app that copy files over a network, the problem is that the size of the files and folders to copy is more than 1 TB.
My method is as follows
public static void SubmitDocsToRepository(string p_FilePaths)
{
IEnumerable<(string,string)> directoryLevels = GetAllFolders(p_FilePaths);
IEnumerable<(string,string)> filesLevels = GetAllFiles(p_FilePaths);
foreach (var tuple in directoryLevels)
Folder copy logic
foreach (var tuple in filesLevels)
File copy logic
}
Which would work fine, but assuming something would happen to the network or remote server or the electric power gets lost for whatever reason what should I add to this code to allow me to continue where I left off, especially how could I retrace my steps to where I was.
It could be something like this:
public static void SubmitDocsToRepository(string p_FilePaths)
{
IEnumerable<(string, string)> directoryLevels = GetAllFolders(p_FilePaths);
IEnumerable<(string, string)> filesLevels = GetAllFiles(p_FilePaths);
foreach (var tuple in directoryLevels)
while (!CopyDirectory(tuple)) ;
foreach (var tuple in filesLevels)
while (!CopyFile(tuple)) ;
}
static bool CopyDirectory((string, string)tuple)
{
try
{
// Copy logic
}
catch
{
// Some logging here
return false;
}
return true;
}
static bool CopyFile((string, string) tuple)
{
try
{
// Copy logic
}
catch
{
// Some logging here
return false;
}
return true;
}
Related
How can I ban a variable from a list without removing it from that list by adding the variable to a list of "banned" variable?
I wish to be able to type in a string. That string is compared to the file names in a folder. If there is a match, the file is read. If I type this same string again, the file should not be read again. There for I want to have a list of "banned" string that is checked whilst typing to avoid the file to be read again.
I have tried a few ways but not getting there. Below is an example of my last attempt.
What would be the best way?
public class test
{
string scl= "test3";
List <string> lsf,lso;
void Start ()
{
lsf=//file names
new List<string>();
lso=//files open
new List<string>();
lsf.Add("test0");
lsf.Add("test1");
lsf.Add("test2");
lsf.Add("test3");
lsf.Add("test4");
lso.Add("idhtk49fngo");//random string
}
void Update ()
{
if
(
Input.GetKeyDown("a")
)
{
for
(
int i=0;
i<lsf.Count;
i++
)
{
if(lsf[i]==scl)
{
Debug.Log
(i+" is read");
for
(
int j=0;
j<lso.Count;
j++
)
{
//how can i avoid reading
//lsf[3] here the second time
//"a" is pressed (by having "test3"
//added to a "ban" list (lso) )
if(scl!=lso[j])
{
lso.Add(lsf[i]);
}
}
}
}
}
}
Michael’s answer is the way to go here but it can be improved using the more appropriate collection available to keep track of opened files; if you want uniqueness use a set, not a list:
HashSet<string> openedFiles = new HashSet<string>();
public static bool TryFirstRead(
string path,
out string result)
{
if (openedFiles.Add(path))
{
result = File.ReadAllText(path);
return true;
}
result = null;
return false;
}
Also, I’d avoid throwing vexing exceptions. Give the consumer a friendly way to know if the file was read or not, don’t make them end up having to use exceptions as a flow control mechanism.
I didn't understand although if you want to replace a value from another list.
You can use the list index to create a new list with the values which you removed.
String list1 = {"hi", "hello", "World"};
String list2 = {"bye", "goodbye", "World"};
List1[1] = list2[1];
I would suggest such way:
public static List<string> openedFiles = new List<string>();
public static string ReadFileAndAddToOpenedList(string path)
{
if (openedFiles.Contains(path))
throw new Exception("File already opened");
// Instead of throwing exception you could for example just log this or do something else, like:
// Consolle.WriteLine("File already opened");
else
{
openedFiles.Add(path);
return File.ReadAllText(path);
}
}
The idea is - on every file read, add file to list, so you can check every time you try read file, if it was already read (or opened). If it is, throw exception (or do something else). Else read a file.
You could instead of making it a string list use your own class
public class MyFile
{
public string Name;
public bool isOpen;
public MyFile(string name)
{
Name = name;
isOpen = false;
}
}
List<MyFile> lsf = new List<MyFile>()
{
new MyFile("test0"),
new MyFile("test1"),
new MyFile("test2"),
new MyFile("test3"),
new MyFile("test4")
};
Than when you read the file set isOpen to true
MyFile[someIndex].isOpen = true;
and later you can check this
// E.g. skip in a loop
if(MyFile[someIndex]) continue;
You could than also use Linq in order to get a list of only unread files:
var unreadFiles = lsf.Select(f => f.Name).Where(file => !file.isOpen);
Im trying to make a process where I unpack a set of rar-files using SharpCompress and then delete them.
I first have a foreach loop which loops over the different set of files to be unpacked like this:
foreach (var package in extractionPackages.Where(package => Extractor.ExtractToFolder(package, destinationFolder)))
{
FileHelper.DeleteExtractionFiles(package);
}
The extraction process is taken straight of the SharpCompress Tests code and goes like this:
public static bool ExtractToFolder(ExtractionPackage extractionPackage, string extractionPath)
{
var fullExtractionPath = Path.Combine(extractionPath, extractionPackage.FolderName);
try
{
using (var reader = RarReader.Open(extractionPackage.ExtractionFiles.Select(p => File.OpenRead(p.FullPath))))
{
while (reader.MoveToNextEntry())
{
reader.WriteEntryToDirectory(fullExtractionPath, ExtractOptions.ExtractFullPath | ExtractOptions.Overwrite);
}
}
return true;
}
catch (Exception)
{
return false;
}
}
As you can see in the first code block I then call to delete the files but there I recieve an error due to that the files are locked by another process:
The process cannot access the file 'file.rar' because it is being used by another process.
If I try to put the deletion after the foreach-loop I am able to delete all but the last set of files if there are more than one. If it is just one the same issue occurs since the last set of files doesn't seems to be "unlocked".
How can I structure the code so that the files is unlocked?
By using an example from Nunrar and modifying it a bit I finally seems to have solved the issue:
public static bool ExtractToFolder(ExtractionPackage extractionPackage, string extractionPath)
{
var fullExtractionPath = Path.Combine(extractionPath, extractionPackage.FolderName);
try
{
using (var reader = RarReader.Open(GetStreams(extractionPackage)))//extractionPackage.ExtractionFiles.Select(p => File.OpenRead(p.FullPath)), Options.None))
{
while (reader.MoveToNextEntry())
{
reader.WriteEntryToDirectory(fullExtractionPath, ExtractOptions.ExtractFullPath | ExtractOptions.Overwrite);
}
}
return true;
}
catch (Exception)
{
return false;
}
}
private static IEnumerable<Stream> GetStreams(ExtractionPackage package)
{
foreach (var item in package.ExtractionFiles)
{
using (Stream input = File.OpenRead(item.FullPath))
{
yield return input;
}
}
}
By default (and probably a flaw) is that I leave the Streams open as I did not create them myself. However, the .NET framework usually breaks this rule with things like StreamReader etc. so it's expected that they'd be closed.
I'm thinking I'm going to change this behavior. In the meantime, you ought to be able to fix this yourself by calling this:
RarReader.Open(extractionPackage.ExtractionFiles.Select(p => File.OpenRead(p.FullPath)), Options.None)
I had a similar trouble. I used SharpCompress for adding files in archive and I need to delete files after archiving. I had tried wrap streams with using() {} - but this solution didn't worked in my case. When I added to my code Application.DoEvents() - it solved the problem.
I'm trying to write a simple Visual Studio 2012 extension. I have generated the extension template and can bring up a dialog box from a tool menu.
I'd like to enumerate all files in the currently open project and then filter them according to some rules. What I'm looking for is a code snippet to return IEnumerable. FileHandle should have the following interface or something similar.
interface IFileHandle {
// Return the string
string Path;
// Open the file in the editor
void OpenEditorFor();
}
FYI I'm trying to build a fuzzy file finder for visual studio. The current file search is less than suitable as you have to have exact match. I can handle writing the indexer and the fuzzy searcher but the interface to Visual Studio extension writing is a bit cryptic at the moment.
This seems to be the simple answer. In the context of a visual
studio extension will return all files.
public IEnumerable<ProjectItem> Recurse(ProjectItems i)
{
if (i!=null)
{
foreach (ProjectItem j in i)
{
foreach (ProjectItem k in Recurse(j))
{
yield return k;
}
}
}
}
public IEnumerable<ProjectItem> Recurse(ProjectItem i)
{
yield return i;
foreach (ProjectItem j in Recurse(i.ProjectItems ))
{
yield return j;
}
}
public IEnumerable<ProjectItem> SolutionFiles()
{
Solution2 soln = (Solution2)_applicationObject.Solution;
foreach (Project project in soln.Projects)
{
foreach (ProjectItem item in Recurse(project.ProjectItems))
{
yield return item;
}
}
}
You can then do neat tricks with it like implement the search function at the core of my CommandT clone.
private static string Pattern(string src)
{
return ".*" + String.Join(".*", src.ToCharArray());
}
private static bool RMatch(string src, string dest)
{
try
{
return Regex.Match(dest, Pattern(src), RegexOptions.IgnoreCase).Success;
}
catch (Exception e)
{
return false;
}
}
private static List<string> RSearch(
string word,
IEnumerable<string> wordList,
double fuzzyness)
{
// Tests have prove that the !LINQ-variant is about 3 times
// faster!
List<string> foundWords =
(
from s in wordList
where RMatch(word, s) == true
orderby s.Length ascending
select s
).ToList();
return foundWords;
}
which is used like
var list = RSearch("bnd", SolutionFiles().Select(x=>x.Name))
On my FTP Server I have the following folder structure
- Parent Directory
-a.txt
-b.txt.old
-SubDirectory1
-c.txt
-NestedSubDirectory1
-d.txt
-SubDirectory2
-e.txt
-f.txt.old
The number of SDs are not fixed. I need a way to get all the files(can be any format) without the .old extension from the Parent Directory.
I'm currently using the 3rd party dll edtFTPnet.
ftpConnection.GetFileInfos()Where(f => !(f.Name.EndsWith(".old"))).ToList();
This helps me get the details of the files and folders at the current working directory level.
Can someone tell me a way to get all the files with the parentdirectory, subdirectories and nested subdirectories.
The solution may or may not use edtFTPnet.
FTPConnection.GetFileInfos() returns an array of FTPFile. The class FTPFile has a boolean property Dir which indicates whether its filename accesses a file (false) or directory (true).
Something like this should work:
void ReadSubDirectories(FTPConncetion connection, FTPFile[] files)
{
foreach (var file in files)
{
if (file.Dir)
{
// Save parent directory
var curDir = connection.ServerDirectory;
// Move into directory
connection.ChangeWorkingDirectory(file.Name)
// Read all files
ReadSubDirectories(connection, connection.GetFileInfos());
// Move back into parent directory
connection.ChangeWorkingDirectory(curDir)
}
else
{
// Do magic with your files
}
}
}
However you might be better off using just .NET's built-in FtpWebRequest class since its methods and naming conventions are clearer, it's better documented and it's easier to find references online.
Try to use extensions like this:
class Program
{
static void Main(string[] args)
{
using (var connection = new FTPConnection
{
ServerAddress = "127.0.0.1",
UserName = "Admin",
Password = "1",
})
{
connection.Connect();
connection.ServerDirectory = "/recursive_folder";
var resultRecursive =
connection.GetFileInfosRecursive().Where(f => !(f.Name.EndsWith(".old"))).ToList();
var resultDefault = connection.GetFileInfos().Where(f => !(f.Name.EndsWith(".old"))).ToList();
}
}
}
public static class FtpClientExtensions
{
public static FTPFile[] GetFileInfosRecursive(this FTPConnection connection)
{
var resultList = new List<FTPFile>();
var fileInfos = connection.GetFileInfos();
resultList.AddRange(fileInfos);
foreach (var fileInfo in fileInfos)
{
if (fileInfo.Dir)
{
connection.ServerDirectory = fileInfo.Path;
resultList.AddRange(connection.GetFileInfosRecursive());
}
}
return resultList.ToArray();
}
}
I have this piece of code that searches for a folder given a starting directory. Once the folder is found I need the name of it's parent. The following piece of code works but it is terribly ugly. I have a flag, "sessionFound" to assist with breaking from nested foreach loops. The following works. I was hoping I could get some eyes on this and see if I could get some suggestions on how to make this less verbose and a bit more concise.
Thanks.
private void SetProjectFolder(string sessionid)
{
//IoC container gives the root directory to begin search.
string[] supportDirs = Directory.GetDirectories(ApplicationContainer.SupportDirectory);
bool sessionFound = false;
foreach (string directory in supportDirs)
{
if (!sessionFound)
{
foreach (string folder in Directory.GetDirectories(directory))
{
if (!sessionFound)
{
foreach (string productSubFolder in Directory.GetDirectories(folder))
{
if (productSubFolder.Contains(sessionid))
{
_productName = Directory.GetParent(productSubFolder).Parent.Name;
sessionFound = true;
break;
}
}
}
else
{
break;
}
}
}
else
{
break;
}
}
}
Try this:
private void SetProjectFolder(string sessionid)
{
//This will simulate the contains statement
string searchPattern = string.Format("*{0}*", sessionid);
string[] supportDirs = Directory.GetDirectories(ApplicationContainer.SupportDirectory, searchPattern);
foreach (string filteredFolder in supportDirs)
{
_productName = Directory.GetParent(filteredFolder).Name;
break;
}
}
By removing sessionFound you end up with a pretty clear and straightforward code:
private void SetProjectFolder(string sessionid)
{
//IoC container gives the root directory to begin search.
string[] supportDirs = Directory.GetDirectories(ApplicationContainer.SupportDirectory);
// search for product subfolder
foreach (string directory in supportDirs)
{
foreach (string folder in Directory.GetDirectories(directory))
{
foreach (string productSubFolder in Directory.GetDirectories(folder))
{
if (productSubFolder.Contains(sessionid))
{
// product sub-folder found, set it and exit
_productName = Directory.GetParent(productSubFolder).Parent.Name;
return;
}
}
}
}
// product sub-folder not found
!!! handle error path
}
In case your directory structure is static in the sense of the abovementioned search algorithm, I don't think there's something wrong with the nested foreach statements. Maybe you could extract the code starting with the middle foreach into a separate method like string FindProductSubfolderInFolder(Directory).