I have made an implementation of File/Folder lookup in Google Drive v3 using their own Google API for .NET.
Code works fine, but to be honest I'm not sure if it's really on a standard-efficient way of doing this.
Logic:
I have to get to each and every folder and download specific files on it.
Structure can be A > B > C > D , basically a folder within a folder within a folder and so on.
I can't use a static-predefined directory schema as a long term solution as it can change anytime the owner wants to modify it, for now, the folders are at least 4 levels deep.
The only way I can navigate to the subfolders is to get its own Google Drive ID and use that to see its contents. It is like you need a KEY first before you can unlock/open the next subfolder.
In short, I can't do LOOKUP on subfolders content the easy way, unless there's someone that can give us better alternatives, I'll be glad to take any criticisms on my aproach and open to all your suggestions.
Thank you.
Update
Thank you all for providing links and examples,
I believed Recursion solution is the best so far on my current scenario,
And also, since this is a heavy IO Operations, I did apply ASYNC operations for downloading files and to the rest as possible, so I made sure to follow the ASYNC ALL THE WAY rule to prevent blocking.
To Call the Recursion Method
var parentID = "<folder id>";
var folderLevel = 0;
var listRequest = service.Files.List();
await MyTask(listRequest, id, count, folderLevel);
This is the Recursion Method, it will search all possible folders from the set root parent id that was defined...
private async Task RecursionTask(ListRequest listRequest, string parentId, int count, int folderLevel)
{
// This method do the Folder search
listRequest.Q = $"('{parentId}' in parents) and (mimeType = 'application/vnd.google-apps.folder') and trashed = false";
listRequest.Fields = "files(id,name)";
var filesTask = await listRequest.ExecuteAsync();
var files = filesTask.Files;
count = files.Count(); // Keep track of recursion flow
count--;
// Keep track of how deep recursion is diving on subfolders
folderLevel++;
var tasks = new List<Task>();
foreach(var file in files)
{
tasks.Add(InnerTask(file, listRequest, name, folderLevel)); // Create Array Of Tasks for IMAGE Search
if (count > 1) // Loop until I exhausted the value of count
{
// Return recursion flow
await RecursionTask(listRequest, file.Name, file.Id, count, folderLevel);
}
}
await Task.WhenAll(tasks); // Wait all tasks to finish
}
This is the innerTask that will handle the Downloading of drive files
private async Task InnerTask(File file, ListRequest listRequest, string name,int folderLevel)
{
// This method do the IMAGE SEARCH
listRequest.Q = $"('{file.Id}' in parents) and (mimeType = 'image/jpeg' or mimeType = 'image/png')";
listRequest.Fields = "files(id,name)";
var subFiles = await listRequest.ExecuteAsync();
foreach (var subFile in subFiles.Files)
{
// Do Async task for downloading images on Google Drive
}
}
Related
I have an application running in multiple servers applying some ACL's.
Problem is when more than one server is applying on the same folder structure (i.e. three levels), usually only levels one and three have the ACL's applied, but there's no exception.
I've created a test with parallel tasks (to simulate the different servers):
[TestMethod]
public void ApplyACL()
{
var baseDir = Path.Combine(Path.GetTempPath(), "ACL-PROBLEM");
if (Directory.Exists(baseDir))
{
Directory.Delete(baseDir, true);
}
var paths = new[]
{
Path.Combine(baseDir, "LEVEL-1"),
Path.Combine(baseDir, "LEVEL-1", "LEVEL-2"),
Path.Combine(baseDir, "LEVEL-1", "LEVEL-2", "LEVEL-3")
};
//create folders and files, so the ACL takes some time to apply
foreach (var dir in paths)
{
Directory.CreateDirectory(dir);
for (int i = 0; i < 1000; i++)
{
var id = string.Format("{0:000}", i);
File.WriteAllText(Path.Combine(dir, id + ".txt"), id);
}
}
var sids = new[]
{
"S-1-5-21-448539723-725345543-1417001333-1111111",
"S-1-5-21-448539723-725345543-1417001333-2222222",
"S-1-5-21-448539723-725345543-1417001333-3333333"
};
var taskList = new List<Task>();
for (int i = 0; i < paths.Length; i++)
{
taskList.Add(CreateTask(i + 1, paths[i], sids[i]));
}
Parallel.ForEach(taskList, t => t.Start());
Task.WaitAll(taskList.ToArray());
var output = new StringBuilder();
var failed = false;
for (int i = 0; i < paths.Length; i++)
{
var ok = Directory.GetAccessControl(paths[i])
.GetAccessRules(true, false, typeof(SecurityIdentifier))
.OfType<FileSystemAccessRule>()
.Any(f => f.IdentityReference.Value == sids[i]);
if (!ok)
{
failed = true;
}
output.AppendLine(paths[i].Remove(0, baseDir.Length + 1) + " --> " + (ok ? "OK" : "ERROR"));
}
Debug.WriteLine(output);
if (failed)
{
Assert.Fail();
}
}
private static Task CreateTask(int i, string path, string sid)
{
return new Task(() =>
{
var start = DateTime.Now;
Debug.WriteLine("Task {0} start: {1:HH:mm:ss.fffffff}", i, start);
var fileSystemAccessRule = new FileSystemAccessRule(new SecurityIdentifier(sid),
FileSystemRights.Modify | FileSystemRights.Synchronize,
InheritanceFlags.ContainerInherit | InheritanceFlags.ObjectInherit,
PropagationFlags.None,
AccessControlType.Allow);
var directorySecurity = Directory.GetAccessControl(path);
directorySecurity.ResetAccessRule(fileSystemAccessRule);
Directory.SetAccessControl(path, directorySecurity);
Debug.WriteLine("Task {0} finish: {1:HH:mm:ss.fffffff} ({2} ms)", i, DateTime.Now, (DateTime.Now - start).TotalMilliseconds);
});
}
I'm getting the same problem: usually (but not always) only levels one and three have the ACL's applied.
Why is that and how can I fix this?
Directory.SetAccessControl internally calls the Win32 API function SetSecurityInfo:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa379588.aspx
The important part of the above documentation:
If you are setting the discretionary access control list (DACL) or any elements in the system access control list (SACL) of an object, the system automatically propagates any inheritable access control entries (ACEs) to existing child objects, according to the ACE inheritance rules.
The enumeration of child objects (CodeFuller already described this) is done in the low level function SetSecurityInfo itself. To be more detailed, this function calls into the system DLL NTMARTA.DLL, which does all the dirty work.
The background of this is inheritance, which is a "pseudo inheritance", done for performance reasons. Every object contains not only the "own" ACEs, but also the inherited ACEs (those which are grayed out in Explorer). All this inheritance is done during the ACL setting, not during runtime ACL resolution / checking.
This former decision of Microsoft is also the trigger of the following problem (Windows admins should know this):
If you move a directory tree to another location in the file system where a different ACL is set, the ACLs of the objects of the moved try will not change.
So to say, the inherited permissions are wrong, they do not match the parent’s ACL anymore.
This inheritance is not defined by InheritanceFlags, but instead with
SetAccessRuleProtection.
To add on CodeFuller’s answer:
>>After enumeration is completed, internal directory security record is assigned to directory.
This enumeration is not just a pure reading of the sub-objects, the ACL of every sub-object will be SET.
So the problem is inherent to the inner workings of Windows ACL handling:
SetSecurityInfo checks the parent directory for all ACEs which should be inherited and then does a recursion and applies these inheritable ACEs to all subobjects.
I know about this because I have written a tool which sets the ACLs of complete file systems (with millions of files) which uses what we call a "managed folder". We can have very complex ALCs with automatic calculated list permissions.
For the setting of the ACL to the files and folders I use SetKernelObjectSecurity. This API should not normally be used for file systems, since it does not handle that inheritance stuff. So you have to do this yourself. But, if you know what you do and you do it correctly, it is the only reliable way to set the ACL on a file tree in every situation.
In fact, there can be situations (broken / invalid ACL entries in child objects) where SetSecurityInfo fails to set these objects correctly.
And now to the code from Anderson Pimentel:
It should be clear from the above that the parallel setting can only work if the inheritance is blocked
at each directory level.
However, it does not work to just call
dirSecurity.SetAccessRuleProtection(true, true);
in the task, since this call may come to late.
I got the code working if the above statement is called before starting the task.
The bad news is that this call, done with C# also does a complete recursion.
So it seems that there is no real compelling solution in C#, beside using PInvoke calling the low-level security functions directly.
But that’s another story.
And to the initial problem where different servers are setting the ACL:
If we know about the intent behind and what you want the resulting ALC to be, we perhaps can find a way.
Let me know.
It's a funny puzzle.
I've launched your test and the problem reproduces almost for each run. And ACL are often not applied for LEVEL-3 too.
However the problem does not reproduce if tasks run not in parallel.
Also if directory does not contain those 1000 files, the problem reproduces much less often.
Such behavior is very similar to classic race condition.
I haven't found any explicit information on this topic but it seems like applying ACL on overlapping directory trees is not a thread-safe operation.
To confirm this we need to analyze implementation of SetAccessControl() (or rather underlying Windows API call). But let's try to imagine what it might be.
SetAccessControl() is called for given directory and DirectorySecurity record.
It creates some internal structure (filesystem object) and fills it with provided data.
Then it starts enumeration of child objects (directories and files). Such enumeration is partly confirmed by tasks execution time. It's about 500 ms for task3, 1000 ms for task2 and 1500 ms for task1.
After enumeration is completed, internal directory security record is assigned to directory.
But in parallel, the same is done for SetAccessControl() called on parent directory. Finally it will overwrite the record created on step 4.
Of course, described flow is just an assumption. We need NTFS or Windows internals experts to confirm this.
But observed behavior almost certainly indicates race condition. Just avoid such parallel applying of ACL on overlapping directory trees and sleep well.
Introduce a lock. You have the shared file system available, so use .NET to lock when a process makes changes to a folder:
using (new FileStream(lockFile, FileMode.Open, FileAccess.Read, FileShare.None))
{
// file locked
}
In your code add on initialization:
var lockFile = Path.Combine(baseDir, ".lock"); // just create a file
File.WriteAllText(lockFile, "lock file");
and pass the well-known lock file to your tasks.
Then wait for the file to get unlocked in each of your processes:
private static Task CreateTask(int i, string path, string sid, string lockFile)
{
return new Task(() =>
{
var start = DateTime.Now;
Debug.WriteLine("Task {0} start: {1:HH:mm:ss.fffffff}", i, start);
Task.WaitAll(WaitForFileToUnlock(lockFile, () =>
{
var fileSystemAccessRule = new FileSystemAccessRule(new SecurityIdentifier(sid),
FileSystemRights.Modify | FileSystemRights.Synchronize,
InheritanceFlags.ContainerInherit | InheritanceFlags.ObjectInherit,
PropagationFlags.None,
AccessControlType.Allow);
var directorySecurity = Directory.GetAccessControl(path);
directorySecurity.ResetAccessRule(fileSystemAccessRule);
Directory.SetAccessControl(path, directorySecurity);
}));
Debug.WriteLine("Task {0} finish: {1:HH:mm:ss.fffffff} ({2} ms)", i, DateTime.Now, (DateTime.Now - start).TotalMilliseconds);
});
}
private static async Task WaitForFileToUnlock(string lockFile, Action runWhenUnlocked)
{
while (true)
{
try
{
using (new FileStream(lockFile, FileMode.Open, FileAccess.Read, FileShare.None))
{
runWhenUnlocked();
}
return;
}
catch (IOException exception)
{
await Task.Delay(100);
}
}
}
With those changes the unit test passes.
You can add further locks on the various levels to make the process most efficient - something like an hierarchy lock logic.
I have a UWP application which perform to capture and process images from a camera. This project leverage Microsoft Cognitive Services Face Recognition API and I'm exploring the application's existing functionality for awhile now. My goal is that when the image of a person is identified by the camera (through Face Recognition API service), I want to show the associated image of that person.
With that, the images are captured and stored in a local directory of my machine. I want to retrieve the image file and render it on the screen once the person is identified.
The code below shows the async Task method ProcessCameraCapture
private async Task ProcessCameraCapture(ImageAnalyzer e)
{
if (e == null)
{
this.UpdateUIForNoFacesDetected();
this.isProcessingPhoto = false;
return;
}
DateTime start = DateTime.Now;
await e.DetectFacesAsync();
if (e.DetectedFaces.Any())
{
string names;
await e.IdentifyFacesAsync();
this.greetingTextBlock.Text = this.GetGreettingFromFaces(e, out names);
if (e.IdentifiedPersons.Any())
{
this.greetingTextBlock.Foreground = new SolidColorBrush(Windows.UI.Colors.GreenYellow);
this.greetingSymbol.Foreground = new SolidColorBrush(Windows.UI.Colors.GreenYellow);
this.greetingSymbol.Symbol = Symbol.Comment;
GetSavedFilePhoto(names);
}
else
{
this.greetingTextBlock.Foreground = new SolidColorBrush(Windows.UI.Colors.Yellow);
this.greetingSymbol.Foreground = new SolidColorBrush(Windows.UI.Colors.Yellow);
this.greetingSymbol.Symbol = Symbol.View;
}
}
else
{
this.UpdateUIForNoFacesDetected();
}
TimeSpan latency = DateTime.Now - start;
this.faceLantencyDebugText.Text = string.Format("Face API latency: {0}ms", (int)latency.TotalMilliseconds);
this.isProcessingPhoto = false;
}
In GetSavedFilePhoto, I passed the string names argument once the person is identified.
Code below for the GetSavedFilePhoto method
private void GetSavedFilePhoto(string personName)
{
if (string.IsNullOrWhiteSpace(personName)) return;
var directoryPath = #"D:\PersonImages";
var directories = Directory.GetDirectories(directoryPath);
var filePaths = Directory.GetFiles(directoryPath, "*.jpg", SearchOption.AllDirectories);
}
However, in GetSavedFilePhoto method the variable directories returned an empty string of array when using directoryPath string variable. Directory "D:\PersonImages" is a valid and existing folder in my machine and, it contains subfolders with images inside. I also tried Directory.GetFiles to retrieve the jpg images but still returned an empty string.
I think it should work because I have used Directory class several times but not inside an asyncTask method. Does using async caused the files not returned when using I/O operation?
Sorry for this stupid question, but I really don't understand.
Any help is greatly appreciated.
Using Directory.GetFiles or Directory.GetDirectories method can get the folder/file in the local folder of the Application by the following code. But it could not open D:\.
var directories = Directory.GetDirectories(ApplicationData.Current.LocalFolder.Path);
In UWP app you can only access two locations at default (local folder and install folder), others need capabilities setting or file open picker.Details please reference file access permission.
If you need access to all files in D:\, the user must manually pick the D:\ drive using the FolderPicker, then you have permissions to access to files in this drive.
var picker = new Windows.Storage.Pickers.FileOpenPicker();
picker.ViewMode = Windows.Storage.Pickers.PickerViewMode.Thumbnail;
picker.SuggestedStartLocation =
Windows.Storage.Pickers.PickerLocationId.ComputerFolder;
picker.FileTypeFilter.Add(".jpg");
picker.FileTypeFilter.Add(".jpeg");
picker.FileTypeFilter.Add(".png");
Windows.Storage.StorageFile file = await picker.PickSingleFileAsync();
if (file != null)
{
// Application now has read/write access to the picked file
}
else
{
//do some stuff
}
I am looking for an elegant way to get the folder hierarchy, beginning with my root folder, using the C# Google Drive API V3.
Currently, you can get the root folder and its parents by
var getRequest = driveService.Files.Get("root");
getRequest.Fields = "parents";
var file = getRequest.Execute();
but I am looking for a way to get the children, not the parents, so I can recursively go down the file structure.
Setting getRequest.Fields = 'children' is not a valid field option.
recursively fetching children is a very time consuming way to fetch the full hierarchy. Much better is to run a query to fetch all folders in a single GET (well it might take more than one if you have more than 1,000 folders) and then traverse their parent properties to build up the hierarchy in memory. Bear in mind that (afaik) there is nothing that prevents a folder hierarchy being cyclic, thus folder1 owns folder2 owns folder3 owns folder1, so whichever strategy you follow, check that you aren't in a loop.
If you're new to GDrive, it's important to realise early on that Folders are simply labels, rather than containers. So cyclic relationships and files with multiple parents is quite normal. They were originally called Collections, but got renamed to Folders to appease those members of the community that couldn't get their head around labels.
I hope this is the answer you were looking for. getHeirarchy Recursively digs Google Drive and stores the file titles into a text file.
public System.IO.StreamWriter w = new System.IO.StreamWriter("Hierarchy.txt", false);
string intend = " ";
private void getHierarchy(Google.Apis.Drive.v2.Data.File Res, DriveService driveService)
{
if (Res.MimeType == "application/vnd.google-apps.folder")
{
w.Write(intend + Res.Title + " :" + Environment.NewLine);
intend += " ";
foreach (var res in ResFromFolder(driveService, Res.Id).ToList())
getHierarchy(res, driveService);
intend = intend.Remove(intend.Length - 5);
}
else
{
w.Write(intend + Res.Title + Environment.NewLine);
}
}
You can call the function something like:
w.Write("My Drive:" + Environment.NewLine);
foreach (var Res in ResFromFolder(driveService, "root").ToList())
getHierarchy(Res, driveService);
w.Close();
Here, root can be replaced with the ID of any Directory to get it's structure. This will generate the entire Drive's structure.
The ResFromFolder method returns a list of Google.Apis.Drive.v2.Data.File metadata contained in a directory.
public List<Google.Apis.Drive.v2.Data.File> ResFromFolder(DriveService service, string folderId)
{
var request = service.Children.List(folderId);
request.MaxResults = 1000;
List<Google.Apis.Drive.v2.Data.File> TList = new List<Google.Apis.Drive.v2.Data.File>();
do
{
var children = request.Execute();
foreach (ChildReference child in children.Items)
{
TList.Add(service.Files.Get(child.Id).Execute());
}
request.PageToken = children.NextPageToken;
} while (!String.IsNullOrEmpty(request.PageToken));
return TList;
}
This code produces output something like
However as pinoyyid mentioned, it does consume a good deal of time if Drive contains a large number of files and folders.
Get folder hierarchy with Google Drive API [C# / .NET]
Google.Apis.Drive.v3.DriveService service = GetService();
List<GoogleDriveFile> folderList = new List<GoogleDriveFile>();
Google.Apis.Drive.v3.FilesResource.ListRequest request = service.Files.List();
//https://developers.google.com/drenter code hereive/api/v3/search-shareddrives
request.Q = string.Format("mimeType='application/vnd.google-apps.folder' and '{0}' in parents", folderId)`enter code here`;
request.Fields = "files(id, name)";
Google.Apis.Drive.v3.Data.FileList result = request.Execute();
foreach (var file in result.Files)
{
GoogleDriveFile googleDriveFile = new GoogleDriveFile
{
Id = file.Id,
Name = file.Name,
Size = file.Size,
Version = file.Version,
CreatedTime = file.CreatedTime,
Parents = file.Parents
};
folderList.Add(googleDriveFile);
}
return folderList;
I'm not really into multithreading so probably the question is stupid but it seems I cannot find a way to solve this problem (especially because I'm using C# and I've been using it for a month).
I have a dynamic number of directories (I got it from a query in the DB). Inside those queries there are a certain amount of files.
For each directory I need to use a method to transfer these files using FTP in a cuncurrent way because I have basically no limit in FTP max connections (not my word, it's written in the specifics).
But I still need to control the max amount of files transfered per directory. So I need to count the files I'm transfering (increment/decrement).
How could I do it? Should I use something like an array and use the Monitor class?
Edit: Framework 3.5
You can use the Semaphore class to throttle the number of concurrent files per directory. You would probably want to have one semaphore per directory so that the number of FTP uploads per directory can be controlled independently.
public class Example
{
public void ProcessAllFilesAsync()
{
var semaphores = new Dictionary<string, Semaphore>();
foreach (string filePath in GetFiles())
{
string filePathCapture = filePath; // Needed to perform the closure correctly.
string directoryPath = Path.GetDirectoryName(filePath);
if (!semaphores.ContainsKey(directoryPath))
{
int allowed = NUM_OF_CONCURRENT_OPERATIONS;
semaphores.Add(directoryPath, new Semaphore(allowed, allowed));
}
var semaphore = semaphores[directoryPath];
ThreadPool.QueueUserWorkItem(
(state) =>
{
semaphore.WaitOne();
try
{
DoFtpOperation(filePathCapture);
}
finally
{
semaphore.Release();
}
}, null);
}
}
}
var allDirectories = db.GetAllDirectories();
foreach(var directoryPath in allDirectories)
{
DirectoryInfo directories = new DirectoryInfo(directoryPath);
//Loop through every file in that Directory
foreach(var fileInDir in directories.GetFiles()) {
//Check if we have reached our max limit
if (numberFTPConnections == MAXFTPCONNECTIONS){
Thread.Sleep(1000);
}
//code to copy to FTP
//This can be Aync, when then transfer is completed
//decrement the numberFTPConnections so then next file can be transfered.
}
}
You can try something along the lines above. Note that It's just the basic logic and there are proberly better ways to do this.
I want to be able to get the size of one of the local directories using C#. I'm trying to avoid the following (pseudo like code), although in the worst case scenario I will have to settle for this:
int GetSize(Directory)
{
int Size = 0;
foreach ( File in Directory )
{
FileInfo fInfo of File;
Size += fInfo.Size;
}
foreach ( SubDirectory in Directory )
{
Size += GetSize(SubDirectory);
}
return Size;
}
Basically, is there a Walk() available somewhere so that I can walk through the directory tree? Which would save the recursion of going through each sub-directory.
A very succinct way to get a folder size in .net 4.0 is below. It still suffers from the limitation of having to traverse all files recursively, but it doesn't load a potentially huge array of filenames, and it's only two lines of code. Make sure to use the namespaces System.IO and System.Linq.
private static long GetDirectorySize(string folderPath)
{
DirectoryInfo di = new DirectoryInfo(folderPath);
return di.EnumerateFiles("*.*", SearchOption.AllDirectories).Sum(fi => fi.Length);
}
If you use Directory.GetFiles you can do a recursive seach (using SearchOption.AllDirectories), but this is a bit flaky anyway (especially if you don't have access to one of the sub-directories) - and might involve a huge single array coming back (warning klaxon...).
I'd be happy with the recursion approach unless I could show (via profiling) a bottleneck; and then I'd probably switch to (single-level) Directory.GetFiles, using a Queue<string> to emulate recursion.
Note that .NET 4.0 introduces some enumerator-based file/directory listing methods which save on the big arrays.
Here my .NET 4.0 approach
public static long GetFileSizeSumFromDirectory(string searchDirectory)
{
var files = Directory.EnumerateFiles(searchDirectory);
// get the sizeof all files in the current directory
var currentSize = (from file in files let fileInfo = new FileInfo(file) select fileInfo.Length).Sum();
var directories = Directory.EnumerateDirectories(searchDirectory);
// get the size of all files in all subdirectories
var subDirSize = (from directory in directories select GetFileSizeSumFromDirectory(directory)).Sum();
return currentSize + subDirSize;
}
Or even nicer:
// get IEnumerable from all files in the current dir and all sub dirs
var files = Directory.EnumerateFiles(searchDirectory,"*",SearchOption.AllDirectories);
// get the size of all files
long sum = (from file in files let fileInfo = new FileInfo(file) select fileInfo .Length).Sum();
As Gabriel pointed out this will fail if you have a restricted directory under the searchDirectory!
You could hide your recursion behind an extension method (to avoid the issues Marc has highlighted with the GetFiles() method):
public static class UserExtension
{
public static IEnumerable<FileInfo> Walk(this DirectoryInfo directory)
{
foreach(FileInfo file in directory.GetFiles())
{
yield return file;
}
foreach(DirectoryInfo subDirectory in directory.GetDirectories())
{
foreach(FileInfo file in subDirectory.Walk())
{
yield return file;
}
}
}
}
(You probably want to add some exception handling to this for protected folders etc.)
Then:
using static UserExtension;
long totalSize = 0L;
var startFolder = new DirectoryInfo("<path to folder>");
// iteration
foreach(FileInfo file in startFolder.Walk())
{
totalSize += file.Length;
}
// linq
totalSize = di.Walk().Sum(s => s.Length);
Basically the same code, but maybe a little neater...
First, forgive my poor english ;o)
I had a problem that took me to this page : enumerate files of a directory and his subdirectories without blocking on an UnauthorizedAccessException, and, like the new method of .Net 4 DirectoryInfo.Enumerate..., get the first result before the end of the entire query.
With the help of various examples found here and there on the web, I finally write this method :
public static IEnumerable<FileInfo> EnumerateFiles_Recursive(this DirectoryInfo directory, string searchPattern, SearchOption searchOption, Func<DirectoryInfo, Exception, bool> handleExceptionAccess)
{
Queue<DirectoryInfo> subDirectories = new Queue<DirectoryInfo>();
IEnumerable<FileSystemInfo> entries = null;
// Try to get an enumerator on fileSystemInfos of directory
try
{
entries = directory.EnumerateFileSystemInfos(searchPattern, SearchOption.TopDirectoryOnly);
}
catch (Exception e)
{
// If there's a callback delegate and this delegate return true, we don't throw the exception
if (handleExceptionAccess == null || !handleExceptionAccess(directory, e))
throw;
// If the exception wasn't throw, we make entries reference an empty collection
entries = EmptyFileSystemInfos;
}
// Yield return file entries of the directory and enqueue the subdirectories
foreach (FileSystemInfo entrie in entries)
{
if (entrie is FileInfo)
yield return (FileInfo)entrie;
else if (entrie is DirectoryInfo)
subDirectories.Enqueue((DirectoryInfo)entrie);
}
// If recursive search, we make recursive call on the method to yield return entries of the subdirectories.
if (searchOption == SearchOption.AllDirectories)
{
DirectoryInfo subDir = null;
while (subDirectories.Count > 0)
{
subDir = subDirectories.Dequeue();
foreach (FileInfo file in subDir.EnumerateFiles_Recursive(searchPattern, searchOption, handleExceptionAccess))
{
yield return file;
}
}
}
else
subDirectories.Clear();
}
I use a Queue and a recursive method to keep traditional order (content of directory and then content of first subdirectory and his own subdirectories and then content of the second...). The parameter "handleExceptionAccess" is just a function call when an exception is thrown with a directory; the function must return true to indicate that the exception must be ignored.
With this methode, you can write :
DirectoryInfo dir = new DirectoryInfo("c:\\temp");
long size = dir.EnumerateFiles_Recursive("*", SearchOption.AllDirectories, (d, ex) => true).Sum(f => f.Length);
And here we are : all exception when trying to enumerate a directory will be ignore !
Hope this help
Lionel
PS : for a reason I can't explain, my method is more quick than the framework 4 one...
PPS : you can get my test solutions with source for those methods : here TestDirEnumerate. I write EnumerateFiles_Recursive, EnumerateFiles_NonRecursive (use a queue to avoid recursion) and EnumerateFiles_NonRecursive_TraditionalOrder (use a stack of queue to avoid recursion and keep traditional order). Keep those 3 methods has no interest, I write them only for test the best one. I think to keep only the last one.
I also wrote the equivalent for EnumerateFileSystemInfos and EnumerateDirectories.
Have a look at this post:
http://social.msdn.microsoft.com/forums/en-US/vbgeneral/thread/eed54ebe-facd-4305-b64b-9dbdc65df04e
Basically there is no clean .NET way, but there is a quite straightforward COM approach so if you're happy with using COM interop and being tied to Windows, this could work for you.
the solution is already here https://stackoverflow.com/a/12665904/1498669
as in the duplicate How do I Get Folder Size in C#? shown -> you can do this also in c#
first, add the COM reference "Microsoft Scripting Runtime" to your project and use:
var fso = new Scripting.FileSystemObject();
var folder = fso.GetFolder(#"C:\Windows");
double sizeInBytes = folder.Size;
// cleanup COM
System.Runtime.InteropServices.Marshal.ReleaseComObject(folder);
System.Runtime.InteropServices.Marshal.ReleaseComObject(fso);
remember to cleanup the COM references
I've been looking some time ago for a function like the one you ask for and from what I've found on the Internet and in MSDN forums, there is no such function.
The recursive way is the only I found to obtain the size of a Folder considering all the files and subfolders that contains.
You should make it easy on yourself. Make a method and passthrough the location of the directory.
private static long GetDirectorySize(string location) {
return new DirectoryInfo(location).GetFiles("*.*", SearchOption.AllDirectories).Sum(file => file.Length);
}
-G