How to fast delete many files - c#

I have a folder in Windows Server with subfolders and ≈50000 files. When I click the right mouse button and choose delete (or shift+delete) – all files are deleted in 10-20 seconds.
When I delete files using code – 1500-4000 seconds.
Delete large number of files – don't work for me.
My code:
string folderPath = #"C://myFolder";
DirectoryInfo folderInfo = new DirectoryInfo(folderPath);
folderInfo.Delete(true); // true - recursive, with sub-folders
How to delete files faster?

A much faster way to delete files is to use the Windows functions instead of the .NET ones.
You will need to first import the function:
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool DeleteFile(string lpFileName);
And then you can do this:
string[] files = Directory.EnumerateFiles(path, "*". SearchOption.AllDirectories);
foreach (string file in files)
{
DeleteFile(file);
}
Once the files are deleted, which is the slowest part by using the managed APIs, you can call Directory.DeleteFolder(path, true) to delete the empty folders.

Since the question is actually about deleting network shared folders and it's stated that the explorer based delete is much faster than the C# internal delete mechanism, it might help to just invoke a windows shell based delete.
ProcessStartInfo Info = new ProcessStartInfo();
Info.Arguments = "/C rd /s /q \"<your-path>\"";
Info.WindowStyle = ProcessWindowStyle.Hidden;
Info.CreateNoWindow = true;
Info.FileName = "cmd.exe";
Process.Start(Info);
Ofcourse, you have to replace <your-path>.
However, I don't have the infrastructure and files available to test the performance myself right now.

Not quite sure why the method DirectoryInfo.Delete() takes too much time when deleting folders that have a lot of files and sub-folders. I suspect that the method may also do quite a few things that are unnecessary.
I write a small class to to use Win API without doing too many unnecessary things to test my idea. It takes about 40 seconds to delete a folder that have 50,000 files and sub-folders. So, hope it helps.
I use this PowerScript to generate the testing files.
$folder = "d:\test1";
For ($i=0; $i -lt 50000; $i++)
{
New-Item -Path $folder -Name "test$i.txt" -ItemType "file" -Value $i.ToString();
}
The following is the code in C#.
using System;
using System.Collections.Generic;
//
using System.Runtime.InteropServices;
using System.IO;
//
namespace TestFileDelete
{
class FileDelete
{
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
struct WIN32_FIND_DATAW
{
public FileAttributes dwFileAttributes;
public System.Runtime.InteropServices.ComTypes.FILETIME ftCreationTime;
public System.Runtime.InteropServices.ComTypes.FILETIME ftLastAccessTime;
public System.Runtime.InteropServices.ComTypes.FILETIME ftLastWriteTime;
public UInt32 nFileSizeHigh; // DWORD
public UInt32 nFileSizeLow; // DWORD
public UInt32 dwReserved0; // DWORD
public UInt32 dwReserved1; // DWORD
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 260)]
public String cFileName;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 14)]
public String cAlternateFileName;
};
static readonly IntPtr INVALID_HANDLE_VALUE = new IntPtr(-1);
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
private static extern IntPtr FindFirstFileW(String lpFileName, out WIN32_FIND_DATAW lpFindFileData);
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
private static extern Boolean FindNextFileW(IntPtr hFindFile, out WIN32_FIND_DATAW lpFindFileData);
[DllImport("kernel32.dll")]
private static extern Boolean FindClose(IntPtr handle);
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
public static extern Boolean DeleteFileW(String lpFileName); // Deletes an existing file
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
private static extern Boolean RemoveDirectoryW(String lpPathName); // Deletes an existing empty directory
// This method check to see if the given folder is empty or not.
public static Boolean IsEmptyFolder(String folder)
{
Boolean res = true;
if (folder == null && folder.Length == 0)
{
throw new Exception(folder + "is invalid");
}
WIN32_FIND_DATAW findFileData;
String searchFiles = folder + #"\*.*";
IntPtr searchHandle = FindFirstFileW(searchFiles, out findFileData);
if (searchHandle == INVALID_HANDLE_VALUE)
{
throw new Exception("Cannot check folder " + folder);
}
do
{
if ((findFileData.dwFileAttributes & FileAttributes.Directory) == FileAttributes.Directory)
{
// found a sub folder
if (findFileData.cFileName != "." && findFileData.cFileName != "..")
{
res = false;
break;
}
} // if ((findFileData.dwFileAttributes & FileAttributes.Directory) == FileAttributes.Directory)
else
{
// found a file
res = false;
break;
}
} while (FindNextFileW(searchHandle, out findFileData));
FindClose(searchHandle);
return res;
} // public static Boolean IsEmptyFolder(String folder)
// This method deletes the given folder
public static Boolean DeleteFolder(String folder)
{
Boolean res = true;
// keep non-empty folders to delete later (after we delete everything inside)
Stack<String> nonEmptyFolder = new Stack<String>();
String currentFolder = folder;
do
{
Boolean isEmpty = false;
try
{
isEmpty = IsEmptyFolder(currentFolder);
}
catch (Exception ex)
{
// Something wrong
res = false;
break;
}
if (!isEmpty)
{
nonEmptyFolder.Push(currentFolder);
WIN32_FIND_DATAW findFileData;
IntPtr searchHandle = FindFirstFileW(currentFolder + #"\*.*", out findFileData);
if (searchHandle != INVALID_HANDLE_VALUE)
{
do
{ // for each folder, find all of its sub folders and files
String foundPath = currentFolder + #"\" + findFileData.cFileName;
if ((findFileData.dwFileAttributes & FileAttributes.Directory) == FileAttributes.Directory)
{
// found a sub folder
if (findFileData.cFileName != "." && findFileData.cFileName != "..")
{
if (IsEmptyFolder(foundPath))
{ // found an empty folder, delete it
if (!(res = RemoveDirectoryW(foundPath)))
{
Int32 error = Marshal.GetLastWin32Error();
break;
}
}
else
{ // found a non-empty folder
nonEmptyFolder.Push(foundPath);
}
} // if (findFileData.cFileName != "." && findFileData.cFileName != "..")
} // if ((findFileData.dwFileAttributes & FileAttributes.Directory) == FileAttributes.Directory)
else
{
// found a file, delete it
if (!(res = DeleteFileW(foundPath)))
{
Int32 error = Marshal.GetLastWin32Error();
break;
}
}
} while (FindNextFileW(searchHandle, out findFileData));
FindClose(searchHandle);
} // if (searchHandle != INVALID_HANDLE_VALUE)
}// if (!IsEmptyFolder(folder))
else
{
if (!(res = RemoveDirectoryW(currentFolder)))
{
Int32 error = Marshal.GetLastWin32Error();
break;
}
}
if (nonEmptyFolder.Count > 0)
{
currentFolder = nonEmptyFolder.Pop();
}
else
{
currentFolder = null;
}
} while (currentFolder != null && res);
return res;
} // public static Boolean DeleteFolder(String folder)
};
class Program
{
static void Main(string[] args)
{
DateTime t1 = DateTime.Now;
try
{
Boolean b = FileDelete.DeleteFolder(#"d:\test1");
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
DateTime t2 = DateTime.Now;
TimeSpan ts = t2 - t1;
Console.WriteLine(ts.Seconds);
}
}
}

Related

QueryFullProcessImageName returns error 31

I'm working on an application that should be able to show and retrieve information from all processes in the system.
I try to get the process full path using the function QueryFullProcessImageName but it fails with error 31 (A device attached to the system is not functioning), this happens only with new processes created after the program's first enumeration of the running processes.
This is the method that I use to get the full path:
public static string GetProcessFullPath(SafeProcessHandle Handle)
{
uint MaxPathLength = 1024;
StringBuilder ExePath = new StringBuilder(1024);
if (NativeMethods.Win32ProcessFunctions.QueryFullProcessImageName(Handle.DangerousGetHandle(), 0, ExePath, ref MaxPathLength))
{
return ExePath.ToString();
}
else
{
Win32Exception ex = new Win32Exception(Marshal.GetLastWin32Error());
Logger.WriteEntry(new LogEntry("Non è stato possibile recuperare il percorso completo dell'eseguibile di un processo, codice di errore: " + ex.NativeErrorCode + " (" + ex.Message + ")", EventSource.WindowsAPI, EventSeverity.Error));
return "Non disponibile";
}
}
This is the declaration of the function:
[DllImport("Kernel32.dll", EntryPoint = "QueryFullProcessImageNameW", SetLastError = true, CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool QueryFullProcessImageName(IntPtr ProcessHandle, uint PathType, StringBuilder ExePath, ref uint Characters);
I don't know what could be causing this error, I get notifications of new processes from WMI (Win32_ProcessStartTrace).
This is how I get the process handle that is used in the GetProcessFullPath method (the PID is from the WMI notification):
public static SafeProcessHandle GetProcessHandle(uint ProcessID)
{
IntPtr UnsafeProcessHandle = NativeMethods.Win32ProcessFunctions.OpenProcess(NativeMethods.Win32Enumerations.ProcessAccessRights.PROCESS_ALL_ACCESS, false, ProcessID);
if (UnsafeProcessHandle == IntPtr.Zero)
{
UnsafeProcessHandle = NativeMethods.Win32ProcessFunctions.OpenProcess(NativeMethods.Win32Enumerations.ProcessAccessRights.PROCESS_QUERY_LIMITED_INFORMATION, false, ProcessID);
if (UnsafeProcessHandle == IntPtr.Zero)
{
Win32Exception ex = new Win32Exception(Marshal.GetLastWin32Error());
Logger.WriteEntry(new LogEntry("Non è stato possibile aprire un processo, codice di errore: " + ex.NativeErrorCode + " (" + ex.Message + ")", EventSource.WindowsAPI, EventSeverity.Error));
return new SafeProcessHandle(UnsafeProcessHandle, true);
}
else
{
return new SafeProcessHandle(UnsafeProcessHandle, true);
}
}
else
{
return new SafeProcessHandle(UnsafeProcessHandle, true);
}
}
Edit:
Further testing indicated that the problem occurs even if the process whose full path I'm trying to get was already started when the function is called.
GetProcessImageFileName works but it returns the path in device form
and I need a Win32 path.
You can use GetLogicalDriveStrings and QueryDosDevice convert dos file path to windows file path.
Some code:
// dos file path => windows file path
BOOL DosPathToNtPath(LPTSTR pszDosPath, LPTSTR pszNtPath)
{
TCHAR szDriveStr[500];
TCHAR szDrive[3];
TCHAR szDevName[100];
INT cchDevName;
INT i;
//Check the parameters
if (!pszDosPath || !pszNtPath)
return FALSE;
//Get local disk string
if (GetLogicalDriveStrings(sizeof(szDriveStr), szDriveStr))
{
for (i = 0; szDriveStr[i]; i += 4)
{
if (!lstrcmpi(&(szDriveStr[i]), _T("A:\\")) || !lstrcmpi(&(szDriveStr[i]), _T("B:\\"))) { continue; }
szDrive[0] = szDriveStr[i];
szDrive[1] = szDriveStr[i + 1];
szDrive[2] = '\0';
// Query Dos device name
if (!QueryDosDevice(szDrive, szDevName, 100)) { return FALSE; }
// Hit
cchDevName = lstrlen(szDevName);
if (_tcsnicmp(pszDosPath, szDevName, cchDevName) == 0) {
// Copy drive
lstrcpy(pszNtPath, szDrive);
// Copy path
lstrcat(pszNtPath, pszDosPath + cchDevName);
return TRUE;
}
}
}
lstrcpy(pszNtPath, pszDosPath);
return FALSE;
}
Then simply call,
GetProcessImageFileName(hProcess, szImagePath, MAX_PATH);
DosPathToNtPath(szImagePath,pszFullPath);
You can convert it to C# code.
The QueryFullProcessImageName function fails with ERROR_GEN_FAILURE (31 or 0x1f, "A device attached to the system is not functioning") if the process is a "zombie" process, i.e. the process terminated but not all handles to it were closed. In this case, you can still use QueryFullProcessImageNameW with the PROCESS_NAME_NATIVE flag to get the native path, but you probably just want to skip it since it's no longer running.

.NET 4.5 File.Copy ok, but file does not exist

strange behaviour in our production environment, these days.
I have the following:
try {
var sourcePath = ... // my source path
var destinationPath = ... // guess what? You're right, my destination path
File.Copy(sourcePath, destinationPath);
Log.Debug(string.Format("Image {0} copied successfully", imagename));
}
catch(Exception e) {
// exception handling
}
Both source and destination path are on a network share, a folder on an other (virtual) machine with a large number of files (> 500k).
From the last 2 days, the code above runs, logs the last line (the one stating that image have been copied), but if I check in the destination folder, the supposed destination file does not exist.
I thought that for any I/O error File.Copy would raise an exception, so this thing is driving me mad.
Please note that other code parts that write files in that folder are working correctly. Also, note that all files names are unique (business code not included for brevity is making sure of that), and I think an exception would be thrown or the file would be at least overwritten, in that case.
Has anyone faced the same problem? Possible causes? Any solution?
EDIT 2016-07-01 15:12 (GMT+0200)
Ok, folks, apparently files aren't being deleted at all... simply for apparently no reason at all, after they are copied they're left open in read+write mode from the client connected user.
I found this trying running the reader application on my computer, in debug mode, and trying to open one of the files i knew that were copied recently.
I got an exception stating that the file was opened by someone else, and that seemed weird to me.
Opening Computer Management in the remote server (the one which stores the files), then going to Shared Folders > Open Files, I found that the file was left open in read+write mode from the impersonated user that the web application that copies the files is impersonating to do that job.
Also a whole bunch of other files where in the same conditions, and many others where open in read mode.
I found also in Shared Folders > Sessions, an astronomical long list of session of the impersonated user, all with long idle time.
Since impersonation is used only to copy the files, and then is disposed, I shouldn't expect that, right?
I think maybe there is a problem in the way we impersonate the user during file copy, linked to the large number of files in the destination folder.
I'll check that.
END EDIT
Thanks,
Claudio Valerio
Found a solution, I think.
My problem was the code used for impersonating the user with write permissions on the destination folder.
(In my defence, all this project have been inherited from previous software company, and it's pretty massive, so keeping an eye on everything isn't that easy)
The impersonation process was wrapped in a class implementing IDisposable
public class Impersonator :
IDisposable
{
public Impersonator()
{
string userName = // get username from config
string password = // get password from config
string domainName = // get domain from config
ImpersonateValidUser(userName, domainName, password);
}
public void Dispose()
{
UndoImpersonation();
}
[DllImport("advapi32.dll", SetLastError = true)]
private static extern int LogonUser(
string lpszUserName,
string lpszDomain,
string lpszPassword,
int dwLogonType,
int dwLogonProvider,
ref IntPtr phToken);
[DllImport("advapi32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern int DuplicateToken(
IntPtr hToken,
int impersonationLevel,
ref IntPtr hNewToken);
[DllImport("advapi32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern bool RevertToSelf();
[DllImport("kernel32.dll", CharSet = CharSet.Auto)]
private static extern bool CloseHandle(
IntPtr handle);
private const int LOGON32_LOGON_INTERACTIVE = 2;
private const int LOGON32_PROVIDER_DEFAULT = 0;
private void ImpersonateValidUser(
string userName,
string domain,
string password)
{
WindowsIdentity tempWindowsIdentity = null;
IntPtr token = IntPtr.Zero;
IntPtr tokenDuplicate = IntPtr.Zero;
try
{
if (RevertToSelf())
{
if (LogonUser(
userName,
domain,
password,
LOGON32_LOGON_INTERACTIVE,
LOGON32_PROVIDER_DEFAULT,
ref token) != 0)
{
if (DuplicateToken(token, 2, ref tokenDuplicate) != 0)
{
tempWindowsIdentity = new WindowsIdentity(tokenDuplicate);
impersonationContext = tempWindowsIdentity.Impersonate();
}
else
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
}
else
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
}
else
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
}
finally
{
if (token != IntPtr.Zero)
{
CloseHandle(token);
}
if (tokenDuplicate != IntPtr.Zero)
{
CloseHandle(tokenDuplicate);
}
}
}
private void UndoImpersonation()
{
if (impersonationContext != null)
{
impersonationContext.Undo();
}
}
private WindowsImpersonationContext impersonationContext = null;
}
This class is used this way:
using(new Impersonator())
{
// do stuff with files in here
}
My suspect was that closing the handlers of the impersonated user, somehow, it could break something in the way that windows handles file open by the impersonated user through a network share, as in my case, leaving shared files open in read+write mode, preventing any other process/user to open them.
I modified the Impersonator class as follows:
public class Impersonator :
IDisposable
{
public Impersonator()
{
string userName = // get username from config
string password = // get password from config
string domainName = // get domain from config
ImpersonateValidUser(userName, domainName, password);
}
public void Dispose()
{
UndoImpersonation();
impersonationContext.Dispose();
}
[DllImport("advapi32.dll", SetLastError = true)]
private static extern int LogonUser(
string lpszUserName,
string lpszDomain,
string lpszPassword,
int dwLogonType,
int dwLogonProvider,
ref IntPtr phToken);
[DllImport("advapi32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern int DuplicateToken(
IntPtr hToken,
int impersonationLevel,
ref IntPtr hNewToken);
[DllImport("advapi32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern bool RevertToSelf();
[DllImport("kernel32.dll", CharSet = CharSet.Auto)]
private static extern bool CloseHandle(
IntPtr handle);
private const int LOGON32_LOGON_INTERACTIVE = 2;
private const int LOGON32_PROVIDER_DEFAULT = 0;
private void ImpersonateValidUser(
string userName,
string domain,
string password)
{
WindowsIdentity tempWindowsIdentity = null;
token = IntPtr.Zero;
tokenDuplicate = IntPtr.Zero;
try
{
if (RevertToSelf())
{
if (LogonUser(
userName,
domain,
password,
LOGON32_LOGON_INTERACTIVE,
LOGON32_PROVIDER_DEFAULT,
ref token) != 0)
{
if (DuplicateToken(token, 2, ref tokenDuplicate) != 0)
{
tempWindowsIdentity = new WindowsIdentity(tokenDuplicate);
impersonationContext = tempWindowsIdentity.Impersonate();
}
else
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
}
else
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
}
else
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
}
finally
{
}
}
private void UndoImpersonation()
{
try
{
if (impersonationContext != null)
{
impersonationContext.Undo();
}
}
finally
{
if (token != IntPtr.Zero)
{
CloseHandle(token);
}
if (tokenDuplicate != IntPtr.Zero)
{
CloseHandle(tokenDuplicate);
}
}
}
private WindowsImpersonationContext impersonationContext = null;
private IntPtr token;
private IntPtr tokenDuplicate;
}
Basically I moved the handlers closing in the UndoImpersonation method. Also I had a doubt about leaving the impersonationContext not explicitly disposed, si I disposed it in the Dispose method of the Impersonator class.
Since I put in production this update, I hadn't any other issue with this code, and not any other shared file left open in read+write mode on the destination server.
Maybe not the optimal solution (I still have a whole bunch of sessions in Computer Management > Shared Folders > Sessions, but this seems not harming the system, for now.
If anyone have some additional comment, suggestion or depth study about this situation, I will be please to read.
Thanks,
Claudio

C# Directory.exist always return false on the local network

I'm trying to check wether a directory exist on not or a local network. After some research on stackoverflow and MSDN, I develop my code by using impersonate method. The problem is it's not working very well, The Directory.exists() method always return False Here you have my code (it's nearly the same as the one from MSDN):
public sealed class SafeTokenHandle : SafeHandleZeroOrMinusOneIsInvalid
{
private SafeTokenHandle()
: base(true)
{
}
[DllImport("kernel32.dll")]
[ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success)]
[SuppressUnmanagedCodeSecurity]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool CloseHandle(IntPtr handle);
protected override bool ReleaseHandle()
{
return CloseHandle(handle);
}
}
class Environment
{
[DllImport("advapi32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
public static extern bool LogonUser(String lpszUsername, String lpszDomain, String lpszPassword,
int dwLogonType, int dwLogonProvider, out SafeTokenHandle phToken);
[DllImport("kernel32.dll", CharSet = CharSet.Auto)]
public extern static bool CloseHandle(IntPtr handle);
const int LOGON32_PROVIDER_DEFAULT = 0;
const int LOGON32_LOGON_INTERACTIVE = 2;
private void m_SendAlertes()
{
SafeTokenHandle safeTokenHandle;
string v_pathToDir = "\\192.168.1.199\Clients SiteInternet";
if (!LogonUser("RKalculateur", "SERVEUR2",
"riskedge", LOGON32_LOGON_INTERACTIVE, LOGON32_PROVIDER_DEFAULT, out safeTokenHandle))
{
int ret = Marshal.GetLastWin32Error();
throw new System.ComponentModel.Win32Exception(ret);
}
using (safeTokenHandle)
{
using (WindowsIdentity newId = new WindowsIdentity(safeTokenHandle.DangerousGetHandle()))
{
using (WindowsImpersonationContext impersonatedUser = newId.Impersonate())
{
if (Directory.Exists(#v_pathToDir))
{
// Proceed code here
}
}
}
}
}
}
Here you have a picture of the rights for this directory :
It's probably an issue connected to user permissions.
From MSDN:
If you do not have at a minimum read-only permission to the
directory, the Exists method will return false.
If you're using local account and not domain account, using Directory.Exists() method is problematic.
I had similar problem in the past: I had to check if a net share existed in my network and there was no domain. Your way didn't work for me. In the end, I gave up on Directory.Exists() method and ended up using NET USE command ( http://www.cezeo.com/tips-and-tricks/net-use-command/ )
bool exists = false;
string output = "";
string error = "";
System.Diagnostics.Process process = new System.Diagnostics.Process();
process = new System.Diagnostics.Process();
ExecuteShellCommand(process, "NET USE", "\""+ #path + "\" "+
this.password+ " /USER:"+machinename+"\\"+username + " /PERSISTENT:NO",
ref output, ref error);
Console.WriteLine("\r\n\t__________________________"+
"\r\n\tOutput:" + output.Trim().Replace("\r", " ") +
"\r\n\tError: " + error.Trim().Replace("\r"," "));
if (output.Length>0 && error.Length==0)
{
exists = true;
}
process = new System.Diagnostics.Process();
ExecuteShellCommand(process, "NET USE", " /DELETE " + #path,
ref output, ref error);
....
public void ExecuteShellCommand(System.Diagnostics.Process process, string fileToExecute,
string command, ref string output, ref string error)
{
try
{
string CMD = string.Format(System.Globalization.CultureInfo.InvariantCulture, #"{0}\cmd.exe", new object[] { Environment.SystemDirectory });
string args = string.Format(System.Globalization.CultureInfo.InvariantCulture, "/C {0}", new object[] { fileToExecute });
if (command != null && command.Length > 0)
{
args += string.Format(System.Globalization.CultureInfo.InvariantCulture, " {0}", new object[] { command, System.Globalization.CultureInfo.InvariantCulture });
}
System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo(CMD, args);
startInfo.CreateNoWindow = true;
startInfo.UseShellExecute = false;
startInfo.RedirectStandardOutput = true;
startInfo.RedirectStandardInput = true;
startInfo.RedirectStandardError = true;
process.StartInfo = startInfo;
process.Start();
// timeout
process.WaitForExit(10 * 1000);
output = process.StandardOutput.ReadToEnd();
error = process.StandardError.ReadToEnd();
}
catch (Win32Exception e32)
{
Console.WriteLine("Win32 Exception caught in process: {0}", e32.ToString());
}
catch (Exception e
{
Console.WriteLine("Exception caught in process: {0}", e.ToString());
}
finally
{
// close process and do cleanup
process.Close();
process.Dispose();
process = null;
}
}
I know it's a hack but it worked for me and it's a possibility. (Although you may need to set up a proper net share)

Manipulate Registry Hive files from C#

1.)
How do Load, Edit and Save binary Hive files for registry from C#?
I found this Win32 api.
http://msdn.microsoft.com/en-us/library/ee210770%28VS.85%29.aspx
This guy shared the code to dump the content of binary Hive files to text.
http://www.codeproject.com/KB/recipes/RegistryDumper.aspx
2.)
In addition to manipulating the Hive files, I also search for a method to load the Hive file into registry at runtime using C#
(similar to the Load Hive and Unload Hive commands on the File many in regedit)
/Thanks
Have you looked at the Registry and RegistryKey classes in Microsoft.Win32?
http://msdn.microsoft.com/en-us/library/microsoft.win32.aspx
It sounds like you may need to create your own representation to read the hive file and either queue up or immediately make the corresponding registry changes. Likewise you would need to write your own converter back to disk.
The article below explains how to analyze the registry file without using WinAPI (advapi32.dll). In this particular case the guy is using Mono:
http://volatile-minds.blogspot.com/2011/01/analyzing-windows-nt-registry-without.html
using (FileStream fs = File.OpenRead (path)) {
var data = new byte[checked((int)fs.Length)];
int i = 0;
int read;
using (var ms = new MemoryStream (checked((int)fs.Length))) {
while ((read = fs.Read (data, 0, data.Length)) > 0) {
ms.Write (data, 0, read);
i += read;
}
byte[] hive = ms.ToArray ();
char[] cList = new char[fs.Length];
i = 0;
foreach (byte b in hive)
cList[i++] = (char)b;
string d = new string (cList);
int all = 0;
foreach (Match mx in lf.Matches (d)) { //you can change out the regex you want here.
byte[] bb = new byte[mx.Value.Length];
char[] cb = new char[mx.Value.Length];
for (int k = 0; k < mx.Value.Length; k++) {
bb[k] = (byte)mx.Value[k];
cb[k] = (char)bb[k];
}
all++;
//Console.WriteLine (new string (cb));
}
Console.WriteLine (all.ToString ());
all = 0;
}
}
This is 9 years old, but I figured this could help someone else. I wrote this class that allows you to do something like this:
Hive.AcquirePrivileges() // Acquires the privileges necessary for loading the hive
Hive myregistryhive = Hive.LoadFromFile("hivepathhere") // Loads the hive
// use myregistryhive.RootKey (a RegistryKey), read and/or write to it and its sub keys
myregistryhive.SaveAndUnload() // Unloads the hive
Hive.ReturnPrivileges() // De-elevate back to normal privileges.
The code for the class:
class Hive
{
[DllImport("advapi32.dll", SetLastError = true)]
static extern int RegLoadKey(IntPtr hKey, string lpSubKey, string lpFile);
[DllImport("advapi32.dll", SetLastError = true)]
static extern int RegSaveKey(IntPtr hKey, string lpFile, uint securityAttrPtr = 0);
[DllImport("advapi32.dll", SetLastError = true)]
static extern int RegUnLoadKey(IntPtr hKey, string lpSubKey);
[DllImport("ntdll.dll", SetLastError = true)]
static extern IntPtr RtlAdjustPrivilege(int Privilege, bool bEnablePrivilege, bool IsThreadPrivilege, out bool PreviousValue);
[DllImport("advapi32.dll")]
static extern bool LookupPrivilegeValue(string lpSystemName, string lpName, ref UInt64 lpLuid);
[DllImport("advapi32.dll")]
static extern bool LookupPrivilegeValue(IntPtr lpSystemName, string lpName, ref UInt64 lpLuid);
private RegistryKey parentKey;
private string name;
private string originalPath;
public RegistryKey RootKey;
private Hive() { }
public static Hive LoadFromFile(string Path)
{
Hive result = new Hive();
result.parentKey = RegistryKey.OpenBaseKey(RegistryHive.Users, RegistryView.Default);
result.name = Guid.NewGuid().ToString();
result.originalPath = Path;
IntPtr parentHandle = result.parentKey.Handle.DangerousGetHandle();
RegLoadKey(parentHandle, result.name, Path);
//Console.WriteLine(Marshal.GetLastWin32Error());
result.RootKey = result.parentKey.OpenSubKey(result.name, true);
return result;
}
public static void AcquirePrivileges()
{
ulong luid = 0;
bool throwaway;
LookupPrivilegeValue(IntPtr.Zero, "SeRestorePrivilege", ref luid);
RtlAdjustPrivilege((int)luid, true, false, out throwaway);
LookupPrivilegeValue(IntPtr.Zero, "SeBackupPrivilege", ref luid);
RtlAdjustPrivilege((int)luid, true, false, out throwaway);
}
public static void ReturnPrivileges()
{
ulong luid = 0;
bool throwaway;
LookupPrivilegeValue(IntPtr.Zero, "SeRestorePrivilege", ref luid);
RtlAdjustPrivilege((int)luid, false, false, out throwaway);
LookupPrivilegeValue(IntPtr.Zero, "SeBackupPrivilege", ref luid);
RtlAdjustPrivilege((int)luid, false, false, out throwaway);
}
public void SaveAndUnload()
{
RootKey.Close();
RegUnLoadKey(parentKey.Handle.DangerousGetHandle(), name);
parentKey.Close();
}
}
Edit: Note that this requires administrator privileges.
please see: https://github.com/brandonprry/volatile_reader
It reads offline hives in C# with a GTK interface. No write support yet though.

c# could not get part of file path error

I'm attempting to recursively copy all files and directories and here is my class code. I am getting could not find part of path. I'm going to assume it may be because there is about 20 sub directories to directories (paths like \game\resour.zip\shared\effects\materials\ and such.) Any help would be appreciated.
static IEnumerable<string> GetFiles(string path)
{
Queue<string> queue = new Queue<string>();
queue.Enqueue(path);
while (queue.Count > 0)
{
path = queue.Dequeue();
try
{
foreach (string subDir in Directory.GetDirectories(path))
{
queue.Enqueue(subDir);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
string[] files = null;
try
{
files = Directory.GetFiles(path);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
if (files != null)
{
for (int i = 0; i < files.Length; i++)
{
yield return files[i];
}
}
}
}
In .NET the full path to a file can not be longer than 260 characters, if you wan't to use longer paths you must call the Win API directly using UNC paths and your file paths can be 32,767 characters long, I have added a .NET class that does this, I did not write this code originally but sadly I did not find the original article when looking for it today so I will just enter my modified code here for demonstration purposes, the code is slightly altered from the original code and has some extra functionalty.
/*
All sample code is provided by the Inge Henriksen for illustrative purposes only. These examples have not been thoroughly tested under all conditions. Inge Henriksen, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.
*/
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Win32.SafeHandles;
using System.Runtime.InteropServices;
using System.IO;
using log4net;
namespace MyStuff.DAL.Services.IO.File
{
/// <summary>
/// Class for communicating with the Windows kernel library for low-level disk access.
/// The main purpose of this class is to allow for longer file paths than System.IO.File,
/// this class handles file paths up to 32,767 characters.
/// Note: Always be careful when accessing this class from a managed multi-threaded application
/// as the unmanaged Windows kernel is different, this "crash" causes application unstability
/// if not handled properly.
/// TODO: Look into if there are any significant gains on 64-bit systems using another kind of
/// core component than kernel32.dll.
/// </summary>
public class Win32File
{
ILog Log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
#region DLLImport's
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
static extern bool MoveFile(string lpExistingFileName, string lpNewFileName);
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
static extern bool DeleteFile(string lpFileName);
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
static extern bool CreateDirectoryW(string lpPathName, IntPtr lpSecurityAttributes);
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
static extern SafeFileHandle CreateFileW(string lpFileName, uint dwDesiredAccess,
uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition,
uint dwFlagsAndAttributes, IntPtr hTemplateFile);
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
static extern uint SetFilePointer(SafeFileHandle hFile, long lDistanceToMove, IntPtr lpDistanceToMoveHigh, uint dwMoveMethod);
#endregion
// uint GetMode( FileMode mode )
// Converts the filemode constant to win32 constant
#region GetMode
private uint GetMode(FileMode mode)
{
Log.Debug("Get Win32 file mode");
uint umode = 0;
switch (mode)
{
case FileMode.CreateNew:
umode = Win32FileAttributes.CREATE_NEW;
break;
case FileMode.Create:
umode = Win32FileAttributes.CREATE_ALWAYS;
break;
case FileMode.Append:
umode = Win32FileAttributes.OPEN_ALWAYS;
break;
case FileMode.Open:
umode = Win32FileAttributes.OPEN_EXISTING;
break;
case FileMode.OpenOrCreate:
umode = Win32FileAttributes.OPEN_ALWAYS;
break;
case FileMode.Truncate:
umode = Win32FileAttributes.TRUNCATE_EXISTING;
break;
}
return umode;
}
#endregion
// uint GetAccess(FileAccess access)
// Converts the FileAccess constant to win32 constant
#region GetAccess
private uint GetAccess(FileAccess access)
{
Log.Debug("Get Win32 file access");
uint uaccess = 0;
switch (access)
{
case FileAccess.Read:
uaccess = Win32FileAttributes.GENERIC_READ;
break;
case FileAccess.ReadWrite:
uaccess = Win32FileAttributes.GENERIC_READ | Win32FileAttributes.GENERIC_WRITE;
break;
case FileAccess.Write:
uaccess = Win32FileAttributes.GENERIC_WRITE;
break;
}
return uaccess;
}
#endregion
// uint GetShare(FileShare share)
// Converts the FileShare constant to win32 constant
#region GetShare
private uint GetShare(FileShare share)
{
Log.Debug("Get Win32 file share");
uint ushare = 0;
switch (share)
{
case FileShare.Read:
ushare = Win32FileAttributes.FILE_SHARE_READ;
break;
case FileShare.ReadWrite:
ushare = Win32FileAttributes.FILE_SHARE_READ | Win32FileAttributes.FILE_SHARE_WRITE;
break;
case FileShare.Write:
ushare = Win32FileAttributes.FILE_SHARE_WRITE;
break;
case FileShare.Delete:
ushare = Win32FileAttributes.FILE_SHARE_DELETE;
break;
case FileShare.None:
ushare = 0;
break;
}
return ushare;
}
#endregion
public bool Move(string existingFile, string newFile)
{
Log.Debug(String.Format("Rename the file \"{0}\" to \"{1}\"", existingFile, newFile) );
return Win32File.MoveFile(existingFile, newFile);
}
public bool CreateDirectory(string filepath)
{
Log.Debug(String.Format("Create the directory \"{0}\"", filepath));
// If file path is disk file path then prepend it with \\?\
// if file path is UNC prepend it with \\?\UNC\ and remove \\ prefix in unc path.
if (filepath.StartsWith(#"\\"))
filepath = #"\\?\UNC\" + filepath.Substring(2, filepath.Length - 2);
else
filepath = #"\\?\" + filepath;
return CreateDirectoryW(filepath, IntPtr.Zero);
}
public FileStream Open(string filepath, FileMode mode, uint uaccess)
{
Log.Debug(String.Format("Open the file \"{0}\"", filepath));
//opened in the specified mode and path, with read/write access and not shared
FileStream fs = null;
uint umode = GetMode(mode);
uint ushare = 0; // Not shared
if (mode == FileMode.Append) uaccess = Win32FileAttributes.FILE_APPEND_DATA;
// If file path is disk file path then prepend it with \\?\
// if file path is UNC prepend it with \\?\UNC\ and remove \\ prefix in unc path.
if (filepath.StartsWith(#"\\"))
filepath = #"\\?\UNC\" + filepath.Substring(2, filepath.Length - 2);
else filepath = #"\\?\" + filepath;
SafeFileHandle sh = CreateFileW(filepath, uaccess, ushare, IntPtr.Zero, umode, Win32FileAttributes.FILE_ATTRIBUTE_NORMAL, IntPtr.Zero);
int iError = Marshal.GetLastWin32Error();
if ((iError > 0 && !(mode == FileMode.Append && iError == Win32FileAttributes.ERROR_ALREADY_EXISTS)) || sh.IsInvalid)
{
Log.Error(String.Format("Error opening file; Win32 Error: {0}", iError));
throw new Exception("Error opening file; Win32 Error:" + iError);
}
else
{
fs = new FileStream(sh, FileAccess.ReadWrite);
}
// if opened in append mode
if (mode == FileMode.Append)
{
if (!sh.IsInvalid)
{
SetFilePointer(sh, 0, IntPtr.Zero, Win32FileAttributes.FILE_END);
}
}
Log.Debug(String.Format("The file \"{0}\" is now open", filepath));
return fs;
}
public FileStream Open(string filepath, FileMode mode, FileAccess access)
{
Log.Debug(String.Format("Open the file \"{0}\"", filepath));
//opened in the specified mode and access and not shared
FileStream fs = null;
uint umode = GetMode(mode);
uint uaccess = GetAccess(access);
uint ushare = 0; // Exclusive lock of the file
if (mode == FileMode.Append) uaccess = Win32FileAttributes.FILE_APPEND_DATA;
// If file path is disk file path then prepend it with \\?\
// if file path is UNC prepend it with \\?\UNC\ and remove \\ prefix in unc path.
if (filepath.StartsWith(#"\\"))
filepath = #"\\?\UNC\" + filepath.Substring(2, filepath.Length - 2);
else
filepath = #"\\?\" + filepath;
SafeFileHandle sh = CreateFileW(filepath, uaccess, ushare, IntPtr.Zero, umode, Win32FileAttributes.FILE_ATTRIBUTE_NORMAL, IntPtr.Zero);
int iError = Marshal.GetLastWin32Error();
if ((iError > 0 && !(mode == FileMode.Append && iError != Win32FileAttributes.ERROR_ALREADY_EXISTS)) || sh.IsInvalid)
{
Log.Error(String.Format("Error opening file; Win32 Error: {0}", iError));
throw new Exception("Error opening file; Win32 Error:" + iError);
}
else
{
fs = new FileStream(sh, access);
}
// if opened in append mode
if (mode == FileMode.Append)
{
if (!sh.IsInvalid)
{
SetFilePointer(sh, 0, IntPtr.Zero, Win32FileAttributes.FILE_END);
}
}
Log.Debug(String.Format("The file \"{0}\" is now open", filepath));
return fs;
}
public FileStream Open(string filepath, FileMode mode, FileAccess access, FileShare share)
{
//opened in the specified mode , access and share
FileStream fs = null;
uint umode = GetMode(mode);
uint uaccess = GetAccess(access);
uint ushare = GetShare(share);
if (mode == FileMode.Append) uaccess = Win32FileAttributes.FILE_APPEND_DATA;
// If file path is disk file path then prepend it with \\?\
// if file path is UNC prepend it with \\?\UNC\ and remove \\ prefix in unc path.
if (filepath.StartsWith(#"\\"))
filepath = #"\\?\UNC\" + filepath.Substring(2, filepath.Length - 2);
else
filepath = #"\\?\" + filepath;
SafeFileHandle sh = CreateFileW(filepath, uaccess, ushare, IntPtr.Zero, umode, Win32FileAttributes.FILE_ATTRIBUTE_NORMAL, IntPtr.Zero);
int iError = Marshal.GetLastWin32Error();
if ((iError > 0 && !(mode == FileMode.Append && iError != Win32FileAttributes.ERROR_ALREADY_EXISTS)) || sh.IsInvalid)
{
throw new Exception("Error opening file Win32 Error:" + iError);
}
else
{
fs = new FileStream(sh, access);
}
// if opened in append mode
if (mode == FileMode.Append)
{
if (!sh.IsInvalid)
{
SetFilePointer(sh, 0, IntPtr.Zero, Win32FileAttributes.FILE_END);
}
}
return fs;
}
public FileStream OpenRead(string filepath)
{
Log.Debug(String.Format("Open the file \"{0}\"", filepath));
// Open readonly file mode open(String, FileMode.Open, FileAccess.Read, FileShare.Read)
return Open(filepath, FileMode.Open, FileAccess.Read, FileShare.Read);
}
public FileStream OpenWrite(string filepath)
{
Log.Debug(String.Format("Open the file \"{0}\" for writing", filepath));
//open writable open(String, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None).
return Open(filepath, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
}
public bool Delete(string filepath)
{
Log.Debug(String.Format("Delete the file \"{0}\"", filepath));
// If file path is disk file path then prepend it with \\?\
// if file path is UNC prepend it with \\?\UNC\ and remove \\ prefix in unc path.
if (filepath.StartsWith(#"\\"))
filepath = #"\\?\UNC\" + filepath.Substring(2, filepath.Length - 2);
else
filepath = #"\\?\" + filepath;
Log.Debug(String.Format("The file \"{0}\" has been deleted", filepath));
return DeleteFile(filepath);
}
}
}
to copy recursively use this method
static public void CopyFolder( string sourceFolder, string destFolder )
{
if (!Directory.Exists( destFolder ))
Directory.CreateDirectory( destFolder );
string[] files = Directory.GetFiles( sourceFolder );
foreach (string file in files)
{
string name = Path.GetFileName( file );
string dest = Path.Combine( destFolder, name );
File.Copy( file, dest );
}
string[] folders = Directory.GetDirectories( sourceFolder );
foreach (string folder in folders)
{
string name = Path.GetFileName( folder );
string dest = Path.Combine( destFolder, name );
CopyFolder( folder, dest );
}
}
Not sure if this will be of assistance:
/// <summary>
/// Operates just like GetFileSystemInfos except it is recursive, operates on more than
/// one FileSystemInfo, ensures no paths are duplicateed, and includes the original
/// items in the return.
/// </summary>
/// <param name="files">A collection of files and/or directories to expand.</param>
/// <returns>A Dictionary of all file, directories, and subfiles and subdirectories of
/// those directories and subdirectories. The key is the fullpath, and the value
/// is the FileSystemInfo.</returns>
public static Dictionary<string, FileSystemInfo> ExpandFileSystemInfos(FileSystemInfo[] files)
{
Dictionary<string, FileSystemInfo> resultingFiles = new Dictionary<string,FileSystemInfo>();
//GetFileSystemInfosRecursively will expand everything, except it may contain duplicates
foreach (FileSystemInfo file in GetFileSystemInfosRecursively(files))
{//so we just go through adding them to the Dictionary
if (!resultingFiles.ContainsKey(file.FullName))
{
resultingFiles.Add(file.FullName, file);
}
}
return resultingFiles;
}
//helper method for ExpandFileSystemInfos
private static List<FileSystemInfo> GetFileSystemInfosRecursively(FileSystemInfo[] files)
{
List<FileSystemInfo> resultingFiles = new List<FileSystemInfo>();
foreach (FileSystemInfo file in files)
{
if (file is DirectoryInfo)
{
//get its sub items and pass to another function to process
DirectoryInfo dir = (DirectoryInfo)file;
//recursive call, passing in subdirectories and files of current directory.
//The result returned will be all subdirectories and files nested within the current
//dir. So we add them to our collection of results for each directory we encounter.
resultingFiles.AddRange(GetFileSystemInfosRecursively(dir.GetFileSystemInfos()));
}
//else do nothing, it is a file we already know about
}
//resultingFiles now contains all sub items
//It does not include the directories and files in files however, so we add them as well
resultingFiles.AddRange(files);
return resultingFiles;
}
I used this with a program where the user could choose multiple directories or files to be compressed. I had to handle the case where they explicitely added a directory, and then added a subdirectory/subfile(pointlessly) of that directory. So I wrote this so that you could pass in multiple directory and files in an array, and as it expands them out it will make sure none are duplicated.

Categories

Resources