Develop Reference

filesystemwatcher waiting for file to complete writing

I have an issue with a file system watcher application I am working on. It seems to run fine with a single file but not with multiple files or a folder with files inside.
in the code below, I call IsFileReady to determine if the file has completed writing before trying to copy it to the other folder. However, when a new folder is created with files inside, for some reason it's hanging and not continuing.
I think it's because multiple files are being written at the same time and my code is referencing just a single file, but I am unsure of how to correct this.
Any assistance is appreciated and thank you.
static void Init()
{
string directory = watch_path;
Program._watcher = new FileSystemWatcher(directory);
Program._watcher.Created +=
new FileSystemEventHandler(Program._watcher_Changed);
Program._watcher.EnableRaisingEvents = true;
Program._watcher.IncludeSubdirectories = true;
}
static void _watcher_Changed(object sender, FileSystemEventArgs e)
{
Console.WriteLine("CHANGED, NAME: " + e.Name);
Console.WriteLine("CHANGED, FULLPATH: " + e.FullPath);
try
{
Console.WriteLine("Checking If File Exists at copy location");
if (!File.Exists(copy_path + "\\" + e.Name))
{
Console.WriteLine("Waiting for File to complete write");
WaitForFile(e.FullPath);
Console.WriteLine("Copying file to remote folder");
File.Copy(e.FullPath, copy_path + "\\" + e.Name);
Console.WriteLine("Copy Completed writing to log");
error_handling("File Copy Completed : ", copy_path + "\\" + e.Name, e.FullPath);
}
else
{
Console.WriteLine("Copy Failed Writing to Log");
error_handling("File Copy Skipped, File exists at destination : ", copy_path + "\\" + e.Name, e.FullPath);
}
}
catch (Exception error)
{
Console.WriteLine("Copy process totally failed" + error.Message);
error_handling("File Copy Failed ", "Exception: " + error.Message, "00");
}
}
public static void WaitForFile(string filename)
{
while (!IsFileReady(filename)) { }
}
public static bool IsFileReady(string filename)
{
try
{
using (FileStream inputStream = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.None))
return inputStream.Length > 0;
}
catch (Exception)
{
return false;
}
*Edit
When it hangs, it seems that the WaitForFile never actually finishes up. When copying a single file, it finishes no problem, but I think when there are multiple files it get stuck somehow and it will not go past the
WaitForFile(e.FullPath);
I am intentionally hooking the Created event. the use of this application is to monitor a folder that needs to be identical between multiple web servers. the sync process needs to be as fast as possible as the web servers are load balanced.
the Call to error_handling is just a write to log function as seen below:
public static void error_handling(string message, string fileident, string source)
{
using (System.IO.StreamWriter myFile = new System.IO.StreamWriter(log_folder + "\\" + log_file, true))
{
string finalMessage = string.Format("{0}: {1} SOURCE: {3} - DEST: {2}", DateTime.Now, message, fileident, source, Environment.NewLine);
myFile.WriteLine(finalMessage);
myFile.Close();
}
}

Do we have to check if file exist for each when delete multiple files?

In the code below, I check if file uploaded by only checking on single folder but I noticed that the code stops if the file /new/name.ext is not exist.
Do we have to check for each to continue with code? Is there a way to continue with code without having to check if file exist for every file - or even easier way?
if (System.IO.File.Exists(HttpContext.Current.Server.MapPath("/products/cats/thumb/" + strGuid + strExt))) {
System.IO.File.Delete(HttpContext.Current.Server.MapPath("/products/cats/icons/" + strGuid + strExt));
System.IO.File.Delete(HttpContext.Current.Server.MapPath("/products/cats/thumb/" + strGuid + strExt));
System.IO.File.Delete(HttpContext.Current.Server.MapPath("/products/cats/new/" + strGuid + strExt));
System.IO.File.Delete(HttpContext.Current.Server.MapPath("/products/cats/large/" + strGuid + strExt));
System.IO.File.Delete(HttpContext.Current.Server.MapPath("/products/cats/full/" + strGuid + strExt));
}

Although I normally wouldn't recommend "eating" exceptions, this might be an exception to that exception rule.
I'd write a single method:
void DeleteFile(string filePath)
{
try
{
if(File.Exists(filePath)
{
File.Delete(filePath);
}
}
catch(DirectoryNotFoundException ex)
{
// depending on your environment you might
// be prompted for some comment to indicate
// that you meant to do this because
// it's usually bad.
}
}
Only catch DirectoryNotFoundException. If the directory doesn't exist then the file doesn't exist.
The reason for placing this in a separate method is because if you're concerned about this scenario, then you don't want one Delete to throw an exception that keeps the subsequent deletes from executing.
It doesn't entirely sit well with me to use exception handling for something when there's another way to check. It's better to verify that the file exists. This is only for extreme paranoia.
Here's a question - how likely are these files and directories to get deleted before you delete them? And if one file throws an exception that prevents the others from getting deleted, how serious are the consequences? It might be better to just allow the delete to fail then to get extra paranoid and overdo the checking an exception handling. But it's a short method so if you're worried about it it can't hurt. But this sort of thing can become habit-forming.

Based on UweKeim advice and the source of the ZetaLongPaths
library he mentioned in his comment (Thankful for you) and since I don't need all the Long File Name/Path issue so I have picked few pieces to handle my little application. I didn't fully tested it but it works fine for the code in my question. Hope someone can check and provide better tested code.
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.IO;
using System.Web.UI;
using System.Web.UI.HtmlControls;
//------------------------------------------------------------------------------
// ZetaLongPaths/Source/Runtime/ZlpSafeFileOperations.cs
// https://github.com/UweKeim/ZetaLongPaths/blob/master/Source/Runtime/ZlpSafeFileOperations.cs#L30
//------------------------------------------------------------------------------
namespace MyNameSpace {
public class FileHandling {
//------------------------------------------------------------------------------
///<summary>
/// Simple File Operations Handling
///</summary>
///<remarks>
///
///</remarks>
//------------------------------------------------------------------------------
public static bool SafeFileExists(FileInfo filePath) {
return filePath != null && SafeFileExists(filePath.FullName);
}
public static bool SafeFileExists(string filePath) {
return !string.IsNullOrEmpty(filePath) && System.IO.File.Exists(filePath);
}
public static void SafeMoveFile(FileInfo sourcePath, FileInfo dstFilePath) {
SafeMoveFile(
sourcePath?.FullName.ToString(),
dstFilePath?.FullName);
}
public static void SafeDeleteFile(FileInfo filePath) {
if (filePath != null) {
SafeDeleteFile(filePath.FullName);
}
}
public static void SafeDeleteFile(
string filePath) {
Trace.TraceInformation(#"About to safe-delete file '{0}'.", filePath);
if (!string.IsNullOrEmpty(filePath) && SafeFileExists(filePath)) {
try {
var attributes = System.IO.File.GetAttributes(filePath);
// Remove read-only attributes.
if ((attributes & FileAttributes.ReadOnly) != 0) {
System.IO.File.SetAttributes(
filePath,
attributes & (~(FileAttributes.ReadOnly)));
}
System.IO.File.Delete(filePath);
} catch (UnauthorizedAccessException x) {
var newFilePath =
$#"{filePath}.{Guid.NewGuid():N}.deleted";
Trace.TraceWarning(#"Caught UnauthorizedAccessException while deleting file '{0}'. " +
#"Renaming now to '{1}'. {2}", filePath, newFilePath, x.Message);
try {
System.IO.File.Move(
filePath,
newFilePath);
} catch (Win32Exception x2) {
Trace.TraceWarning(#"Caught IOException while renaming upon failed deleting file '{0}'. " +
#"Renaming now to '{1}'. {2}", filePath, newFilePath, x2.Message);
}
} catch (Win32Exception x) {
var newFilePath =
$#"{filePath}.{Guid.NewGuid():N}.deleted";
Trace.TraceWarning(#"Caught IOException while deleting file '{0}'. " +
#"Renaming now to '{1}'. {2}", filePath, newFilePath, x.Message);
try {
System.IO.File.Move(
filePath,
newFilePath);
} catch (Win32Exception x2) {
Trace.TraceWarning(#"Caught IOException while renaming upon failed deleting file '{0}'. " +
#"Renaming now to '{1}'. {2}", filePath, newFilePath, x2.Message);
}
}
} else {
Trace.TraceInformation(#"Not safe-deleting file '{0}', " +
#"because the file does not exist.", filePath);
}
}
public static void SafeMoveFile(string sourcePath, string dstFilePath) {
Trace.TraceInformation(#"About to safe-move file from '{0}' to '{1}'.", sourcePath, dstFilePath);
if (sourcePath == null || dstFilePath == null) {
Trace.TraceInformation(
string.Format(
#"Source file path or destination file path does not exist. " +
#"Not moving."
));
} else {
if (SafeFileExists(sourcePath)) {
SafeDeleteFile(dstFilePath);
var d = Path.GetDirectoryName(dstFilePath);
if (!System.IO.Directory.Exists(d)) {
Trace.TraceInformation(#"Creating non-existing folder '{0}'.", d);
System.IO.Directory.CreateDirectory(d);
}
System.IO.File.Move(sourcePath, dstFilePath);
} else {
Trace.TraceInformation(#"Source file path to move does not exist: '{0}'.", sourcePath);
}
}
}
}
}
Then tested with
if (MyNameSpace.FileHandling.SafeFileExists(HttpContext.Current.Server.MapPath("/products/cats/thumb/" + strGuid + strExt))) {
MyNameSpace.FileHandling.SafeDeleteFile(HttpContext.Current.Server.MapPath("/products/cats/icons/" + strGuid + strExt).ToString());
MyNameSpace.FileHandling.SafeDeleteFile(HttpContext.Current.Server.MapPath("/products/cats/thumb/" + strGuid + strExt).ToString());
MyNameSpace.FileHandling.SafeDeleteFile(HttpContext.Current.Server.MapPath("/products/cats/new/" + strGuid + strExt).ToString());
MyNameSpace.FileHandling.SafeDeleteFile(HttpContext.Current.Server.MapPath("/products/cats/large/" + strGuid + strExt).ToString());
MyNameSpace.FileHandling.SafeDeleteFile(HttpContext.Current.Server.MapPath("/products/cats/full/" + strGuid + strExt).ToString());
}

Couldn't you just iterate through all the files in folder ..cats/ and its sub folders with right attributes(name / extension)? Then you could just use System.IO.File.Exists().
if (System.IO.File.Exists(Path/to/file.ext)){
System.IO.File.Delete(Path/to/file.ext);
}
The iteration could be done finding all files with
string[] files = Directory.GetFiles(txtFolderPath.Text, "*ProfileHandler.cs", SearchOption.AllDirectories);
and then just doing something like:
foreach(file in files){
if (System.IO.File.Exists(Path/to/file.ext)){
System.IO.File.Delete(Path/to/file.ext);
}
}
E: Sorry for not being able to give you the exact syntax, currently using a computer without a proper IDE to write and test the code.

FileInfo delete() method not deleting files

I have this function to delete files older than X amount of months, but it doesn't seem to delete them when I run the code: any guesses to why? When I check folder they are still there.
public static void deleteFiles()
{
try
{
DirectoryInfo ddi = new DirectoryInfo(destination_path);
if (ddi.Exists)
{
logFile.WriteLine("Log Entry: {0}", String.Format("{0:f}", dt) + System.Environment.NewLine);
foreach (var filename in ddi.EnumerateFiles())
{
FileInfo fi = new FileInfo(filename.ToString());//(destination_path + "\\" + filename);
if (fi.CreationTime < dt.AddMonths(-1) )
{
try
{
fi.Delete();
logFile.WriteLine("{0} was deleted successfully.", destination_path + "\\" + filename);
}
catch (Exception ex)
{
logFile.WriteLine("The deletion process failed: {0}", ex.Message);
}
}
}
logFile.WriteLine(String.Concat(Enumerable.Repeat("-------", 25)));
logFile.WriteLine();
}
}
catch (DirectoryNotFoundException ex)
{
logFile.WriteLine("Log Entry: {0}", String.Format("{0:f}", dt) + System.Environment.NewLine);
logFile.WriteLine("Could not delete files from specified directory: {0}", ex.Message);
logFile.WriteLine(String.Concat(Enumerable.Repeat("-------", 25)));
logFile.WriteLine();
}
}

As "Alessandro D'Andria" points out the documentation says:
https://msdn.microsoft.com/en-us/library/system.io.fileinfo.delete(v=vs.110).aspx
"If the file does not exist, this method does nothing."
This makes you think that the file delete has worked and is masking the issue.
If you look at the line:
foreach (var filename in ddi.EnumerateFiles())
this is returning a FileInfo object which is basically a link to the file you want to delete.
But the next line, creates a new FileInfo object based on the filename.
FileInfo fi = new FileInfo(filename.ToString());
But the filename.ToString() is just returning the filename, there is no path information in here.
If you run through the debugger you will see that the fi object will have a pathname (fi.Directory) which is not your destination_path, but is in fact the path of your running executable.
Therefore, fi.Delete() does not actually find the file, so it does nothing (as per the documentation) but you still write a 'success' message to your log file and no exception is thrown, so you wrongly think that everything has worked.
So you actually want something more like this;
if (filename.CreationTime < dt.AddMonths(-1))
{
try
{
if (!filename.Exists)
throw new Exception("File does not exist");
filename.Delete();
WriteLine("{0} was deleted successfully.", destination_path + "\\" + filename);
}
catch (Exception ex)
{
WriteLine("The deletion process failed: {0}", ex.Message);
}
}

Try with the full path:
string path = Path.Combine(destination_path,filename.ToString());
FileInfo fi = new FileInfo(path);

Try something like.... (Ive left out some of your logic but the principle is the same....)
You don't need to use fileinfo as far as I can see.
{
string[] files = Directory.GetFiles(destination_path);
foreach (string filename in files)
{
if (File.Exists(destination_path + "\\" + filename))
{
try
{
File.Delete(destination_path + "\\" + filename);
}
catch (Exception ex)
{
logFile.WriteLine("The deletion process failed: {0}", ex.Message);
}
}
}
}

Extracting JPG Image From MP4 Video

I am using the below method to extract an image from a uploaded video and placing the image into a folder.
private string GeneratePreviewImageMP4(string FileName, HttpPostedFile file, string ProperPath)
{
string inputfile = System.IO.Path.Combine(Server.MapPath(ProperPath), file.FileName);
string ext = System.IO.Path.GetExtension(FileName);
string thumbpath = AppDomain.CurrentDomain.BaseDirectory + "Reports\\TrainingLibrary\\Videothumbnails\\";
string thumbname = thumbpath + FileName.Replace(ext, "") + ".jpg";
string thumbargs = "-i " + inputfile + " -ss 00:00:25.435 -qscale:v 2 -vframes 1 " + thumbname;
Process thumbproc = new Process();
thumbproc.StartInfo.FileName = "C:\\FFMPEG\\Bin\\ffmpeg.exe";
thumbproc.StartInfo.Arguments = thumbargs;
thumbproc.StartInfo.UseShellExecute = false;
thumbproc.StartInfo.CreateNoWindow = false;
thumbproc.StartInfo.RedirectStandardOutput = false;
try
{
thumbproc.Start();
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
thumbproc.WaitForExit();
thumbproc.Close();
return FileName.Replace(ext, ".jpg");
}
The issue, is that once the image is extracted, I get locked out of the folder. No body has admin rights to that folder. We have to restart the Server just to restore access to that folder.
This is only at certain times, most of the time it works great, but one out of ten times there will be an issue.
Anyone now why this is happening? Does it play the video to extract the image, but does not stop the video?

Try calling Dispose on thumbproc after you are done using it
Try using a lock:
lock (_aStaticObject)
{
//All the code above
}
A word of warning, it can be bad for performance to use a lock inside of a http request (thou shall not mess with thy server's threads), which is why this type of thing belongs in a job.

Creating a new directory structure and moving files using ASP.NET and C#

Can someone tell me what is going to happen in this code when an error is encountered? Ideally it should continue the foreach statement until it gets to the last record, but I suspect it's stopping in the middle of the operation because when I check the number of files moved it's off by 225. If it is in fact stopping because of an error, what can I do to make it continue the loop?
I'm creating a new upload manager for our software and need to clean the old files up. There are about 715 orphaned files equaling around 750 MB after a year and a half of use because the original developers didn't write the code to correctly overwrite old files when a new one was uploaded. They also saved the files in a single directory. I can't stand that so I'm moving all of the files into a structure - Vessel Name - ServiceRequesetID - files uploaded for that service. I'm also giving the users a gridview to view and delete files they no longer need as they work the service.
protected void Button1_Click(object sender, EventArgs e)
{
GridViewRow[] rowArray = new GridViewRow[gv_Files.Rows.Count];
gv_Files.Rows.CopyTo(rowArray, 0);
int i = -1;
foreach(GridViewRow row in rowArray)
{
i++;
string _serviceRequestID = ((Label)gv_Files.Rows[row.RowIndex].FindControl("lbl_SRID")).Text;
string _vesselName = ((Label)gv_Files.Rows[row.RowIndex].FindControl("lbl_VesselID")).Text;
string _uploadDIR = Server.MapPath("uploadedFiles");
string _vesselDIR = Server.MapPath("uploadedFiles" + "\\" + _vesselName);
string _fileName = ((Label)gv_Files.Rows[row.RowIndex].FindControl("lbl_FileName")).Text;
DirectoryInfo dInfo = new DirectoryInfo(_uploadDIR);
DirectoryInfo dVessel = new DirectoryInfo(_vesselDIR);
DirectoryInfo dSRID = new DirectoryInfo(_serviceRequestID);
dInfo.CreateSubdirectory(_vesselName);
dVessel.CreateSubdirectory(_serviceRequestID);
string _originalFile = _uploadDIR + "\\" + _fileName;
string _fileFullPath = Path.Combine(Server.MapPath("uploadedFiles/" + _vesselName + "/" + _serviceRequestID + "/"), _fileName);
FileInfo NewFile = new FileInfo(_fileFullPath);
string _fileUploadPath = _vesselName + "/" + _serviceRequestID + "/" + _fileName;
string sourceFile = _originalFile;
FileInfo _source = new FileInfo(sourceFile);
string destinationFile = _fileFullPath;
try
{
{
File.Move(sourceFile, destinationFile);
movefiles.InsertNewUploadPath(Convert.ToDecimal(_serviceRequestID), 1, _fileUploadPath);
}
}
catch (Exception ex)
{
CreateLogFiles Err = new CreateLogFiles();
Err.ErrorLog(Server.MapPath("Logs/ErrorLog"), ex.Message);
}
}
_utility.MessageBox("Completed processing files.");
}

As long as the error encountered occurs within the try catch clause, the code will continue executing within the foreach loop. However, if the error occurs outside of the try catch, the function will exit and throw an error. How many files does your error log report??