I am using FileSystemWatcher to monitor a folder that will be used to do some file renaming.
The only thing that will be copied will be folders containing files. There will not be single files put into the monitored folder. This is the code for setting up the FileSystemWatcher
watcher.Path = path;
watcher.NotifyFilter = NotifyFilters.DirectoryName | NotifyFilters.FileName;
watcher.IncludeSubdirectories = true;
watcher.Filter = "*.*";
watcher.Created += new FileSystemEventHandler(watcher_Created);
watcher.Changed += new FileSystemEventHandler(watcher_Changed);
watcher.Renamed += new RenamedEventHandler(watcher_Renamed);
watcher.EnableRaisingEvents = true;
There doesn't seem to be any issues with this setup..
The folders being copied can be between 50-200mb big. Is there a way to check/make sure that all the files have completed copying before starting the renaming process.
I tried this thinking that i would get an IOException if the copying was still happening when the GetFiles() was called.
bool finishedCopying = false;
while (!finishedCopying)
{
try
{
List<FileInfo> fileList = directoryInfo.GetFiles().ToList();
AlbumSearch newAlbum = new AlbumSearch(directoryInfo);
return newAlbum;
}
catch (IOException)
{
finishedCopying = false;
}
}
If anymore information is required, just ask an i can provide.
Ta.
I gave this a go using a timer. It may not be the prettiest solution out there but at first testing it seems to be working so far. Essentially what this does is when a folder is copied to the monitored folder it will add the folder path to the AlbumList. The files in that folder will trigger the Created event. This waits for the file to finish copying. Once finished it starts a timer. If a new Created event gets triggered the timer will reset itself.
When the timer.elapsed event is triggered it assumes (and I know assumption is the mother of all f*&k ups) that there are no more files to be copied and can start to process the fully copied folder..
System.Timers.Timer eventTimer = new System.Timers.Timer();
List<string> AlbumList = new List<string>();
private void watcher_Created(object sender, FileSystemEventArgs e)
{
if (Directory.Exists(e.FullPath))
{
AlbumList.Add(e.FullPath);
}
if (File.Exists(e.FullPath))
{
eventTimer.Stop();
FileInfo newTrack = new FileInfo(e.FullPath);
while (IsFileLocked(newTrack))
{
// File is locked. Do Nothing..
}
eventTimer.Start();
}
}
private void eventTimer_Elapsed(object sender, ElapsedEventArgs e)
{
List<string> ItemToRemove = new List<string>();
foreach (var item in AlbumList)
{
DirectoryInfo di = new DirectoryInfo(item);
AlbumSearch newAlbum = new AlbumSearch(di);
if (DoSomethingMethod(newAlbum))
{
ItemToRemove.Add(item);
}
else
{
// why did it fail
}
}
foreach (var path in ItemToRemove)
{
AlbumList.Remove(path);
}
}
private bool DoSomethingMethod(AlbumSearch as)
{
// Do stuff here
return true;
}
This is a small demo app that check files at the beginning, and then uses two hashsets to track copied files. This will only work if the source directory is known. There is no way to know if a file was created from a file copy or from a direct creation, so you can only compare two known directories with Directory.GetFiles. And, as already said in comments, you will still have to check if during the copy process, other files were added / removed / renamed in the old directory
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApplication1
{
class Program
{
static HashSet<string> oldDirFiles = new HashSet<string>();
static HashSet<string> newDirFiles = new HashSet<string>();
static string oldDir = "C:\\New Folder";
static string newDir = "C:\\New Folder 2";
static System.Threading.ManualResetEvent resetEvent = new System.Threading.ManualResetEvent(false);
static void Main(string[] args)
{
System.IO.FileSystemWatcher watcher = new System.IO.FileSystemWatcher();
watcher.Path = newDir;
watcher.NotifyFilter = NotifyFilters.DirectoryName | NotifyFilters.FileName;
watcher.IncludeSubdirectories = true;
watcher.Filter = "*.*";
watcher.Created += watcher_Created;
watcher.Changed += watcher_Changed;
watcher.Renamed += watcher_Renamed;
watcher.EnableRaisingEvents = true;
//get all files in old directory
var oldFiles = Directory.GetFiles(oldDir, "*.*", SearchOption.AllDirectories);
foreach (var file in oldFiles)
oldDirFiles.Add(file);
resetEvent.WaitOne();
//now launch the directory copy
//then you have to check if in the meaning time, new files were added or renamed
//that could be done also with a watcher in the old directory
}
static void watcher_Renamed(object sender, RenamedEventArgs e)
{
throw new NotImplementedException();
}
static void watcher_Changed(object sender, FileSystemEventArgs e)
{
throw new NotImplementedException();
}
static void watcher_Created(object sender, FileSystemEventArgs e)
{
//check if the copied file was in the old directory before starting
if (oldDirFiles.Contains(e.FullPath.Replace(newDir, oldDir)))
{
newDirFiles.Add(e.FullPath);
//if all the files have been copied, the file count will be the same in the two hashsets
//the resetevent.Set() signal the waiting thread and the program can proceed
if (newDirFiles.Count == oldDirFiles.Count)
resetEvent.Set();
}
}
}
}
Related
I posted a question here but deleted it after I found a rather tedious solution
I am trying to write an app that can monitor multiple folders. I looked at the following solutions:
FIleSystemWatcher multiple folders (Dynamically)
Create multiple instances of the same FileSystemWatcher
Monitor multiple folders using FileSystemWatcher
I tried all their solutions. But what would happen is that it only worked for local folders. Not network drives.
My last implementation was a combination of all three:
public static void StartMultipleWatchers(string path)
{
string[] paths = path.Split(',');
foreach (string folderPath in paths)
{
try
{
string folderPathtrim = folderPath.Trim();
WatchFile(folderPathtrim);
}
catch(Exception ex)
{
Logger.Error(ex);
}
}
}
private static void WatchFile(string monitoredDir)
{
FileSystemWatcher fsw = new FileSystemWatcher(monitoredDir, "*.gz");
fsw.Changed += new FileSystemEventHandler(OnChanged);
fsw.Created += new FileSystemEventHandler(OnCreated);
fsw.EnableRaisingEvents = true;
fsw.IncludeSubdirectories = true;
Logger.Info($"Started loop Monitor of Folder: {monitoredDir}");
}
private static void OnCreated(object sender, FileSystemEventArgs e)
{
FileInfo fileInfo = new FileInfo(e.FullPath);
string value = $"Created: {e.FullPath}";
Logger.Info(value);
}
Again this solution only worked for local folder. Or if I only made one watcher that watched one network folder.
Then I tried this very tedious solution:
public static void TestManualWatchers()
{
var fsw1 = new FileSystemWatcher(#"\\lap.org.com\tool_data_odp_ws\metrology\CIM\DATA_READY\");
var fsw2 = new FileSystemWatcher(#"C:\TestPath\");
fsw1.Changed += OnChanged;
fsw1.Created += OnCreated;
fsw1.EnableRaisingEvents = true;
fsw1.IncludeSubdirectories = true;
fsw2.Changed += OnChanged;
fsw2.Created += OnCreated;
fsw2.EnableRaisingEvents = true;
fsw2.IncludeSubdirectories = true;
Logger.Info($"Started watching manual double files");
}
Where I manually create each watcher and it's own properties. What is the difference between this tedious way and the dynamic above?
Is there a way to actually dynamically create individual watchers?
I'm monitoring a Folder for File creation(Copied) event using FileSystem Watcher. I only want the program to process image Files.
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Filter = "*.*";
watcher.Created += new FileSystemEventHandler(watcher_FileCreated);
watcher.Path = path;
So I try to create a Bitmap and avoid the file if an exception is thrown
private static void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
try
{
using (Bitmap test = new Bitmap(Bitmap.FromFile(e.FullPath)))
{
mytoprocesslist.add(e.FullPath);
}
//do my processing with image
Console.WriteLine(e.FullPath);
}
catch (Exception error)
{
Console.WriteLine("File Error");
}
}
This throws Out of Memory exception even when a valid image file is copied, which I think happens because the event was raised before the file was copied completely. How can I get over this? I only want to add the valid image files to a to do list and I will process these images one by one later.
A bit cleaner solution than a Try-Catch might be this one.
Im using this code without any exceptions raised.
private static bool IsImage(string path) {
try {
var result = false;
using (var stream = new FileStream(path, FileMode.Open)) {
stream.Seek(0, SeekOrigin.Begin);
var jpg = new List<string> { "FF", "D8" };
var bmp = new List<string> { "42", "4D" };
var gif = new List<string> { "47", "49", "46" };
var png = new List<string> { "89", "50", "4E", "47", "0D", "0A", "1A", "0A" };
var imgTypes = new List<List<string>> { jpg, bmp, gif, png };
var bytesIterated = new List<string>();
for (var i = 0; i < 8; i++) {
var bit = stream.ReadByte().ToString("X2");
bytesIterated.Add(bit);
var isImage = imgTypes.Any(img => !img.Except(bytesIterated).Any());
if (isImage) {
result = true;
break;
}
}
}
return result;
} catch (UnauthorizedAccessException) {
return false;
}
}
Usage of code
foreach (var file in Directory.EnumerateFiles(#"pathToFlowersFolder"))
{
Console.WriteLine($"File: {file} Result:{IsImage(file)}");
}
Edit
After playing around i got an IO-Exception (File already in use)
After reading this i'd offer you the following solution:
private void button1_Click(object sender, EventArgs e)
{
var watcher = new FileSystemWatcher();
watcher.Created += new FileSystemEventHandler(fileSystemWatcher1_Changed);
watcher.Path = #"c:\temp";
watcher.NotifyFilter = NotifyFilters.FileName | NotifyFilters.Size;
watcher.EnableRaisingEvents = true;
}
private void fileSystemWatcher1_Changed(object sender, System.IO.FileSystemEventArgs e)
{
Thread.Sleep(100); // <- give the Creator some time. Increase value for greate pause
if (IsImage(e.FullPath))
{
Console.WriteLine("success----------->" + e.FullPath);
}
}
Note
This piece of code properly works on my machine. My HDD is an SSD, so you might need to increase the thread-sleeping time. It properly works for all images (jpg, bmp, gif, png) up to a size of 7 Mb (im quite sure and greater).
If this code doesnt works for you, please post the exception rather than uploading your code.
For the first requirement: "I only want the program to process image files"
private static void fileSystemWatcher1_Changed(object sender, FileSystemEventArgs e)
{
string strFileExt = getFileExt(e.FullPath);
// filter file types
if (Regex.IsMatch(strFileExt, #"\.png|\.jpg", RegexOptions.IgnoreCase))
{
//here Process the image file
}
}
For the second requirement: "Out of Memory Exception"
Here what happens is, when the file is created (only file name and some attributes) the system is calling the created event. Then the file changed event is also called
So you have to do the processing in the changed event. Also to prevent duplicate calling you have to add a filter to your watcher.
The following is the complete code.
private void fileSystemWatcher1_Changed(object sender, System.IO.FileSystemEventArgs e)
{
FileInfo fileInfo = new FileInfo(e.FullPath);
string strFileExt = fileInfo.Extension;
// filter file types
if (Regex.IsMatch(strFileExt, #"\.png|\.jpg", RegexOptions.IgnoreCase))
{
//here Process the image file
try
{
using (Bitmap test = new Bitmap(Bitmap.FromFile(e.FullPath)))
{
//Do your code here.
}
}
catch (Exception error)
{
Console.WriteLine("File Error");
}
}
}
private void Form1_Load(object sender, EventArgs e)
{
fileSystemWatcher1.Path = #"C:\Users\Christlin\Desktop\res";
//To Prevent duplicated calling of changed event
fileSystemWatcher1.NotifyFilter = NotifyFilters.FileName | NotifyFilters.Size;
}
My program needs to monitor multiple locations, but trigger the same code for each location. As a single FileSystemWatcher can't monitor multiple locations, but is it possible to create multiple instances of it and pass in a folder path for each?
I can't hard code each FileSystemWatcher as more and more locations will need to be added in time and this needs to be done by the end users, as it is highly impractical for me to have to manually hard code a new FileSystemWatcher each time. So my plan was to have the folder paths saved to a file and the program just creates a FileSystemWatcher for each path in the list. But I have no idea if this is possible in the slightest.
Going on the Factory Method Pattern suggestion here the attempt:
I get the errors: "'List' does not contain a definition for 'add'
public void StartWatchers()
{
string[] ArrayPaths = new string[2];
List<FileSystemWatcher> watchers = new List<FileSystemWatcher>();
ArrayPaths[0] = #"K:\Daily Record Checker\Test\Test1";
ArrayPaths[1] = #"K:\Daily Record Checker\Test\Test2";
int i = 0;
foreach (String String in ArrayPaths)
{
watcher.add(MyWatcherFatory(ArrayPaths[i]));
i++;
}
//Do other stuff....
//....
//Start my watchers...
foreach (FileSystemWatcher watcher in watchers)
{
Watcher.EnableRaisingEvents = true;
i++;
}
}
FileSystemWatcher MyWatcherFatory(string path)
{
FileSystemWatcher watcher = new FileSystemWatcher(path);
watcher.Changed += Watcher_Created;
watcher.Path = path;
watcher.Filter = "*.csv";
return watcher;
}
private void Watcher_Created(object sender, FileSystemEventArgs e)
{
System.Threading.Thread.Sleep(1000);
FileInfo fileInfo = new FileInfo(e.FullPath);
if (!IsFileLocked(fileInfo))
{
CheckNumberOfRecordsInFile(e.FullPath);
}
}
Use the factory method pattern.
FileSystemWatcher MyWatcherFatory(string path, object additionalParameters)
{
FileSystemWatcher watcher = new FileSystemWatcher(path);
watcher.Changed += myWatcherChangedMethod;//Attach them to the same listeners,
//Set additional parameters...
return watcher.
}
EDIT: Based on the information you further provided:
public void StartWatchers()
{
string[] ArrayPaths = new string[2];
List<FileSystemWatcher> watchers = new List<FileSystemWatcher>();
ArrayPaths[0] = #"K:\Daily Record Checker\Test\Test1";
ArrayPaths[1] = #"K:\Daily Record Checker\Test\Test2";
int i = 0;
foreach (String String in ArrayPaths)
{
watchers.Add(MyWatcherFatory(ArrayPaths[i]));
i++;
}
//Do other stuff....
//....
//Start my watchers...
foreach (FileSystemWatcher watcher in watchers )
{
watcher.EnableRaisingEvents = true;;
i++;
}
}
FileSystemWatcher MyWatcherFatory(string path)
{
FileSystemWatcher watcher = new FileSystemWatcher(path);
watcher.Changed += Watcher_Created;
watcher.Path = path;
watcher.Filter = "*.csv";
return watcher;
}
private void Watcher_Created(object sender, FileSystemEventArgs e)
{
System.Threading.Thread.Sleep(1000);
FileInfo fileInfo = new FileInfo(e.FullPath);
if (!IsFileLocked(fileInfo))
{
CheckNumberOfRecordsInFile(e.FullPath);
}
}
My task is to delete file once the processing is completed . I am using FileWatcher to complete this task. It is watching specific folder . Suppose If i copy one file and put that in filewatcher folder it is deleting. Second time when i copy the same file and paste that in the same watching folder. This time it says that Another process is using that file . and exception is throwing . I think i am missing something. Here is my code
private static void Main(string[] args)
{
var fw = new FileSystemWatcher(EmailSetting.DataFolder)
{
IncludeSubdirectories = false
,
EnableRaisingEvents = true
};
fw.Created += (sender, e) =>
{
File.Delete(e.FullPath);
};
Console.ReadLine();
}
You receive the Created event when the file was created (hence the name). But at this point in time the other process that is actually creating it, didn't finish writing content into that file. So the file might be already there, but the other is still working on it (imagine you would copy a 8 GB file).
It would be wiser to simply write the path of the file into a list within the event and let another thread regularly check this concurrent bag (e.g. once a second). First it checks if the file exists and if yes, try to delete it. If succeeded, remove it from the bag, otherwise try again next time.
Code example
private static readonly ConcurrentQueue<FileInfo> _FileCandidates = new ConcurrentQueue<FileInfo>();
private static void Main(string[] args)
{
var watcher = new FileSystemWatcher
{
Path = #"R:\TestFolder",
IncludeSubdirectories = false,
Filter = "*.*",
};
Console.WriteLine("Start watching folder... " + watcher.Path);
watcher.Created += OnFileCreated;
watcher.EnableRaisingEvents = true;
var timer = new Timer
{
AutoReset = true,
Interval = 1000,
};
timer.Elapsed += OnTimerElapsed;
timer.Enabled = true;
Console.ReadKey();
}
static void OnTimerElapsed(object sender, ElapsedEventArgs e)
{
FileInfo file;
var stillInUseFiles = new List<FileInfo>();
Console.WriteLine("Check for file candidates...");
while (_FileCandidates.TryDequeue(out file))
{
try
{
Console.WriteLine("Delete " + file.FullName);
if (file.Exists)
file.Delete();
}
catch (IOException)
{
Console.WriteLine("Could not delete file, try again next time.");
stillInUseFiles.Add(file);
}
}
foreach (var unhappyFile in stillInUseFiles)
{
_FileCandidates.Enqueue(unhappyFile);
}
}
static void OnFileCreated(object sender, FileSystemEventArgs e)
{
Console.WriteLine("Found new file candidate " + e.FullPath);
_FileCandidates.Enqueue(new FileInfo(e.FullPath));
}
I just started with C#, and i tried to create create a FileWatcher, which should print the content of a File, if it is changed:
{
public static void watch()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = "Path";
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "Filter";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
}
public static void OnChanged(object source, FileSystemEventArgs e)
{
using (TextReader r = File.OpenText("Path")) {
while ((s = r.ReadLine()) != null) {
Console.WriteLine(s);
}
r.Close();
}
}
static void Main()
{
watch();
}
}
So far the FileWatcher is working fine, but if I try to print the content it works once and no matter how long I wait the programm will stop working on the second change.
As far as I understood the "using" statement should free the file. The close command does not change anything at all.
The file is a very small text file and should not be a problem.
Is there anyway to force the program to free the File?
The following code works fine:
using System;
using System.IO;
namespace FileReadTest
{
internal class Program
{
public static FileSystemWatcher watch()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = "d:\\";
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "test.txt";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
return watcher;
}
public static void OnChanged(object source, FileSystemEventArgs e)
{
string s;
using (StreamReader r = new StreamReader(File.Open("d:\\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite)))
{
while ((s = r.ReadLine()) != null)
{
Console.WriteLine(s);
}
}
}
static void Main()
{
var watcher = watch();
Console.ReadKey();
watcher.Dispose();
}
}
}
Note that I've changed file reading routine to avoid some reading problems when a file is opened by an other program. FileSystemWatcher will also not get out of scope and will not be disposed accidentally.
I think instead of watcher.EnableRaisingEvents = true, in your case want to use watcher.WaitForChanged(WatcherChangeTypes.All), to make your program wait for changes indefinitely.
BTW, the statement r.Close() is redundant because you are already implicitly call Dispose() via using, which in turn calls Close().
EDIT: To be more specific: WaitForChanged of course just waits for one change and then returns, so if you want to wait for more changes, you can use a loop. Note that no event handler is needed if you use it this way.
while(true)
{
watcher.WaitForChanged(WaitForChanged.All);
// Do stuff with the changed file here, no event handler needed
using(var sr = new StreamReader(filePath))
{
// ...
}
}
the first thing to do is debug and verify if your method OnChanged gets called only once or also at the following edits of the monitored file, then you already know if the issue is with the lifetime / scope of the watcher or somewhere else.
Is this a console application? does it close or stays open after you call the watch method in the main?
You need to dispose the FileSystemWatcher.
public static void OnChanged(object source, FileSystemEventArgs e)
{
using (TextReader r = File.OpenText("Path")) {
while ((s = r.ReadLine()) != null) {
Console.WriteLine(s);
}
r.Close();
}
File((FileSystemWatcher)sender).Dispose();
}