I have a filesystemwatcher that will trigger an event when a file is modified. I want to read from that file once the lock has been removed. At the moment I am just trying to open the file once the event is triggered, when A large file is being copied the file lock stays on for a while after the events have been sent, preventing the file from being opened for read access.
Any suggestions?
This one's actually a bit of a doozie, unless the problem space has changed significantly since I last had to deal with it.
The easiest way is to simply try to open the file, catch the resulting IOException, and if the file is locked, add it to a queue to be checked later. You can't just try to process every file that comes in because there are all kinds of cases where multiple events will be generated for the same file, so setting up a retry loop on every single received event can turn into a disaster, fast. You need to queue them up instead and check the queue at a regular interval.
Here is a basic class template that should help you out with this problem:
public class FileMonitor : IDisposable
{
private const int PollInterval = 5000;
private FileSystemWatcher watcher;
private HashSet<string> filesToProcess = new HashSet<string>();
private Timer fileTimer; // System.Threading.Timer
public FileMonitor(string path)
{
if (path == null)
throw new ArgumentNullException("path");
watcher = new FileSystemWatcher();
watcher.Path = path;
watcher.NotifyFilter = NotifyFilters.FileName;
watcher.Created += new FileSystemEventHandler(FileCreated);
watcher.EnableRaisingEvents = true;
fileTimer = new Timer(new TimerCallback(ProcessFilesTimer),
null, PollInterval, Timeout.Infinite);
}
public void Dispose()
{
fileTimer.Dispose();
watcher.Dispose();
}
private void FileCreated(object source, FileSystemEventArgs e)
{
lock (filesToProcess)
{
filesToProcess.Add(e.FullPath);
}
}
private void ProcessFile(FileStream fs)
{
// Your code here...
}
private void ProcessFilesTimer(object state)
{
string[] currentFiles;
lock (filesToProcess)
{
currentFiles = filesToProcess.ToArray();
}
foreach (string fileName in currentFiles)
{
TryProcessFile(fileName);
}
fileTimer.Change(PollInterval, Timeout.Infinite);
}
private void TryProcessFile(string fileName)
{
FileStream fs = null;
try
{
FileInfo fi = new FileInfo(fileName);
fs = fi.OpenRead();
}
catch (IOException)
{
// Possibly log this error
return;
}
using (fs)
{
ProcessFile(fs);
}
lock (filesToProcess)
{
filesToProcess.Remove(fileName);
}
}
}
(Note - I'm recalling this from memory here so it might not be perfect - let me know if it's buggy.)
Related
Is there an Event I can capture for when a known file has been closed by an external application?
For example, a user is editing a workbook in Excel and I want to read that file as soon as the user finishes working on it and closes the file.
My current solution is to use a combination of FileSystemWatcher and Timer. The FileSystemWatcher will detect when changes have been made to a file, and start a new thread running a Timer to check when the file has closed (via try-catch) However I don't feel as though this is a good solution. If the user forgot to close the file and heads home for the weekend, it feels wasteful for my Timer to be running the whole time. If I increase the interval on my Timer, then my program won't be as responsive. Is there a solution that doesn't involve polling?
EDIT: updated with code example of what I have
private System.Windows.Forms.Timer processTimer;
private string blockedFile;
// Starts here. File changes were detected.
private void OnFileSystemWatcher_Changed(object source, FileSystemEventArgs e)
{
FileSystemWatcher fsw = (FileSystemWatcher)source;
string fullpath = Path.Combine(fsw.Path, fsw.Filter);
StartFileProcessing(fullpath);
}
private void StartFileProcessing(string filePath)
{
if (isFileOpen(new FileInfo(filePath)))
{
blockedFile = filePath;
processTimer = new System.Windows.Forms.Timer();
processTimer.Interval = 1000; // 1 sec
processTimer.Tick += new EventHandler(processTimer_Elapsed);
processTimer.Enabled = true;
processTimer.Start();
}
else
ProcessFile(filePath);
}
private void ProcessFile(string filePath)
{
// Do stuff, read + writes to the file.
}
// GOAL: Without polling, how can I get rid of this step just know right away when the file has been closed?
private void processTimer_Elapsed(object sender, EventArgs e)
{
if (isFileOpen(new FileInfo(blockedFile)) == false)
{
// The file has been freed up
processTimer.Enabled = false;
processTimer.Stop();
processTimer.Dispose();
ProcessFile(blockedFile);
}
}
// Returns true if the file is opened
public bool isFileOpen(FileInfo file)
{
FileStream str = null;
try
{
str = file.Open(FileMode.Open, FileAccess.Read, FileShare.None);
}
catch (IOException)
{
return true;
}
finally
{
if (str != null)
str.Close();
}
return false;
}
My task is to delete file once the processing is completed . I am using FileWatcher to complete this task. It is watching specific folder . Suppose If i copy one file and put that in filewatcher folder it is deleting. Second time when i copy the same file and paste that in the same watching folder. This time it says that Another process is using that file . and exception is throwing . I think i am missing something. Here is my code
private static void Main(string[] args)
{
var fw = new FileSystemWatcher(EmailSetting.DataFolder)
{
IncludeSubdirectories = false
,
EnableRaisingEvents = true
};
fw.Created += (sender, e) =>
{
File.Delete(e.FullPath);
};
Console.ReadLine();
}
You receive the Created event when the file was created (hence the name). But at this point in time the other process that is actually creating it, didn't finish writing content into that file. So the file might be already there, but the other is still working on it (imagine you would copy a 8 GB file).
It would be wiser to simply write the path of the file into a list within the event and let another thread regularly check this concurrent bag (e.g. once a second). First it checks if the file exists and if yes, try to delete it. If succeeded, remove it from the bag, otherwise try again next time.
Code example
private static readonly ConcurrentQueue<FileInfo> _FileCandidates = new ConcurrentQueue<FileInfo>();
private static void Main(string[] args)
{
var watcher = new FileSystemWatcher
{
Path = #"R:\TestFolder",
IncludeSubdirectories = false,
Filter = "*.*",
};
Console.WriteLine("Start watching folder... " + watcher.Path);
watcher.Created += OnFileCreated;
watcher.EnableRaisingEvents = true;
var timer = new Timer
{
AutoReset = true,
Interval = 1000,
};
timer.Elapsed += OnTimerElapsed;
timer.Enabled = true;
Console.ReadKey();
}
static void OnTimerElapsed(object sender, ElapsedEventArgs e)
{
FileInfo file;
var stillInUseFiles = new List<FileInfo>();
Console.WriteLine("Check for file candidates...");
while (_FileCandidates.TryDequeue(out file))
{
try
{
Console.WriteLine("Delete " + file.FullName);
if (file.Exists)
file.Delete();
}
catch (IOException)
{
Console.WriteLine("Could not delete file, try again next time.");
stillInUseFiles.Add(file);
}
}
foreach (var unhappyFile in stillInUseFiles)
{
_FileCandidates.Enqueue(unhappyFile);
}
}
static void OnFileCreated(object sender, FileSystemEventArgs e)
{
Console.WriteLine("Found new file candidate " + e.FullPath);
_FileCandidates.Enqueue(new FileInfo(e.FullPath));
}
I just started with C#, and i tried to create create a FileWatcher, which should print the content of a File, if it is changed:
{
public static void watch()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = "Path";
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "Filter";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
}
public static void OnChanged(object source, FileSystemEventArgs e)
{
using (TextReader r = File.OpenText("Path")) {
while ((s = r.ReadLine()) != null) {
Console.WriteLine(s);
}
r.Close();
}
}
static void Main()
{
watch();
}
}
So far the FileWatcher is working fine, but if I try to print the content it works once and no matter how long I wait the programm will stop working on the second change.
As far as I understood the "using" statement should free the file. The close command does not change anything at all.
The file is a very small text file and should not be a problem.
Is there anyway to force the program to free the File?
The following code works fine:
using System;
using System.IO;
namespace FileReadTest
{
internal class Program
{
public static FileSystemWatcher watch()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = "d:\\";
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "test.txt";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
return watcher;
}
public static void OnChanged(object source, FileSystemEventArgs e)
{
string s;
using (StreamReader r = new StreamReader(File.Open("d:\\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite)))
{
while ((s = r.ReadLine()) != null)
{
Console.WriteLine(s);
}
}
}
static void Main()
{
var watcher = watch();
Console.ReadKey();
watcher.Dispose();
}
}
}
Note that I've changed file reading routine to avoid some reading problems when a file is opened by an other program. FileSystemWatcher will also not get out of scope and will not be disposed accidentally.
I think instead of watcher.EnableRaisingEvents = true, in your case want to use watcher.WaitForChanged(WatcherChangeTypes.All), to make your program wait for changes indefinitely.
BTW, the statement r.Close() is redundant because you are already implicitly call Dispose() via using, which in turn calls Close().
EDIT: To be more specific: WaitForChanged of course just waits for one change and then returns, so if you want to wait for more changes, you can use a loop. Note that no event handler is needed if you use it this way.
while(true)
{
watcher.WaitForChanged(WaitForChanged.All);
// Do stuff with the changed file here, no event handler needed
using(var sr = new StreamReader(filePath))
{
// ...
}
}
the first thing to do is debug and verify if your method OnChanged gets called only once or also at the following edits of the monitored file, then you already know if the issue is with the lifetime / scope of the watcher or somewhere else.
Is this a console application? does it close or stays open after you call the watch method in the main?
You need to dispose the FileSystemWatcher.
public static void OnChanged(object source, FileSystemEventArgs e)
{
using (TextReader r = File.OpenText("Path")) {
while ((s = r.ReadLine()) != null) {
Console.WriteLine(s);
}
r.Close();
}
File((FileSystemWatcher)sender).Dispose();
}
I am trying to write a simple serial port data logging app which monitors the serial port and writes the data to a file.
If there is a long gap between the data the the file is closed and a new one started.
I use a timer to determine when to close the file. and the serial event handler to open the file.
The problem is that when the file is closed the file object seems to be null; The file itself is created by another object as it does other stuff deleted from my example code.
For some reason the object created by the serial event handler is valid but the file within the object is null.
I am a very experienced C and linux kernal programmer but new to C# so the syntax and object nature still catches me out
but I cannot see any reason why this should not work unless I am losing an object context somewhere.
That might be a problem as there are four serial port objects open at the same time writing to four different files.
I already solved my timer problem which wouldn't work when called from the serial handler by chaning from a FOrms.timer to Timers.timer.
so I am wondering if its a threading issue.
Any light you can shed would be welcome.
class CommGroup
{
private SerialPort _comPort;
private System.Timers.Timer myTimer;
private ResultsLog logFile = null;
public CommGroup()
{
_comPort = new SerialPort(string name);
_comPort.PortName = name;
_comPort.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandler);
_comPort.Open();
myTimer = new System.Timers.Timer();
myTimer.Elapsed += new System.Timers.ElapsedEventHandler(TimerEventProcessor);
}
private void TimerEventProcessor(Object myObject,
EventArgs myEventArgs)
{
System.Timers.Timer timer = (System.Timers.Timer) myObject ;
timer.Stop();
if (logFile != null)
{
logFile.Close(); /* this fails due to null object */
}
}
private void DataReceivedHandler(object sender,SerialDataReceivedEventArgs e)
{
SerialPort sp = (SerialPort)sender;
if (myTimer.Enabled == false)
{
myTimer.Interval = 5000;
myTimer.Start();
}
else
{
myTimer.Stop();
myTimer.Interval = 5000;
myTimer.Start();
}
if (logFile == null)
{
logFile = new ResultsLog("Filename");
}
logFile.LogResult(sp.ReadExisting());
}
}
public class ResultsLog
{
private StreamWriter resultFile;
public ResultsLog(string filename)
{
StreamWriter resultFile = new StreamWriter(filename, true);
}
public void Close(string errorname)
{
if (this.resultFile != null)
{
resultFile.Flush();
resultFile.Close();
}
else
{
MessageBox.Show("File NULL Error " + errorname, "File Close");
}
}
public void LogResult(string result)
{
if (resultFile != null)
{
resultFile.Write(result);
}
}
}
Is your resultFile the troublesome null?
You are re-declaring locally in your ResultsLog constructor. Should be:
public ResultsLog(string filename)
{
resultFile = new StreamWriter(filename, true);
}
Definately could be threading because you have multiple threads accessing the log object without syncronization.
Use lock around your accesses to the varibale and see if it helps.
I need to create a C# Console Application that will parse the file from SFTP directory when the new file created. For that I implemented FileSystemWatcher with FileCreated event which enqueue the new file path and create a new thread and parse the file.
I read in the blogs that some times FileSystemWatcher may fail to detect new files, for that I implemented Timer which will fire every 1 hr and if the FileSystemWatcher thread is in waitsleep state then will read the IMCOMING SFTP folder and parse the file.
Below is the code i written for FileSystemWatcher and Timer, but its not working properly and I think filesystemwatcher is not in Multithreading. Please help me to get right solution.
MAIN
static void Main(string[] args)
{
try
{
string path = incomingFilePath;
if (Directory.Exists(path))
{
#region Initiate Timer
Thread t = new Thread(new ParameterizedThreadStart(ThreadLoop));
t.Start((Action)fileProcessor.StartTimer);
#endregion
#region FileSystemWatcher
watcher = new FileSystemWatcher { Path = path, Filter = "*.CUST", IncludeSubdirectories = true };
watcher.Created += new
FileSystemEventHandler(watcher_FileCreated);
watcher.Error += new
ErrorEventHandler(watcher_OnError);
watcher.EnableRaisingEvents = true;
#endregion
}
}
catch (Exception Err)
{
}
}
FILESYSTEMWATCHER CODE:
private static void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
if (e.FullPath.ToUpper().Contains("INCOMING"].ToString()))
{
fileProcessor.EnqueueFile(e.FullPath);
lock (lockObject)
{
files.Enqueue(path);
}
if (FileWacherThread == null || shouldStop)
{
FileWacherThread = new Thread(new ThreadStart(Work));
FileWacherThread.Start();
}
// If the thread is waiting then start it
else if (FileWacherThread.ThreadState == ThreadState.WaitSleepJoin)
{
waitHandle.Set();
}
}
}
private void Work()
{
while (!shouldStop)
{
string path = String.Empty;
lock (lockObject)
{
if (files.Count > 0)
{
path = files.Dequeue();
}
}
if (!String.IsNullOrEmpty(path))
{
// Process the file
ParseFile(path);
}
else
{
// If no files are left to process then wait
waitHandle.WaitOne();
}
}
}
TIMER CODE
public void StartTimer()
{
lock (lockObject)
{
if (FileWacherThread == null || FileWacherThread.ThreadState == ThreadState.WaitSleepJoin)
{
if (files.Count == 0)
{
IEnumerable<string> result = new List<string>(Directory.GetFiles(incomingFilePath, "*.CUST", SearchOption.AllDirectories)).Where(s => s.Contains(incomingFilePrefix));
foreach (string path in result)
{
ParseFile(path);
}
}
}
}
}
Things to check...
is waitHandle an AutoResetEvent or a ManualResetEvent? (from the way that you are using it, it should be an AutoResetEvent
If shouldStop is true, is FileWacherThread(sic) set to null when Work() exits...
How are you protecting access to FileWacherThread? if it is accessed from multiple threads (to check its state, assign etc, then it too should be protected with a lock).
You shouldn't worry about the state of the FileWacherThread when you set the event. If you want to signal to that thread, just set it, (i.e. build your multithreaded code such that the publisher doesn't know/care about the current state of the subscriber).
Currently there are states that your FileWacherThread can be in where it isn't waiting but it might still need to be signaled. If you always set the event, the worst that can happen is that it loops one time unnecessarily.