I'm trying to implement a file watcher that will raise an event if the file content was changed.
The problem that once the file I am watching was modified, I am getting 2 events.
(I want to get it only once)
_automationStatusWatcher = new FileSystemWatcher(fileInfo.Directory.FullName,
fileInfo.Name);
_automationStatusWatcher.NotifyFilter = NotifyFilters.LastWrite;
_automationStatusWatcher.Changed += OnAutomationStatusChanged;
_automationStatusWatcher.EnableRaisingEvents = true;
The file that i'm watching is not intended to be recreated/moved/deleted/whatever. its purpose is to be an xml database file that i need to monitor it once it changes. i want to get only 1 event when the file is modified and to ignore the other events.
How can I do that?
Manual:
Note
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
I had to cope with this problem some time ago. I wanted to be notified when a file was created. The problem was that the event is triggered when the file is still empty or still being written to, not when the new file is closed.
The solution was to create the file using a temporary name and when finished renaming the file to its final name. I then watched for the rename-event.
You could name the file to myfile.tmp while writing to it and when finished rename it to myfile.xml and watch the rename-event.
You could attempt to unhook your event handler once you catch a change until you are ready to accept more changes:
_automationStatusWatcher.Changed -= OnAutomationStatusChanged;
But that is not necessarily thread safe and you could get multiple firings anyway.
Another option is to use a semaphore variable to see if you are handling the change:
private bool _inStatusChange = false;
private void OnAutomationStatusChanged(object sender, args...)
{
if (_inStatusChange)
{
return;
}
else
{
_inStatusChange = true;
//do work
_inStatusChange = false;
}
}
You should use appropriate locking to keep access to the semaphore variable thread safe.
i meet this problem too, now i found out this way :
new Thread(() => {
while (true) {
var r = watch.WaitForChanged(WatcherChangeTypes.All);
this.Invoke(new ThreadStart(() => {
listBox1.Items.Add(string.Format("{0} {1} {2}", DateTime.Now, r.Name, r.ChangeType));
}));
}
}) { IsBackground = true }.Start();
its very similar to nio in java
I had a similar problem for checking updates in logging configurations.
I read about the multiple events problem of FileSystemWatcher. So I decided to implement another solution.
I check for every access to my configuration file if it has changed by comparing modified date. Later I added a latence (2 seconds) to avoid too many accesses to filesystem. Maybe you can also use that way.
If you look at the documentation for FileSystemWatcher it says that it will fire multiple times.
Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
I would recommend doing some kind of queuing mechanism so that you don't have multiple actions on the same file.
Related
I'm working on a service where in my OnStartmethod I have the following lines of code to set up my FileSystemWatcher
Log.Info($"File location {_location}");
var watcher = new FileSystemWatcher(_location);
watcher.Changed += new FileSystemEventHandler(OnChanged);
Then in my OnChanged method I am wanting to start a timer like so:
private void OnChanged(object source, FileSystemEventArgs e)
{
Log.Info($"A file has been placed in {_location} starting timer");
OnTimer(null, null); //run immediately at startup
StartEventTimer();
}
The timer code works, so I know that isn't an issue, likewise in my log I know it is checking for the correct location. What is it that I'm missing?
All I'm wanting my code to do is to trigger my timer, the moment a file is placed in my target location yet I've not been able to do so. Am I correct in that I should be using FileSystemWatcherto do this, or should I use something else as this code is within a service?
You might well find that the Changed event is firing more than once on a new file, a common problem with some applications, which might create unwanted side effects later on. Have a look and try changing to Created instead.
If you're looking for a new file appearing in a folder, you should use:
watcher.NotifyFilter = NotifyFilters.FileName;
watcher.Created += OnCreated;
Gist demonstrating it firing twice using Changed on LastWrite and a for predictable behaviour, a Gist demonstrating single fire on file create using Created and NotifyFilter.FileName
Just run it up in a Console App and copy a file into c:\temp.
There are a couple of things it could be based on what you said there.
The first thing of note is that the declaration for var watcher looks like it's not a class variable and will go out of scope when it exits OnStart(). You'll need to move the declaration outside of that.
The second item of interest is that it looks like EnableRaisingEvents isn't being set. A working example of the FileSystemWatcher is below.
public class SomeService
{
private FileSystemWatcher _watcher;
public void OnStart()
{
// set up the watcher
_watcher = new FileSystemWatcher(_location);
_watcher.Path = path;
_watcher.NotifyFilter = NotifyFilters.LastWrite;
_watcher.Filter = "*.*";
_watcher.Changed += new FileSystemEventHandler(OnChanged);
_watcher.EnableRaisingEvents = true;
}
}
EDIT
As Ben Hall mentioned, it is possible that multiple events can be raised for the same file when a file is moved into the folder. As per the MSDN documentation;
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher
I'm making a program that controls a game server. One of the functions I'm making, is a live server logfile monitor.
There is a logfile (a simple textfile) that gets updated by the server as it runs.
How do I continuously check the logfile and output it's content in a RichTextBox?
I did this simple function just try and get the content of the log. It will of course just get the text row by row and output it to my textbox. Also it will lock the program for as long as the loop runs, so I know it's useless.
public void ReadLog()
{
using (StreamReader reader = new StreamReader("server.log"))
{
String line;
// Read and display lines from the file until the end of the file is reached.
while ((line = reader.ReadLine()) != null)
{
monitorTextBox.AppendText(line + "\n");
CursorDown();
}
}
}
But how would you go about solving the live monitoring as simple as possible?
*** EDIT ***
I'm using Prescots solution. great stuff.
At the moment I'm using a sstreamreader to put the text from the file to my textbox. I ran into the problem is that, whenever I tried to access any of the gui controls in my event handler the program just stopped with no error or warnings.
I found out that it has to do with threading. I solved that like this:
private void OnChanged(object source, FileSystemEventArgs e)
{
if (monitorTextField.InvokeRequired)
{
monitorTextField.Invoke((MethodInvoker)delegate { OnChanged(source, e); });
}
else
{
StreamReader reader = new StreamReader("file.txt");
monitorTextField.Text = "";
monitorTextField.Text = reader.ReadToEnd();
reader.Close();
CursorDown();
}
}
Now my only problem is that the file.txt is used by the server so I can't access it, since it's "being used by another process". I can't control that process, so maybe I'm out of luck.
But the file can be opened in notepad while the server is running, so somehow it must be possible. Perhaps I can do a temp copy of the file when it updates and read the copy. I don't know.
Check out the System.IO.FileSystemWatcher class:
public static Watch()
{
var watch = new FileSystemWatcher();
watch.Path = #"D:\tmp";
watch.Filter = "file.txt";
watch.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite; //more options
watch.Changed += new FileSystemEventHandler(OnChanged);
watch.EnableRaisingEvents = true;
}
/// Functions:
private static void OnChanged(object source, FileSystemEventArgs e)
{
if(e.FullPath == #"D:\tmp\file.txt")
{
// do stuff
}
}
Edit: if you know some details about the file, you could handle the most efficent way to get the last line. For example, maybe when you read the file, you can wipe out what you've read, so next time it's updated, you just grab whatever is there and output. Perhaps you know one line is added at a time, then your code can immediately jump to the last line of the file. Etc.
Although the FileSystemWatcher is the most simple solution I have found it to be unreliable in reality.. often a file can be updated with new contents but the FileSystemWatcher does not fire an event until seconds later and often never.
The only reliable way I have found to approach this is to check for changes to the file on a regular basis using a System.Timers.Timer object and checking the file size.
I have written a small class that demonstrates this available here:
https://gist.github.com/ant-fx/989dd86a1ace38a9ac58
Example Usage
var monitor = new LogFileMonitor("c:\temp\app.log", "\r\n");
monitor.OnLine += (s, e) =>
{
// WARNING.. this will be a different thread...
Console.WriteLine(e.Line);
};
monitor.Start();
The only real disadvantage here (apart from a slight performance delay caused by file size checking) is that because it uses a System.Timers.Timer the callback comes from a different thread.
If you are using a Windows Forms or WPF app you could easily modify the class to accept a SynchronizingObject which would ensure the event handler events are called from the same thread.
As #Prescott suggested, use a FileSystemWatcher. And make sure, you open the file with the appropriate FileShare mode (FileShare.ReadWrite seems to be appropriate), since the file might still be opened by the server. If you try to open the file exclusively while it is still used by another process, the open operation will fail.
Also in order to gain a bit of performance, you could remember the last position up to which you already have read the file and only read the new parts.
Use this answer on another post c# continuously read file.
This one is quite efficient, and it checks once per second if the file size has changed.
You can either run it on another thread (or convert to async code), but in any case you would need to marshall the text back to the main thread to append to the textbox.
Try adding a Timer and have the Timer.Tick set to an Interval of 1 second. On Timer.Tick you run the function.
private void myTimer_Tick(object sender, EventArgs e)
{
ReadLog();
}
(I know It's a common problem but I couldn't find an exact answer)
I need to write a windows service that monitors a directory, and upon the arrival of a file, opens it, parses the text, does something with it and moves it to another directory afterwards. I used IsFileLocked method mentioned in this post to find out if a file is still been written. My problem is that I don't know how much it takes for another party to complete writing into the file. I could wait a few seconds before opening the file but this is not a perfect solution since I don't know in which rate is the file written to and a few seconds may not suffice.
here's my code:
while (true)
{
var d = new DirectoryInfo(path);
var files = d.GetFiles("*.txt").OrderBy(f => f);
foreach (var file in files)
{
if (!IsFileLocked(file))
{
//process file
}
else
{
//???
}
}
}
I think you might use a FileSystemWatcher (more info about it here: http://msdn.microsoft.com/it-it/library/system.io.filesystemwatcher(v=vs.110).aspx ).
Specificially you could hook to the OnChanged event and after it raises you can check IsFileLocked to verify if it's still being written or not.
This strategy should avoid you to actively wait through polling.
I am writing some C# code and I need to detect if a specific folder on my windows file system has been opened while the application is running. Is there any way to do it? WinAPI maybe?
There are three API things I think you should check out:
FindFirstChangeNotification() http://msdn.microsoft.com/en-us/library/aa364417%28VS.85%29.aspx
That gives you a handle you can wait on and use to find changes to a file in a particular file, directory, or tree of directories. It won't tell you when a directory is browsed, but it will tell you when a file is saved, renamed, and so on and so forth.
SetWindowsHookEx() http://msdn.microsoft.com/en-us/library/ms644990%28v=VS.85%29.aspx
You can set that up to give you a callback when any number of events occur - in fact I'm pretty positive that you CAN get this callback when a directory is opened, but it will probably be inordinately difficult because you'll be intercepting messages to explorer's window. So you'll be rebooting during debugging.
Windows Shells http://msdn.microsoft.com/en-us/library/bb776778%28v=VS.85%29.aspx
If that wasn't painful enough, you can try writing a shell program.
If you're trying to write a rootkit, I suppose you don't want me to spoil the details for you. If you're NOT trying to write a rootkit, I suggest you look it up - carefully. There are open source rootkits, and they all basically have to monitor file access this way to hide from the user / OS.
Go with the Windows Shell Extensions. You can use Shell Namespace Extensions to make a "virtual" folder that isn't there (or hides a real one), like the GAC (C:\Windows\assembly)
Here are several examples of Shell Extension coding in .Net 4.0.
A Column Handler would let you know when a folder is "Opened", and even let you provide extra data for each of the files (new details columns).
Check out the FileSystemWatcher class.
The closest thing that I can think of, that may be useful to you, is using the static Directory class. It provides methods to determine the last time a file or directory was accessed. You could setup a BackgroundWorker to monitor if the directory was accessed during a specified interval. Keep track of the start and end of the interval by using DateTime, and if the last access time falls between those, then you can use the BackgroundWorker's ProgressChanged event to notify the application.
BackgroundWorker folderWorker = new BackgroundWorker();
folderWorker.WorkerReportsProgress = true;
folderWorker.WorkerSupportsCancellation = true;
folderWorker.DoWork += FolderWorker_DoWork;
folderWorker.ProgressChanged += FolderWorker_ProgressChanged;
folderWorker.RunWorkerAsync();
void FolderWorker_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = (BackgroundWorker)sender;
while(!worker.CancellationPending)
{
DateTime lastAccess = Directory.GetLastAccessTime(DIRECTORY_PATH);
//Check to see if lastAccess falls between the last time the loop started
//and came to end.
if(/*your check*/)
{
object state; //Modify this if you need to send back data.
worker.ReportProgress(0, state);
}
}
}
void FolderWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
//Take action here from the worker.ReportProgress being invoked.
}
You could use the FileSystemInfo's LastAccessProperty. The problem though is that it can be cached.
FileSystemInfo: http://msdn.microsoft.com/en-us/library/975xhcs9.aspx
LastAccessTime Property: http://msdn.microsoft.com/en-us/library/system.io.filesysteminfo.lastaccesstimeutc.aspx
As noted that this can be pre-cached.
"The value of the LastAccessTimeUtc property is pre-cached if the current instance of the FileSystemInfo object was returned from any of the following DirectoryInfo methods:
GetDirectories
GetFiles
GetFileSystemInfos
EnumerateDirectories
EnumerateFiles
EnumerateFileSystemInfos
To get the latest value, call the Refresh method."
Therefore call the Refresh method but it still might not be up to date due to Windows caching the value. (This is according to msdn doc "FileSystemInfo.Refresh takes a snapshot of the file from the current file system. Refresh cannot correct the underlying file system even if the file system returns incorrect or outdated information. This can happen on platforms such as Windows 98." - link: http://msdn.microsoft.com/en-us/library/system.io.filesysteminfo.refresh.aspx
I think the only way you can reliably achieve this is by monitoring the currently running processes and watch closely for new Explorer.exe instances and/or new Explorer.exe spawned threads (the "Run every window on a separate process" setting gets in the way here).
I admit I don't have a clue about how to code this, but that's what I would look for.
My application writes a log file (currently using log4net). I'd like to setup a timer and a background worker to read the log file and print its content into some control in my form, while it's being written.
I can't use the FileSystemWatcher class because seems broken: sometimes the event "changed" fires, sometimes do not. And it has an extremely low "pooling rate".
So I created a Timer and a FileSystemWatcher. On the "tick" event of the timer, the background worker does its job.
The question is: how to read only the lines that are added since the last check of the worker?
public LogForm()
{
InitializeComponent();
logWatcherTimer.Start();
}
private void logWatcherTimer_Tick(object sender, EventArgs e)
{
FileInfo log = new FileInfo(#"C:\log.txt");
if(!logWorker.IsBusy) logWorker.RunWorkerAsync(log);
}
private void logWorker_DoWork(object sender, DoWorkEventArgs e)
{
// Read only new lines since last check.
FileInfo log = (FileInfo) e.Argument;
// Here is the main question!
}
EDIT: Code Solution (maybe there is a more elegant way?):
private void logWatherWorker_DoWork(object sender, DoWorkEventArgs e)
{
// retval
string newLines = string.Empty;
FileInfo log = (FileInfo) e.Argument;
// Just skip if log file hasn't changed
if (lastLogLength == log.Length) return;
using (StreamReader stream = new StreamReader(log.FullName))
{
// Set the position to the last log size and read
// all the content added
stream.BaseStream.Position = lastLogLength;
newLines = stream.ReadToEnd();
}
// Keep track of the previuos log length
lastLogLength = log.Length;
// Assign the result back to the worker, to be
// consumed by the form
e.Result = newLines;
}
Check and store the file size each time you read the log, then start your text reader (or whatever you're using) at that location the next time you read.
You could keep track of the index of the last character read from the stream, and subsequently seek to that position.
Edit: see http://dotnetperls.com/seek for examples.
If all you want is to view you log file on a form as it is being written, why not do something simple like write your own Appender that is backed by a TextBox, RichTextBox, or whatever.
Here are some links that I found just doing a quick Google search for "log4net textbox appender":
http://www.nimblecoder.com/blog/archive/2009/01/30/using-a-delegate-and-custom-appender-with-log4net-to-display.aspx (This one looks pretty cool because it allows you to specify a delegate to execute on every log message, so you would not even be tied to a TextBox. You could write different delegates depending on where you wanted your log output to go).
http://www.l4ndash.com/Log4NetMailArchive%2Ftabid%2F70%2Fforumid%2F1%2Fpostid%2F15133%2Fview%2Ftopic%2FDefault.aspx
http://weblogs.asp.net/psteele/archive/2010/01/25/live-capture-of-log4net-logging.aspx
http://www.l4ndash.com/Log4NetMailArchive%2Ftabid%2F70%2Fforumid%2F1%2Fpostid%2F14923%2Fview%2Ftopic%2FDefault.aspx (This one is an Appender that raises an event for every message that is logged).
http://markmail.org/message/ma62bdjpmab3cn7y (relatively recent - posted in 2008 - uses RichTextBox to generated ColoredConsoleAppender-style output)
http://www.claassen.net/geek/blog/2005/06/log4net-scrollingtextbox.html (This one uses the MemoryAppender to capture the log messages and then writes those messages to a TextBox)
http://code.google.com/p/devdefined-tools/source/browse/trunk/projects/common/DevDefined.Common/Appenders/TextBoxAppender.cs?r=90
I have not tried any of these, so I can't vouch for their quality. But, I think that the approach of using a custom Appender backed by a TextBox seems like a much better approach than trying to watch the log file, read it, and then put the messages in a TextBox.
Some common themes that I noticed while looking briefly over these Appenders:
When you write to the TextBox from the Appender, you might need to use BeginInvoke.
One tricky part seems to be telling the Appender which TextBox to write to. In most cases, the Appender is configured via the config file and then the TextBox is added to the Appender programmatically AFTER the logging system has been initialized (I think you have to either retrieve at least one logger or log at least one message to force all of the lazy initialization to happen).
Be careful about constantly adding lines to the TextBox. You could use up a lot of memory, cause performance issues, or exceed the limit on the TextBox (if there is one). Several of these Appenders include code that removes "old" lines from the TextBox periodically.