Develop Reference

Reducing the number of monitored files via FileSystemWatcher

My Windows Service application is writing logs to files in a folder every second. I want to monitor the log activities, if there are no logging for some time, it will signal error.
Below is an example of the log file names. Only the latest log file is being written into at a time, other files are not written into. The Monitor app cares about only the last log file.
MyApplication.MachineName.2018-06-05.log
MyApplication.MachineName.2018-06-04.001.log
Below is the code that monitors the log activities.
private void WatchFileChanges()
{
// Create a new FileSystemWatcher and set its properties.
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = #"C:\Logs";
/* Watch for changes in LastWrite times */
watcher.NotifyFilter = NotifyFilters.LastWrite;
// Only watch text files.
watcher.Filter = "*.log";
// Add event handlers.
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.Created += new FileSystemEventHandler(OnChanged);
// Begin watching.
watcher.EnableRaisingEvents = true;
}
// Define the event handlers.
private void OnChanged(object source, FileSystemEventArgs e)
{
Console.WriteLine("File: " + e.FullPath + " " + e.ChangeType);
}
Question:
Is there a way to reduce the number of log files being monitored under the folder? For example, monitor only log files for today, or last two days, rather than all files in the folder. Because the internal buffer is limited, excess log files might cause the monitoring not working.

You need to get a little bit creative here, and the simplest approach is just the FileSystemWatcher.Filter Property
Gets or sets the filter string used to determine what files are
monitored in a directory.
Remarks
To watch changes in all files, set the Filter property to an empty
string (""). To watch a specific file, set the Filter property to the
file name. For example, to watch for changes in the file MyDoc.txt,
set the Filter property to "MyDoc.txt". You can also watch for changes
in a certain type of file. For example, to watch for changes in any
text files, set the Filter property to "*.txt". Use of multiple
filters such as "*.txt|*.doc" is not supported.
Some further examples
*.* : All files (default). An empty string ("") also watches all files.
*.txt : All files with a "txt" extension.
*recipe.doc : All files ending in "recipe" with a "doc" extension.
win*.xml : All files beginning with "win" with an "xml" extension.
Sales*200?.xls :
Matches the following:
Sales July 2001.xls
Sales Aug 2002.xls
Sales March 2004.xls
but does not match:
Sales Nov 1999.xls
MyReport.Doc : Watches only MyReport.doc
Now with this information, you can easily determine if you can create a filter for the current days logs, and if you can, then you can change the filter dynamically each day to target those logs. As noted in the documentation
The Filter property can be changed after the FileSystemWatcher object
has started receiving events.
Or as mentioned in the comments,
Either put the days logs in a different folder
If you are using a logging framework name your logs different for the current day so they "are" targetable
Or increase the FileSystemWatcher buffer and monitor everything
This is your mission, if you choose to accept it.

How to write FileSystemWatcher without triggering an infinite loop

How do I write a file in C# to a folder path that is watched by FileSystemWatcher?
My fileSystemWatcher settings are as follows:
public FileSystemWatcher CreateAndExecute(string path)
{
Console.WriteLine("Watching " + path);
//Create new watcher
FileSystemWatcher fileSystemWatcher = new FileSystemWatcher();
fileSystemWatcher.Path = path;
fileSystemWatcher.IncludeSubdirectories = false;
fileSystemWatcher.NotifyFilter = NotifyFilters.LastWrite |
NotifyFilters.FileName | NotifyFilters.DirectoryName;
fileSystemWatcher.Filter = "*.txt";
fileSystemWatcher.Changed += new FileSystemEventHandler(OnChange);
fileSystemWatcher.InternalBufferSize = 32768;
//Execute
fileSystemWatcher.EnableRaisingEvents = true;
}
private void OnChange(object source, FileSystemEventArgs e)
{
//Replace modified file with original copy
}
I want to replace a modified file's contents with a back up copy from a database whenever an unauthorized write (out of the program) happens to the file.
However, when i write using File.WriteAllText() to the modified file, it triggers a Change event by the FileSystemWatcher as the action is registered as a write again.
This causes the program to run in an endless loop of overwriting the file it has just wrote.
How can I replace the modified file with the backup copy, without triggering another event to write by FileSystemWatcher?

Apart from the fact that you can probably solve the problem in a better/different way by using the OS file security f.e., you have some options:
Temporarily disable the watcher, in which case you can loose events in case of a brute force attack, which might not be what you want.
Keep a list with files you've rewritten and ignore one change for files on the list and remove it from the list afterwards -> can also be abused if the malicious program knows this
Store a SHA1 or SHA256 (or other hash) of the file content and only replace the file if the hash is different -> probably the best approach for this problem

is there any way for me to know if a file has been modified?

I am making a program that detects if a file has been modified. but the file is not from me. I wanna know if it is possible to know if the files I'm getting were already modified or not.
Is there a way for me to know that? I've tried the creation date and date modified but sometimes when I modify a file their values will be the same.
P.S. I don't have the original file. I wanna know if it is possible to know if the I'm getting were unchanged before i get it.

The following example creates a FileSystemWatcher to watch the directory specified at run time. The component is set to watch for changes in LastWrite and LastAccess time, the creation, deletion, or renaming of text files in the directory. If a file is changed, created, or deleted, the path to the file prints to the console. When a file is renamed, the old and new paths print to the console.
Use the System.Diagnostics and System.IO namespaces for this example.
using System;
using System.IO;
using System.Security.Permissions;
public class Watcher
{
public static void Main()
{
Run();
}
[PermissionSet(SecurityAction.Demand, Name="FullTrust")]
public static void Run()
{
string[] args = System.Environment.GetCommandLineArgs();
// If a directory is not specified, exit program.
if(args.Length != 2)
{
// Display the proper way to call the program.
Console.WriteLine("Usage: Watcher.exe (directory)");
return;
}
// Create a new FileSystemWatcher and set its properties.
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = args[1];
/* Watch for changes in LastAccess and LastWrite times, and
the renaming of files or directories. */
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
// Only watch text files.
watcher.Filter = "*.txt";
// Add event handlers.
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.Created += new FileSystemEventHandler(OnChanged);
watcher.Deleted += new FileSystemEventHandler(OnChanged);
watcher.Renamed += new RenamedEventHandler(OnRenamed);
// Begin watching.
watcher.EnableRaisingEvents = true;
// Wait for the user to quit the program.
Console.WriteLine("Press \'q\' to quit the sample.");
while(Console.Read()!='q');
}
// Define the event handlers.
private static void OnChanged(object source, FileSystemEventArgs e)
{
// Specify what is done when a file is changed, created, or deleted.
Console.WriteLine("File: " + e.FullPath + " " + e.ChangeType);
}
private static void OnRenamed(object source, RenamedEventArgs e)
{
// Specify what is done when a file is renamed.
Console.WriteLine("File: {0} renamed to {1}", e.OldFullPath, e.FullPath);
}
}
Reference: Microsoft MSDN

If you want to know if the content of a file changed, compute its hash:
using System.IO;
using System.Security.Cryptography;
using (HashAlgorithmalgorithm = new .SHA512Managed())
{
using (Stream fileStream = new FileStream(#"path", FileMode.Open))
{
byte[] hash = algorithm.ComputeHash(fileStream);
}
}
Persist the hash as you like on the first run, then recompute it and match it upon the saved value. If they are different, the file changed.

I think this question has a few issues, as there are multiple ways to answer this.
1) If the question is, "I have received a file (that previously was not on the computer/network/location), and want to know if it has been modified...", then the answer is - no, because you're looking at a just a chunk of bits, then all of those bits are subject to being modified and adjusted in any way the sender wants - timestamps, file-status indicators (For instance, the 'archive' tag on Windows systems, or structured metadata on newer file systems) or anything else about that file can be set to what they want.
2) If the question is, "I have a file (on the local network or file-system that I can 'poke' every so often, or locally on the drive that I can 'poke' every so often) and I want to know if it has been modified between the times my application has run", then probably the easiest way to do this is to store a computed hash of that file (As answered by #Albireo).
3) If the question is, "I have a file (on the local network or file-system) and I want to know if it was modified prior to any running of my application", then you're looking at something that may or may not be possible, and will not be possible in a completely reliable way - having some understanding of the underlying structure and what you're expecting in that file will be necessary, and you'd be better off updating your question with more details of what the file consists of and why you're looking for changes programatically so that we can determine the best way of doing this.

Using FileSystemWatcher to monitor file creation and copy it before deleted

We have a 3rd party application that writes a file to a directory and then deletes it.
We want to copy that file before it is deleted.
We have this:
FileSystemWatcher watcher;
private void WatchForFileDrop()
{
watcher = new FileSystemWatcher();
watcher.Path = "c:\\FileDrop";
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.FileName;
watcher.Filter = "*.txt";
watcher.Created += new FileSystemEventHandler(OnCreated);
watcher.EnableRaisingEvents = true;
}
private void OnCreated(object source, FileSystemEventArgs e)
{
//Copy the file to the file drop location
System.IO.File.Copy(e.FullPath, "C:\\FileDropCopy\\" + e.Name);
}
The FileSystemWatcher does work. It will see that the file has been created and it goes to the OnCreated(). The file is created in the directory.
The only problem is the file is empty and the file size is 0kb.
I wanted to double check my thinking of why the file is empty. Is it because the file is deleted so quickly by the 3rd party application that it doesn't have the chance to do a proper copy?
thanks for taking a look.

Options 1:
Instead of looking at FileSystemWatcher, you should look to hook your code to delete event.
You can look this this comment on Stack Overflow: https://stackoverflow.com/a/4395147/442470
Option 2:
As soon as your FileSystemWatcher realizes that a file is created, change permission of the file so that it cannot be deleted.

Windows: How to determine if a file has been modified since a given date

I have a utility which goes through a processes a set of files in a directory - the process is relatively slow (and there are a lot of files) and so I've tried to optimise the process by only processes files that have a "last modified" later than the last processing date.
Usually this works well however I've found that as copying a file doesn't change the last modified date, and so there are various scenarios involving copying files in which certain files that have changed are skipped by the process, for example:
The user processes the directory at 9:00.
A file is then copied from this directory and modified so that it has a last modified date of 9:30
The directory is then processed again at 10:00
The modified file is then copied back into the directory at 10:30
Finally the directory is processed again at 11:00
As the modified date of the given file is 9:30, and the directory was last processed at 10:00 the file is skipped when it shouldn't be.
Unfortunately the above tends to happen far too often in certain situations (such as in a collaborative environment with source control etc...). Clearly my logic is flawed - what I really need is a "last modified or copied" date. does such a thing exist?
Failing that, is there another way to quickly determine with reasonable reliability if a given file has changed?

You might want to look at using the FileSystemWatcher class. This class lets you monitor a directory for changes and will fire an event when something is modified. Your code can then handle the event and process the file.
From MSDN:
// Create a new FileSystemWatcher and set its properties.
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = args[1];
/* Watch for changes in LastAccess and LastWrite times, and
the renaming of files or directories. */
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
// Only watch text files.
watcher.Filter = "*.txt";
// Add event handlers.
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.Created += new FileSystemEventHandler(OnChanged);
watcher.Deleted += new FileSystemEventHandler(OnChanged);
watcher.Renamed += new RenamedEventHandler(OnRenamed);

Have you thought of running MD5 checksums on the files and storing them later for comparison? If your always processing a certain directory, this might be feasible.

You can use the FileInfo class to get the required change information (which you might be already using). You need to check two properties of a file, which are LastWriteTime and CreationTime. If either of them is higher than your last processing date, you need to copy the file. It is a common misconception that CreationTime is always less than LastWriteTime. It's not. If a file is copied to another file, the new file retains the LastWriteTime of the source but the CreationTime will be the time of the copy.

Have you considered adding a process to watch your directory instead? Using a FileSystemWatcher? Then you move from using a batch process and a real time system for monitoring your files.

As you've observed, copying a file to an existing destination file keeps the existing file's CreationTime, and sets LastWriteTime to the source file's LastWriteTime, rather than current system time when doing the copy. Two possible solutions:
Do a delete-and-copy, ensuring a destination CreationTime will be system's current time.
Check for file's Archived attribute as well, and clear it while processing. When copying source->dest, dest +A attribute will be set.