detecting that a file is currently being written to - c#

(I know It's a common problem but I couldn't find an exact answer)
I need to write a windows service that monitors a directory, and upon the arrival of a file, opens it, parses the text, does something with it and moves it to another directory afterwards. I used IsFileLocked method mentioned in this post to find out if a file is still been written. My problem is that I don't know how much it takes for another party to complete writing into the file. I could wait a few seconds before opening the file but this is not a perfect solution since I don't know in which rate is the file written to and a few seconds may not suffice.
here's my code:
while (true)
{
var d = new DirectoryInfo(path);
var files = d.GetFiles("*.txt").OrderBy(f => f);
foreach (var file in files)
{
if (!IsFileLocked(file))
{
//process file
}
else
{
//???
}
}
}

I think you might use a FileSystemWatcher (more info about it here: http://msdn.microsoft.com/it-it/library/system.io.filesystemwatcher(v=vs.110).aspx ).
Specificially you could hook to the OnChanged event and after it raises you can check IsFileLocked to verify if it's still being written or not.
This strategy should avoid you to actively wait through polling.

Related

Delete .xlsx or .pdf after closing file

I'm trying to delete .xlsx or .pdf files after using them. When files are created I display them, but then users want automatic file deletion after closing them.
I've tried couple of things, but none of them seem to work properly. Issue:
When opened multiple files (.xlsx or .pdf) I can't terminate a single process, like just a single file. Instead what happens is that file get's deleted only when I close all same processes (Excel or PDF files). As I investigated this happens because Excel or PDF works as one instance only. However code works as expected when I have only one file opened...
This is what I have so far:
var process= Process.Start(file_path); //file_path is global variable
Set_event(process);
private void Set_event(Process process)
{
process.EnableRaisingEvents = true;
process.Exited += new EventHandler(Delete_File);
}
public void Delete_File(object sender, EventArgs e)
{
//Delete file on close
File.Delete(file_path);
}
I've also tried with DeleteOnClose method of FileOptions, but unfortunally that doesn't display file to user and doesn't quite delete file immediately after using them, only after my win app is closed. That isn't my desired output, but at least files are deleted, so If I could fix that I would be partially satisfied too. Here is my line for that:
var open_file = new FileStream(file_path,FileMode.Open, FileAccess.ReadWrite,FileShare.ReadWrite, 512, FileOptions.DeleteOnClose);
With all that said, are there any other options I missed ? Thanks for help in advance.
I've tried almost everything I could find (different variations of Exited_Event for Process, monitoring with FileSystemWatcher, creating files with DeleteOnClose - even API), but none of them worked as expected.
Everything ends or fails with issue I described in first place - some apps, like Microsoft Excel or Adobe Acrobat uses one instance to open a file (.pdf or .xls/.xlsx), so you can't just reference a single file as object while you have opened more files. That means you either end up with an error when trying to assign Exited_event to single file, or no error but file gets deleted only when you close all files with same type...
BUT fortunate enough I figured out one thing: WHEN you have opened more than one file in question (.pdf or .xlsx) something happens in background of OS: If you loop through processes of same type at that time, you'll get a list of particular instance that is in use.
In other words, while you have 2 Excel files opened, loop through processes is showing you only a file which is currently active for "EXCEL" process.
So, that leaded me to a completely new approach that might solve this issue. In order to have a complete solution for this you have to:
1. Create a method to check whether file is no longer in use.
2. Set a Timer with a delay of 2 seconds, to make sure process really ends. Maybe this should be incremented for different purposes...
3. Set a Timer_tick event, where you loop processes to see whether particular file is listed as active, and If user has already closed this file. As described by other users this method isn't quite accurate, but with setting delay for Timer I think there shouldn't be any problems anymore.
Here is a complete code for this (for .pdf and .xlsx - that is what I needed):
//as global variable
System.Windows.Forms.Timer delete_file = new System.Windows.Forms.Timer();
Process.Start(file_path); //file_path is global variable
delete_file.Tick += new EventHandler(timer_Tick);
delete_file.Interval = (2000);
delete_file.Enabled = true;
delete_file.Start();
private void timer_Tick(object sender, EventArgs e)
{
Boolean file_is_opened = false;
// Loop processes and list active files in use
foreach (var process in Process.GetProcesses())
{
if (process.MainWindowTitle.Contains(Path.GetFileName(file_path)))
{
file_is_opened = true;
}
}
//If our file is not listed under active processes we check
//whether user has already closed file - If so, we finally delete It
if (file_is_opened==false)
{
if (!File_In_Use(new FileInfo(file_path)))
{
File.Delete(file_path);
delete_file.Enabled = false;
delete_file.Stop();
return;
}
}
}
private bool File_In_Use(FileInfo file)
{
//Method to check whether file is in use
FileStream stream = null;
try
{
//If file doesn't exist
if (!file.Exists)
{
return false;
}
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch (IOException)
{
//File is unavailable:
//because someone writes to It, or It's being processed
return true;
}
finally
{
if (stream!=null)
{
stream.Close();
}
}
//File not locked
return false;
}
This is how I did It. It might not be a perfect solution, but that works for me on Win 10 with no errors so far.
If someone has a suggestion to fix upper code, please let me know. Otherwise I hope this will help someone in future as I noticed there were already some questions about this in past, with no proper answer.

Windows: Delete EXE after execution

I am working on an application in C# which does the following things:
Write an EXE to disk
Execute the EXE through Process.Start()
I am now trying to ensure that the EXE will be deleted once it is closed.
The easiest way to do so is to set the FileOptions.DeleteOnClose parameter when creating the EXE using File.Create.
However, this means that the EXE can not be executed while is in use. Once the file handle is closed, the EXE is immediately deleted before it can be executed.
Is there any way to retain a "weak reference" to the EXE in my application which does not lock the file and allows it to be executed? Alternatively, is there any way to unlock the EXE for execution with the file handle still open? Are there any other obvious solutions I am missing?
CLARIFICATION A: I am aware of other methods to delete files in use which will delete the file eventually (e.g. upon reboot). I am however looking for a method to delete the file immediately once it starts executing which is handled by the OS (e.g. when running a batch that first executes the file and then deletes it, the file would remain on disk if the batch job is terminated).
CLARIFICATION B: To explain the bigger picture: The application receives and decrypts an executable file. After decryption, the file should be executed. However, I want to make sure the decrypted version of the EXE does not stay on disk. Ideally, I also want to prevent users from copying the decrypted EXE. However, since the decryption application runs as the same user, this will be impossible to achieve in a truly secure fashion as both have the same privileges on the system.
You could use Process.WaitForExit:
var process = Process.Start(processPath);
process.WaitForExit();
// File.Delete(processPath); // Not strong enough (thanks to Binary Worrier)
DeleteOrDie(processPath); // Will attempts anything to delete the file.
But it gives the possibility to copy the exe from where you writed it.
A good solution is to run it from memory.
If your target exe is a CLR program, you can use the Assembly.Load function:
// read the file and put data in bin
...
Assembly a = Assembly.Load(bin);
MethodInfo method = a.EntryPoint;
if (method == null) throw new NoEntryPointException();
object o = a.CreateInstance(method.Name);
method.Invoke(o, null);
More details here.
If you want to load/execute any exe in memory, you could use the Nebbett’s Shuttle approach but you will need to code it in C/C++ and make a call to it from C#.
Also it looks like Microsoft doesn't like it (security issues) and I don't think you can achieve it from C# only. Good anti-virus will probably detect it.
In a not very good way but a way that can give you what you want, I suggest this solution:
(I use a Console Application with some input arguments for this solution)
[1: ] Write a function to check opened processes:
/// <summary>
/// Check application is running by the name of its process ("processName").
/// </summary>
static bool IsProcessOpen(string processName)
{
foreach (Processing.Process clsProcess in Processing.Process.GetProcesses())
{
if (clsProcess.ProcessName.ToUpper().Contains(processName.ToUpper()))
return true;
}
return false;
}
[2: ] Define some variables:
static bool IamRunning = true;
static int checkDuration = 1; // in seconds
[3: ] Use a Thread for run a loop for checking:
Thread t = new Thread(delegate() {
DateTime lastCheck = DateTime.MinValue;
while (IamRunning)
{
var now = DateTime.Now;
int dd = (now.Hour - lastCheck.Hour) * 3600 + (now.Minute - lastCheck.Minute) * 60 + now.Second - lastCheck.Second;
if (dd >= checkDuration)
if (!IsProcessOpen("ProcessName"))
{
delApplication(); // You have a function to delete ...
break;
}
}
});
t.SetApartmentState(ApartmentState.STA);
t.Start();
[4: ] Use a loop at the end of the program:
while (t.ThreadState == ThreadState.Running)
{
// just wait.
}
Note: This solution by Console Application in my low computer have 50% usage of CPU.

Is it possible, through FileSystemWatcher or something else, to be able to retrieve only the new entries of a text/log file that is being watched?

I have a folder where I am watching though FileSystemWatcher the contents of any log file, i.e. watching *.log within a particular directory. I'm now realizing that FileSystemWatcher is rather limited and will only be able to kick off events based off changes, however it doesn't return nor know of what changes were made. Is there a library that can control this somewhere or something else I should be using?
When I detect the changes to this DNS Log, I want to get the new entries that were added to the end of the file. I can't even get line numbers from StreamReader which I thought might of helped. What are my options?
You have several options.Read whole file on each change and each time compare new lines with previous ones.This would be overkill if the file is too large. If your file is changed by appending text to the end only, then you can Read the file for first time, then you can keep the line count, and in every change you can Skip previous lines and get new ones.And update the counter.For example:
private static int counter;
private static string[] currentLines;
private static void Main(string[] args)
{
FileSystemWatcher watcher = new FileSystemWatcher("myfile.txt");
watcher.Changed += fileChanged;
currentLines = File.ReadLines("myFile.txt").ToArray();
counter = currentLines.Length;
Console.ReadLine();
}
private static void fileChanged(object sender, FileSystemEventArgs e)
{
var temp = File.ReadLines("myFile.txt").Skip(counter).ToArray();
if (temp.Any())
{
currentLines = temp;
counter += temp.Length;
}
}
Right solution is to employ the filesystem filter driver and intercept file write operations thus getting the data being written right after (or even before) they reach the file.
You can write a filesystem driver yourself (this is quite tricky) or use our CallbackFilter library which includes a ready-to-use driver.
Also if the file is not opened when you detect the change or is opened in the mode that allows reading, you can read the known data from it as Selman22 described in his answer. Note, that it's a bad idea to read text lines - instead you need to read the written data as binary (and read only the data after the position that you have remembered during previous read) and split it to lines in your code.

Upload file still in use *sometimes* when trying to move

After an image is uploaded to my server, my code moves it into a specific folder given by the user details. Sometimes I think it tries to move the file too fast or the upload file is still in use so 9/10 the function won't perform the move.
Is there a way to add a 'wait' or a way to check if a file is in use and possibly perform a while loop until the file is allowed to be moved?
Current move function in my controller:
while (!File.Exists(uploadedPath))
{
}
File.Move(uploadedPath, savePath);
PS. I intend to add in a counter to ensure the while loop doesn't get stuck and has a timeout.
If you have control over the code receiving the file, I would update it to notify the moving code when the file is received completely. Alternatively I would move the file from there or even save the file where it should be eventually.
Otherwise, it will be a hack. You need
Try to move the file,
Catch the exception if it doesn't move
Use Thread.Sleep for a few sec
Go To 1
Something along the lines:
bool success = false;
for (var count = 0; !success && count < 10; ++count)
{
try
{
File.Move(uploadedPath, savePath);
success = true;
}
catch (IOException)
{
Thread.Wait(1000);
}
}
You also need to handle the situation when it cannot move the file at all. So it is a hack and should not be done in general if there are other ways to notify the moving code.
Also note:
From File.Move msdn:
If you try to move a file across disk volumes and that file is in use,
the file is copied to the destination, but it is not deleted from the
source.
which means that your file will remain in the received files directory after moving.
Are UploadFile and MoveFile 2 different components that are independent of each other. If so I don't think it's a good architecture. I would recommend a way so as to have the UploadFile pass the control to MoveFile once it's part is done. This way you can avoid multiple processes trying to access the same file.

FileSystemWatcher getting too many event

I'm trying to implement a file watcher that will raise an event if the file content was changed.
The problem that once the file I am watching was modified, I am getting 2 events.
(I want to get it only once)
_automationStatusWatcher = new FileSystemWatcher(fileInfo.Directory.FullName,
fileInfo.Name);
_automationStatusWatcher.NotifyFilter = NotifyFilters.LastWrite;
_automationStatusWatcher.Changed += OnAutomationStatusChanged;
_automationStatusWatcher.EnableRaisingEvents = true;
The file that i'm watching is not intended to be recreated/moved/deleted/whatever. its purpose is to be an xml database file that i need to monitor it once it changes. i want to get only 1 event when the file is modified and to ignore the other events.
How can I do that?
Manual:
Note
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
I had to cope with this problem some time ago. I wanted to be notified when a file was created. The problem was that the event is triggered when the file is still empty or still being written to, not when the new file is closed.
The solution was to create the file using a temporary name and when finished renaming the file to its final name. I then watched for the rename-event.
You could name the file to myfile.tmp while writing to it and when finished rename it to myfile.xml and watch the rename-event.
You could attempt to unhook your event handler once you catch a change until you are ready to accept more changes:
_automationStatusWatcher.Changed -= OnAutomationStatusChanged;
But that is not necessarily thread safe and you could get multiple firings anyway.
Another option is to use a semaphore variable to see if you are handling the change:
private bool _inStatusChange = false;
private void OnAutomationStatusChanged(object sender, args...)
{
if (_inStatusChange)
{
return;
}
else
{
_inStatusChange = true;
//do work
_inStatusChange = false;
}
}
You should use appropriate locking to keep access to the semaphore variable thread safe.
i meet this problem too, now i found out this way :
new Thread(() => {
while (true) {
var r = watch.WaitForChanged(WatcherChangeTypes.All);
this.Invoke(new ThreadStart(() => {
listBox1.Items.Add(string.Format("{0} {1} {2}", DateTime.Now, r.Name, r.ChangeType));
}));
}
}) { IsBackground = true }.Start();
its very similar to nio in java
I had a similar problem for checking updates in logging configurations.
I read about the multiple events problem of FileSystemWatcher. So I decided to implement another solution.
I check for every access to my configuration file if it has changed by comparing modified date. Later I added a latence (2 seconds) to avoid too many accesses to filesystem. Maybe you can also use that way.
If you look at the documentation for FileSystemWatcher it says that it will fire multiple times.
Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
I would recommend doing some kind of queuing mechanism so that you don't have multiple actions on the same file.

Categories

Resources