I am busy writing an additional module to an existing C# application. In my module I have to generate some data and add it to ZIP archives created by the main program. (Ionic.Zip library is used). In most cases I can successfully update existing archives and store them again. But sometimes (and quite often) this does not work, since, as far as I understand, something in the main program opens a stream to the concerned archive file and fails to close it before I try to access it.
It is absolutely clear that the process that locks the files is the main program itself (tested with a tool based on How do I find out which process is locking a file using .NET?).
I am not aware of any technical possibility to trace down the point in the code where a file is opened without being (somewhere) closed, but it is my hope that such a technique may exist however and that someone could share some information on it.
TIA
Related
Here's the scenario:
Some application reads contents of a file. It does so by opening it for shared reading, and then at some point closing it (as I've seen it in Procmon.exe, see image). Right after the file is closed, I want to run my own code editing that file.
The only answer I've found so far is looping File.Open() in a try block and firing an event when it throws an exception, and that won't work for shared read access, let alone how ugly it is.
Is there a package or a custom class I can use?
Edit: The application doesn't lock the file. To make sure, I tried running File.OpenRead(); Thread.Sleep(); before the application launch, and it loaded the file as intenteded. It crashed with File.OpenWrite() though, so I'm convinced there are no locks placed by the application.
Edit: Tried writing some test programs with Restart Manager API implementations, none of them seem to be capturing non-locking reading.
Language used: C#
Theory:
I want to create a file with the flag FileOptions.DeleteOnClose in a temporary folder.
The file is successfully created and I write dato onto it, the next step is to launch the application associated with the file Process.Start(...) and allow the user to inspect the document, finally I close my handle and as soon as other process close the handle to the temporary file, the file is deleted by operating system.
My problem is that other processes cannot open the file, even for reading, despite if I add FileShare.ReadWrite | FileShare.Delete to the sharing mode.
Any suggestions?
The other processes need to specify FileShare.Delete when they open the DeleteOnClose file
From the MSDN CreateFile docs:
"FILE_FLAG_DELETE_ON_CLOSE... Subsequent open requests for the file fail, unless the FILE_SHARE_DELETE share mode is specified."
Check this:
You need to make sure that all processes are opening the file with FileShare.ReadWrite and FileShare.Delete.
Even if the creator opens with share-readwrite, if a second program tries to open with share-read, the second program is basically saying no-one else can write. But the first program already has that power so the second open fails.
Switch to Linux scnr
Ok, seriously now: That is a flaw in the Windows operating system which can't really be worked around. Each program opening the file must agree on other programs having the file open in the same time. That was a problem I got many years back when I still used Windows as well. It doesn't suffice to open a file and say: Let anyone else open this as well. The others must also say open this file even if it's open already.
On Linux on the contrary, the operating system doesn't allow any file locking in the way Windows does at all. Here, if any file is used by more than one program simultaneously, the programs itself must make sure, that concurrent accesses get locked out. Additionally, on Linux, we can just create the file, make sure the other process has been started and opened the file and then just delete the file (while it is open). The filename is then removed from the file system immediatelly, but the file is still maintained by the file system driver until the last link (including open file handles) got removed.
Back to your problem: As all of this doen't work on Windows, you could do two other approaches:
Register the file to be deleted on next boot (in the Win3x days, there was a section in the win.ini for that. Newer Windows version still support that, I just can't recall any longer, how it's done now).
Start the other process, wait for it to open the file, close the file and then try each minute to delete the file until deletion succeeds ...
Regards, Bodo
I have written a Java application that has to monitor a live log file. However I have not thought of the possibly of the file being locked, and that is exactly the case now.
Is there a way to disable the ability to add filelocks in a directory under Windows? If I would succeed, what would happen to the program writing to the logs (I think it's written in C#).
I am suspecting the logger locks the file to prevent it from being deleted, I cannot see any reason why it would need to prevent reads on it.
Does anyone perhaps know another solution to "cancel" the read-part of a file lock from within Java, if that even makes sense?
The entire idea of a lock is that when you acquire it, you have some sort of guarantee that you have exclusive access (of the desired type(s)) to the file. Perhaps the programmer of the other application made a mistake or did not give much thought to how he/she locked the file.
I don't think there are clean ways to bypass this problem (except hooking the OS calls?). You could open the file before running the other application, but chances are the application wouldn't even start up in that case.
I am implementing an event handler that must open and process the content of a file created by a third part application over which I have no control. I am warned by a note in "C# 4.0 in a nutshell" (page 495) about the risk to open a file before it is fully populated; so I am wondering how to manage this occurrence. To keep at minimum the load on the event handler, I am considering to have the handler simply insert in a queue the file names and then to have a different thread to manage the processing, but, anyways, how may I make sure that the write is completed and the file read is safe? The file size could be arbitrary.
Some idea? Thanks
A reliable way to achieve what you want might be to use FileSystemWatcher + NTFS USN journal.
Maybe more complicated than you expected, but FileSystemWatcher alone won't tell you for sure that the newly created file has been closed
-first, the FileSystemWatcher, to know when a file is created. From there you have the complete file path, and are 1 or 2 pinvokes away from getting the file unique ID (which can help you to track it during its whole lifetime).
-then, read the USN journal, which tracks everything that occurs on your drive. Filter on entries corresponding to your new file's ID, and read the journal until reaching the entry with the 'Close' event.
From there, unless your file is manipulated in special ways (opened and closed multiple times by the application that generates it), you can assume it is safe to read it and do whatever you wanted to do with it.
A really great C# implementation of an USN journal parser is StCroixSkipper's work, available here:
http://mftscanner.codeplex.com/
If you are interested I can give you more help about USN journal, as I use it in my project.
Our workaround is to watch for a specific extension. When a file is uploaded, the extension is ".tmp". When its done uploading, it's renamed to have the proper extension.
Another alternative is to have the server try to move the file in a try/catch block. If the fie isn't done being uploaded, the attempt to move the file will throw an exception, so we wait and try again.
Realistically, you can't know. If the other applications "write" operation is to open the file denying write access to everyone else then when it's done, close the file. When you get a notification then you could simply open the file requesting write access and if that fails, you know the operation isn't complete. But, if the "write" operation is to open the file, write, close the file, open the file again, and write again, etc., then you're pretty much out of luck.
The best solution I've seen is to set a timer after the last notification. When the timer elapses, try to open the file for write--if you can, assume the "operation" is done and do what you need to do. If the open fails, assume the operation is still in progress and wait some more.
Of course, nothing is foolproof. Despite the above, another operation could start while you're doing what you want with the file and cause interaction problems.
I have a thread which polls a folder for new files. The problem is that it sees a new file and starts working on it even before the file has been completely copied by another process. Because of this the poller gets file used by another process error.
Is there a way to check the file is free to use or get notified? We can certainly use exception handling code, but is there a better way?
Tech: .NET 2.0/C#
Update:
Found out from other answers that if we have access to the app writing the file then better design is to start with some other extension .tmp and then rename it after copying.
The FileStream.Lock could be used if we don't control the source application
We attempt to get a lock on the file before processing it and handle the IOException rather than a generic exception during the attempt to read the file.
See FileStream.Lock on MSDN.