I have a thread which polls a folder for new files. The problem is that it sees a new file and starts working on it even before the file has been completely copied by another process. Because of this the poller gets file used by another process error.
Is there a way to check the file is free to use or get notified? We can certainly use exception handling code, but is there a better way?
Tech: .NET 2.0/C#
Update:
Found out from other answers that if we have access to the app writing the file then better design is to start with some other extension .tmp and then rename it after copying.
The FileStream.Lock could be used if we don't control the source application
We attempt to get a lock on the file before processing it and handle the IOException rather than a generic exception during the attempt to read the file.
See FileStream.Lock on MSDN.
Related
I am busy writing an additional module to an existing C# application. In my module I have to generate some data and add it to ZIP archives created by the main program. (Ionic.Zip library is used). In most cases I can successfully update existing archives and store them again. But sometimes (and quite often) this does not work, since, as far as I understand, something in the main program opens a stream to the concerned archive file and fails to close it before I try to access it.
It is absolutely clear that the process that locks the files is the main program itself (tested with a tool based on How do I find out which process is locking a file using .NET?).
I am not aware of any technical possibility to trace down the point in the code where a file is opened without being (somewhere) closed, but it is my hope that such a technique may exist however and that someone could share some information on it.
TIA
I'm trying to use the following code to delete a read-only file.
var fileInfo = new FileInfo(saveLocation);
fileInfo.IsReadOnly = false;
fileInfo.Delete();
When it gets to the third line, the following exception is thrown
Message: The process cannot access the file '\\filepath\filename.pdf' because it is being used by another process.
Note: \\filepath\filename.pdf is not the actual file path, I'm just using it to replace a longer path
I've checked the file, and before the code runs, it is set to read-only, and after the code runs, it is not anymore.
Am I incorrect in thinking that when a file is opened as read-only it is not considered to by in use? I'm pretty sure that is true for Microsoft office files suck as .xlsx files, but maybe not for PDFs?
Ultimately, my goal is to be able to push an updated version of this file to a shared location even if some user has the file open on their machine, which is why I initially set it to be read-only.
Message: The process cannot access the file '\filepath\filename.pdf' because it is being used by another process.
This is not the same as the file being read-only.
You can find out in code which process is locking the file
https://stackoverflow.com/a/20623311/141172
You can also find out from the command line
UPDATE
Based on your comments, it seems like you may want an exclusive lock on the file for the duration that you are processing it
open file in exclusive mode in C#
Command-line tool for finding out who is locking a file
I'm using a FileSystemWatcher to watch a directory. I created a _Created() event handler to fire when a file is moved to this folder. My problem is the following:
The files in this directory get created when the user hits a "real life button" (a button in our stock, not in the application). The FileSystemWatcher take this file, do some stuff in the system and then delete it. That wouldn't be a problem when the application runs only once. But it is used by 6 clients. So every application on every client is trying to delete it. If one client is too slow, it will throw an exception because the file is already deleted.
What I'm asking for is: Is there a way to avoid this?
I tried using loops and check if the file still exists, but without any success.
while (File.Exists(file))
{
File.Delete(file);
Thread.Sleep(100);
}
Can someone give me a hint how it could probably work?
Design
If you want a file to be processed by a single instance only (for example, the first instance that reacts gets the job), then you should implement a locking mechanism. Only the instance that is able to obtain a lock on the file is allowed to process and remove it, all other instances should skip the file.
If you're fine with all instances processing the file, and only care that at least one of them succeeds, then you need to figure out which exceptions indicate a genuine failure and which ones indicate a failure caused by the actions of another instance.
Locking
To 'lock' a file, you can open it with share-mode FileShare.None. This prevents other processes from opening it until you close the file. However, you'll then need to close the file before you can delete it, which leaves a small gap during which another instance could open the file.
A better solution is to create a separate lock file for that purpose. Create it with file-mode FileMode.Create and share-mode FileShare.None and keep it open until the whole process is finished, including the removal of the processed file. Then the lock file can be closed and optionally removed.
Exception
As for the UnauthorizedAccessException you got, according to the documentation, that means one of 4 things:
You don't have the required permission
The file is an executable file that is in use
The path is a directory
The file is read-only
1 and 4 seem most likely in this case (if the file was open in another process you'd get an IOException).
If you want to synchronize access between multiple clients on the same computer you should use a Named Mutex.
I am implementing an event handler that must open and process the content of a file created by a third part application over which I have no control. I am warned by a note in "C# 4.0 in a nutshell" (page 495) about the risk to open a file before it is fully populated; so I am wondering how to manage this occurrence. To keep at minimum the load on the event handler, I am considering to have the handler simply insert in a queue the file names and then to have a different thread to manage the processing, but, anyways, how may I make sure that the write is completed and the file read is safe? The file size could be arbitrary.
Some idea? Thanks
A reliable way to achieve what you want might be to use FileSystemWatcher + NTFS USN journal.
Maybe more complicated than you expected, but FileSystemWatcher alone won't tell you for sure that the newly created file has been closed
-first, the FileSystemWatcher, to know when a file is created. From there you have the complete file path, and are 1 or 2 pinvokes away from getting the file unique ID (which can help you to track it during its whole lifetime).
-then, read the USN journal, which tracks everything that occurs on your drive. Filter on entries corresponding to your new file's ID, and read the journal until reaching the entry with the 'Close' event.
From there, unless your file is manipulated in special ways (opened and closed multiple times by the application that generates it), you can assume it is safe to read it and do whatever you wanted to do with it.
A really great C# implementation of an USN journal parser is StCroixSkipper's work, available here:
http://mftscanner.codeplex.com/
If you are interested I can give you more help about USN journal, as I use it in my project.
Our workaround is to watch for a specific extension. When a file is uploaded, the extension is ".tmp". When its done uploading, it's renamed to have the proper extension.
Another alternative is to have the server try to move the file in a try/catch block. If the fie isn't done being uploaded, the attempt to move the file will throw an exception, so we wait and try again.
Realistically, you can't know. If the other applications "write" operation is to open the file denying write access to everyone else then when it's done, close the file. When you get a notification then you could simply open the file requesting write access and if that fails, you know the operation isn't complete. But, if the "write" operation is to open the file, write, close the file, open the file again, and write again, etc., then you're pretty much out of luck.
The best solution I've seen is to set a timer after the last notification. When the timer elapses, try to open the file for write--if you can, assume the "operation" is done and do what you need to do. If the open fails, assume the operation is still in progress and wait some more.
Of course, nothing is foolproof. Despite the above, another operation could start while you're doing what you want with the file and cause interaction problems.
I have a requirement to move certain files after they has been processed. Another process access the file and I am not sure when it releases them. Is there any way I can find out when the handle to the file has been released so I can move them at that time.
I am using Microsoft C# and .Net framework 3.5.
Cheers,
Hamid
If you have control of both the producer of the file and the consumer, the old trick to use is create the file under a different name, and rename it once complete.
For example, say the producer is creating files always called file_.txt, and your consumer is scanning for all files beginning file_, then the producer can do this:
1. Create the file called tmpfile_.txt
2. When the file is written, the producer simply renames the file to file_.txt
The rename operation is atomic, so once your consumer sees its available, it is safe to open it.
Of course, this answer depends on if you are writing both programs.
HTH
Dermot.
Just contniually try to open the file for exclusive writing? (e.g. pass FileShare.None to the FileStream constructor). Once you have opened it, you know no one else is using it. However, this might not be the best way to do what you're doing.
If you're after two way communication, see if the other program can be talked to via a pipe.
If you have control of both of the sources, use a named mutex (which works across processes) to control access to the files rather than locking the file at the filesystem level. This way, you don't have to catch the exception raised by attempting to lock a locked file and loop on that, which is rather inelegant.