I have a program that logs it's progress and other data to an XML file. I want to be able to open this XML file without blocking out the writer program (not a .NET program, and I have no control over it), and to read the XML as it comes, waiting for more when it is all processed, until the EOF is received.
How can this be achieved in C#?
Note that there are 2 problems:
Keeping a reading stream open without blocking the other process.
Knowing when there is more input and waiting when there isn't.
If I needed to do this I would do something like the following:
Use a FileSystemWatcher to get notified when the file changes. Then just read the file and parse the XML as you require.
I would go down this route as it will be difficult to read the stream as and when the external application writes to the file.
I did soemthing similar in past yielding in an OS program called Tailf.
Just check the code if you want to do it yourself, or grab all from it, it should almost work for you as well, a part the fact I just care about text files.
You can open a file stream without locking it by passing in the following flags:
new FileStream(logFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
As far as waiting for the "EOF", if the other program is only writing data intermittently, you may have to put some sort of heuristics into your progress (ie. stop peeking for new data only if there's nothing new for X minutes).
Related
I've got a program that will be running over a long duration (hours), and regularly writing output to a text file.
I'm looking to use a TextWriter implementation to write to the file, and I am concerned that keeping the file locked open during the entire length of operations may be problematic.
First question: Will there be performance problems (or other kinds) for keeping a stream open to a file for an extended duration?
If so, will a StreamWriter (opened with File Name constructor) manage opening and closing the file on a regular buffered basis for me, or will it hold the file open for the duration of its existence?
Lastly, is there a built in option for handling these more long-duration writes? Or will I need a custom Writer/Stream implementation?
Personally I would use one of the File.Appendxxx routines, which open the file, append the data and then close it again.
If I'm writing at such a rate that the cost of all this opening and closing is too high, then I add some kind of memory-based queue and flush it periodically.
If you're doing text-based logging, you might look at one of the umpteen logging frameworks for .NET, which can do this sort of stuff for you, along with things like rotating filenames, etc.
StreamWriter / FileStream, etc, will generally hold the file open until you dispose them.
Having a file opened indefinitely for it to be written to is not advised because this can cause issues when you need to back-up the file, edit the file, read the file, or even if the system crashes while it is in the process of writing to the file inevitably corrupting the data.
Will's answer does provide a solution of opening and appending the file when needed. The point here is if the file is not being written to 24/7 then there is no reason why you shouldn't close it.
To directly answer your question in the sense of the file needing to be open 24/7. I would use asynchronous methods to write to the file using tasks. This way you can then invoke another task to write a back-up of the file, say everyday. The back-up file will allow you to view the information of the production file.
I want to read the content of a file which is opened (and locked?) by a other process.
I tried it with File.ReadAllText() and with new StreamReader(new FileStream(path, FileMode.Open, FileAccess.Read)) but both methods trigger a IOException.
For example I can open the file with Notepad++ and the content is shown so I think it must be possible too with c#.
You need to use the FileStream constructor overload that takes a FileShare argument. And pass FileShare.ReadWrite. You can only open the file if you permit write access since the other program already acquired that right. Otherwise the reason that your attempts failed so far, they used FileShare.Read. Can't work, you cannot deny write access because the other program already got that.
Dealing with the program writing to the file while you are reading it is entirely up to you. Results can be quite random. Anything is possible, but in general for a log file you'll get a partially written last line that's trailing behind the actual output of the program, some of which is still in the program's file buffer. A buffer size of 4096 bytes is a common choice.
I have a process where I am writing byte array to a file (StorageFile). The process runs in periodic intervals and during the same time I need to read from the beginning of the file in another process. The two processes are in different classes. When I start reading the file, the write operation fails with access denied error.
Here the file is a StorageFile inside the app folder.
The write method calls:
IRandomAccessStream randomStream = await targetFile.OpenAsync(FileAccessMode.ReadWrite);
Stream stream = randomStream.AsStreamForWrite();
The read method calls:
IRandomAccessStream randomStream = await targetFile.OpenAsync(FileAccessMode.Read);
Stream stream = randomStream.AsStreamForRead();
Both operations take place simultaneously from different methods and it results in an access denied error. Seems I can use the read/write inside the same method while opening for write. But how to do that from different methods??
Do we have anything similar to the earlier System.IO.FileShare so that I can explicitly say that this file needs to be accessed from another location in the same app?
Besides switching to a database type setup, I would suggest buffering your writes to an array and writing them in chunks based on the buffer filling and/or certain about of time elapsed. Unless you need the latest and greatest performance down to the millisecond, I doubt people will notice a performance impact. You will have to play with the buffer size to make sure you give your reader enough time to actually read the file.
On the reader side, you could implement a FileSystemWatcher object on the file.
I had a webservice that reads .txt files using StreamReader and sends back responses to the user. If multiple people call my webservice, will the .txt files be "locked" and thus only be able to handle processing one request at a time?
Thanks.
Files are not locked during reads using streamreader.
I think this depends. Are any changes being made to the text file? If not, then the file should not be locked, because once it is opened, it should be read into the stream using a stremreader object, and then closed. Refer to
How do I open an already opened file with a .net StreamReader?
and
http://msdn.microsoft.com/en-us/library/db5x7c0d.aspx
This code creates a StreamReader that points to MyFile.txt through a
call to File.OpenText. StreamReader.ReadLine returns each line as a
string. When there are no more characters to read, a message is
displayed to that effect, and the stream is closed.
I wanted to ask if it is possible to redirect writes to a specific file, to a memory stream in my application so I can immediately read it.
If it is possible please explain how to do so.
Thanks.
Make sure your application works with Stream, and then you can use the MemoryStream concrete implementation to accomplish streaming without a file.
What you need is a stream which writes to a file as well as to another stream which you are watching - a stream splitter, commonly known in the Unix world as tee.
For .NET you can use this implementation of a stream splitter. Pass it a FileStream (so that the file gets written to) and for the second stream, pass a Stream implementation which does whatever you want when the stream is written to (for example, a subclass of MemoryStream with an overridden Write method).
Update: I see from your comment that my answer isn't what you need - I didn't realise from your original question that you had no control over the app writing to file.
In order to get this kind of control, for finest control you will need to go to a low-level driver which intercepts system calls - much like FileMon does. For slightly less control, you can use System.IO.FileSystemWatcher.
I don't know how to do this in the way you say but you could use a FileWatcher to hook file. If its written to you'll be notified and you can read it.