Develop Reference

Repeatedly reading small file throws OutOfMemory

I have two programs that work together. To coordinate their operations I use a small settings file. This setting file contains two words separated by a ';'. So in the one program I repeatedly read the words in the file using a while loop. By repeatedly I mean once every second. The loop only terminates when the program terminates; when the user turns off the pc.
But with each iteration of the loop the program size in memory increases until the program throws an OutOfMemory exception. I have tried two different methods of reading the files but both causes the program to 'grow' in memory.
FileStream FS = new FileStream("br.stat", FileMode.Open);
StreamReader SR = new StreamReader(FS);
string s = SR.ReadToEnd();
FS.Dispose();
SR.Dispose();
and
string S = File.ReadAllText("br.stat");
Is there a way to read a file repeatedly with out this happening?
Thanks.

the problem is the design, more so than the implementation. You only need to read from the file once when the app starts and again when the file changes. You can use FileSystemWatcher to detect changes to the file and reload the settings.
This uses drastically less resources than reading the file indefinitely.
also, you'll want to take advantage of the using keyword to ensure you properly dispose of the file stream and reader. In fact I would simplify and just use File.ReadAllText(filename).

I think you're doing it wrong. Realistically there is no need to read the file every iteration unless its changed.
Instead it would be better to use a FileSystemWatcher http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.changed(v=vs.85).aspx to read the values and change your control values.
You should also use the following to circumvent your memory leak issues
using (var FS = new FileStream("br.stat", FileMode.Open))
{
using (var SR = new StreamReader(FS))
{
var s = SR.ReadToEnd();
}
}

I would recomend you look at using an using statement
File and Font are examples of managed types that access unmanaged
resources (in this case file handles and device contexts). There are
many other kinds of unmanaged resources and class library types that
encapsulate them. All such types must implement the IDisposable
interface.
As a rule, when you use an IDisposable object, you should declare and
instantiate it in a using statement. The using statement calls the
Dispose method on the object in the correct way, and (when you use it
as shown earlier) it also causes the object itself to go out of scope
as soon as Dispose is called. Within the using block, the object is
read-only and cannot be modified or reassigned.
so something like
using (FileStream FS = new FileStream("br.stat", FileMode.Open))
using (StreamReader SR = new StreamReader(FS))
{
string s = SR.ReadToEnd();
}

Some trouble with locking files for a longer time

I have a function that is reading a file and adding some of the string in a list and returning this list. Because I wanted that nobody and nothing could change, delete or whatever the current file that I was reading I locked it. Everything was fine, I did it somehow like this:
public static List<string> Read(string myfile)
{
using (FileStream fs = File.Open(myfile, FileMode.Open, FileAccess.Read, FileShare.None))
{
//read lines, add string to a list
//return list
}
}
Thats fine. Now I have another function in another class that is doing stuff with the list and calling other functions and so on. Now sometimes I want to move the file that I was reading. And here is the problem: because Im now in a new function and the function Read(string myfile) is already processed, there is no more lock in the file.
//in another class
public static void DoStuff(/*somefile*/)
{
List<string> = Read(/*somefile*/);
//the file (somefile) is not more locked!
//do stuff
if (something)
Move(/*somefile*/) //could get an error, file maybe not more there or changed...
}
So another function/user could change the file, rename it, deleting it or whatever => Im not able to move this file. Or I will move the changed file, but I dont what that. If I would use threading, another thread with the same function could lock the file again and I could not move it.
Thats why I somehow need to lock this file for a longer time. Is there an easy way? Or do I have to replace my using (FileStream fs = File.Open(myfile, FileMode.Open, FileAccess.Read, FileShare.None) code? Any suggestions? thank you

If you want to keep the file locked for longer then you need to refactor your code so that the Stream object is kept around for longer - I would change the Read method to accept a FileStream, a little bit like this
using (FileStream fs = File.Open(myfile, FileMode.Open, FileAccess., FileShare.None))
{
List<string> = Read(fs);
if (something)
{
File.Move(/* somefile */)
}
}
The problem you are going to have is that File.Move method is going to fail as this file is already locked (by you, but File.Move doesn't know that).
Depending on what exactly it is you want to do it might be possible to work out a way of keeping the file locked while also "moving" the file, (for example if you know something in advance you could open the file specifying FileOptions.DeleteOnClose and write a new file with the same contents in the desired destination), however this isn't really the same as moving the file and so it all depends on what exactly it is you are trying to do.
In general such things are almost always going to be more trouble than they are worth - you are better off just unlocking the file just before you move it and catching/ handling any exception that is thrown as a result of the move.

The only way you could keep it locked is to keep it exclusively open, like you have done in your code.
Maybe you need to //do stuff within your using statement, and then straight after call Move

No amount of locking will prevent this. A lock only protects the data in the file. The user (or any other program) can still move or rename the file. The file's attributes, including name, time stamps and file attributes are stored separately and can be changed at will.
This is just something you'll have to deal with in any Windows program. It is rare enough that simply catching the exception is good enough to let you know that something happened to the file. The user will rarely be surprised. If you really need to know up front then you can use FileSystemWatcher to get a notification when it happens.

You are locking the file only when Read method is called.
If you want to keep it locked and release it only when you decide, make your methods OpenFile(string filename) and CloseFile(string filename). Then remove the using statement from Read method.
Open it when you start working (lock). Read it when you need it. When you have to move it, simply create a new file with the same name and copy the content. Close the original file (unlock) and delete it.

Open a file that has just been closed

I'm writing an application that manipulates a text file. The first half of my function reads the textfile, while the second half writes to (optionally) the same file. Although I call .close() on the StreamReader object before opening the StreamWriter object, I still get a IOException: The process cannot access the file "file.txt" because it is being used by another process.
How do I force my program to release the file before continuing?
public static void manipulateFile(String fileIn, String fileOut,String obj)
{
StreamReader sr = new StreamReader(fileIn);
String line;
while ((line = sr.ReadLine()) != null)
{
//code to split up file into part1, part2, and part3[]
}
sr.Close();
//Write the file
if (fileOut != null)
{
StreamWriter sw = new StreamWriter(fileOut);
sw.Write(part1 + part2);
foreach (String s in part3)
{
sw.WriteLine(s);
}
sw.Close();
}
}

Your code as posted runs fine - I don't see the exception.
However calling Close() manually like that is a bad idea - if an exception is thrown your call to Close() might never be made. You should use a finally block, or better yet : a using statement.
using (StreamReader sr = new StreamReader(fileIn))
{
// ...
}
But the actual problem you are experiencing might not be specifically with this method, but a general problem with forgetting to close files properly in using blocks. I suggest you go through all your code base and look for all the places in your code where you use IDisposable objects and check that you dispose them correctly even when there could be exceptions.

Getting read access to a file that's already opened elsewhere isn't usually difficult. Most code would open a file for reading with FileShare.Read, allowing somebody else to read the file as well. StreamReader does so for example.
Getting write access is an entirely different ball of wax. That same FileShare.Read does not include FileShare.Write, allowing you to write the file while somebody else is reading it. That's very troublesome, you're jerking the mat out from under that somebody else, suddenly providing entirely different data.
All you have to do is find out who that 'somebody else' might be. SysInternals' Handles utility can tell you. Hopefully it is your own program, you could do something about that.

May sound like a stupid question, but are you sure you didn't edit the file with another application, which didn't release the file? I've had this situation before, mostly with Excel files where Excel didn't completely unloading from memory (or me just being dumb enough not to close the other application sometimes). Might happen with whatever application you use for .txt files, if any. Just a suggestion.

.NET File.Create , can't delete file afterwards

Using method: System.IO.File.Create()
After the file gets created, it still remains used by a process, and I can't delete it.
Any idea how I can better create the file, should be a 0byte file, and then somehow close and dispose?

JL,
You should wrap your call to .Create in a using statement so that the FileStream that .Create returns will be closed properly. IE:
using (File.Create("path")){...}

The Create method not only creates the file, it opens it and return a FileStream object that you can use to write to the file.
You have to close the file after yourself, otherwise it will not be closed before the garbage collector cleans up the FileStream object.
The easiest way is to simply close the file using the reference that the Create method returns:
File.Create(fileName).Close();

nikmd23 has the short answer, the long answer is: the FileStream that File.Create(...) is returning is not being deterministically disposed of, therefore it's file handle is not closed when you're trying to delete it.
As nikmd23 put it, wrapping your File.Create(...) call will with a using statement will make sure the stream is closed and disposed of:
using (FileStream fs = File.Create(path)) {
// do anything with the stream if need-be...
}
File.Delete(path); //after it's been disposed of.
The using(...) block is really just compiler-sugar for:
FileStream fs = File.Create(path);
try {
// do anything with the stream if need-be...
}
finally {
fs.Dispose();
}
File.Delete(path)

You should use nikmd23's answer in almost all cases. If you can't, because you need to pass the FileStream somewhere else, make sure to call the FileStream.Close method eventually. Preferably you would have the class that 'owns' the FileStream implement IDisposable itself, and close the stream in its Dispose method.
For more information on implementing IDisposable, refer to the MSDN documentation. Easier reading, and more up to date, is Joe Duffy's post on the subject.

using(FileStream f = File.Create(file_path))
{
// ... do something with file
f.Close();
}
The "f.Close();" line closing file immediately. If not close manually, Disposing may not close it.

See System.IO.File.Create(String) Method paramter and return value description
Parameters
path
Type: System.String
The path and name of the file to create.
Return Value
Type: System.IO.FileStream
A FileStream that provides read/write access to the file specified in path.
The FileStream return value is there for IO access to the created file. If you are not interested in writing (or reading) the newly created file, close the stream. That is what the using block is ensuring.

Is it necessary to wrap StreamWriter in a using block?

A few days ago I posted some code like this:
StreamWriter writer = new StreamWriter(Response.OutputStream);
writer.WriteLine("col1,col2,col3");
writer.WriteLine("1,2,3");
writer.Close();
Response.End();
I was told that instead I should wrap StreamWriter in a using block in case of exceptions. Such a change would make it look like this:
using(StreamWriter writer = new StreamWriter(Response.OutputStream))
{
writer.WriteLine("col1,col2,col3");
writer.WriteLine("1,2,3");
writer.Close(); //not necessary I think... end of using block should close writer
}
Response.End();
I am not sure why this is a valuable change. If an exception occurred without the using block, the writer and response would still be cleaned up, right? What does the using block gain me?

Nope the stream would stay open in the first example, since the error would negate the closing of it.
The using operator forces the calling of Dispose() which is supposed to clean the object up and close all open connections when it exits the block.

I'm going to give the dissenting opinion. The answer to the specific question "Is it necessary to wrap StreamWriter in a using block?" is actually No. In fact, you should not call Dispose on a StreamWriter, because its Dispose is badly designed and does the wrong thing.
The problem with StreamWriter is that, when you Dispose it, it Disposes the underlying stream. If you created the StreamWriter with a filename, and it created its own FileStream internally, then this behavior would be totally appropriate. But if, as here, you created the StreamWriter with an existing stream, then this behavior is absolutely The Wrong Thing(tm). But it does it anyway.
Code like this won't work:
var stream = new MemoryStream();
using (var writer = new StreamWriter(stream)) { ... }
stream.Position = 0;
using (var reader = new StreamReader(stream)) { ... }
because when the StreamWriter's using block Disposes the StreamWriter, that will in turn throw away the stream. So when you try to read from the stream, you get an ObjectDisposedException.
StreamWriter is a horrible violation of the "clean up your own mess" rule. It tries to clean up someone else's mess, whether they wanted it to or not.
(Imagine if you tried this in real life. Try explaining to the cops why you broke into someone else's house and started throwing all their stuff into the trash...)
For that reason, I consider StreamWriter (and StreamReader, which does the same thing) to be among the very few classes where "if it implements IDisposable, you should call Dispose" is wrong. Never call Dispose on a StreamWriter that was created on an existing stream. Call Flush() instead.
Then just make sure you clean up the Stream when you should. (As Joe pointed out, ASP.NET disposes the Response.OutputStream for you, so you don't need to worry about it here.)
Warning: if you don't Dispose the StreamWriter, then you do need to call Flush() when you're done writing. Otherwise you could have data still being buffered in memory that never makes it to the output stream.
My rule for StreamReader is, pretend it doesn't implement IDisposable. Just let it go when you're done.
My rule for StreamWriter is, call Flush where you otherwise would have called Dispose. (This means you have to use a try..finally instead of a using.)

If an exception occurs without the using block and kills the program, you will be left with openconnections. The using block will always close the connection for you, similiar to if you were to use try{}catch{}finally{}

Eventually, the writer will be cleaned up. When this happens is up to the garbage collector, who will notice that the Dispose for the command has not been called, and invoke it. Of course, the GC may not run for minutes, hours or days depending on the situation. If the writer is holding an exclusive lock on say, a file, no other process will be able to open it, even though you're long finished.
The using block ensures the Dispose call is always made, and hence that the Close is always called, regardless of what control flow occurs.

Wrapping the StreamWriter in a using block is pretty much equivalent of the following code:
StreamWriter writer;
try
{
writer = new StreamWriter(Response.OutputStream);
writer.WriteLine("col1,col2,col3");
writer.WriteLine("1,2,3");
}
catch
{
throw;
}
finally
{
if (writer != null)
{
writer.Close();
}
}
While you could very well write this code yourself, it is so much easier to just put it in a using block.

In my opinion it's necessary to wrap any class that implements IDisposable in a using block. The fact that a class implements IDisposable means that the class has resources that need to be cleaned up.

My rule of thumb is, if I see Dispose listed in intellisense, I wrap it in a using block.

the using block calls dispose() when it ends. It's just a handy way of ensuring that resources are cleaned up in a timely manner.

In almost every case, if a class implements IDisposable, and if you're creating an instance of that class, then you need the using block.

While it's good practice to always dipose disposable classes such as StreamWriter, as others point out, in this case it doesn't matter.
Response.OutputStream will be disposed by the ASP.NET infrastructure when it's finished processing your request.
StreamWriter assumes it "owns" a Stream passed to the constructor, and will therefore Close the stream when it's disposed. But in the sample you provide, the stream was instantiated outside your code, so there will be another owner who is responsible for the clean-up.