Develop Reference

I operate with file descriptors correct, but I still get an error

string fileText;
using (var reader = File.OpenText(pathToSave)) {
fileText = await reader.ReadToEndAsync();
reader.Close();
}
using (var stream = File.Open(pathToSave, FileMode.Append, FileAccess.Write, FileShare.ReadWrite)) {
Byte[] text = new UTF8Encoding(true).GetBytes("test1" + Environment.NewLine);
stream.Write(text, 0, text.Length);
text = new UTF8Encoding(true).GetBytes("test2" + Environment.NewLine);
stream.Write(text, 0, text.Length);
stream.Close();
}
I don't operate with files anywhere else.
I always close file descriptors after reading / writing, but I still get an error, that the file is used by another process.
What do I do wrong?
The error appears on FileMode.Append.

I tried your code and it seems fileText = await reader.ReadToEndAsync(); locks the file, so you need to complete this task before using this file again.
You can use ReadToEnd instead of ReadToEndAsync to run synchronously:
string fileText;
using (var reader = File.OpenText(pathToSave)) {
fileText = reader.ReadToEnd();
reader.Close();
}

#John Doe
Replying to your comment, sadly I don't have enough reputation to do it in the comments. Yes of course you can! Check this out https://stackoverflow.com/a/9396092/13294196 (and also the question in that thread to understand it if you don't)
EDIT: and make sure to see Joelius's advice about using streamreader and streamwriter, it's a much better solution than operating on bytes.

Few last lines missing in CSV downloaded [duplicate]

Here is my code. :
FileStream fileStreamRead = new FileStream(pathAndFileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.None);
FileStream fileStreamWrite = new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
StreamWriter sw = new StreamWriter(fileStreamWrite);
int readIndex = 0;
using (StreamReader sr = new StreamReader(fileStreamRead))
{
while (!sr.EndOfStream) {
Console.WriteLine("eof" + sr.EndOfStream);
readIndex++;
Console.WriteLine(readIndex);
string currentRecord = "";
currentRecord = sr.ReadLine();
if (currentRecord.Trim() != "")
{
Console.WriteLine("Writing " + readIndex);
sw.WriteLine(currentRecord);
}
else {
Console.WriteLine("*******************************************spaces ***********************");
}
}
It is cutting off 2 lines with one test file and half a line, and then 1 line and half a line with the other test file I am running it against.
I am not a streamreader/writer expert you can probably see.
Any ideas or suggestions would be greatly appreciated as this is driving me batty. I am sure it is me using these incorrectly.

You are missing Flush/Close or simply using for your writer.
using(FileStream fileStreamWrite =
new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
{
using(StreamWriter sw = new StreamWriter(fileStreamWrite))
{
// .... write everything here
}
}

Right after the closing brace of the using statement, do this:
sw.Flush();
sw.Close();
There, that should do it.

You need to Flush your StreamWriter. A StreamWriter has a buffer, and it writes to disk only when the buffer is full. By flushing at the end you make sure all the text in the buffer is written to the disk.

In addition to other answers (use using, and/or flush/close), would say that they do not actually respond to the question: "why it may cut several lines."
I have an idea on subject that it is related to a fact that you use StreamReader and call EndOfStream twice: in a while loop header, and another inside it.
The only possible way of understanding if the stream ends is try to read some data from it. So I suspect EnfOfStream does it, and reading it twice, may create a problem in stream processing.
To resolve an issue:
Or use simple TextReader, considering that you are reading text file (seems to me)
Or change your logic to call only once, so no more call to Console.WriteLine("eof" + sr.EndOfStream);
Or change your logic, so do not use EndOFStream at all, but read line by line till the line is null.

You're not using StreamWriter properly. Also, since you're always reading lines, I would use a method that already does all that for you (and manages it properly).
using (var writer = new StreamWriter("path"))
{
foreach(var line in File.ReadLines("path"))
{
if (string.IsNullOrWhiteSpace(line))
{ /**/ }
else
{ /**/ }
}
}
... or ...
/* do not call .ToArray or something that will evaluate this _here_, let WriteAllLines do that */
var lines = File.ReadLines("path")
.Select(line => string.IsNullOrWhiteSpace(line) ? Stars : line);
var encoding = Encoding.ASCII; // whatever is appropriate for you.
File.WriteAllLines("path", lines, encoding);

Problems with Streams in C#

I have this code for c# to read all the lines in TestFile.txt but when i finish reading i want to read it again and then put it in a string array (not a List) but when i try do that again it says that the file is already in use. I want to reset the stream or do something like sr.Close() because first time i read it i want to count how many lines are there in the Testfile.txt.
using (StreamReader sr = new StreamReader("TestFile.txt"))
{
string line;
while ((line = sr.ReadLine()) != null)
{
Console.WriteLine(line);
}
}
I already tried to put after the while loop if(line == null) sr.Close() but it doesn't work.

Why not just read it into a List<string> and then build an array from that? Or more simply still, just call File.ReadAllLines:
string[] lines = File.ReadAllLines("TestFile.txt");
While you could reset the underlying stream and flush the buffer in the reader, I wouldn't do so - I'd just read it all once in a way that doesn't require you to know the size up-front.
(In fact, I'd try to use a List<string> instead of a string[] anyway - they're generally more pleasant to use. Read Eric Lippert's blog post on the subject for more information.)

You can do it by setting the BaseStream Position property to 0.
If you cannot (example would be a HttpWebResponse stream) then a good option would be to copy the stream to a MemoryStream...there you can set Position to 0 and restart the Stream as much as you want.
Stream s = new MemoryStream();
StreamReader sr = new StreamReader(s);
// later... after we read stuff
s.Position = 0;

Close a filestream without Flush()

Can I close a file stream without calling Flush (in C#)? I understood that Close and Dispose calls the Flush method first.

MSDN is not 100% clear, but Jon Skeet is saying "Flush", so do it before close/dispose. It won't hurt, right?
From FileStream.Close Method:
Any data previously written to the buffer is copied to the file before
the file stream is closed, so it is not necessary to call Flush before
invoking Close. Following a call to Close, any operations on the file
stream might raise exceptions. After Close has been called once, it
does nothing if called again.
Dispose is not as clear:
This method disposes the stream, by writing any changes to the backing
store and closing the stream to release resources.
Remark: the commentators might be right, it's not 100% clear from the Flush:
Override Flush on streams that implement a buffer. Use this method to
move any information from an underlying buffer to its destination,
clear the buffer, or both. Depending upon the state of the object, you
might have to modify the current position within the stream (for
example, if the underlying stream supports seeking). For additional
information see CanSeek.
When using the StreamWriter or BinaryWriter class, do not flush the
base Stream object. Instead, use the class's Flush or Close method,
which makes sure that the data is flushed to the underlying stream
first and then written to the file.
TESTS:
var textBytes = Encoding.ASCII.GetBytes("Test123");
using (var fileTest = System.IO.File.Open(#"c:\temp\fileNoCloseNoFlush.txt", FileMode.CreateNew))
{
fileTest.Write(textBytes,0,textBytes.Length);
}
using (var fileTest = System.IO.File.Open(#"c:\temp\fileCloseNoFlush.txt", FileMode.CreateNew))
{
fileTest.Write(textBytes, 0, textBytes.Length);
fileTest.Close();
}
using (var fileTest = System.IO.File.Open(#"c:\temp\fileFlushNoClose.txt", FileMode.CreateNew))
{
fileTest.Write(textBytes, 0, textBytes.Length);
fileTest.Flush();
}
using (var fileTest = System.IO.File.Open(#"c:\temp\fileCloseAndFlush.txt", FileMode.CreateNew))
{
fileTest.Write(textBytes, 0, textBytes.Length);
fileTest.Flush();
fileTest.Close();
}
What can I say ... all files got the text - maybe this is just too little data?
Test2
var rnd = new Random();
var size = 1024*1024*10;
var randomBytes = new byte[size];
rnd.NextBytes(randomBytes);
using (var fileTest = System.IO.File.Open(#"c:\temp\fileNoCloseNoFlush.bin", FileMode.CreateNew))
{
fileTest.Write(randomBytes, 0, randomBytes.Length);
}
using (var fileTest = System.IO.File.Open(#"c:\temp\fileCloseNoFlush.bin", FileMode.CreateNew))
{
fileTest.Write(randomBytes, 0, randomBytes.Length);
fileTest.Close();
}
using (var fileTest = System.IO.File.Open(#"c:\temp\fileFlushNoClose.bin", FileMode.CreateNew))
{
fileTest.Write(randomBytes, 0, randomBytes.Length);
fileTest.Flush();
}
using (var fileTest = System.IO.File.Open(#"c:\temp\fileCloseAndFlush.bin", FileMode.CreateNew))
{
fileTest.Write(randomBytes, 0, randomBytes.Length);
fileTest.Flush();
fileTest.Close();
}
And again - every file got its bytes ... to me it looks like it's doing what I read from MSDN: it doesn't matter if you call Flush or Close before dispose ... any thoughts on that?

You don't have to call Flush() on Close()/Dispose(), FileStream will do it for you as you can see from its source code:
http://referencesource.microsoft.com/#mscorlib/system/io/filestream.cs,e23a38af5d11ddd3
[System.Security.SecuritySafeCritical] // auto-generated
protected override void Dispose(bool disposing)
{
// Nothing will be done differently based on whether we are
// disposing vs. finalizing. This is taking advantage of the
// weak ordering between normal finalizable objects & critical
// finalizable objects, which I included in the SafeHandle
// design for FileStream, which would often "just work" when
// finalized.
try {
if (_handle != null && !_handle.IsClosed) {
// Flush data to disk iff we were writing. After
// thinking about this, we also don't need to flush
// our read position, regardless of whether the handle
// was exposed to the user. They probably would NOT
// want us to do this.
if (_writePos > 0) {
FlushWrite(!disposing); // <- Note this
}
}
}
finally {
if (_handle != null && !_handle.IsClosed)
_handle.Dispose();
_canRead = false;
_canWrite = false;
_canSeek = false;
// Don't set the buffer to null, to avoid a NullReferenceException
// when users have a race condition in their code (ie, they call
// Close when calling another method on Stream like Read).
//_buffer = null;
base.Dispose(disposing);
}
}

I've been tracking a newly introduced bug that seems to indicate .NET 4 does not reliably flush changes to disk when the stream is disposed (unlike .NET 2.0 and 3.5, which always did so reliably).
The .NET 4 FileStream class has been heavily modified in .NET 4, and while the Flush*() methods have been rewritten, similar attention seems to have been forgotten for .Dispose().
This is resulting in incomplete files.

Since you've stated that you understood that close & dispose called the flush method if it was not called explicitly by user code, I believe that (by close without flush) you actually want to have a possibility to discard changes made to a FileStream, if necessary.
If that is correct, using a FileStream alone won't help. You will need to load this file into a MemoryStream (or an array, depending on how you modify its contents), and then decide whether you want to save changes or not after you're done.
A problem with this is file size, obviously. FileStream uses limited size write buffers to speed up operations, but once they are depleted, changes need to be flushed. Due to .NET memory limits, you can only expect to load smaller files in memory, if you need to hold them entirely.
An easier alternative would be to make a disk copy of your file, and work on it using a plain FileStream. When finished, if you need to discard changes, simply delete the temporary file, otherwise replace the original with a modified copy.

Wrap the FileStream in a BufferedStream and close the filestream before the buffered stream.
var fs = new FileStream(...);
var bs = new BufferedStream(fs, buffersize);
bs.Write(datatosend, 0, length);
fs.Close();
try {
bs.Close();
}
catch (IOException) {
}

Using Flush() is worthy inside big Loops.
when you have to read and write a big File inside one Loop. In other case the buffer or the computer is big enough, and doesn´t matter to close() without making one Flush() before.
Example: YOU HAVE TO READ A BIG FILE (in one format) AND WRITE IT IN .txt
StreamWriter sw = .... // using StreamWriter
// you read the File ...
// and now you want to write each line for this big File using WriteLine ();
for ( .....) // this is a big Loop because the File is big and has many Lines
{
sw.WriteLine ( *whatever i read* ); //we write here somrewhere ex. one .txt anywhere
sw.Flush(); // each time the sw.flush() is called, the sw.WriteLine is executed
}
sw.Close();
Here it is very important to use Flush(); beacause otherwise each writeLine is save in the buffer and does not write it until the buffer is frull or until the program reaches sw.close();
I hope this helps a little to understand the function of Flush

I think it is safe to use simple using statement, which closes the stream after the call to GetBytes();
public static byte[] GetBytes(string fileName)
{
byte[] buffer = new byte[4096];
using (FileStream fs = new FileStream(fileName))
using (MemoryStream ms = new MemoryStream())
{
fs.BlockCopy(ms, buffer, 4096); // extension method for the Stream class
fs.Close();
return ms.ToByteArray();
}
}

Advanced TextReader to EndOfFile

I have a textReader that in a specific instance I want to be able to advance to the end of file quickly so other classes that might hold a reference to this object will not be able to call tr.ReadLine() without getting a null.
This is a large file. I cannot use TextReader.ReadToEnd() as it will often lead to an OutOfMemoryException
I thought I would ask the community if there was a way SEEK the stream without using TextReader.ReadToEnd() which returns a string of all data in the file.
Current method, inefficient.
The following example code is a mock up. Obviously I am not opening a file with an if statement directly following it asking if I want to read to the end.
TextReader tr = new StreamReader("Largefile");
if(needToAdvanceToEndOfFile)
{
while(tr.ReadLine() != null) { }
}
Desired solution (Note this code block contains fake 'concept' methods or methods that cannot be used due to risk of outofmemoryexception)
TextReader tr = new StreamReader("Largefile");
if(needToAdvanceToEndOfFile)
{
tr.SeekToEnd(); // A method that does not return anything. This method does not exist.
// tr.ReadToEnd() not acceptable as it can lead to OutOfMemoryException error as it is very large file.
}
A possible alternative is to read through the file in bigger chunks using tr.ReadBlock(args).
I poked around ((StreamReader)tr).BaseStream but could not find anything that worked.
As I am new to the community I figured I would see if someone knew the answer off the top of their head.

You have to discard any buffered data if you have read any file content - since data is buffered you might get content even if you seek the underlying stream to the end - working example:
StreamReader sr = new StreamReader(fileName);
string sampleLine = sr.ReadLine();
//discard all buffered data and seek to end
sr.DiscardBufferedData();
sr.BaseStream.Seek(0, SeekOrigin.End);
The problem as mentioned in the documentation is
The StreamReader class buffers input from the underlying stream when
you call one of the Read methods. If you manipulate the position of
the underlying stream after reading data into the buffer, the position
of the underlying stream might not match the position of the internal
buffer. To reset the internal buffer, call the DiscardBufferedData
method

Use
reader.BaseStream.Seek(0, SeekOrigin.End);
Test:
using (StreamReader reader = new StreamReader(#"Your Large File"))
{
reader.BaseStream.Seek(0, SeekOrigin.End);
int read = reader.Read();//read will be -1 since you are at the end of the stream
}
Edit: Test it with your code:
using (TextReader tr = new StreamReader("C:\\test.txt"))//test.txt is a file that has data and lines
{
((StreamReader)tr).BaseStream.Seek(0, SeekOrigin.End);
string foo = tr.ReadLine();
Debug.WriteLine(foo ?? "foo is null");//foo is null
int read = tr.Read();
Debug.WriteLine(read);//-1
}