I have a simple program to read a file using the StreamReader and process it line by line. But the file I am reading may sometimes locate in a network folder. I came across while doing some testing with such a file, that if the network connection lost at some point while I am reading, it'll stay in the same line again and again looping in an infinite loop by resulting the same line as the result from stream.ReadLine().
Is there a way I can find when the fileHandle is not available from the stream itself? I was expecting a FileNotAvailableException kind of an exception would fire when the filehandle is lost from the StreamReader.
Here's my code snippet...
string file = #"Z://1601120903.csv"; //Network file
string line;
StringBuilder stb = new StringBuilder();
StreamReader stream = new StreamReader(file, Encoding.UTF8, true, 1048576);
do
{
line = stream.ReadLine();
// Do some work here
} while (line != "");
Compare with null not with empty string:
https://msdn.microsoft.com/en-us/library/system.io.streamreader.readline(v=vs.110).aspx
Return Value Type: System.String The next line from the input stream,
or null if the end of the input stream is reached.
do
{
line = stream.ReadLine();
// Do some work here
} while (line != null);
A better approach, however, is to let .Net do the work (line by line file reading) for you and drop all readers:
foreach (String line in File.ReadLines(file)) {
// Do some work here
}
Correct approach 1 (EndOfStream) :
using(StreamReader sr = new StreamReader(...)) {
while(!sr.EndOfStream) {
string line = sr.ReadLine();
Console.WriteLine(line);
}
}
Correct approach 2 (Peek)
using(StreamReader sr = new StreamReader(...)) {
while(sr.Peek() >= 0) {
string line = sr.ReadLine();
}
}
Note: that it is incorrect to threat an empty string as end of file.
if the network connection lost at some point while I am reading,
it'll stay in the same line again and again looping in an infinite
loop by resulting the same line as the result from stream.ReadLine()
I've checked this scenario right now - the System.IO.IOException ("The network path was not found."} should be thrown in this case.
Wrapping this with a try catch block will not fix my problem, will it?
In this case you can break the reading as follows:
string line;
do {
try {
line = sr.ReadLine();
// Do some work here
}
catch(System.IO.IOException) {
break;
}
} while(line != null);
If you write it with a while-loop:
while ((line = sr.ReadLine()) != null)
{
Console.WriteLine(line);
}
Source
One more way would be to use File.ReadAllLines() and it will take care of opening file and reading all lines and closig the file and may also handle scenario when network connection is lost.
var lines = File.ReadAllLines("Z://1601120903.csv");
foreach(line in lines)
{
// Do some work
}
Assuming the file shouldn't change while you reading it and it's not huge, you might want to consider to copy it to a temp file (locally) and then work on it without interference.
If you want to get index of the place you reached this might help:
How to know position(linenumber) of a streamreader in a textfile?
If your stream is a NetworkStream, the ReadLine method will expect more content from the stream, if it reached at end, indefinitely. I think, and according to the StreamReader documentation, it is designed to work only with local file streams. In this case, you can read bytes directly from the NetworkStream.
https://learn.microsoft.com/pt-br/dotnet/api/system.net.sockets.networkstream.read?view=netcore-3.1#System_Net_Sockets_NetworkStream_Read_System_Span_System_Byte__
Related
Hi everyone beginner here looking for some advice with a program I'm writing in C#. I need to be able to open a text document, read the first line of text (that is not blank), save this line of text to another text document and finally overwrite the read line with an empty line.
This is what I have so far, everything works fine until the last part where I need to write a blank line to the original text document, I just get a full blank document. Like I mentioned above I'm new to C# so I'm sure there is an easy solution to this but I can't figure it out, any help appreciated:
try
{
StreamReader sr = new StreamReader(#"C:\Users\Stephen\Desktop\Sample.txt");
line = sr.ReadLine();
while (line == "")
{
line = sr.ReadLine();
}
sr.Close();
string path = (#"C:\Users\Stephen\Desktop\new.txt");
if (!File.Exists(path))
{
File.Create(path).Dispose();
TextWriter tw = new StreamWriter(path);
tw.WriteLine(line);
tw.Close();
}
else if (File.Exists(path))
{
TextWriter tw = new StreamWriter(path, true);
tw.WriteLine(line);
tw.Close();
}
StreamWriter sw = new StreamWriter(#"C:\Users\Stephen\Desktop\Sample.txt");
int cnt1 = 0;
while (cnt1 < 1)
{
sw.WriteLine("");
cnt1 = 1;
}
sw.Close();
}
catch (Exception e)
{
Console.WriteLine("Exception: " + e.Message);
}
finally
{
Console.WriteLine("Executing finally block.");
}
else
Console.WriteLine("Program Not Installed");
Console.ReadLine();
Unfortunately, you do have to go through the painstaking process of rewriting the file. In most cases, you could get away with loading it into memory and just doing something like:
string contents = File.ReadAllText(oldFile);
contents = contents.Replace("bad line!", "good line!");
File.WriteAllText(newFile, contents);
Remember that you'll have to deal with the idea of line breaks here, since string.Replace doesn't innately pay attention only to whole lines. But that's certainly doable. You could also use a regex with that approach. You can also use File.ReadAllLines(string) to read each line into an IEnumerable<string> and test each one while you write them back to the new file. It just depends on what exactly you want to do and how precise you want to be about it.
using (var writer = new StreamWriter(newFile))
{
foreach (var line in File.ReadAllLines(oldFile))
{
if (shouldInsert(line))
writer.WriteLine(line);
}
}
That, of course, depends on the predicate shouldInsert, but you can modify that as you see so fit. But the nature of IEnumerable<T> should make that relatively light on resources. You could also use a StreamReader for a bit lower-level of support.
using (var writer = new StreamWriter(newFile))
using (var reader = new StreamReader(oldFile))
{
string line;
while ((line = reader.ReadLine()) != null)
{
if (shouldInsert(line))
writer.WriteLine(line);
}
}
Recall, of course, that this could leave you with an extra, empty line at the end of the file. I'm too tired to say that with the certainty I should be able to, but I'm pretty sure that's the case. Just keep an eye out for that, if it really matters. Of course, it normally won't.
That all said, the best way to do it would be to have a bit of fun and do it without wasting the memory, by writing a function to read the FileStream in and write out the appropriate bytes to your new file. That's, of course, the most complicated and likely over-kill way, but it'd be a fun undertaking.
See: Append lines to a file using a StreamWriter
Add true to the StreamWriter constructor to set it to "Append" mode. Note that this adds a line at the bottom of the document, so you may have to fiddle a bit to insert or overwrite it at the top instead.
And see: Edit a specific Line of a Text File in C#
Apparently, it's not that easy to just insert or overwrite a single line and the usual method is just to copy all lines while replacing the one you want and writing every line back to the file.
So, let's say i have a text file with 20 lines, with on each line different text.
i want to be able to have a string that has the first line in it, but when i do NextLine(); i want it to be the next line. I tried this but it doesn't seem to work:
string CurrentLine;
int LastLineNumber;
Void NextLine()
{
System.IO.StreamReader file = new System.IO.StreamReader("c:\\test.txt");
CurrentLine = file.ReadLine(LastLineNumber + 1);
LastLineNumber++;
}
How would i be able to do this?
Thanks in advance.
In general, it would be better if you could design this in a way to leave your file open, and not try to reopen the file each time.
If that is not practical, you'll need to call ReadLine multiple times:
string CurrentLine;
int LastLineNumber;
void NextLine()
{
// using will make sure the file is closed
using(System.IO.StreamReader file = new System.IO.StreamReader("c:\\test.txt"))
{
// Skip lines
for (int i=0;i<LastLineNumber;++i)
file.ReadLine();
// Store your line
CurrentLine = file.ReadLine();
LastLineNumber++;
}
}
Note that this can be simplified via File.ReadLines:
void NextLine()
{
var lines = File.ReadLines("C:\\test.txt");
CurrentLine = lines.Skip(LastLineNumber).First();
LastLineNumber++;
}
One simple call should do it:
var fileLines = System.IO.File.ReadAllLines(fileName);
You will want to validate the file exists and of course you still need to watch for blank lines or invalid values but that should give you the basics. To loop over the file you can use the following:
foreach (var singleLine in fileLines) {
// process "singleLine" here
}
One more note - you won't want to do this with large files since it processes everything in memory.
Well, if you really don't mind re-opening the file each time, you can use:
CurrentLine = File.ReadLines("c:\\test.txt").Skip(LastLineNumber).First();
LastLineNumber++;
However, I'd advise you to just read the whole thing in one go using File.ReadAllLines, or perhaps File.ReadLines(...).ToList().
The ReadLine method already reads the next line in the StreamReader, you don't need the counter, or your custom function for that matter. Just keep reading until you reach your 20 lines or until the file ends.
You can't pass a line number to ReadLine and expect it to find that particular line. If you look at the ReadLine documentation, you'll see it doesn't accept any parameters.
public override string ReadLine()
When working with files, you must treat them as streams of data. Every time you open the file, you start at the very first byte/character of the file.
var reader = new StreamReader("c:\\test.txt"); // Starts at byte/character 0
You have to keep the stream open if you want to read more lines.
using (var reader = new StreamReader("c:\\test.txt"))
{
string line1 = reader.ReadLine();
string line2 = reader.ReadLine();
string line3 = reader.ReadLine();
// etc..
}
If you really want to write a method NextLine, then you need to store the created StreamReader object somewhere and use that every time. Somewhat like this:
public class MyClass : IDisposable
{
StreamReader reader;
public MyClass(string path)
{
this.reader = new StreamReader(path);
}
public string NextLine()
{
return this.reader.ReadLine();
}
public void Dispose()
{
reader.Dispose();
}
}
But I suggest you either loop through the stream:
using (var reader = new StreamReader("c:\\test.txt"))
{
while (some_condition)
{
string line = reader.ReadLine();
// Do something
}
}
Or get all the lines at once using the File class ReadAllLines method:
string[] lines = System.IO.File.ReadAllLines("c:\\test.txt");
for (int i = 0; i < lines.Length; i++)
{
string line = lines[i];
// Do something
}
What is the quickest way to read a text file into a string variable?
I understand it can be done in several ways, such as read individual bytes and then convert those to string. I was looking for a method with minimal coding.
How about File.ReadAllText:
string contents = File.ReadAllText(#"C:\temp\test.txt");
A benchmark comparison of File.ReadAllLines vs StreamReader ReadLine from C# file handling
Results. StreamReader is much faster for large files with 10,000+
lines, but the difference for smaller files is negligible. As always,
plan for varying sizes of files, and use File.ReadAllLines only when
performance isn't critical.
StreamReader approach
As the File.ReadAllText approach has been suggested by others, you can also try the quicker (I have not tested quantitatively the performance impact, but it appears to be faster than File.ReadAllText (see comparison below)). The difference in performance will be visible only in case of larger files though.
string readContents;
using (StreamReader streamReader = new StreamReader(path, Encoding.UTF8))
{
readContents = streamReader.ReadToEnd();
}
Comparison of File.Readxxx() vs StreamReader.Readxxx()
Viewing the indicative code through ILSpy I have found the following about File.ReadAllLines, File.ReadAllText.
File.ReadAllText - Uses StreamReader.ReadToEnd internally
File.ReadAllLines - Also uses StreamReader.ReadLine internally with the additionally overhead of creating the List<string> to return as the read lines and looping till the end of file.
So both the methods are an additional layer of convenience built on top of StreamReader. This is evident by the indicative body of the method.
File.ReadAllText() implementation as decompiled by ILSpy
public static string ReadAllText(string path)
{
if (path == null)
{
throw new ArgumentNullException("path");
}
if (path.Length == 0)
{
throw new ArgumentException(Environment.GetResourceString("Argument_EmptyPath"));
}
return File.InternalReadAllText(path, Encoding.UTF8);
}
private static string InternalReadAllText(string path, Encoding encoding)
{
string result;
using (StreamReader streamReader = new StreamReader(path, encoding))
{
result = streamReader.ReadToEnd();
}
return result;
}
string contents = System.IO.File.ReadAllText(path)
Here's the MSDN documentation
For the noobs out there who find this stuff fun and interesting, the fastest way to read an entire file into a string in most cases (according to these benchmarks) is by the following:
using (StreamReader sr = File.OpenText(fileName))
{
string s = sr.ReadToEnd();
}
//you then have to process the string
However, the absolute fastest to read a text file overall appears to be the following:
using (StreamReader sr = File.OpenText(fileName))
{
string s = String.Empty;
while ((s = sr.ReadLine()) != null)
{
//do what you have to here
}
}
Put up against several other techniques, it won out most of the time, including against the BufferedReader.
Take a look at the File.ReadAllText() method
Some important remarks:
This method opens a file, reads each line of the file, and then adds
each line as an element of a string. It then closes the file. A line
is defined as a sequence of characters followed by a carriage return
('\r'), a line feed ('\n'), or a carriage return immediately followed
by a line feed. The resulting string does not contain the terminating
carriage return and/or line feed.
This method attempts to automatically detect the encoding of a file
based on the presence of byte order marks. Encoding formats UTF-8 and
UTF-32 (both big-endian and little-endian) can be detected.
Use the ReadAllText(String, Encoding) method overload when reading
files that might contain imported text, because unrecognized
characters may not be read correctly.
The file handle is guaranteed to be closed by this method, even if
exceptions are raised
string text = File.ReadAllText("Path"); you have all text in one string variable. If you need each line individually you can use this:
string[] lines = File.ReadAllLines("Path");
System.IO.StreamReader myFile =
new System.IO.StreamReader("c:\\test.txt");
string myString = myFile.ReadToEnd();
if you want to pick file from Bin folder of the application then you can try following and don't forget to do exception handling.
string content = File.ReadAllText(Path.Combine(System.IO.Directory.GetCurrentDirectory(), #"FilesFolder\Sample.txt"));
#Cris sorry .This is quote MSDN Microsoft
Methodology
In this experiment, two classes will be compared. The StreamReader and the FileStream class will be directed to read two files of 10K and 200K in their entirety from the application directory.
StreamReader (VB.NET)
sr = New StreamReader(strFileName)
Do
line = sr.ReadLine()
Loop Until line Is Nothing
sr.Close()
FileStream (VB.NET)
Dim fs As FileStream
Dim temp As UTF8Encoding = New UTF8Encoding(True)
Dim b(1024) As Byte
fs = File.OpenRead(strFileName)
Do While fs.Read(b, 0, b.Length) > 0
temp.GetString(b, 0, b.Length)
Loop
fs.Close()
Result
FileStream is obviously faster in this test. It takes an additional 50% more time for StreamReader to read the small file. For the large file, it took an additional 27% of the time.
StreamReader is specifically looking for line breaks while FileStream does not. This will account for some of the extra time.
Recommendations
Depending on what the application needs to do with a section of data, there may be additional parsing that will require additional processing time. Consider a scenario where a file has columns of data and the rows are CR/LF delimited. The StreamReader would work down the line of text looking for the CR/LF, and then the application would do additional parsing looking for a specific location of data. (Did you think String. SubString comes without a price?)
On the other hand, the FileStream reads the data in chunks and a proactive developer could write a little more logic to use the stream to his benefit. If the needed data is in specific positions in the file, this is certainly the way to go as it keeps the memory usage down.
FileStream is the better mechanism for speed but will take more logic.
well the quickest way meaning with the least possible C# code is probably this one:
string readText = System.IO.File.ReadAllText(path);
you can use :
public static void ReadFileToEnd()
{
try
{
//provide to reader your complete text file
using (StreamReader sr = new StreamReader("TestFile.txt"))
{
String line = sr.ReadToEnd();
Console.WriteLine(line);
}
}
catch (Exception e)
{
Console.WriteLine("The file could not be read:");
Console.WriteLine(e.Message);
}
}
string content = System.IO.File.ReadAllText( #"C:\file.txt" );
You can use like this
public static string ReadFileAndFetchStringInSingleLine(string file)
{
StringBuilder sb;
try
{
sb = new StringBuilder();
using (FileStream fs = File.Open(file, FileMode.Open))
{
using (BufferedStream bs = new BufferedStream(fs))
{
using (StreamReader sr = new StreamReader(bs))
{
string str;
while ((str = sr.ReadLine()) != null)
{
sb.Append(str);
}
}
}
}
return sb.ToString();
}
catch (Exception ex)
{
return "";
}
}
Hope this will help you.
you can read a text from a text file in to string as follows also
string str = "";
StreamReader sr = new StreamReader(Application.StartupPath + "\\Sample.txt");
while(sr.Peek() != -1)
{
str = str + sr.ReadLine();
}
I made a comparison between a ReadAllText and StreamBuffer for a 2Mb csv and it seemed that the difference was quite small but ReadAllText seemed to take the upper hand from the times taken to complete functions.
I'd highly recommend using the File.ReadLines(path) compare to StreamReader or any other File reading methods. Please find below the detailed performance benchmark for both small-size file and large-size file.
I hope this would help.
File operations read result:
For small file (just 8 lines)
For larger file (128465 lines)
Readlines Example:
public void ReadFileUsingReadLines()
{
var contents = File.ReadLines(path);
}
Note : Benchmark is done in .NET 6.
This comment is for those who are trying to read the complete text file in winform using c++ with the help of C# ReadAllText function
using namespace System::IO;
String filename = gcnew String(charfilename);
if(System::IO::File::Exists(filename))
{
String ^ data = gcnew String(System::IO::File::RealAllText(filename)->Replace("\0", Environment::Newline));
textBox1->Text = data;
}
I can currently remove the last line of a text file using:
var lines = System.IO.File.ReadAllLines("test.txt");
System.IO.File.WriteAllLines("test.txt", lines.Take(lines.Length - 1).ToArray());
Although, how is it possible to instead remove the beginning of the text file?
Instead of lines.Take, you can use lines.Skip, like:
var lines = File.ReadAllLines("test.txt");
File.WriteAllLines("test.txt", lines.Skip(1).ToArray());
to truncate at the beginning despite the fact that the technique used (read all text and write everything back) is very inefficient.
About the efficient way: The inefficiency comes from the necessity to read the whole file into memory. The other way around could easily be to seek in a stream and copy the stream to another output file, delete the original, and rename the old. That one would be equally fast and yet consume much less memory.
Truncating a file at the end is much easier. You can just find the trunaction position and call FileStream.SetLength().
Here is an alternative:
using (var stream = File.OpenRead("C:\\yourfile"))
{
var items = new LinkedList<string>();
using (var reader = new StreamReader(stream))
{
reader.ReadLine(); // skip one line
string line;
while ((line = reader.ReadLine()) != null)
{
//it's far better to do the actual processing here
items.AddLast(line);
}
}
}
Update
If you need an IEnumerable<string> and don't want to waste memory you could do something like this:
public static IEnumerable<string> GetFileLines(string filename)
{
using (var stream = File.OpenRead(filename))
{
using (var reader = new StreamReader(stream))
{
reader.ReadLine(); // skip one line
string line;
while ((line = reader.ReadLine()) != null)
{
yield return line;
}
}
}
}
static void Main(string[] args)
{
foreach (var line in GetFileLines("C:\\yourfile.txt"))
{
// do something with the line here.
}
}
var lines = System.IO.File.ReadAllLines("test.txt");
System.IO.File.WriteAllLines("test.txt", lines.Skip(1).ToArray());
Skip eliminates the given number of elements from the beginning of the sequence. Take eliminates all but the given number of elements from the end of the sequence.
To remove fist line from a text file
System.IO.StreamReader file = new System.IO.StreamReader(filePath);
string data = file.ReadToEnd();
file.Close();
data = Regex.Replace(data, "<.*\n", "");
System.IO.StreamWriter file = new System.IO.StreamWriter(filePath, false);
file.Write(data);
file.Close();
can do in one line also
File.WriteAllLines(origialFilePath,File.ReadAllLines(originalFilePath).Skip(1));
Assuming you are passing your filePath as parameter to the function.
This is the way I read file:
public static string readFile(string path)
{
StringBuilder stringFromFile = new StringBuilder();
StreamReader SR;
string S;
SR = File.OpenText(path);
S = SR.ReadLine();
while (S != null)
{
stringFromFile.Append(SR.ReadLine());
}
SR.Close();
return stringFromFile.ToString();
}
The problem is it so long (the .txt file is about 2.5 megs). Took over 5 minutes. Is there a better way?
Solution taken
public static string readFile(string path)
{
return File.ReadAllText(path);
}
Took less than 1 second... :)
S = SR.ReadLine();
while (S != null)
{
stringFromFile.Append(SR.ReadLine());
}
Of note here, S is never set after that initial ReadLine(), so the S != null condition never triggers if you enter the while loop. Try:
S = SR.ReadLine();
while (S != null)
{
stringFromFile.Append(S = SR.ReadLine());
}
or use one of the other comments.
If you need to remove newlines, use string.Replace(Environment.NewLine, "")
Leaving aside the horrible variable names and the lack of a using statement (you won't close the file if there are any exceptions) that should be okay, and certainly shouldn't take 5 minutes to read 2.5 megs.
Where does the file live? Is it on a flaky network share?
By the way, the only difference between what you're doing and using File.ReadAllText is that you're losing line breaks. Is this deliberate? How long does ReadAllText take?
return System.IO.File.ReadAllText(path);
Marcus Griep has it right. IT's taking so long because YOU HAVE AN INFINITE LOOP. copied your code and made his changes and it read a 2.4 M text file in less than a second.
but I think you might miss the first line of the file. Try this.
S = SR.ReadLine();
while (S != null){
stringFromFile.Append(S);
S = SR.ReadLine();
}
Do you need the entire 2.5 Mb in memory at once?
If not, I would try to work with what you need.
Use System.IO.File.RealAllLines instead.
http://msdn.microsoft.com/en-us/library/system.io.file.readalllines.aspx
Alternatively, estimating the character count and passing that to StringBuilder's constructor as the capacity should speed it up.
Try this, should be much faster:
var str = System.IO.File.ReadAllText(path);
return str.Replace(Environment.NewLine, "");
By the way: Next time you're in a similar situation, try pre-allocating memory. This improves runtime drastically, regardless of the exact data structures you use. Most containers (StringBuilder as well) have a constructor that allow you to reserve memory. This way, less time-consuming reallocations are necessary during the read process.
For example, you could write the following if you want to read data from a file into a StringBuilder:
var info = new FileInfo(path);
var sb = new StringBuilder((int)info.Length);
(Cast necessary because System.IO.FileInfo.Length is long.)
ReadAllText was a very good solution for me. I used following code for 3.000.000 row text file and it took 4-5 seconds to read all rows.
string fileContent = System.IO.File.ReadAllText(txtFilePath.Text)
string[] arr = fileContent.Split('\n');
The loop and StringBuilder may be redundant; Try using
ReadToEnd.
To read a text file fastest you can use something like this
public static string ReadFileAndFetchStringInSingleLine(string file)
{
StringBuilder sb;
try
{
sb = new StringBuilder();
using (FileStream fs = File.Open(file, FileMode.Open))
{
using (BufferedStream bs = new BufferedStream(fs))
{
using (StreamReader sr = new StreamReader(bs))
{
string str;
while ((str = sr.ReadLine()) != null)
{
sb.Append(str);
}
}
}
}
return sb.ToString();
}
catch (Exception ex)
{
return "";
}
}
Hope this will help you. and for more info, please visit to the following link-
Fastest Way to Read Text Files