I have txt file(65mb)i need to read line by line and change each line,
For example i have many lines
User=value Password=value Phone=123456789
User=value Password=value Phone=123456789
User=value Password=value Phone=123456789
and i need to change first number of credit card/Phone to*(security reason), and get text like this and save it, or just to change origin text file.
User=value Password=value Phone=*****6789
User=value Password=value Phone=*****6789
User=value Password=value Phone=*****6789
I build new string and add to there line(changed) by line than save, but it take me to many time this is my code
string NewPath = "";
string lineOfText;
string NewTextFile = "";
using (var filestream = new FileStream(FilePath,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
{
var file = new StreamReader(filestream, Encoding.UTF8, true, 128);
while ((lineOfText = file.ReadLine()) != null)//here i reading line by line
{
NewTextFile += lineOfText.Substring(0, 124) + "************" +
lineOfText.Substring(136, lineOfText.Length - 136);
NewTextFile += Environment.NewLine;//here i make new string
}
}
NewPath = FilePatharr[1] + "\\temp.txt";
System.IO.File.WriteAllText(NewPath, NewTextFile);//here i save him
Do any one know better way to do this,my code is taking to long to save this big file.
UPDATE
Why do i get -2 for this question? Whats wrong with this question? I see here only wrong answers about how to pass sensitive data and more things that not belong to this questions When the question was -->Fast way to change txt file and save it
Any way i find out how to do this speed of saving file speedUp from 100kb\sec to 3MB\sec now it taking me 20sec and not 20min like before
Your primary problem here is that you're appending to a string. And that gets expensive very quickly. You should be able to process that 65 MB in about five seconds. Here's what I would do:
string outputFileName = "temp.txt";
using (var outputFile = new StreamWriter(outputFileName))
{
foreach (var line in File.ReadLines(inputFileName))
{
var newLine = line.Substring(0, 124) + "************" +
line.Substring(136, lineOfText.Length - 136);
outputFile.WriteLine(newLine);
}
}
This is going to be a lot faster than appending strings. If you really want to do it all in memory, then use a StringBuilder. Instead of
string NewTextFile = "";
Use
StringBuilder NewTextFile = new StringBuilder();
And when you're composing the output, replace the string concatenation with:
NewTextFile.AppendLine(
lineOfText.Substring(0, 124) + "************" +
lineOfText.Substring(136, lineOfText.Length - 136));
Finally, to write it to the file:
System.IO.File.WriteAllText(NewPath, NewTextFile.ToString());
You could keep WriteAllLines and use ReadAllLines:
ReadAllLines to get all lines as an array of strings
Enumerate the array and use string.Replace to replace its content, line by line
WriteAllLines and overwrite existing file
Or you could use stream reader/writer and read/write line by line. You currently are building a string with millions of concatenations, that's the performance issue.
That being said, you probably have an obvious security issue here. Why was critical information such as credit card numbers stored as plain text in the first place?
public static string ChangeCardNumber(string FilePath, string txtFileName)
{
string NewPath = "";
string lineOfText;
string NewTextFile = "";
NewPath = FilePatharr[0] + "\\temp_" + txtFileName;
using (var filestream = new FileStream(FilePath,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
{
var file = new StreamReader(filestream, Encoding.UTF8, true, 128);
System.IO.File.Create(NewPath).Close();
int res = 0;
while ((lineOfText = file.ReadLine()) != null)
{
res++;
if (!lineOfText.Contains("EOF"))
{
NewTextFile += lineOfText.Substring(0, 124) + "************" +
lineOfText.Substring(136, lineOfText.Length - 136);
NewTextFile += Environment.NewLine;
}
if (res % 200 == 0 || lineOfText.Contains("EOF"))//check if endline and if in NewTextFile 200lines
{
System.IO.File.AppendAllText(NewPath, NewTextFile);
NewTextFile = "";
}
}
}
return NewPath;
}
Well I Guess You Could Do It Another Way And This Way Is The Way I Use For All of my saving needs.
if you want to save it all at the same time then this would work! or you could have it loaded in VARIABLES and then save it but this is the code to save it. it is very simple and easy to edit
There You Go
Basic Code To Add A New Line With Text
echo your text here >>filename.txt
For Explaination Please Visit The DropBox Link: https://www.dropbox.com/s/nufkecx1b8088cu/HOW%20TO%20USE%20THIS.txt
Related
I want to count the number of some strings and store it into a csv file. I've tried it but I don't know if this is the correct way and in addition, there are two problems.
First of all, here is my method:
public void CountMacNames(String macName)
{
string path = #"D:\Counter\macNameCounter.csv";
if (!File.Exists(path))
{
File.Create(path).Close();
}
var lines = File.ReadLines(path);
foreach (var line in lines)
{
bool isExists = line.Split(',').Any(x => x == macName);
if (isExists)
{
// macName exists, increment it's value by 1
}
else
{
// macName does not exists, add macName to CSV file and start counter by 1
var csv = new StringBuilder();
var newLine = string.Format("{0},{1}", macName, 1);
csv.AppendLine(newLine);
File.WriteAllText(path, csv.ToString());
}
}
}
The first problem is this IOException:
The process cannot access the file 'D:\Counter\macNameCounter.csv'
because it is being used by another process.
The second problem is, that I don't know how to increment the value by one, if a macName exists in the csv file (see first comment)
EDIT: Example for method "CountMacNames" call:
CountMacNames("Cansas");
CountMacNames("Wellback");
CountMacNames("Newton");
CountMacNames("Cansas");
CountMacNames("Princet");
Then, the CSV file should contain:
Cansas, 2
Wellback, 1
Newton, 1
Princet, 1
OK, this is what I'd do:
public void CountMacNames(String macName)
{
string path = #"D:\Counter\macNameCounter.csv";
// Read all lines, but only if file exists
string[] lines = new string[0];
if (File.Exists(path))
lines = File.ReadAllLines(path);
// This is the new CSV file
StringBuilder newLines = new StringBuilder();
bool macAdded = false;
foreach (var line in lines)
{
string[] parts = line.Split(',');
if (parts.Length == 2 && parts[0].Equals(macName))
{
int newCounter = Convert.ToIn32(parts[1])++;
newLines.AppendLine(String.Format("{0},{1}", macName, newCounter));
macAdded = true;
}
else
{
newLines.AppendLine(line.Trim());
}
}
if (!macAdded)
{
newLines.AppendLine(String.Format("{0},{1}", macName, 1));
}
File.WriteAllText(path, newLines.ToString());
}
This code does this:
Read all the lines from file only if it exists - otherwise we start a new file
Iterate over all the lines
If the first part of a 2-part line equals the mac, add 1 to counter and add line to output
If the first part doesn't match or the line format is wrong, add the line to output as is
If we didn't find the mac in any line, add a new line for the mac with counter 1
Write the file back
You can't read and write to the same file at the same time (in a simple way).
For small files, there are already answers.
If your file is really large (too big to fit in memory) you need another approach:
Read input file line by line
optinally modify the current line
write line to a temporary file
If finished delete input file, rename temporary file
For the first problem you can either read all the lines into memory and work there then write it all out again, or use streams.
using (FileStream fs = File.Open(filePath, FileMode.Create, FileAccess.ReadWrite))
{
var sw = new StreamWriter(fs);
var sr = new StreamReader(fs);
while(!streamReader.EndOfStream)
{
var line = sr.ReadLine();
//Do stuff with line.
//...
if (macExists)
{
//Increment the number, Note that in here we can only replace characters,
//We can't insert extra characters unless we rewrite the rest of the file
//Probably more hassle than it's worth but
//You could have a fixed number of characters like 000001 or 1
//Read the number as a string,
//Int.Parse to get the number
//Increment it
//work out the number of bytes in the line.
//get the stream position
//seek back to the beginning of the line
//Overwrite the whole line with the same number of bytes.
}
else
{
//Append a line, also harder to do with streams like this.
//Store the current position,
//Seek to the end of the file,
//WriteLine
//Seek back again.
}
}
}
You need to read the file in and release it, like this, to avoid the IO exception:
string[] lines = null;
using (var sr = new System.IO.StreamReader(path))
lines = sr.ReadToEnd().Split(new string[] {"\r", "\n"}, StringSplitOptions.RemoveEmptyEntries);
As for the count, you can just add an int value, change the method return type as int, too.
public int CountMacNames(String macName, String path)
{
if (!File.Exists(path))
{
File.Create(path).Close();
}
string[] lines = null;
using (var sr = new System.IO.StreamReader(path))
lines = sr.ReadToEnd().Split(new string[] {"\r", "\n"}, StringSplitOptions.RemoveEmptyEntries);
return lines.Where(p => p.Split(',').Contains(macName)).Count();
}
and inside the method that calls it:
var path = #"<PATH TO FILE>";
var cnt = CountMacNames("Canvas", path);
if (cnt > 0)
{
using (var sw = new StreamWriter(path, true, Encoding.Unicode))
sw.WriteLine(string.Format("Canvas,{0}", cnt));
}
Now, var res = CountMacNames("Canvas","PATH"); will return 2, and the lines "Canvas,2" or "Newton,1" will be appended to the file, without overwriting it.
I'm writing text to files using StreamWriter using the following code:
path == #"Desktop\";
filepath1 = path + "1.txt";
StreamWriter _sw = new StreamWriter(filepath1, true);
_sw.WriteLine("some Text");
_sw.Close();
if size of textfile exceeds 500kb I want to create text files dynamically. I'm tryng following code:
var size = (path.Length)/1024;
if(size>=500)
{
int i = (size/500)+1;
var filepath2 = path + i + ".txt";
if (File.Exists(filepath2))
{
StreamWriter _sw = new StreamWriter(filepath2, true);
_sw.WriteLine("Some message");
_sw.Close();
}
}
else
{
FileStream fs = File.Create(filepath2);
StreamWriter _sw = new StreamWriter(filepath2, true);
_sw.WriteLine(ex);
_sw.Close();
}
My question is if file 2.txt also exceeds 500kb I want to create 3.txt,4.txt..... and so on..
I want to create all these dynamically - how to solve this problem?
First thing you need to do the SIZE comparison for the data length of File not the File Path.
Here is Function which dose what you want to achieve, Please make appropriate changes for your path.
//Public variable to manage file names
int FileCounter = 1;
string FileName;
// Call this function to Add text to file
private void WriteToFile(string writeText)
{
FileName = "MyFile_"+FileCounter +".txt";
if (File.Exists(FileName))
{
string str = File.ReadAllText(FileName);
if ((str.Length + writeText.Length) / 1024 > 500) // check for limit
{
// Create new File
FileCounter++;
FileName = "MyFile_" + FileCounter + ".txt";
StreamWriter _sw = new StreamWriter(FileName, true);
_sw.WriteLine(writeText);
_sw.Close();
}
else // use exixting file
{
StreamWriter _sw = new StreamWriter(FileName, true);
_sw.WriteLine(writeText);
_sw.Close();
}
}
}
Where to start..
You are writing it as one big long procedural script. You need to break it down into chunks that can be reused using functions. As it is, it will get out of control way too quickly.
path == #"Desktop\"; is not valid. 1 too many =
Use Path.Combine() to combine your folder and filenames.
I'm sure this is all just test/rough/scratch code but just in case it's not, also check out Try/Except to wrap your file handling. You should also look up using() to dispose of your streams/writers.
My last comment would be that I see a lot of this sort of code a lot and it's often replaceable with something like Nlog for a whole lot less friction.
I would have commented but this login has no rep.
I am working on C# on Win7.
I need to use Streamwriter to write to a txt file.
StreamWriter outfile = new StreamWriter(MY_PATH, true);
foreach(a line of strings)
{
// process the line
outfile.Write(String.Format(WIDTH + " " + WIDTH, num1Str+"\t", num2Str+"\t"));
}
if all elements in line are "0"
// do not write anything to the file, clear outfile buffer
// WIDTH are constants. num1Str and num2Str are variables.
How to clear the contents written in the stream buffer ?
Flush is not a solution because I do not want to write the file if all elements are 0.
Any help would be appreciated.
I believe you're looking for outfile.Flush();.
UPDATE: so now that the question is clearer, you don't want a StreamWriter, you want to leverage something like a MemoryStream instead. Consider the following snippet:
var writeToDisk = false;
var outfile = new MemoryStream();
foreach(a line of strings)
{
// process the line
// BTW: the `String.Format` you have here is exceptionally confusing
// and may be attributing to why everything is \0
outfile.Write(...);
// set the flag to `true` on some condition to let yourself know
// you DO want to write
if (someCondition) { writeToDisk = true; }
}
if (writeToDisk)
{
var bytes = new byte[outfile.Length];
outfile.Read(bytes, 0, outfile.Length);
File.WriteAllBytes(MY_PATH, bytes);
}
I think what you want is the Any for checking if any is not "0", but also using using would be nice so that you can dispose properly.
if(someString.Any(a=> a != '0')) //if any elements in line are not '0'
{
using(StreamWriter outfile = new StreamWriter(MY_PATH, true))
{
foreach(char a in someString)
{
outfile.Write(WIDTH + " " + WIDTH, num1Str+"\t", num2Str+"\t");
}
}
}
if all elements in line are "0"
// do not write anything to the file, clear outfile buffer
Then why don't you check your line's content, before you write it ?
// process the line
string line = String.Format(WIDTH + " " + WIDTH, num1Str+"\t", num2Str+"\t");
if(!line.Trim().All(c => c == '0'))
outfile.Write(line);
I am making a project that uses streamreader and streamwriter, Is it possible that I only replace or save a text in an specific line only without affecting the other lines?
if I make like this
streamreader sr = new streamreader(#"txtfile");
list<string> lines = new list<string>();
while (!sr.EndOfStream)
sr.readline();
{
lines.Add(sr.ReadLine();
}
//put in textbox
sr.close();
{
streamwriter sw = new streamwriter(#"txtfile");
sw.WriteLine(textBox1.text);
sw.close();
}
this is just a sample, but Is it possible that I use list also un streamwriter?
If you want a one line solution (code golf :) ) you can use
string path = #"C:\Test.txt";
string lineToReplace = "Relpace This Line";
string newLineValue = "I Replaced This Line";
File.WriteAllLines(path, File.ReadAllLines(path).Select(line => line.Equals(lineToReplace) ? newLineValue : line));
You cannot just change a line as such but you can to ReadAllLines, find the line you want to change, change it and write all of it to the file again :
StringBuilder newFile = new StringBuilder();
string temp = "";
string[] file = File.ReadAllLines(#"txtfile");
foreach (string line in file)
{
if (line.Contains("string you want to replace"))
{
temp = line.Replace("string you want to replace", "New String");
newFile.Append(temp + "\r\n");
continue;
}
newFile.Append(line + "\r\n");
}
File.WriteAllText(#"txtfile", newFile.ToString());
Read the file into memory, changing the line(s) you want to change, close the reader, open the file for writing, write the new contents of the file out.
I have two text files, Source.txt and Target.txt. The source will never be modified and contain N lines of text. So, I want to delete a specific line of text in Target.txt, and replace by an specific line of text from Source.txt, I know what number of line I need, actually is the line number 2, both files.
I haven something like this:
string line = string.Empty;
int line_number = 1;
int line_to_edit = 2;
using StreamReader reader = new StreamReader(#"C:\target.xml");
using StreamWriter writer = new StreamWriter(#"C:\target.xml");
while ((line = reader.ReadLine()) != null)
{
if (line_number == line_to_edit)
writer.WriteLine(line);
line_number++;
}
But when I open the Writer, the target file get erased, it writes the lines, but, when opened, the target file only contains the copied lines, the rest get lost.
What can I do?
the easiest way is :
static void lineChanger(string newText, string fileName, int line_to_edit)
{
string[] arrLine = File.ReadAllLines(fileName);
arrLine[line_to_edit - 1] = newText;
File.WriteAllLines(fileName, arrLine);
}
usage :
lineChanger("new content for this line" , "sample.text" , 34);
You can't rewrite a line without rewriting the entire file (unless the lines happen to be the same length). If your files are small then reading the entire target file into memory and then writing it out again might make sense. You can do that like this:
using System;
using System.IO;
class Program
{
static void Main(string[] args)
{
int line_to_edit = 2; // Warning: 1-based indexing!
string sourceFile = "source.txt";
string destinationFile = "target.txt";
// Read the appropriate line from the file.
string lineToWrite = null;
using (StreamReader reader = new StreamReader(sourceFile))
{
for (int i = 1; i <= line_to_edit; ++i)
lineToWrite = reader.ReadLine();
}
if (lineToWrite == null)
throw new InvalidDataException("Line does not exist in " + sourceFile);
// Read the old file.
string[] lines = File.ReadAllLines(destinationFile);
// Write the new file over the old file.
using (StreamWriter writer = new StreamWriter(destinationFile))
{
for (int currentLine = 1; currentLine <= lines.Length; ++currentLine)
{
if (currentLine == line_to_edit)
{
writer.WriteLine(lineToWrite);
}
else
{
writer.WriteLine(lines[currentLine - 1]);
}
}
}
}
}
If your files are large it would be better to create a new file so that you can read streaming from one file while you write to the other. This means that you don't need to have the whole file in memory at once. You can do that like this:
using System;
using System.IO;
class Program
{
static void Main(string[] args)
{
int line_to_edit = 2;
string sourceFile = "source.txt";
string destinationFile = "target.txt";
string tempFile = "target2.txt";
// Read the appropriate line from the file.
string lineToWrite = null;
using (StreamReader reader = new StreamReader(sourceFile))
{
for (int i = 1; i <= line_to_edit; ++i)
lineToWrite = reader.ReadLine();
}
if (lineToWrite == null)
throw new InvalidDataException("Line does not exist in " + sourceFile);
// Read from the target file and write to a new file.
int line_number = 1;
string line = null;
using (StreamReader reader = new StreamReader(destinationFile))
using (StreamWriter writer = new StreamWriter(tempFile))
{
while ((line = reader.ReadLine()) != null)
{
if (line_number == line_to_edit)
{
writer.WriteLine(lineToWrite);
}
else
{
writer.WriteLine(line);
}
line_number++;
}
}
// TODO: Delete the old file and replace it with the new file here.
}
}
You can afterwards move the file once you are sure that the write operation has succeeded (no excecption was thrown and the writer is closed).
Note that in both cases it is a bit confusing that you are using 1-based indexing for your line numbers. It might make more sense in your code to use 0-based indexing. You can have 1-based index in your user interface to your program if you wish, but convert it to a 0-indexed before sending it further.
Also, a disadvantage of directly overwriting the old file with the new file is that if it fails halfway through then you might permanently lose whatever data wasn't written. By writing to a third file first you only delete the original data after you are sure that you have another (corrected) copy of it, so you can recover the data if the computer crashes halfway through.
A final remark: I noticed that your files had an xml extension. You might want to consider if it makes more sense for you to use an XML parser to modify the contents of the files instead of replacing specific lines.
When you create a StreamWriter it always create a file from scratch, you will have to create a third file and copy from target and replace what you need, and then replace the old one.
But as I can see what you need is XML manipulation, you might want to use XmlDocument and modify your file using Xpath.
You need to Open the output file for write access rather than using a new StreamReader, which always overwrites the output file.
StreamWriter stm = null;
fi = new FileInfo(#"C:\target.xml");
if (fi.Exists)
stm = fi.OpenWrite();
Of course, you will still have to seek to the correct line in the output file, which will be hard since you can't read from it, so unless you already KNOW the byte offset to seek to, you probably really want read/write access.
FileStream stm = fi.Open(FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None);
with this stream, you can read until you get to the point where you want to make changes, then write. Keep in mind that you are writing bytes, not lines, so to overwrite a line you will need to write the same number of characters as the line you want to change.
I guess the below should work (instead of the writer part from your example). I'm unfortunately with no build environment so It's from memory but I hope it helps
using (var fs = File.Open(filePath, FileMode.Open, FileAccess.ReadWrite)))
{
var destinationReader = StreamReader(fs);
var writer = StreamWriter(fs);
while ((line = reader.ReadLine()) != null)
{
if (line_number == line_to_edit)
{
writer.WriteLine(lineToWrite);
}
else
{
destinationReader .ReadLine();
}
line_number++;
}
}
The solution works fine. But I need to change single-line text when the same text is in multiple places. For this, need to define a trackText to start finding after that text and finally change oldText with newText.
private int FindLineNumber(string fileName, string trackText, string oldText, string newText)
{
int lineNumber = 0;
string[] textLine = System.IO.File.ReadAllLines(fileName);
for (int i = 0; i< textLine.Length;i++)
{
if (textLine[i].Contains(trackText)) //start finding matching text after.
traced = true;
if (traced)
if (textLine[i].Contains(oldText)) // Match text
{
textLine[i] = newText; // replace text with new one.
traced = false;
System.IO.File.WriteAllLines(fileName, textLine);
lineNumber = i;
break; //go out from loop
}
}
return lineNumber
}