Develop Reference

c# load txt file and split it to X files based on number of lines

this is the code that i've written so far...
it doesnt do the job except re-write every line on the same file over and over again...
*RecordCntPerFile = 10K
*FileNumberName = 1 (file number one)
*Full File name should be something like this: 1_asci_split
string FileFullPath = DestinationFolder + "\\" + FileNumberName + FileNamePart + FileExtension;
using (System.IO.StreamReader sr = new System.IO.StreamReader(SourceFolder + "\\" + SourceFileName))
{
for (int i = 0; i <= (RecordCntPerFile - 1); i++)
{
using (StreamWriter sw = new StreamWriter(FileFullPath))
{
{ sw.Write(sr.Read() + "\n"); }
}
}
FileNumberName++;
}
Dts.TaskResult = (int)ScriptResults.Success;
}

If I understood correctly, you want to split a big file in smaller files with maximum of 10k lines. I see 2 problems on your code:
You never change the FullFilePath variable. So you will always rewrite on the same file
You always read and write the whole source file to the target file.
I rewrote your code to fit the behavior I said earlier. You just have to modify the strings.
int maxRecordsPerFile = 10000;
int currentFile = 1;
using (StreamReader sr = new StreamReader("source.txt"))
{
int currentLineCount = 0;
List<string> content = new List<string>();
while (!sr.EndOfStream)
{
content.Add(sr.ReadLine());
if (++currentLineCount == maxRecordsPerFile || sr.EndOfStream)
{
using (StreamWriter sw = new StreamWriter(string.Format("file{0}.txt", currentFile)))
{
foreach (var line in content)
sw.WriteLine(line);
}
content = new List<string>();
currentFile++;
currentLineCount = 0;
}
}
}
Of course you can do better than that, as you don't need to create that string and do that foreach loop. I just made this quick example to give you the idea. To improve the performance is up to you

How do I edit/replace a specific value in a .txt file using StreamWriter?

I have code in which I am reading in my .txt file. The text file contains values of first and last name, and a number.
"Daniel"
"Murrow"
"1234"
I am trying to use StreamWriter to edit a specific value. However, my code as of now edits the whole .txt file and replaces everything. I would only like to replace the number value in my case.
static classMates[] readClassMates(classMates[] classMateInfo)
{
StreamReader sr = new StreamReader(#"C:\Users\Callum\Documents\class.txt");
int count = 0;
while (!sr.EndOfStream)
{
classMateInfo[count].first = sr.ReadLine();
classMateInfo[count].last = sr.ReadLine();
string idTemp = sr.ReadLine();
classMateInfo[count].ID = Convert.ToInt32(idTemp);
count++;
}
sr.Close();
return classMateInfo;
static void editClassMates(classMates[] classMateInfo)
{
Console.Write("Who's number would you like to change? ");
string classMateInput = Console.ReadLine();
for (int i = 0; i < classMateInfo.Length; i++)
{
if (classMateInfo[i].last.Equals(classMateInput))
{
Console.Write("Enter new number: ");
string temp = Console.ReadLine();
int classMateNumber = Convert.ToInt32(temp);
StreamWriter sw = new StreamWriter(#"C:\Users\Callum\Documents\class.txt");
for (int j = 0; j < classMateInfo.Length; j++)
{
sw.WriteLine(classMateNumber);
}
sw.Close();
}
}
}
So my question is, how do I edit a specific value in my .txt file? I've tried my best, but I've been stuck on this for ages!

If the file is not very big (and therefore memory usage is not an issue) how about this:
var text = File.ReadAllText(#"C:\Users\Callum\Documents\class.txt");
var updatedText = text.Replace(...); // Do your string replacement here
File.WriteAllText(#"C:\Users\Callum\Documents\class.txt");

How to incorporate a text qualifier in a file name used as a variable?

I have a C# script which takes in two CSV files as input, combines the two files, performs numerous calculations on them, and writes the result in a new CSV file.
These two input CSV file names are declared as variables and are used in the C# script by accessing those variable names.
The data in the input CSV files looks like this:
Since the data has values in thousands and millions, line splits in the C# code are truncating the data incorrectly. For instance a value of 11,861 appears only as 11 and 681 goes in the next columns.
Is there any way in C#, by which I can specify a text qualifier (" in this case) for the two files ?
Here is the C# code snippet:
string[,] filesToProcess = new string[2, 2] { {(String)Dts.Variables["csvFileNameUSD"].Value,"USD" }, {(String)Dts.Variables["csvFileNameCAD"].Value,"CAD" } };
string headline = "CustType,CategoryType,CategoryValue,DataType,Stock QTY,Stock Value,Floor QTY,Floor Value,Order Count,Currency";
string outPutFile = Dts.Variables["outputFile"].Value.ToString();
//Declare Output files to write to
FileStream sw = new System.IO.FileStream(outPutFile, System.IO.FileMode.Create);
StreamWriter w = new StreamWriter(sw);
w.WriteLine(headline);
//Loop Through the files one by one and write to output Files
for (int x = 0; x < filesToProcess.GetLength(1); x++)
{
if (System.IO.File.Exists(filesToProcess[x, 0]))
{
string categoryType = "";
string custType = "";
string dataType = "";
string categoryValue = "";
//Read the input file in memory and close after done
StreamReader sr = new StreamReader(filesToProcess[x, 0]);
string fileText = sr.ReadToEnd();
string[] lines = fileText.Split(Convert.ToString(System.Environment.NewLine).ToCharArray());
sr.Close();
where csvFileNameUSD and csvFileNameCAD are variables with values pointing to their locations.

Well, based on the questions you have answered, this ought to do what you want to do:
public void SomeMethodInYourCodeSnippet()
{
string[] lines;
using (StreamReader sr = new StreamReader(filesToProcess[x, 0]))
{
//Read the input file in memory and close after done
string fileText = sr.ReadToEnd();
lines = fileText.Split(Convert.ToString(System.Environment.NewLine).ToCharArray());
sr.Close(); // redundant due to using, but just to be safe...
}
foreach (var line in lines)
{
string[] columnValues = GetColumnValuesFromLine(line);
// Do whatever with your column values here...
}
}
private string[] GetColumnValuesFromLine(string line)
{
// Split on ","
var values = line.Split(new string [] {"\",\""}, StringSplitOptions.None);
if (values.Count() > 0)
{
// Trim leading double quote from first value
var firstValue = values[0];
if (firstValue.Length > 0)
values[0] = firstValue.Substring(1);
// Trim the trailing double quote from the last value
var lastValue = values[values.Length - 1];
if (lastValue.Length > 0)
values[values.Length - 1] = lastValue.Substring(0, lastValue.Length - 1);
}
return values;
}
Give that a try and let me know how it works!

You posted a very similar looking question few days ago. Did that solution not help you?
If so, what issues are you facing on that. We can probably help you troubleshoot that as well.

Replace data/value of csv file

I want to count the number of some strings and store it into a csv file. I've tried it but I don't know if this is the correct way and in addition, there are two problems.
First of all, here is my method:
public void CountMacNames(String macName)
{
string path = #"D:\Counter\macNameCounter.csv";
if (!File.Exists(path))
{
File.Create(path).Close();
}
var lines = File.ReadLines(path);
foreach (var line in lines)
{
bool isExists = line.Split(',').Any(x => x == macName);
if (isExists)
{
// macName exists, increment it's value by 1
}
else
{
// macName does not exists, add macName to CSV file and start counter by 1
var csv = new StringBuilder();
var newLine = string.Format("{0},{1}", macName, 1);
csv.AppendLine(newLine);
File.WriteAllText(path, csv.ToString());
}
}
}
The first problem is this IOException:
The process cannot access the file 'D:\Counter\macNameCounter.csv'
because it is being used by another process.
The second problem is, that I don't know how to increment the value by one, if a macName exists in the csv file (see first comment)
EDIT: Example for method "CountMacNames" call:
CountMacNames("Cansas");
CountMacNames("Wellback");
CountMacNames("Newton");
CountMacNames("Cansas");
CountMacNames("Princet");
Then, the CSV file should contain:
Cansas, 2
Wellback, 1
Newton, 1
Princet, 1

OK, this is what I'd do:
public void CountMacNames(String macName)
{
string path = #"D:\Counter\macNameCounter.csv";
// Read all lines, but only if file exists
string[] lines = new string[0];
if (File.Exists(path))
lines = File.ReadAllLines(path);
// This is the new CSV file
StringBuilder newLines = new StringBuilder();
bool macAdded = false;
foreach (var line in lines)
{
string[] parts = line.Split(',');
if (parts.Length == 2 && parts[0].Equals(macName))
{
int newCounter = Convert.ToIn32(parts[1])++;
newLines.AppendLine(String.Format("{0},{1}", macName, newCounter));
macAdded = true;
}
else
{
newLines.AppendLine(line.Trim());
}
}
if (!macAdded)
{
newLines.AppendLine(String.Format("{0},{1}", macName, 1));
}
File.WriteAllText(path, newLines.ToString());
}
This code does this:
Read all the lines from file only if it exists - otherwise we start a new file
Iterate over all the lines
If the first part of a 2-part line equals the mac, add 1 to counter and add line to output
If the first part doesn't match or the line format is wrong, add the line to output as is
If we didn't find the mac in any line, add a new line for the mac with counter 1
Write the file back

You can't read and write to the same file at the same time (in a simple way).
For small files, there are already answers.
If your file is really large (too big to fit in memory) you need another approach:
Read input file line by line
optinally modify the current line
write line to a temporary file
If finished delete input file, rename temporary file

For the first problem you can either read all the lines into memory and work there then write it all out again, or use streams.
using (FileStream fs = File.Open(filePath, FileMode.Create, FileAccess.ReadWrite))
{
var sw = new StreamWriter(fs);
var sr = new StreamReader(fs);
while(!streamReader.EndOfStream)
{
var line = sr.ReadLine();
//Do stuff with line.
//...
if (macExists)
{
//Increment the number, Note that in here we can only replace characters,
//We can't insert extra characters unless we rewrite the rest of the file
//Probably more hassle than it's worth but
//You could have a fixed number of characters like 000001 or 1
//Read the number as a string,
//Int.Parse to get the number
//Increment it
//work out the number of bytes in the line.
//get the stream position
//seek back to the beginning of the line
//Overwrite the whole line with the same number of bytes.
}
else
{
//Append a line, also harder to do with streams like this.
//Store the current position,
//Seek to the end of the file,
//WriteLine
//Seek back again.
}
}
}

You need to read the file in and release it, like this, to avoid the IO exception:
string[] lines = null;
using (var sr = new System.IO.StreamReader(path))
lines = sr.ReadToEnd().Split(new string[] {"\r", "\n"}, StringSplitOptions.RemoveEmptyEntries);
As for the count, you can just add an int value, change the method return type as int, too.
public int CountMacNames(String macName, String path)
{
if (!File.Exists(path))
{
File.Create(path).Close();
}
string[] lines = null;
using (var sr = new System.IO.StreamReader(path))
lines = sr.ReadToEnd().Split(new string[] {"\r", "\n"}, StringSplitOptions.RemoveEmptyEntries);
return lines.Where(p => p.Split(',').Contains(macName)).Count();
}
and inside the method that calls it:
var path = #"<PATH TO FILE>";
var cnt = CountMacNames("Canvas", path);
if (cnt > 0)
{
using (var sw = new StreamWriter(path, true, Encoding.Unicode))
sw.WriteLine(string.Format("Canvas,{0}", cnt));
}
Now, var res = CountMacNames("Canvas","PATH"); will return 2, and the lines "Canvas,2" or "Newton,1" will be appended to the file, without overwriting it.

Writing a string array to a File but it just prints System.String[]

I have written this code but has some problems:
const int maxPeopleInFile = 2;
using (StreamReader reader = new StreamReader(#"c:\mytest\SortedTest.txt"))
{
string[] columnheaders = reader.ReadLine().Split(',');
List<string> listKeeper = new List<string>();
int fileNumber = 1;
while (reader.Peek() > 0)
{
string[] currentRowValues = reader.ReadLine().Split(',');
string id = currentRowValues[2];
if (listKeeper.Count < maxPeopleInFile || (listKeeper.Count() <= maxPeopleInFile && listKeeper.Contains(id)))
{
if (!listKeeper.Contains(id))
{
listKeeper.Add(id);
}
var writer = File.CreateText("file_" + fileNumber + ".txt");
writer.Write(currentRowValues);
writer.Close();
}
else // new file
{
fileNumber++;
listKeeper = new List<string>();
var writer = File.CreateText("file_" + fileNumber + ".txt");
writer.Write(currentRowValues);
}
}
}
Problems:
1: The files generated don't have the line I have read in string[] currentRowValues = reader.ReadLine().Split(',');
What is being written to the file is one line and it is the text System.String[]

Since currentRowValues is an array, this call
writer.Write(currentRowValues);
is equivalent to
writer.Write(currentRowValues.ToString());
which produces the output that you see (i.e. System.String[]) because ToString() does not iterate the individual string values.
You can write the whole array into a file in one shot using File.WriteAllLines, like this:
File.WriteAllLines("file_" + fileNumber + ".txt", currentRowValues);
(From a comment) [I] want to put [the exact row that I have read from original file] in a new file exactly like it used to be in the original file
Then you should use string.Join to undo the effects of the string.Split:
writer.Write(string.Join(",", currentRowValues));

It's because currentRowValues won't be serialized back to a comma-separated list of values by just giving it to FileStream.Write, because it writes the return value of ToString which is the object type name.
One possible approach might be using string.Join:
writer.Write(string.Join(",", currentRowValues));

your problem is in this snippet:
writer.Write(currentRowValues);
try:
writer.Write(currentRowValues[0]);
or
writer.Write(currentRowValues[1]);
depending on your needs

You need to provide a string value to the Write() method, not a string[].
I would suggest the following:
StringBuilder builder = new StringBuilder();
foreach (var currentValue in currentRowValues)
builder.Append(currentValue);
writer.Write(builder.ToString());
Note: Don't forget to add the namespace to be able to use StringBuilder.