I have a method that writing a csv from list content. If the csv file size reach to specific size like 5MB, I want to create a new csv file so it won't grow too big if the list content is huge. Here is my code below but it is not working. I am getting error "file is being used by another process". Looks like this is not the correct way to create the new csv file.
public static void WriteCSV<T>(IEnumerable<T> items, string path)
{
Type itemType = typeof(T);
var props = itemType.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.OrderBy(p => p.Name);
int filenumber = 0;
using (var writer = new StreamWriter(path))
{
writer.WriteLine(string.Join(", ", props.Select(p => p.Name)));
foreach (var item in items)
{
writer.WriteLine(string.Join(", ", props.Select(p => p.GetValue(item, null))));
if (writer.BaseStream.Length > 5120)
{
writer.Close();
writer.Dispose();
filenumber++;
var writer2 = new StreamWriter(string.Format(path, filenumber), false);
}
}
}
}
Your problem is two-fold.
The method string.Format(path, filenumber) does not do what you expect it to. It actually expects a format string as the first argument like "value goes here {0}" and a variable to put into the place of the {0}. Since you give it only a path it will return simply the path, which is the same name as your previous file. Hence your get the error, because you are trying to access the same file again.
Solution to 1):
1.1) you need to break the path into the path and the filename,
1.2) then take the filename without extension,
1.3) then concatenate the number to the filename
1.4) reconstruct the path.
Problem 2)
The iteration over the collection should not be inside the using block of the writer. It even does not depend on the writer. This makes it impossible for you to use another writer without specifying a new instance each time. This would make the code ugly and unnecessarily complicated.
Remove the using block and dispose the writer by hand as you already do:
Here is an examplary programm that does the trick.
void Main()
{
string path = #"C:\WorkSpace\temp\test\fileName.csv";
List<string> items = Enumerable.Range(1, 1200).Select(x => $"Something: {x} else {x+2} ").ToList();
int filenumber = 0;
var writer = new StreamWriter(path);
string pathOnly = Path.GetDirectoryName(path);
string fileName = Path.GetFileNameWithoutExtension(path);
string extension = Path.GetExtension(path);
foreach (var item in items)
{
writer.WriteLine(string.Join(", ", item.Split(' ')));
if (writer.BaseStream.Length > 120)
{
writer.Close();
writer.Dispose();
filenumber++;
writer = new StreamWriter(Path.Combine(pathOnly, $"{fileName}{filenumber}{extension}"), false);
}
}
// don't forget to close and dispose the last instance of the writer
writer.Close();
writer.Dispose();
}
Related
I am making an app in C# where I am searching if the file exists in the text file or not. If it does not exist then, it would add it in the text file and then append it in a List. But, for some reason the list only takes one file and ends at that point. So, can someone help me with what is the problem in this foreach loop?
static void CheckNewFile()
{
string path_f = #"File_Address_where_Text_file_exists";
var new_file = new List<string>();
if (!File.Exists(path_f)) # Checking if the text file exists or not and then creating it
{
var myFile = File.Create(path_f);
myFile.Close();
}
DirectoryInfo hdDirectoryInWhichToSearch = new DirectoryInfo(#"File_Address_in_which_Files_need_to_be_searched");
FileInfo[] filesInDir = hdDirectoryInWhichToSearch.GetFiles("AC" + "*" + "*.*" + "AC"); # Format of the file to be searched
foreach (FileInfo foundFile in filesInDir) # foreach for the files in the directory
{
string fullName = foundFile.FullName;
int flag = 0;
var lines = File.ReadLines(path_f);
foreach (var line in lines) # Reading line by line and checking if the file exists in the text file before
{
if (String.Equals(line, fullName))
{
flag += 1;
break;
}
}
if (flag < 1)
{
if (new FileInfo(path_f).Length == 0) # File Address is appended in the File
{
//TextWriter tw = new StreamWriter(path_f);
//tw.WriteLine(fullName);
//tw.Close();
}
else
{
//using (var tw = new StreamWriter(path_f, true))
//{
// tw.WriteLine(fullName);
//}
}
new_file.Add(fullName.ToString()); # Adding File Address to the list
flag = 0;
break;
}
}
}
Remove the last break. It is causing the program flow to leave the enclosing foreach loop with the file names.
As the other poster mentioned, you're breaking out of your loop early in the if block.
However, there isn't really a need for the flag (or loop or if block) at all. Your method could be simplified greatly by using a little System.Linq and just using Directory to find the new files by comparing their paths to the contents of the input file.
For example:
static List<string> CheckForNewFiles(string filePath, string searchDir,
string searchPattern)
{
// Create file if it doesn't exist
if (!File.Exists(filePath)) using (File.Create(filePath)) ;
// Get list of files that match search pattern which aren't contained in our file
var newFiles = Directory
.GetFiles(searchDir, searchPattern)
.Where(match => !File.ReadLines(filePath).Contains(match))
.ToList();
// Add the new file paths to our file
File.AppendAllLines(filePath, newFiles);
// Return the list of new files (?)
return newFiles;
}
In use it migth look something like:
public static void Main()
{
Console.WriteLine("Checking for new files...");
var newFiles = CheckForNewFiles(#"c:\temp\paths.txt", #"c:\temp\temp", "*.png");
Console.WriteLine($"{newFiles.Count} files found since last search.");
if (newFiles.Any())
{
Console.WriteLine(" -> " + string.Join(Environment.NewLine + " -> ", newFiles));
}
Console.ReadLine();
}
I am using the file helpers library to construct a csv but I need to create one csv file per record.
var engine = new FileHelperEngine(typeof(SalesOrders));
int count = 0;
foreach (SalesOrders item in purchaseOrder)
{
count++;
string fileName = "SalesOrder_" + item.OrderNumber.ToString() + ".csv";
var feng = new FileHelperEngine<SalesOrders>();
string str1 = feng.WriteString(purchaseOrder);
List<SalesOrdersItems> _orderItemsForExport = SopOrderItems.Where(a => a.ORDER_NUMBER.ToString() == item.OrderNumber).ToList();
}
However this line
string str1 = feng.WriteString(purchaseOrder);
Is printing out all the contents instead of the item per csv. I think its just my logic is not correct.
foreach (SalesOrders item in purchaseOrder)
{
using (StreamWriter sw = new StreamWriter($"SalesOrder_{item.OrderNumber.ToString()}.csv"))
{
var lines = TakeAllStuffFrom(item); // like a collection of $"{SalesOrder.FirstField.Trim()};{SalesOrder.SecondField.Trim()}";
foreach (var line in lines)
{
sw.WriteLine(line);
}
}
}
Write example of what you expect to appear in your csv file and we will figure something out :)
How can I read line by line for a list of files?
I have a directory with a number of files, I need to save all the files in a list and process them one by one, line by line.
So far I have done the following :
//fetching all files from directory
DirectoryInfo d = new DirectoryInfo("path");
Dictionary<int, FileInfo> DatFiles = new Dictionary<int, FileInfo>();
int filecounter = 1;
foreach (var dat in d.EnumerateFiles())
{
DatFiles.Add(filecounter, dat);
filecounter++;
}
Console.WriteLine(filecounter);
foreach (var fileName in DatFiles)
{
foreach (var line in File.ReadLines(fileName.Value.OpenText().ToString()))
{
//run some methods
}
}
When executing, I'm getting an exception, file not found. Even though the list is full of file names.
EnumerateFiles() returns a list of FileInfo objects, and File.ReadLines() takes a string path argument; you probably want to use File.ReadLines(fileName.Value.FullName) in your foreach as that gives the path to the actual file; OpenText() returns a StreamReader object.
This smaller code should do the task
var files = Directory.GetFiles("path");
foreach(var f in files)
{
using(StreamReader reader = new StreamReader(f))
{
var lines = reader.ReadToEnd().Split(new string[]{Environment.NewLine});
// Do here
}
}
When all else fails refer to the documentation
// Open the stream and read it back.
using (StreamReader sr = fi.OpenText())
{
string s = "";
while ((s = sr.ReadLine()) != null)
{
Console.WriteLine(s);
}
}
I want to count the number of some strings and store it into a csv file. I've tried it but I don't know if this is the correct way and in addition, there are two problems.
First of all, here is my method:
public void CountMacNames(String macName)
{
string path = #"D:\Counter\macNameCounter.csv";
if (!File.Exists(path))
{
File.Create(path).Close();
}
var lines = File.ReadLines(path);
foreach (var line in lines)
{
bool isExists = line.Split(',').Any(x => x == macName);
if (isExists)
{
// macName exists, increment it's value by 1
}
else
{
// macName does not exists, add macName to CSV file and start counter by 1
var csv = new StringBuilder();
var newLine = string.Format("{0},{1}", macName, 1);
csv.AppendLine(newLine);
File.WriteAllText(path, csv.ToString());
}
}
}
The first problem is this IOException:
The process cannot access the file 'D:\Counter\macNameCounter.csv'
because it is being used by another process.
The second problem is, that I don't know how to increment the value by one, if a macName exists in the csv file (see first comment)
EDIT: Example for method "CountMacNames" call:
CountMacNames("Cansas");
CountMacNames("Wellback");
CountMacNames("Newton");
CountMacNames("Cansas");
CountMacNames("Princet");
Then, the CSV file should contain:
Cansas, 2
Wellback, 1
Newton, 1
Princet, 1
OK, this is what I'd do:
public void CountMacNames(String macName)
{
string path = #"D:\Counter\macNameCounter.csv";
// Read all lines, but only if file exists
string[] lines = new string[0];
if (File.Exists(path))
lines = File.ReadAllLines(path);
// This is the new CSV file
StringBuilder newLines = new StringBuilder();
bool macAdded = false;
foreach (var line in lines)
{
string[] parts = line.Split(',');
if (parts.Length == 2 && parts[0].Equals(macName))
{
int newCounter = Convert.ToIn32(parts[1])++;
newLines.AppendLine(String.Format("{0},{1}", macName, newCounter));
macAdded = true;
}
else
{
newLines.AppendLine(line.Trim());
}
}
if (!macAdded)
{
newLines.AppendLine(String.Format("{0},{1}", macName, 1));
}
File.WriteAllText(path, newLines.ToString());
}
This code does this:
Read all the lines from file only if it exists - otherwise we start a new file
Iterate over all the lines
If the first part of a 2-part line equals the mac, add 1 to counter and add line to output
If the first part doesn't match or the line format is wrong, add the line to output as is
If we didn't find the mac in any line, add a new line for the mac with counter 1
Write the file back
You can't read and write to the same file at the same time (in a simple way).
For small files, there are already answers.
If your file is really large (too big to fit in memory) you need another approach:
Read input file line by line
optinally modify the current line
write line to a temporary file
If finished delete input file, rename temporary file
For the first problem you can either read all the lines into memory and work there then write it all out again, or use streams.
using (FileStream fs = File.Open(filePath, FileMode.Create, FileAccess.ReadWrite))
{
var sw = new StreamWriter(fs);
var sr = new StreamReader(fs);
while(!streamReader.EndOfStream)
{
var line = sr.ReadLine();
//Do stuff with line.
//...
if (macExists)
{
//Increment the number, Note that in here we can only replace characters,
//We can't insert extra characters unless we rewrite the rest of the file
//Probably more hassle than it's worth but
//You could have a fixed number of characters like 000001 or 1
//Read the number as a string,
//Int.Parse to get the number
//Increment it
//work out the number of bytes in the line.
//get the stream position
//seek back to the beginning of the line
//Overwrite the whole line with the same number of bytes.
}
else
{
//Append a line, also harder to do with streams like this.
//Store the current position,
//Seek to the end of the file,
//WriteLine
//Seek back again.
}
}
}
You need to read the file in and release it, like this, to avoid the IO exception:
string[] lines = null;
using (var sr = new System.IO.StreamReader(path))
lines = sr.ReadToEnd().Split(new string[] {"\r", "\n"}, StringSplitOptions.RemoveEmptyEntries);
As for the count, you can just add an int value, change the method return type as int, too.
public int CountMacNames(String macName, String path)
{
if (!File.Exists(path))
{
File.Create(path).Close();
}
string[] lines = null;
using (var sr = new System.IO.StreamReader(path))
lines = sr.ReadToEnd().Split(new string[] {"\r", "\n"}, StringSplitOptions.RemoveEmptyEntries);
return lines.Where(p => p.Split(',').Contains(macName)).Count();
}
and inside the method that calls it:
var path = #"<PATH TO FILE>";
var cnt = CountMacNames("Canvas", path);
if (cnt > 0)
{
using (var sw = new StreamWriter(path, true, Encoding.Unicode))
sw.WriteLine(string.Format("Canvas,{0}", cnt));
}
Now, var res = CountMacNames("Canvas","PATH"); will return 2, and the lines "Canvas,2" or "Newton,1" will be appended to the file, without overwriting it.
I have written this code but has some problems:
const int maxPeopleInFile = 2;
using (StreamReader reader = new StreamReader(#"c:\mytest\SortedTest.txt"))
{
string[] columnheaders = reader.ReadLine().Split(',');
List<string> listKeeper = new List<string>();
int fileNumber = 1;
while (reader.Peek() > 0)
{
string[] currentRowValues = reader.ReadLine().Split(',');
string id = currentRowValues[2];
if (listKeeper.Count < maxPeopleInFile || (listKeeper.Count() <= maxPeopleInFile && listKeeper.Contains(id)))
{
if (!listKeeper.Contains(id))
{
listKeeper.Add(id);
}
var writer = File.CreateText("file_" + fileNumber + ".txt");
writer.Write(currentRowValues);
writer.Close();
}
else // new file
{
fileNumber++;
listKeeper = new List<string>();
var writer = File.CreateText("file_" + fileNumber + ".txt");
writer.Write(currentRowValues);
}
}
}
Problems:
1: The files generated don't have the line I have read in string[] currentRowValues = reader.ReadLine().Split(',');
What is being written to the file is one line and it is the text System.String[]
Since currentRowValues is an array, this call
writer.Write(currentRowValues);
is equivalent to
writer.Write(currentRowValues.ToString());
which produces the output that you see (i.e. System.String[]) because ToString() does not iterate the individual string values.
You can write the whole array into a file in one shot using File.WriteAllLines, like this:
File.WriteAllLines("file_" + fileNumber + ".txt", currentRowValues);
(From a comment) [I] want to put [the exact row that I have read from original file] in a new file exactly like it used to be in the original file
Then you should use string.Join to undo the effects of the string.Split:
writer.Write(string.Join(",", currentRowValues));
It's because currentRowValues won't be serialized back to a comma-separated list of values by just giving it to FileStream.Write, because it writes the return value of ToString which is the object type name.
One possible approach might be using string.Join:
writer.Write(string.Join(",", currentRowValues));
your problem is in this snippet:
writer.Write(currentRowValues);
try:
writer.Write(currentRowValues[0]);
or
writer.Write(currentRowValues[1]);
depending on your needs
You need to provide a string value to the Write() method, not a string[].
I would suggest the following:
StringBuilder builder = new StringBuilder();
foreach (var currentValue in currentRowValues)
builder.Append(currentValue);
writer.Write(builder.ToString());
Note: Don't forget to add the namespace to be able to use StringBuilder.