Develop Reference

Get the values for rows for specific columns using TextFieldParser and c# from a csv file

I am trying to use TextFieldParser with c# to parse a csv file and get the values of rows for specific columns.
For example a csv with..
Name,Age,Color,Sport
Tom,24,Blue,Football
Dick,21,Red,Hockey
Jane,19,Green,Basketball
string fullpath = "C:\myfile.csv"
using (TextFieldParser parser = new TextFieldParser(fullPath))
{
parser.SetDelimiters(",");
while (!parser.EndOfData)
{
// Processing row
string[] fields = parser.ReadFields();
int sportcount = 0;
for (int i = 0; i < fields.Length; ++i)
{
string column = fields[i];
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
if (column.Contains("Sport"))
{
Console.WriteLine("Found Sport at {0}", i);
sportcount = i;
}
}
string sportCol = fields[sportCount];
// if that column contains a value
if (!(sportCol.Equals("")) {
// ignore the column header row
if (parser.LineNumber > 2) {
if (sportCol.Equals("Football")) {
Console.WriteLine("Found somebody playing football");
}
}
}
}
}
So idealy i'd like a list or array containing all the sports of those in the csv file. So it should contain (Football, Hockey, Basketball). Bearing in mind, the actual csv file has hundreds of columns and thousands of rows.
Update:
I've update the above to something that is working for me. By assigning i to a variable when it finds a 'Sport' column, then I can search for that columns value down through the csv file.

I guess by "trying" you mean it's something you've come across that can read csv and you're not wedded to it. As such I wanted to point out how it would look if you were to instead read this file with CsvHelper:
public class Person
{
public int Age { get; set; }
public string Name { get; set; }
public string Color { get; set; }
public string Sport { get; set; }
}
//in a method somewhere
using var reader = new StreamReader("path\\to\\file.csv"));
using var csv = new CsvReader(reader, CultureInfo.InvariantCulture));
foreach(var person in csv.GetRecords<Person>()){
//put code here
Console.WriteLine(person.Name);
}
the actual csv file has hundreds of columns
You'll need to make a property for each column
idealy i'd like a list or array containing all the sports of those in the csv file
A distinct list of every sport?
Instead do:
csv.GetRecords<Person>().Select(p => p.Sport).Distinct().ToArray()
Bear in mind that reading a CSV in this way is forward only, so you can't call getRecords a second time without resetting the reader to the start of the file. Perhaps then if you have more to do you should:
var people = csv.GetRecords<Person>().ToList();
to load it all into memory then you can repeatedly query:
people.Sum(p => p.Age); //sum of all ages
people.Avg(p => p.Age); //avg age
people.Count(p => p.Color == "Red"); //count of reds

C# Reading CSV to DataTable and Invoke Rows/Columns

i am currently working on a small Project and i got stuck with a Problem i currently can not manage to solve...
I have multiple ".CSV" Files i want to read, they all have the same Data just with different Values.
Header1;Value1;Info1
Header2;Value2;Info2
Header3;Value3;Info3
While reading the first File i Need to Create the Headers. The Problem is they are not splited in Columns but in rows (as you can see above Header1-Header3).
Then it Needs to read the Value 1 - Value 3 (they are listed in the 2nd Column) and on top of that i Need to create another Header -> Header4 with the data of "Info2" which is always placed in Column 3 and Row 2 (the other values of Column 3 i can ignore).
So the Outcome after the first File should look like this:
Header1;Header2;Header3;Header4;
Value1;Value2;Value3;Info2;
And after multiple files it sohuld be like this:
Header1;Header2;Header3;Header4;
Value1;Value2;Value3;Value4;
Value1b;Value2b;Value3b;Value4b;
Value1c;Value2c;Value3c;Value4c;
I tried it with OleDB but i get the Error "missing ISAM" which i cant mange to fix. The Code i Used is the following:
public DataTable ReadCsv(string fileName)
{
DataTable dt = new DataTable("Data");
/* using (OleDbConnection cn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\"" +
Path.GetDirectoryName(fileName) + "\";Extendet Properties ='text;HDR=yes;FMT=Delimited(,)';"))
*/
using (OleDbConnection cn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
Path.GetDirectoryName(fileName) + ";Extendet Properties ='text;HDR=yes;FMT=Delimited(,)';"))
{
using(OleDbCommand cmd = new OleDbCommand(string.Format("select *from [{0}]", new FileInfo(fileName).Name,cn)))
{
cn.Open();
using(OleDbDataAdapter adapter = new OleDbDataAdapter(cmd))
{
adapter.Fill(dt);
}
}
}
return dt;
}
Another attempt i did was using StreamReader. But the Headers are in the wrong place and i dont know how to Change this + do this for every file. the Code i tried is the following:
public static DataTable ReadCsvFilee(string path)
{
DataTable oDataTable = new DataTable();
var fileNames = Directory.GetFiles(path);
foreach (var fileName in fileNames)
{
//initialising a StreamReader type variable and will pass the file location
StreamReader oStreamReader = new StreamReader(fileName);
// CONTROLS WHETHER WE SKIP A ROW OR NOT
int RowCount = 0;
// CONTROLS WHETHER WE CREATE COLUMNS OR NOT
bool hasColumns = false;
string[] ColumnNames = null;
string[] oStreamDataValues = null;
//using while loop read the stream data till end
while (!oStreamReader.EndOfStream)
{
String oStreamRowData = oStreamReader.ReadLine().Trim();
if (oStreamRowData.Length > 0)
{
oStreamDataValues = oStreamRowData.Split(';');
//Bcoz the first row contains column names, we will poluate
//the column name by
//reading the first row and RowCount-0 will be true only once
// CHANGE TO CHECK FOR COLUMNS CREATED
if (!hasColumns)
{
ColumnNames = oStreamRowData.Split(';');
//using foreach looping through all the column names
foreach (string csvcolumn in ColumnNames)
{
DataColumn oDataColumn = new DataColumn(csvcolumn.ToUpper(), typeof(string));
//setting the default value of empty.string to newly created column
oDataColumn.DefaultValue = string.Empty;
//adding the newly created column to the table
oDataTable.Columns.Add(oDataColumn);
}
// SET COLUMNS CREATED
hasColumns = true;
// SET RowCount TO 0 SO WE KNOW TO SKIP COLUMNS LINE
RowCount = 0;
}
else
{
// IF RowCount IS 0 THEN SKIP COLUMN LINE
if (RowCount++ == 0) continue;
//creates a new DataRow with the same schema as of the oDataTable
DataRow oDataRow = oDataTable.NewRow();
//using foreach looping through all the column names
for (int i = 0; i < ColumnNames.Length; i++)
{
oDataRow[ColumnNames[i]] = oStreamDataValues[i] == null ? string.Empty : oStreamDataValues[i].ToString();
}
//adding the newly created row with data to the oDataTable
oDataTable.Rows.Add(oDataRow);
}
}
}
//close the oStreamReader object
oStreamReader.Close();
//release all the resources used by the oStreamReader object
oStreamReader.Dispose();
}
return oDataTable;
}
I am thankful for everyone who is willing to help. And Thanks for reading this far!
Sincerely yours

If I understood you right, there is a strict parsing there like this:
string OpenAndParse(string filename, bool firstFile=false)
{
var lines = File.ReadAllLines(filename);
var parsed = lines.Select(l => l.Split(';')).ToArray();
var header = $"{parsed[0][0]};{parsed[1][0]};{parsed[2][0]};{parsed[1][0]}\n";
var data = $"{parsed[0][1]};{parsed[1][1]};{parsed[2][1]};{parsed[1][2]}\n";
return firstFile
? $"{header}{data}"
: $"{data}";
}
Where it would return - if first file:
Header1;Header2;Header3;Header2
Value1;Value2;Value3;Value4
if not first file:
Value1;Value2;Value3;Value4
If I am correct, rest is about running this against a list file of files and joining the results in an output file.
EDIT: Against a directory:
void ProcessFiles(string folderName, string outputFileName)
{
bool firstFile = true;
foreach (var f in Directory.GetFiles(folderName))
{
File.AppendAllText(outputFileName, OpenAndParse(f, firstFile));
firstFile = false;
}
}
Note: I missed you want a DataTable and not an output file. Then you could simply create a list and put the results into that list making the list the datasource for your datatable (then why would you use semicolons in there? Probably all you need is to simply attach the array values to a list).

(Adding as another answer just to make it uncluttered)
void ProcessMyFiles(string folderName)
{
List<MyData> d = new List<MyData>();
var files = Directory.GetFiles(folderName);
foreach (var file in files)
{
OpenAndParse(file, d);
}
string[] headers = GetHeaders(files[0]);
DataGridView dgv = new DataGridView {Dock=DockStyle.Fill};
dgv.DataSource = d;
dgv.ColumnAdded += (sender, e) => {e.Column.HeaderText = headers[e.Column.Index];};
Form f = new Form();
f.Controls.Add(dgv);
f.Show();
}
string[] GetHeaders(string filename)
{
var lines = File.ReadAllLines(filename);
var parsed = lines.Select(l => l.Split(';')).ToArray();
return new string[] { parsed[0][0], parsed[1][0], parsed[2][0], parsed[1][0] };
}
void OpenAndParse(string filename, List<MyData> d)
{
var lines = File.ReadAllLines(filename);
var parsed = lines.Select(l => l.Split(';')).ToArray();
var data = new MyData
{
Col1 = parsed[0][1],
Col2 = parsed[1][1],
Col3 = parsed[2][1],
Col4 = parsed[1][2]
};
d.Add(data);
}
public class MyData
{
public string Col1 { get; set; }
public string Col2 { get; set; }
public string Col3 { get; set; }
public string Col4 { get; set; }
}

I don't know if this is the best way to do this. But what i would have done in your case, is to rewrite the CSV's the conventionnal way while reading all the files, then create a stream containing the new CSV created.
It would look like something like this :
var csv = new StringBuilder();
csv.AppendLine("Header1;Header2;Header3;Header4");
foreach (var item in file)
{
var newLine = string.Format("{0},{1},{2},{3}", item.value1, item.value2, item.value3, item.value4);
csv.AppendLine(newLine);
}
//Create Stream
MemoryStream stream = new MemoryStream();
StreamReader reader = new StreamReader(stream);
//Fill your data table here with your values
Hope this will help.

Converting Data Table values into array of string values

method1:
public void method1(DataTable ServerGroupIds)
{
obj.method2(ServerGroupIds);
}
method2 :
public static void method2(string[] servergroups)
{
obj.Message = userName + " has Un-Restricted the Project.";
}
Now I want to pass the DataTable values into the method2 String[] servergroups
How can I pass my DataTable values into an array of string values?
Sorry, Forget to mention I have 2 columns in my DataTable. !st column is ProjectId and Second Column is Server Group Id. Now I need only ServerGroup Id's in my array of string

Try this
public void method1(DataTable ServerGroupIds)
{
string [] serverGroups = ServerGroupIds.AsEnumerable().Select(t => t.Field<string>("ID")).ToArray<string>();
obj.method2(serverGroups );
}
Don't forget to include System.Linq
in t.Field<string>("ID"). Replace "ID" with the name of the column in the data table you want to put into the array

For a single row you can do this:
var rowAsString = string.Join(", ", ServerGroupIds.Rows[0].ItemArray);
Now add all the rows (by looping through your DataTable) rowAsString to a list:
List<string> list = new List<string>();
for (int i = 0; i < ServerGroupIds.Rows.Count; i++)
{
string rowAsString = string.Join(", ", ServerGroupIds.Rows[i].ItemArray);
list .Add(rowAsString );
}
string[] array = list.ToArray();
And pass to method2:
obj.method2(array);

Remove a certain value from string which keeps on changing

I'm trying to make a utility to generate an insert script of SQL tables along with relational table.
I got all the values in C#.
Now I want to remove the one column name and its value from the script.most probably the identity column.
For example: the string I have (which keeps on changing with table name and varies)
INSERT INTO Core.Customers ([customerId], [customername], [customeradress],[ordernumber])
VALUES (123, N'Rahul', N'244 LIZ MORN', 2334)
NOW I know I have to remove CustomerId (sometimes need to be replaces with #somevariable).
Please give me an efficient way how to retrieve customerId value and deleting column name and value.
I was looking for a method to find column value by column Name.
What I am doing is below - I know it's inefficient and can cause problem but for now it is working smoothly.
public string GetColumnValueToForDesiredColumnName(string row, TableInfo tableinfo, string NameofColumnTOfindvalueFor)
{
Dictionary<string, string> ValueTypedictionary = new Dictionary<string, string>();
string value = null;
// this code is quite messy - I need some suggestion on this one
string[] plusseperatedinsert = row.Replace("INSERT " + "[" + tableinfo.Schema + "].[" + tableinfo.TableName + "]", string.Empty).Trim().Replace("VALUES", "+").Split('+');
string[] columnvalues = plusseperatedinsert[0].Replace("(", string.Empty).Replace(")", string.Empty).Replace("(", string.Empty).Replace("[", string.Empty).Replace("]", string.Empty).Trim().Split(',');
string[] valuesfield = plusseperatedinsert[1].Replace("(", string.Empty).Replace(")", string.Empty).Replace("(", string.Empty).Replace("[", string.Empty).Replace("]", string.Empty).Trim().Split(',');
for (int index = 0; index < columnvalues.Length; index++)
{
ValueTypedictionary.Add(columnvalues[index], valuesfield[index]);
}
ValueTypedictionary.TryGetValue(NameofColumnTOfindvalueFor, out value);
return value;
}
This returns 123 as value.
And then I am using
string.Replace("[customerId],", string.empty).Replace(123, string.empty);

Create a special clas InsertQuery which stores pairs of column names with column values and then if you always need to remove or change an Id of a table you will know that it is on the first index of a list/array/whatever you use to store these pairs.
Define a method for removing this column and you are good to go.
So here is the code. You will probably change it somehow, it is just a proof of concept.
public class InsertQuery
{
private class Column
{
public string Name { get; set; }
public string Value { get; set; }
}
private readonly List<Column> columns = new List<Column>();
private readonly string tableName;
public InsertQuery(string tableName)
{
this.tableName = tableName;
}
public void AddColumn(string name, string value)
{
columns.Add(new Column { Name = name, Value = value });
}
public string RemoveColumnByName(string columnName)
{
var column = columns.First(c => c.Name == columnName);
var value = column.Value;
columns.Remove(column);
return value;
}
public string RemoveIdColumn()
{
var column = columns.First();
var value = column.Value;
columns.RemoveAt(0);
return value;
}
public override string ToString()
{
var sb = new StringBuilder();
sb.Append("INSERT INTO ");
sb.Append(tableName);
sb.Append(" (");
// append first all column names and then their values
return sb.ToString();
}
}
Trust me. Building such a tool using just string operations is not quite a good idea. In time you will want to add more functionality and you will be stuck with code that can be hardly extended. Encapsulation is a way to go.

Need to convert comma separated data from a file into a list of object

I need to read data from a .csv file and store the header and the content in my object in the following format. A list of the below mentioned class.
public class MappingData
{
private string ColumnName { get; set; }
private List<string> Data { get; set; }
}
So for example say I have a table like as shown below,
| Name | Phone | City |
|:-----------|------------:|:------------:|
| Yassser | 32342342234 | Mumbai
| Sachin | 32342342234 | Surat
| Will | 32342342234 | London
So for the above data my class should have 3 objects, first object will have the following details
ColumnName : 'Name'
Data: ['Yasser', 'Sachin', 'Will']
so, this is what I am trying to do, Below is what I have started with. I am reading the file using stream reader and spliting each line with comma seperator.
private List<MappingData> GetData(string filename)
{
var data = new List<MappingData>();
string fullPath = GetFilePath(filename);
StreamReader reader = new StreamReader(fullPath);
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
if (!String.IsNullOrWhiteSpace(line))
{
string[] values = line.Split(',');
}
}
return data;
}
can some one please help me mold this data into the required format. Thanks.

You should create a small int variable (or bool if you prefer) to determine whether the first row has been completed :
And create each of the list objects you will need (mpName, mpPhone, mpCity), on the first row you set the ColumnName property and on the subsequent rows you add to the MappingData's data list.
Then you add each of the lists (mpName, mpPhone, mpCity) to the data list for the method and return this.
private List<MappingData> GetData(string filename) {
List<MappingData> data = new List<MappingData>();
int NumRow = 0;
MappingData mpName = new MappingData();
MappingData mpPhone = new MappingData();
MappingData mpCity = new MappingData();
string fullPath = GetFilePath(filename);
StreamReader reader = new StreamReader(fullPath);
while (!reader.EndOfStream) {
string line = reader.ReadLine();
if (!String.IsNullOrWhiteSpace(line)) {
string[] values = line.Split(',');
if (NumRow == 0) {
mpName.ColumnName = values[0];
mpPhone.ColumnName = values[1];
mpCity.ColumnName = values[2];
NumRow = 1;
} else {
mpName.Data.Add(values[0]);
mpPhone.Data.Add(values[1]);
mpCity.Data.Add(values[2]);
}
}
}
data.Add(mpName);
data.Add(mpPhone);
data.Add(mpCity);
return data;
}
Hope this helps.

There's an excellent library for processing the CSV files.
KentBoogard
It is very easy with the above third party library to read content from the CSV files.
I would suggest this because you just seem to be starting and don't re invent the wheel.
Still, if you want to process the file in your own way, here's one working implementation. Enjoy
var csvData = File.ReadAllLines("d:\\test.csv");
var dataRows = csvData.Skip(1).ToList();
var csvHeaderColumns = csvData.First().Split(',').ToList();
var outputList = new List<MappingData>();
foreach (var columnName in csvHeaderColumns)
{
var obj = new MappingData { columnName = columnName, data = new List<string>() };
foreach (var rowStrings in dataRows.Select(dataRow => dataRow.Split(',').ToList()))
{
obj.data.Add(rowStrings[csvHeaderColumns.IndexOf(columnName)]);
}
outputList.Add(obj);
}
This will populate your MappingData class.

Assuming the rest of your method is correct then try this:
private List<MappingData> GetData(string filename)
{
var raw = new List<string[]>();
var data = new List<MappingData>();
string fullPath = GetFilePath(filename);
using(var reader = new StreamReader(fullPath))
{
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
if (!String.IsNullOrWhiteSpace(line))
{
raw.Add(line.Split(','));
}
}
}
Func<int, MappingData> extract =
n => new MappingData()
{
ColumnName = raw[0][n],
Data = raw.Skip(1).Select(x => x[n]).ToList(),
};
data.Add(extract(0));
data.Add(extract(1));
data.Add(extract(2));
return data;
}
You'd have to make you MappingData properties accessible though.