Populating a dataset from a CSV file - c#

I would like to read the contents of a CSV file and create a dataset.
I am trying like this:
var lines = File.ReadAllLines("test.csv").Select(a => a.Split(';'));
DataSet ds = new DataSet();
ds.load(lines);
but apparently this is not correct.

You need to add the reference Microsoft.VisualBasic.dll to use TextFieldParser Class.
private static DataTable GetDataTabletFromCSVFile(string csv_file_path)
{
DataTable csvData = new DataTable();
try
{
using(TextFieldParser csvReader = new TextFieldParser(csv_file_path))
{
csvReader.SetDelimiters(new string[] { "," });
csvReader.HasFieldsEnclosedInQuotes = true;
string[] colFields = csvReader.ReadFields();
foreach (string column in colFields)
{
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
while (!csvReader.EndOfData)
{
string[] fieldData = csvReader.ReadFields();
//Making empty value as null
for (int i = 0; i < fieldData.Length; i++)
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
}
csvData.Rows.Add(fieldData);
}
}
}
catch (Exception ex)
{
}
return csvData;
}
}
See this article for more info : http://www.morgantechspace.com/2013/08/how-to-read-data-from-csv-file-in-c.html

You need to run a SELECT statement against the CSV file to fill the dataset:
Edit: here's some sample code from http://carllbrown.blogspot.co.uk/2007/09/populate-dataset-from-csv-delimited_18.html
string FileName = ...
OleDbConnection conn = new OleDbConnection
("Provider=Microsoft.Jet.OleDb.4.0; Data Source = " +
Path.GetDirectoryName(FileName) +
"; Extended Properties = \"Text;HDR=YES;FMT=Delimited\"");
conn.Open();
OleDbDataAdapter adapter = new OleDbDataAdapter
("SELECT * FROM " + Path.GetFileName(FileName), conn);
DataSet ds = new DataSet("Temp");
adapter.Fill(ds);
conn.Close();

You can use Library like Fast CSV Reader then
using System.IO;
using LumenWorks.Framework.IO.Csv;
void ReadCsv()
{
// open the file "data.csv" which is a CSV file with headers
using (CsvReader csv = new CsvReader(
new StreamReader("data.csv"), true))
{
myDataRepeater.DataSource = csv;
myDataRepeater.DataBind();
}
}

Comma (,) Problem Solved in This Code
Works Even If you add Commas(,) in between a cell
Reading CSV file CODE:
public MainWindow()
{
InitializeComponent();
DataTable dtDataSource = new DataTable();
string[] fileContent = File.ReadAllLines(#"..\\Book1.csv");
if (fileContent.Count() > 0)
{
//Create data table columns dynamically
string[] columns = fileContent[0].Split(',');
for (int i = 0; i < columns.Count(); i++)
{
dtDataSource.Columns.Add(columns[i]);
}
//Add row data dynamically
for (int i = 1; i < fileContent.Count(); i++)
{
string[] rowData = fileContent[i].Split(',');
string[] realRowData = new string[columns.Count()];
StringBuilder collaboration = new StringBuilder();
int v = 0;
//this region solves the problem of a cell containing ",".
#region CommaSepProblem
for (int j = 0, K = 0; j < rowData.Count(); j++, K++)
{
if ((rowData[j].Count(x => x == '"') % 2 == 0))//checks if the string contains even number of DoubleQuotes
{
realRowData[K] = quotesLogic((rowData[j]));
}
else if ((rowData[j].Count(x => x == '"') % 2 != 0))//If Number of DoubleQuotes are ODD
{
int c = rowData[j].Count(x => x == '"');
v = j;
while (c % 2 != 0)//Go through all the next array cell till it makes EVEN Number of DoubleQuotes.
{
collaboration.Append(rowData[j] + ",");
j++;
c += rowData[j].Count(x => x == '"');
}
collaboration.Append(rowData[j]);
realRowData[K] = quotesLogic(collaboration.ToString());
}
else { continue; }
}
#endregion
dtDataSource.Rows.Add(realRowData);
}
if (dtDataSource != null)
{
//dataGridView1 = new DataGridView();
dataGrid1.ItemsSource = dtDataSource.DefaultView;
}
}
}
Method Need to be added:
string quotesLogic(string collaboration)
{
StringBuilder after = new StringBuilder(collaboration);
if (after.ToString().StartsWith("\"") && after.ToString().EndsWith("\""))//removes 1st and last quotes as those are system generated
{
after.Remove(0, 1);
after.Remove(after.Length - 1, 1);
int count = after.Length - 1;
//FACT: if you try to add DoubleQuote in a cell in excel. It'll save that quote as 2 times DoubleQuote(Like "") which means first DoubleQuote is to give instruction to CPU that the next DoubleQuote is not system generated.
while (count > 0)//This loop find twice insertion of 2 DoubleQuotes and neutralise them to One DoubleQuote.
{
if (after[count] == '"' && after[count - 1] == '"')
{
after.Remove(count, 1);
}
count--;
}
}
return after.ToString();
}

If you just want to quickly create a DataTable filled with sample data from a CSV file (or pasted directly from Excel) to play around or prototype, then you can use my fork of Shan Carter's Mr. Data Converter -- I recently added the ability to output comma- and tab-delimited data to a C# DataTable.
http://thdoan.github.io/mr-data-converter/

I have written five methods below that will turn a Csv file into a DataTable.
They have been designed to take into account optional quote marks (e.g. " symbols) and to be as versatile as possible without using other libraries:
public static DataTable GetDataTabletFromCSVFile(string filePath, bool isHeadings)
{
DataTable MethodResult = null;
try
{
using (TextFieldParser TextFieldParser = new TextFieldParser(filePath))
{
if (isHeadings)
{
MethodResult = GetDataTableFromTextFieldParser(TextFieldParser);
}
else
{
MethodResult = GetDataTableFromTextFieldParserNoHeadings(TextFieldParser);
}
}
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
public static DataTable GetDataTableFromCsvString(string csvBody, bool isHeadings)
{
DataTable MethodResult = null;
try
{
MemoryStream MemoryStream = new MemoryStream();
StreamWriter StreamWriter = new StreamWriter(MemoryStream);
StreamWriter.Write(csvBody);
StreamWriter.Flush();
MemoryStream.Position = 0;
using (TextFieldParser TextFieldParser = new TextFieldParser(MemoryStream))
{
if (isHeadings)
{
MethodResult = GetDataTableFromTextFieldParser(TextFieldParser);
}
else
{
MethodResult = GetDataTableFromTextFieldParserNoHeadings(TextFieldParser);
}
}
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
public static DataTable GetDataTableFromRemoteCsv(string url, bool isHeadings)
{
DataTable MethodResult = null;
try
{
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
StreamReader StreamReader = new StreamReader(httpWebResponse.GetResponseStream());
using (TextFieldParser TextFieldParser = new TextFieldParser(StreamReader))
{
if (isHeadings)
{
MethodResult = GetDataTableFromTextFieldParser(TextFieldParser);
}
else
{
MethodResult = GetDataTableFromTextFieldParserNoHeadings(TextFieldParser);
}
}
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
private static DataTable GetDataTableFromTextFieldParser(TextFieldParser textFieldParser)
{
DataTable MethodResult = null;
try
{
textFieldParser.SetDelimiters(new string[] { "," });
textFieldParser.HasFieldsEnclosedInQuotes = true;
string[] ColumnFields = textFieldParser.ReadFields();
DataTable dt = new DataTable();
foreach (string ColumnField in ColumnFields)
{
DataColumn DataColumn = new DataColumn(ColumnField);
DataColumn.AllowDBNull = true;
dt.Columns.Add(DataColumn);
}
while (!textFieldParser.EndOfData)
{
string[] Fields = textFieldParser.ReadFields();
for (int i = 0; i < Fields.Length; i++)
{
if (Fields[i] == "")
{
Fields[i] = null;
}
}
dt.Rows.Add(Fields);
}
MethodResult = dt;
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
private static DataTable GetDataTableFromTextFieldParserNoHeadings(TextFieldParser textFieldParser)
{
DataTable MethodResult = null;
try
{
textFieldParser.SetDelimiters(new string[] { "," });
textFieldParser.HasFieldsEnclosedInQuotes = true;
bool FirstPass = true;
DataTable dt = new DataTable();
while (!textFieldParser.EndOfData)
{
string[] Fields = textFieldParser.ReadFields();
if(FirstPass)
{
for (int i = 0; i < Fields.Length; i++)
{
DataColumn DataColumn = new DataColumn("Column " + i);
DataColumn.AllowDBNull = true;
dt.Columns.Add(DataColumn);
}
FirstPass = false;
}
for (int i = 0; i < Fields.Length; i++)
{
if (Fields[i] == "")
{
Fields[i] = null;
}
}
dt.Rows.Add(Fields);
}
MethodResult = dt;
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
If, like me, you're saving from reporting services then you should use it like this:
Warning[] warnings;
string[] streamids;
string mimeType;
string encoding;
string filenameExtension;
byte[] bytes = rvMain.ServerReport.Render("csv", null, out mimeType, out encoding, out filenameExtension, out streamids, out warnings);
string CsvBody = System.Text.Encoding.UTF8.GetString(bytes);
DataTable dt = GetDataTableFromCsvString(CsvBody,true);
Otherwise, all you need do is:
bool IsHeadings = true; //Does the data include a heading row?
DataTable dt = GetDataTableFromCsvString(CsvBody, IsHeadings);
Or to use directly from a csv file
bool IsHeadings = true; //Does the data include a heading row?
DataTable dt = GetDataTabletFromCsvFile(FilePath, IsHeadings)
Or to use a csv file that is stored remotely
bool IsHeadings = true; //Does the data include a heading row?
DataTable dt = GetDataTabletFromRemoteCsv(Url, IsHeadings)
A Dataset is a collection of DataTables, so create one like so:
DataSet ds = new DataSet();
ds.Tables.Add(dt);

Related

export to excel in c# where first row and 4 columns are in string & rest is in integer

protected void ibtnExport_Click(object sender, ImageClickEventArgs e)
{
try
{
DataTable dt = new DataTable();
dt = createDataTable();
dt.Columns.RemoveAt(0);
ClosedXML.Excel.XLWorkbook wbook = new ClosedXML.Excel.XLWorkbook();
wbook.Worksheets.Add(dt,lblStartDate.Text+"to"+lblEndDate.Text);
// Prepare the response
HttpResponse httpResponse = Response;
httpResponse.Clear();
httpResponse.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
//Provide you file name here
httpResponse.AddHeader("content-disposition", "attachment;filename=\""+lblStartDate.Text+"to"+lblEndDate.Text+".xlsx\"");
string style = #"<style> .integermode {mso-number-format:General} </style>";
Response.Write(style);
// Flush the workbook to the Response.OutputStream
using (MemoryStream memoryStream = new MemoryStream())
{
wbook.SaveAs(memoryStream);
memoryStream.WriteTo(httpResponse.OutputStream);
memoryStream.Close();
}
httpResponse.End();
}
catch
{
Response.Write("<script>alert('Something went wrong..')</script>");
}
}
This is the code which i am using to export excel. Here the header is in string but the rest of the column is also in string, which I will need to change into a number or integer for doing addition which is ha tic for the user to perform any action on it.
Please do help me to resolve this. If you need any other part of the code please ask.
Edit:
This is the createDataTable code:
private DataTable createDataTable()
{
DataSet _ds = new DataSet();
_ds = (DataSet)ViewState["DataSet"];
DataTable dt = new DataTable();
dt.Columns.Add("ProjectID");
dt.Columns.Add("Projects");
dt.Columns.Add("Network");
dt.Columns.Add("Activity");
dt.Columns.Add("SOP");
for (int i = 0; i < _ds.Tables[1].Rows.Count; i++)
{
dt.Columns.Add(_ds.Tables[1].Rows[i]["Name"].ToString() + "(" + _ds.Tables[1].Rows[i]["SapCode"].ToString() + ")");
}
for (int i = 0; i < _ds.Tables[0].Rows.Count; i++)
{
dt.Rows.Add(_ds.Tables[0].Rows[i]["ProjectID"].ToString(), _ds.Tables[0].Rows[i]["BaseProject"].ToString(), _ds.Tables[0].Rows[i]["ProjectName"].ToString(), _ds.Tables[0].Rows[i]["Network"].ToString(), _ds.Tables[0].Rows[i]["Activity"].ToString(), _ds.Tables[0].Rows[i]["SOP"].ToString());
}
for (int j = 0; j < _ds.Tables[2].Rows.Count; j++)
{
for (int i = 0; i < _ds.Tables[0].Rows.Count; i++)
{
if(dt.Columns.Contains(_ds.Tables[2].Rows[j]["EmployeeData"].ToString()))
//if (dt.Columns[_ds.Tables[2].Rows[j]["EmployeeData"].ToString()].ColumnName!= null && dt.Columns[_ds.Tables[2].Rows[j]["EmployeeData"].ToString()].ColumnName.ToString() != "")
{
if (dt.Rows[i]["ProjectID"].ToString() == _ds.Tables[2].Rows[j]["ProjectID"].ToString() && dt.Columns[_ds.Tables[2].Rows[j]["EmployeeData"].ToString()].ColumnName.ToString() == _ds.Tables[2].Rows[j]["EmployeeData"].ToString())
{
dt.Rows[i][_ds.Tables[2].Rows[j]["EmployeeData"].ToString()] = Convert.ToDouble(_ds.Tables[2].Rows[j]["Hourspent"].ToString());
}
}
}
}
return dt;
}

I am trying to export an excel and make it password protected. My code is given below.But i am getting error

I am trying to export an excel and make it password protected.
My code is given below.
But i am getting error:
Excel completed file level validation and repair.
Some parts of this workbook may have been repaired or discarded.
I DON'T KNOW WHAT I AM DOING WRONG .
In-case i do it without the save As line for package then this error doesn't appear.
In my controller:
[HttpGet]
public FileStreamResult ExportToExcel()
{
_objService = new ServiceBAL();
List<ReconcilationEntity> Objmodel = new List<ReconcilationEntity>();
Objmodel = _objService.GetCreditsudharLeads();
String URL = string.Empty;
if (!Directory.Exists(Server.MapPath("~/TempExcel")))
{
System.IO.Directory.CreateDirectory(Server.MapPath("~/TempExcel"));
}
String Filepath = Server.MapPath("~/TempExcel");
string date = DateTime.Now.ToShortDateString().Replace("/", "_") + "_" + DateTime.Now.ToShortTimeString().Replace(" ", "_").Replace(":", "_").Trim();
String FileName = "Creditsudhar_" + date + ".xlsx";
Filepath = Filepath + "\\" + FileName;
string[] columns = { "AffName", "AffPhone", "AffEmail", "ProductName", "ContactName", "Status", "CreatedOn", "Commission", "IsCommissionPaid", "Accountname", "AccountNumber", "BankName", "BankBranch", "IFSCCode", "PanNumber" };
var file = ExcelExportHelper.ExportExcel(ExcelExportHelper.ListToDataTable(Objmodel), Filepath, "Creditsudhar Reconcillation Sheet " + DateTime.Now.ToShortDateString(), true, columns);
var memStream = new MemoryStream(file);
return this.File(memStream, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", FileName);
}
public static string ExcelContentType
{
get
{ return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"; }
}
public static DataTable ListToDataTable<T>(List<T> data)
{
PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(typeof(T));
DataTable dataTable = new DataTable();
for (int i = 0; i < properties.Count; i++)
{
PropertyDescriptor property = properties[i];
dataTable.Columns.Add(property.Name, Nullable.GetUnderlyingType(property.PropertyType) ?? property.PropertyType);
}
object[] values = new object[properties.Count];
foreach (T item in data)
{
for (int i = 0; i < values.Length; i++)
{
values[i] = properties[i].GetValue(item);
}
dataTable.Rows.Add(values);
}
return dataTable;
}
public static byte[] ExportExcel(DataTable dataTable, String Filepath, string heading = "", bool showSrNo = false, params string[] columnsToTake)
{
string fullPath = string.Empty;
byte[] ret;
DeleteUploadedFile(Filepath);
String result = String.Empty;
using (ExcelPackage package = new ExcelPackage())
{
ExcelWorksheet workSheet = package.Workbook.Worksheets.Add(String.Format("{0} Data", heading));
int startRowFrom = String.IsNullOrEmpty(heading) ? 1 : 3;
if (showSrNo)
{
DataColumn dataColumn = dataTable.Columns.Add("#", typeof(int));
dataColumn.SetOrdinal(0);
int index = 1;
foreach (DataRow item in dataTable.Rows)
{
item[0] = index;
index++;
}
}
// add the content into the Excel file
workSheet.Cells["A" + startRowFrom].LoadFromDataTable(dataTable, true);
// autofit width of cells with small content
int columnIndex = 1;
foreach (DataColumn column in dataTable.Columns)
{
try
{
ExcelRange columnCells = workSheet.Cells[workSheet.Dimension.Start.Row, columnIndex, workSheet.Dimension.End.Row, columnIndex];
int maxLength = columnCells.Max(cell => cell.Value.ToString().Count());
if (maxLength < 150)
{
workSheet.Column(columnIndex).AutoFit();
}
columnIndex++;
}
catch (Exception ex)
{
if (!(ex is System.Threading.ThreadAbortException))
{
//Log other errors here
}
}
}
// format header - bold, yellow on black
using (ExcelRange r = workSheet.Cells[startRowFrom, 1, startRowFrom, dataTable.Columns.Count])
{
r.Style.Font.Color.SetColor(System.Drawing.Color.White);
r.Style.Font.Bold = true;
r.Style.Fill.PatternType = OfficeOpenXml.Style.ExcelFillStyle.Solid;
r.Style.Fill.BackgroundColor.SetColor(System.Drawing.ColorTranslator.FromHtml("#1fb5ad"));
}
// format cells - add borders
using (ExcelRange r = workSheet.Cells[startRowFrom + 1, 1, startRowFrom + dataTable.Rows.Count, dataTable.Columns.Count])
{
r.Style.Border.Top.Style = ExcelBorderStyle.Thin;
r.Style.Border.Bottom.Style = ExcelBorderStyle.Thin;
r.Style.Border.Left.Style = ExcelBorderStyle.Thin;
r.Style.Border.Right.Style = ExcelBorderStyle.Thin;
r.Style.Border.Top.Color.SetColor(System.Drawing.Color.Black);
r.Style.Border.Bottom.Color.SetColor(System.Drawing.Color.Black);
r.Style.Border.Left.Color.SetColor(System.Drawing.Color.Black);
r.Style.Border.Right.Color.SetColor(System.Drawing.Color.Black);
}
// removed ignored columns
for (int i = dataTable.Columns.Count - 1; i >= 0; i--)
{
if (i == 0 && showSrNo)
{
continue;
}
if (!columnsToTake.Contains(dataTable.Columns[i].ColumnName))
{
workSheet.DeleteColumn(i + 1);
}
}
if (!String.IsNullOrEmpty(heading))
{
workSheet.Cells["A1"].Value = heading;
workSheet.Cells["A1"].Style.Font.Size = 20;
workSheet.InsertColumn(1, 1);
workSheet.InsertRow(1, 1);
workSheet.Column(1).Width = 5;
}
System.IO.FileInfo fileinfo2 = new System.IO.FileInfo(Filepath);
DeleteUploadedFile(Filepath);
workSheet.Protection.SetPassword("myPassword");
workSheet.Protection.IsProtected = true;
workSheet.Protection.AllowSelectUnlockedCells = false;
workSheet.Protection.AllowSelectLockedCells = false;
package.SaveAs(fileinfo2, "myPassword");
ret = package.GetAsByteArray();
return ret;
}
}
public static void DeleteUploadedFile(String filePath)
{
try
{
if (System.IO.File.Exists(filePath))
{
System.IO.File.Delete(filePath);
}
}
catch (Exception ex)
{ }
}
public static byte[] ExportExcel<T>(List<T> data, String Filepath, string Heading = "", bool showSlno = false, params string[] ColumnsToTake)
{
return ExportExcel(ListToDataTable<T>(data), Filepath, Heading, showSlno, ColumnsToTake);
}
An answer mentioned SaveAs close the package, so the correct steps will be returning the saved file as array instead of using GetAsByteArray afterwards. Or simply use GetAsByteArray(passwords) without SaveAs.

Delete row in a CSV file and show in DataGridView using c#

I have a problem when I want to delete a row in a CSV File, I have this code but only deletes the field that contains the line.
Example:
CSV File:
ID,Name,Lastname,Country
1,David,tod,UK
2,Juan,Perez,Germ
3,Pepe,Lopez,Col
First iteration, sending the id 1 to delete the line:
ID,Name,Lastname,Country
David,tod,UK
2,Juan,Perez,Germ
3,Pepe,Lopez,Arg
Just delete the id I want, but not the whole line
The expected result would be that like this:
ID,Name,Lastname,Country
2,Juan,Perez,Arg
3,Pepe,Lopez,Col
this is my code, What am I doing wrong? I have never used csv in C# :(
string searchid = "1";
string[] values = File.ReadAllText("C:\\registros.csv").Split(new char[] { ',' });
StringBuilder ObjStringBuilder = new StringBuilder();
for (int i = 0; i < values.Length; i++)
{
if (values[i].Contains(searchid))
continue;
ObjStringBuilder.Append(values[i] + ",");
}
ObjStringBuilder.ToString().Remove(ObjStringBuilder.Length - 1);
File.WriteAllText("\\registros.csv", ObjStringBuilder.ToString());
Another question is how can I show the CSV file in a datagridview in Windows Forms. I have this logic, don't know if this is correct, but how I can show it?
public DataTable ConvertCSVtoDataTable()
{
StreamReader sr = new StreamReader("\\registros.csv");
string[] headers = sr.ReadLine().Split(',');
DataTable dt = new DataTable();
foreach (string header in headers)
{
dt.Columns.Add(header);
}
while (!sr.EndOfStream)
{
string[] rows = Regex.Split(sr.ReadLine(), ",(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)");
DataRow dr = dt.NewRow();
for (int i = 0; i < headers.Length; i++)
{
dr[i] = rows[i];
}
dt.Rows.Add(dr);
}
return dt;
}
Thanks!
You can delete row from CSV using below link
Delete rows from CSV
and
You can convert the CSV into DataTable using the below code. If your csv file uses delimiter as ,
public DataTable ReadCSV(String FilePath, Boolean IsHeader)
{
string strConn = null;
string folderpath = null;
try
{
folderpath = FilePath.Substring(0, FilePath.LastIndexOf("\\") + 1);
string FileName = Path.GetFileName(FilePath);
if (IsHeader == true)
{
strConn = "Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source=" + folderpath + ";" + "Extended Properties=\"text;HDR=YES\"";
}
else
{
strConn = "Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source=" + folderpath + ";" + "Extended Properties=\"text;HDR=NO\"";
}
OleDbConnection Conn = new OleDbConnection();
Conn.ConnectionString = strConn;
Conn.Open();
string s1 = "select * from [" + FileName + "]";
OleDbDataAdapter da1 = new OleDbDataAdapter(s1, Conn);
DataSet dtall = new DataSet();
da1.Fill(dtall);
Conn.Close();
return dtall.Tables[0].Copy();
}
catch (Exception ex)
{
Exception excep = new Exception("CSV : " + ex.Message);
throw excep;
}
}
Reading and writing CSV files is not as trivial as it first seems. Cells can have embedded commas, and even new line characters. The following is one implementation of a CSV reader which can optionally be run asynchronously as a background worker. This implementation returns a standard DataTable which can easily be bound to a DataGridView:
grid.DataSource = dataTable;
The CsvReader class:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.IO;
using System.Linq;
using System.Text;
namespace CsvReaderExample
{
public class CsvReader
: BackgroundWorker
{
string[] m_lines;
public DataTable DataTable { get; private set; }
public CsvReader(string[] lines)
{
m_lines = lines;
WorkerReportsProgress = true;
WorkerSupportsCancellation = true;
}
public DataTable RunWorker()
{
return DataTable = ParseCsvLines();
}
protected override void OnDoWork(DoWorkEventArgs e)
{
base.OnDoWork(e);
e.Result = DataTable = ParseCsvLines();
}
private DataTable ParseCsvLines()
{
if (m_lines.Length == 0)
return null;
var table = new DataTable();
var columns = table.Columns;
var columnNames = GetRowValues(m_lines[0]);
foreach (var columnName in columnNames)
{
var name = columnName;
int number = 2;
while (columns.Contains(name))
name += " " + number++;
columns.Add(name);
}
var rows = table.Rows;
for (int index = 1, linesCount = m_lines.Length; index < linesCount; index++)
{
if (CancellationPending)
return null;
var line = m_lines[index];
var values = GetRowValues(line);
int valueCount = values.Count;
if (valueCount > columns.Count)
{
int columnNumber = columns.Count;
while (columns.Contains(columnNumber.ToString()))
columnNumber++;
columns.Add(columnNumber.ToString());
}
rows.Add(values.ToArray());
if (WorkerReportsProgress)
ReportProgress(100 * index / linesCount);
}
return table;
}
const char COMMA = ',',
DOUBLE_QUOTE = '"',
VERTICAL_BAR = '|';
private List<string> GetRowValues(string line)
{
var builder = new StringBuilder();
var values = new List<string>();
var inDoubleQuotes = false;
var maxIndex = line.Length - 1;
for (int index = 0; index <= maxIndex; index++)
{
char c = line[index];
if (c == DOUBLE_QUOTE)
{
if (index == 0)
{
inDoubleQuotes = true;
continue;
}
if (index < maxIndex)
{
var nextIndex = index + 1;
if (nextIndex < maxIndex)
{
if (line[nextIndex] == DOUBLE_QUOTE)
{
index++;
if (inDoubleQuotes)
builder.Append(DOUBLE_QUOTE);
continue;
}
}
}
inDoubleQuotes = !inDoubleQuotes;
continue;
}
if (c == COMMA)
{
if (inDoubleQuotes)
{
builder.Append(c);
continue;
}
values.Add(builder.ToString());
builder = new StringBuilder();
continue;
}
builder.Append(c);
}
values.Add(builder.ToString());
return values;
}
#region Sanitise cells with new line characters
public static void SanitiseCellsWithNewLineCharacters(string fileName)
{
var text = File.ReadAllText(fileName, Encoding.Default);
text = text.Replace("\r\n", "\n");
text = text.Replace("\r", "\n");
using (var writer = File.CreateText(fileName))
{
var inDoubleQuotes = false;
foreach (char c in text)
{
if (c == '\n' && inDoubleQuotes)
{
writer.Write(VERTICAL_BAR);
continue;
}
if (c == DOUBLE_QUOTE)
{
if (inDoubleQuotes)
inDoubleQuotes = false;
else
inDoubleQuotes = true;
}
writer.Write(c);
}
}
}
#endregion
}
}
You can read the DataTable synchronously as follows:
var lines = File.ReadAllLines("C:\\registros.csv");
var csvReader = new CsvReader(lines);
var dataTable = csvReader.RunWorker();
You could then remove row(s) from the DataTable with a method such as:
private static void RemoveById(DataTable dataTable, int id)
{
var column = dataTable.Columns["ID"];
if (column == null)
return;
var rows = dataTable.Rows;
for (int index = rows.Count - 1; index >= 0; index--)
{
var row = rows[index];
var value = row ["ID"];
if (value == null)
continue;
if (value.Equals(id))
{
rows.RemoveAt(index);
return;
}
}
}
Call it:
RemoveById(dataTable, 1);
The first thing that is wrong with your implementation is that you use ',' as the separator. You should either split on the new-line character '\n' or read the file line by line as follows:
var lines = new List<string>();
var file = new System.IO.StreamReader("c:\\registros.csv");
string line;
while((line = file.ReadLine()) != null)
{
lines.Add(line);
}
file.Close();
You could then look for the line that starts with the id you are looking for. When you find it, remove the line from the list.
for(int i=0; i++; i<lines.Count)
{
if (lines[i].StartsWith(searchid))
{
lines.RemoveAt(i);
break;
}
}
Next step is to write the result back to the file:
File.WriteAllLines("c:\\registros.csv", lines);
Regarding your second question, I found a similar Q/A on stackoverflow here.
First step is creating the DataTable, then you'll have to bind the table to the table control that will show the data.
SIMPLE & UNDERSTANDABLE!`
Solution For your First Problem is:
****Reading & Writing back to CSV File!****
string searchid = "1";
string[] values = File.ReadAllText(#"Complete Path Of File").Split(new char[] { '\n' });
StringBuilder ObjStringBuilder = new StringBuilder();
for (int i = 0; i < values.Length - 1; i++)
{
if (values[i].StartsWith(searchid) == false)
{
ObjStringBuilder.Append(values[i]+"\n");
}
}
File.WriteAllText(#"Complete Path Of File", ObjStringBuilder.ToString());
}
Answer to your Second Doubt:
****Populating DataGridView dynamically from CSV File!****
Comma(,) Problem SOLVED:
DataTable dtDataSource = new DataTable();
string[] fileContent = File.ReadAllLines(#"..\\Book1.csv");
if (fileContent.Count() > 0)
{
//Create data table columns dynamically
string[] columns = fileContent[0].Split(',');
for (int i = 0; i < columns.Count(); i++)
{
dtDataSource.Columns.Add(columns[i]);
}
//Add row data dynamically
for (int i = 1; i < fileContent.Count(); i++)
{
string[] rowData = fileContent[i].Split(',');
string[] realRowData = new string[columns.Count()];
StringBuilder collaboration = new StringBuilder();
int v = 0;
//this region solves the problem of a cell containing ",".
#region CommaSepProblem
for (int j = 0, K = 0; j < rowData.Count(); j++, K++)
{
//After splitting the line with commas. The cells containing commas will also be splitted.
//Fact: if a cell contains special symbol in excel that cell will be saved in .csv contained in quotes E.g A+B will be saved "A+B" or A,B will be saved as "A,B"
//Our code splits everything where comma is found. So solution is:
//Logic: After splitting if a string contains even number of DoubleQuote then its perfect cell otherwise, it is splitted in multiple cells of array.
if ((rowData[j].Count(x => x == '"') % 2 == 0))//checks if the string contains even number of DoubleQuotes
{
realRowData[K] = quotesLogic((rowData[j]));
}
else if ((rowData[j].Count(x => x == '"') % 2 != 0))//If Number of DoubleQuotes are ODD
{
int c = rowData[j].Count(x => x == '"');
v = j;
while (c % 2 != 0)//Go through all the next array cell till it makes EVEN Number of DoubleQuotes.
{
collaboration.Append(rowData[j] + ",");
j++;
c += rowData[j].Count(x => x == '"');
}
collaboration.Append(rowData[j]);
realRowData[K] = quotesLogic(collaboration.ToString());
}
else { continue; }
}
#endregion
dtDataSource.Rows.Add(realRowData);
}
if (dtDataSource != null)
{
dataGrid1.ItemsSource = dtDataSource.DefaultView;
}
}
Add This Method Too:
string quotesLogic(string collaboration)
{
StringBuilder after = new StringBuilder(collaboration);
if (after.ToString().StartsWith("\"") && after.ToString().EndsWith("\""))//removes 1st and last quotes as those are system generated
{
after.Remove(0, 1);
after.Remove(after.Length - 1, 1);
int count = after.Length - 1;
//FACT: if you try to add DoubleQuote in a cell in excel. It'll save that quote as 2 times DoubleQuote(Like "") which means first DoubleQuote is to give instruction to CPU that the next DoubleQuote is not system generated.
while (count > 0)//This loop find twice insertion of 2 DoubleQuotes and neutralise them to One DoubleQuote.
{
if (after[count] == '"' && after[count - 1] == '"')
{
after.Remove(count, 1);
}
count--;
}
}
return after.ToString();
}

To download and convert csv file to Datatable

I have a requirement where i need to download a csv file from given url and then export the data from file to database.
Is there any way to download and convert the file directly to datatable rather than saving it to harddisk and then reading it to datatable.
Hope my question make some sense.
Thanks in advance
You can use StreamReader class to download csv without the need to save it locally:
public string DownloadCSV(string url)
{
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse httpWebResponse = (HttpWebResponse)httpWebRequest .GetResponse();
StreamReader streamReader = new StreamReader(httpWebResponse .GetResponseStream());
string results = streamReader.ReadToEnd();
streamReader .Close();
return results;
}
string FileName = ...
OleDbConnection conn = new OleDbConnection
("Provider=Microsoft.Jet.OleDb.4.0; Data Source = " +
Path.GetDirectoryName(FileName) +
"; Extended Properties = \"Text;HDR=YES;FMT=Delimited\"");
conn.Open();
OleDbDataAdapter adapter = new OleDbDataAdapter
("SELECT * FROM " + Path.GetFileName(FileName), conn);
DataSet ds = new DataSet("Temp");
adapter.Fill(ds);
conn.Close();
I have written five methods below.
They have been designed to take into account optional quote marks (e.g. " symbols) and to be as versatile as possible without using other libraries:
public static DataTable GetDataTabletFromCSVFile(string filePath, bool isHeadings)
{
DataTable MethodResult = null;
try
{
using (TextFieldParser TextFieldParser = new TextFieldParser(filePath))
{
if (isHeadings)
{
MethodResult = GetDataTableFromTextFieldParser(TextFieldParser);
}
else
{
MethodResult = GetDataTableFromTextFieldParserNoHeadings(TextFieldParser);
}
}
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
public static DataTable GetDataTableFromCsvString(string csvBody, bool isHeadings)
{
DataTable MethodResult = null;
try
{
MemoryStream MemoryStream = new MemoryStream();
StreamWriter StreamWriter = new StreamWriter(MemoryStream);
StreamWriter.Write(csvBody);
StreamWriter.Flush();
MemoryStream.Position = 0;
using (TextFieldParser TextFieldParser = new TextFieldParser(MemoryStream))
{
if (isHeadings)
{
MethodResult = GetDataTableFromTextFieldParser(TextFieldParser);
}
else
{
MethodResult = GetDataTableFromTextFieldParserNoHeadings(TextFieldParser);
}
}
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
public static DataTable GetDataTableFromRemoteCsv(string url, bool isHeadings)
{
DataTable MethodResult = null;
try
{
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
StreamReader StreamReader = new StreamReader(httpWebResponse.GetResponseStream());
using (TextFieldParser TextFieldParser = new TextFieldParser(StreamReader))
{
if (isHeadings)
{
MethodResult = GetDataTableFromTextFieldParser(TextFieldParser);
}
else
{
MethodResult = GetDataTableFromTextFieldParserNoHeadings(TextFieldParser);
}
}
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
private static DataTable GetDataTableFromTextFieldParser(TextFieldParser textFieldParser)
{
DataTable MethodResult = null;
try
{
textFieldParser.SetDelimiters(new string[] { "," });
textFieldParser.HasFieldsEnclosedInQuotes = true;
string[] ColumnFields = textFieldParser.ReadFields();
DataTable dt = new DataTable();
foreach (string ColumnField in ColumnFields)
{
DataColumn DataColumn = new DataColumn(ColumnField);
DataColumn.AllowDBNull = true;
dt.Columns.Add(DataColumn);
}
while (!textFieldParser.EndOfData)
{
string[] Fields = textFieldParser.ReadFields();
for (int i = 0; i < Fields.Length; i++)
{
if (Fields[i] == "")
{
Fields[i] = null;
}
}
dt.Rows.Add(Fields);
}
MethodResult = dt;
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
private static DataTable GetDataTableFromTextFieldParserNoHeadings(TextFieldParser textFieldParser)
{
DataTable MethodResult = null;
try
{
textFieldParser.SetDelimiters(new string[] { "," });
textFieldParser.HasFieldsEnclosedInQuotes = true;
bool FirstPass = true;
DataTable dt = new DataTable();
while (!textFieldParser.EndOfData)
{
string[] Fields = textFieldParser.ReadFields();
if(FirstPass)
{
for (int i = 0; i < Fields.Length; i++)
{
DataColumn DataColumn = new DataColumn("Column " + i);
DataColumn.AllowDBNull = true;
dt.Columns.Add(DataColumn);
}
FirstPass = false;
}
for (int i = 0; i < Fields.Length; i++)
{
if (Fields[i] == "")
{
Fields[i] = null;
}
}
dt.Rows.Add(Fields);
}
MethodResult = dt;
}
catch (Exception ex)
{
ex.HandleException();
}
return MethodResult;
}
If, like me, you're saving from reporting services then you should use it like this:
Warning[] warnings;
string[] streamids;
string mimeType;
string encoding;
string filenameExtension;
byte[] bytes = rvMain.ServerReport.Render("csv", null, out mimeType, out encoding, out filenameExtension, out streamids, out warnings);
string CsvBody = System.Text.Encoding.UTF8.GetString(bytes);
DataTable dt = GetDataTableFromCsvString(CsvBody,true);
Otherwise, all you need do is:
bool IsHeadings = true; //Does the data include a heading row?
DataTable dt = GetDataTableFromCsvString(CsvBody, IsHeadings);
Or to use directly from a csv file
bool IsHeadings = true; //Does the data include a heading row?
DataTable dt = GetDataTabletFromCsvFile(FilePath, IsHeadings)
Or to use a csv file that is stored remotely
bool IsHeadings = true; //Does the data include a heading row?
DataTable dt = GetDataTabletFromRemoteCsv(Url, IsHeadings)

using paging in DataTable for showing in GridView

I want 30 data in GridView. I want to read all the files in my folder and show in GridView. I am using the following code.
string folderPath = #"C:\Folder\Folder-2.0\New folder";
DataTable dt = new DataTable();
//Creating DataTable
foreach (string fileName in Directory.EnumerateFiles(folderPath, "*.txt"))
{
string contents = File.ReadAllText(fileName);
string str = string.Empty;
string s;
if (File.Exists(fileName))
{
StreamReader sr = new StreamReader(fileName);
String line;
int k = 0;
while ((line = sr.ReadLine()) != null)
{
string[] text;
if (line.Trim() != string.Empty)
{
//text = line.Split(new char[] { '^' }, StringSplitOptions.RemoveEmptyEntries);
text = line.Split('^');
if (text[0].Contains("START FOOTER"))
{
break;
}
else
{
DataRow dr;
dr = dt.NewRow();
for (int i = 0; i <= text.Length - 1; i++)
{
dr[i] = text[i];
}
dt.Rows.Add(dr);
k = k + 1;
}
}
}
Response.Write(k);
// Response.End;
GridView1.DataSource = dt;
GridView1.DataBind();
}
else
{
s = "File does not exists";
}
}
But the above Code throws error System.OutOfMemoryException. Error is pointing at code GridView1.DataBind(); There are about 2000 files in the folder each file is of size between 1 to 2 MB. Thats why I want to use paging using DataTable. Through DataTable I want to show 30 records.
Thanks,
Please define PageSize as 30 and implement PageIndexChanging event
protected void GridView1_PageIndexChanging(object sender, GridView1PageEventArgs e)
{
// here you need create one method of your above code and call here
GridView1.PageIndex = e.NewPageIndex;
GridView1.DataBind();
}

Categories

Resources