I have downloaded data that is contained in a List<Row> Rows like this:
class Row
{
string[] Items { get; set; }
public Row(string[] Items)
{
this.Items = Items;
}
}
The rows are basically comma delimited entries (.csv)
using (var reader = new StreamReader(spreadSheetStream))
{
string header = reader.ReadLine(); //This is the header
Rows.Add(new Row(header.Split(',')));
while (!reader.EndOfStream)
{
string tickerInfo = reader.ReadLine(); //This is a data entry
Rows.Add(new Row(tickerInfo.Split(',')));
}
}
I convert the List<Row> into a Datatable like this
DataTable historicalDataTable = ToDataTable<Row>(Rows);
The first element of List<Row> Rows contains the names of the columns, seven of them. Then each element thereafter is an actual data element.
public static DataTable ToDataTable<T>(List<T> items)
{
DataTable dataTable = new DataTable(typeof(T).Name);
//Get all the properties
PropertyInfo[] Props = typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (PropertyInfo prop in Props)
{
//Setting column names as Property names
dataTable.Columns.Add(prop.Name);
}
foreach (T item in items)
{
var values = new object[Props.Length];
for (int i = 0; i < Props.Length; i++)
{
//inserting property values to datatable rows
values[i] = Props[i].GetValue(item, null);
}
dataTable.Rows.Add(values);
}
//put a breakpoint here and check datatable
return dataTable;
}
When I try to write out the contents of the table, I see the right number of rows, but there is nothing in ItemArray
foreach (DataRow dataRow in historicalDataTable.Rows)
{
Console.WriteLine(dataRow.ToString());
foreach (var item in dataRow.ItemArray)
{
Console.WriteLine(item);
}
}
Your code is a bit contradictory. You are trying to copy properties as column names, however your csv code actually populates the first row as the column names. You have no distinction between header rows and data rows
You can just read it straight into a datatable with something like :-
(though you may want to do better error checking)
var dt = new DataTable("Rows");
string data = "a,b,c\r\n1,2,3\r\n4,5,6";
var stream = GenerateStreamFromString(data); // http://stackoverflow.com/questions/1879395/how-to-generate-a-stream-from-a-string
using (var reader = new StreamReader(stream))
{
reader.ReadLine()?.Split(',').ToList().ForEach(h => dt.Columns.Add(h));
while (!reader.EndOfStream)
{
dt.Rows.Add(reader.ReadLine()?.Split(',').ToArray());
}
}
foreach (DataColumn dataColumn in dt.Columns)
{
Console.Write($"{dataColumn.ColumnName} ");
}
Console.WriteLine();
foreach (DataRow dataRow in dt.Rows)
{
Console.Write("Row: ");
foreach (var item in dataRow.ItemArray)
{
Console.Write(item + " ");
}
Console.WriteLine();
}
Related
I've retrieved a table from outlook as html body then I've parse it to a datatable but when I run the code, all I get is System.Data.DataRow
static void Main(string[] args)
{
var mails = OutlookEmails.ReadMailItems();
foreach (var mail in mails)
{
StringBuilder builder = new StringBuilder();
builder.Append(mail.EmailBody.ToString());
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(builder.ToString());
var nodes = doc.DocumentNode.SelectNodes("//table//tr");
DataTable dataTable = new DataTable();
var headers = nodes[0]
.Elements("th")
.Select(th => th.InnerText.Trim());
foreach (var header in headers)
{
dataTable.Columns.Add(header);
}
var rows = nodes.Skip(1).Select(tr => tr
.Elements("td")
.Select(td => td.InnerText.Trim())
.ToArray());
foreach (var row in rows)
{
dataTable.Rows.Add(row);
}
Console.WriteLine(dataTable.Rows);
Console.ReadLine();
}
}
Because you are just printing out the type of the object.
What else did you expect?
If you want to print out every column for every row in your dataTable, you must specify it.
Try this:
foreach (DataRow row in dataTable.Rows)
{
Console.WriteLine();
foreach (DataColumn col in dataTable.Columns)
{
Console.Write(row[col] + " ");
}
}
For further information: MS DataTable Docs
I have 10 reports in my application which I let users to export to excel. I have never written CSV files. In my existing application, I convert the results from the stored procedure to an HTML table and write it to Excel. Some of my results from the stored procedures have dynamic columns so I use dapper. My new requirement is to provide CSV export as well.
So should I first convert html datatable to excel and convert it to CSV or write HTML datatable to CSV. I dont want to manually parse because there are 10 different reports with different columns and some of the reports have dynamic columns so I cant manually parse.
Stored procs returning Dapper, Dynamic Columns
EFDbContext db = new EFDbContext();
var recordDate = StartDate.Date;
var cnn = new SqlConnection(db.Database.Connection.ConnectionString);
cnn.Open();
var p = new DynamicParameters();
p.Add("#StartDate", StartDate);
p.Add("#UserRoleID", UserRoleID);
p.Add("#SelectedSystemIDs", SelectedSystemIDs);
p.Add("#SelectedPartIDs", SelectedPartIDs);
p.Add("#SelectedSubSystems", SelectedSubsystems);
p.Add("#SelectedServiceTypes", SelectedServiceTypes);
var obs = cnn.Query(sql: "spExportInstrumentConfigAll", param: p, commandType: CommandType.StoredProcedure);
var dt = ToDataTable(obs);
return ExportDatatableToHtml(dt);
public static DataTable ToDataTable(IEnumerable<dynamic> items)
{
if (items == null) return null;
var data = items.ToArray();
if (data.Length == 0) return null;
var dt = new DataTable();
foreach (var pair in ((IDictionary<string, object>)data[0]))
{
dt.Columns.Add(pair.Key, (pair.Value ?? string.Empty).GetType());
}
foreach (var d in data)
{
dt.Rows.Add(((IDictionary<string, object>)d).Values.ToArray());
}
return dt;
}
public static string ExportDatatableToHtml(DataTable dt)
{
StringBuilder strHTMLBuilder = new StringBuilder();
strHTMLBuilder.Append("<html >");
strHTMLBuilder.Append("<head>");
strHTMLBuilder.Append("</head>");
strHTMLBuilder.Append("<body>");
strHTMLBuilder.Append("<table border='1px' cellpadding='1' cellspacing='1 style='font-family:Garamond; font-size:medium'>");
strHTMLBuilder.Append("<tr >");
foreach (DataColumn myColumn in dt.Columns)
{
strHTMLBuilder.Append("<td >");
strHTMLBuilder.Append(myColumn.ColumnName);
strHTMLBuilder.Append("</td>");
}
strHTMLBuilder.Append("</tr>");
foreach (DataRow myRow in dt.Rows)
{
strHTMLBuilder.Append("<tr >");
foreach (DataColumn myColumn in dt.Columns)
{
strHTMLBuilder.Append("<td >");
strHTMLBuilder.Append(myRow[myColumn.ColumnName].ToString());
strHTMLBuilder.Append("</td>");
}
strHTMLBuilder.Append("</tr>");
}
//Close tags.
strHTMLBuilder.Append("</table>");
strHTMLBuilder.Append("</body>");
strHTMLBuilder.Append("</html>");
string Htmltext = strHTMLBuilder.ToString();
return Htmltext;
}
Non-Dynamic Columns mapped to entity
return db.Database.SqlQuery<ServiceEntryPartExportDataRow>("[dbo].[spExportServiceParts] #parm1, #parm2, #parm3, #parm4,#parm5,#parm6",
new SqlParameter("parm1", StartDate),
new SqlParameter("parm2", EndDate),
new SqlParameter("parm3", Reconciled),
new SqlParameter("parm4", ServiceTypes),
new SqlParameter("parm5", SelectedSystemIDs),
new SqlParameter("parm6", UserRoleID)
).ToList().ToHTMLTable();
public static string ToHTMLTable<T>(this IList<T> data)
{
PropertyDescriptorCollection props =
TypeDescriptor.GetProperties(typeof(T));
StringBuilder builder = new StringBuilder();
builder.Append("<table border=\"1\">");
builder.Append("<tr>");
for (int i = 0; i < props.Count; i++)
{
builder.Append("<td>");
PropertyDescriptor prop = props[i];
builder.Append(prop.Name);
builder.Append("</td>");
}
builder.Append("</tr>");
object[] values = new object[props.Count];
foreach (T item in data)
{
builder.Append("<tr>");
for (int i = 0; i < values.Length; i++)
{
builder.Append("<td>");
builder.Append(props[i].GetValue(item));
builder.Append("</td>");
}
builder.Append("</tr>");
}
builder.Append("</table>");
return "<html><body>" + builder.ToString() + "</body></html";
}
Current code Sending to Excel
return new PostActionResult(htmlTable, "ServiceEntryHistory", submit);
public PostActionResult(string htmlTable, string typeName, string submit) { this.htmlTable = htmlTable; this.typeName = typeName; this.submit = submit; }
public PostActionResult(DataTable dataTable, string typeName, string submit) { this.dataTable = dataTable; this.typeName = typeName; this.submit = submit; }
public override void ExecuteResult(ControllerContext context)
{
if (submit == "Excel")
{
ExcelHelpers.ExportToExcel(context.HttpContext, typeName, htmlTable);
}
if (submit == "CSV")
{
ExcelHelpers.ExportToExcelCSV(context.HttpContext, typeName, htmlTable);
}
}
public static void ExportToExcel(HttpContextBase httpBase, string fileNamePrefix, string table)
{
string TimeStamp = DateTime.Now.ToLocalTime().ToString();
string fileName = string.Format("attachment;filename={0}_{1}.xls", fileNamePrefix, TimeStamp);
httpBase.Response.ClearHeaders();
httpBase.Response.ClearContent();
httpBase.Response.Clear();
httpBase.Response.AddHeader("content-disposition", fileName);
httpBase.Response.ContentType = "application/vnd.ms-excel";
httpBase.Response.Write(table);
httpBase.Response.End();
}
You already have code to build an HTML table from the data. Building a CSV is very nearly identical. For brevity, let's simplify the HTML table pseudo-code:
builder.Append("<table>");
// header
builder.Append("<tr>");
foreach (var column in columns)
builder.Append("<th>" + column.name + "</th>");
builder.Append("</tr>");
// rows
foreach (var row in rows)
{
builder.Append("<tr>");
foreach (var column in row.columns)
builder.Append("<td>" + column.value + "</td>");
builder.Append("</tr>");
}
builder.Append("</table>");
Building a CSV is the exact same structure:
// header
foreach (var column in columns)
builder.Append("\"" + column.name + "\",");
// there's now an extra comma at the end. remove it, or use a
// different method to have built the row, such as string.Join.
// rows
foreach (var row in rows)
{
foreach (var column in row.columns)
builder.Append("\"" + column.value + "\",");
// there's now an extra comma at the end. remove it, or use a
// different method to have built the row, such as string.Join.
builder.Append(Environment.NewLine);
}
Remember that this is free-hand pseudo-code, there are some clean-ups you can employ. You might also check the column types to determine if you need those escaped quotes or not, since numeric types wouldn't want them. But the point is that the structure is the same. A CSV is text in the same way that HTML is text. It's only the dressing around the values that's different.
Side note: This is actually a classic example of the Template Method Pattern.
am searching and cracking my brain on how to convert a dynamic list to a databale,
c#, please advise, thanks
List<dynamic>dlist=new List<dynamic>
to
DataTable
I think you looking something like this. Hope it's working for you.
From dynamic list to DataTable:
List<dynamic> dlist=new List<dynamic>
var json = JsonConvert.SerializeObject(dlist);
DataTable dataTable = (DataTable)JsonConvert.DeserializeObject(json, (typeof(DataTable)));
Also to get JsonString from DataTable:
string JSONresult = JsonConvert.SerializeObject(dataTable);
The following is the method through which you can convert any list object to datatable..
public DataTable ConvertToDataTable<T>(IList<T> data)
{
PropertyDescriptorCollection properties =
TypeDescriptor.GetProperties(typeof(T));
DataTable table = new DataTable();
foreach (PropertyDescriptor prop in properties)
table.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ?? prop.PropertyType);
foreach (T item in data)
{
DataRow row = table.NewRow();
foreach (PropertyDescriptor prop in properties)
row[prop.Name] = prop.GetValue(item) ?? DBNull.Value;
table.Rows.Add(row);
}
return table;
}
http://social.msdn.microsoft.com/Forums/vstudio/en-US/6ffcb247-77fb-40b4-bcba-08ba377ab9db/converting-a-list-to-datatable?forum=csharpgeneral
public DataTable ToDataTable<T>(dynamic items)
{
DataTable dtDataTable = new DataTable();
if (items.Count == 0) return dtDataTable;
((IEnumerable)items[0]).Cast<dynamic>().Select(p => p.Name).ToList().ForEach(col => { dtDataTable.Columns.Add(col); });
((IEnumerable)items).Cast<dynamic>().ToList().
ForEach(data =>
{
DataRow dr = dtDataTable.NewRow();
((IEnumerable)data).Cast<dynamic>().ToList().ForEach(Col => { dr[Col.Name] = Col.Value; });
dtDataTable.Rows.Add(dr);
});
return dtDataTable;
}
I don't know why you need this, however, you can use this ObjectShredder using reflection which can convert anything to DataTable, so even dynamic or anonymous types:
Implement CopyToDataTable<T> Where the generic Type T Is Not a DataRow
However, my suggestion is to not name that extension method CopyToDataTable but for example CopyAnyToDataTable to avoid conflicts with the existing extension method CopyToDataTable.
Use this function ,
public static DataTable ConvertToDatatable<T>(this IList<T> data)
{
PropertyDescriptorCollection props =
TypeDescriptor.GetProperties(typeof(T));
DataTable table = new DataTable();
for(int i = 0 ; i < props.Count ; i++)
{
PropertyDescriptor prop = props[i];
table.Columns.Add(prop.Name, prop.PropertyType);
}
object[] values = new object[props.Count];
foreach (T item in data)
{
for (int i = 0; i < values.Length; i++)
{
values[i] = props[i].GetValue(item);
}
table.Rows.Add(values);
}
return table;
}
If the underlying type is ExpandoObject, then you need to check for IDictionary instead of going via reflection. Hopefully this helps someone else in the future:
public static DataTable ConvertToDataTable<T>(IEnumerable<T> data)
{
DataTable table = new DataTable();
foreach (T item in data)
{
if (item is IDictionary<string, object> dict)
{
foreach (var key in dict)
{
table.Columns.Add(key.Key, key.Value?.GetType() ?? typeof(object));
}
break;
}
foreach (var prop in typeof(T).GetProperties())
{
table.Columns.Add(prop.Name, prop.PropertyType);
}
break;
}
DataRow row = null;
foreach (T item in data)
{
if (item is IDictionary<string, object> dict)
{
row = table.NewRow();
foreach (var key in dict)
{
row[key.Key] = key.Value;
}
table.Rows.Add(row);
continue;
}
row = table.NewRow();
foreach (var prop in typeof(T).GetProperties())
{
row[prop.Name] = prop.GetValue(item);
}
table.Rows.Add(row);
}
return table;
}
To Convert the Dynamic List object into DataTable using C#
public DataTable DynamicToDT(List<object> objects)
{
DataTable dt = new DataTable("StudyRecords"); // Runtime Datatable
string[] arr = { "Name", "Department", "CollegeName", "Address" };// Column Name for DataTable
if (objects != null && objects.Count > 0)
{
for (int i = 0; i < objects.Count; i++)
{
dt.Columns.Add(arr[i]);
if (i == 0)
{
var items = objects[0] as IEnumerable<string>;
foreach (var itm in items)
{
DataRow dr1 = dt.NewRow(); // Adding values to Datatable
dr1[arr[i]] = itm;
dt.Rows.Add(dr1);
}
}
else
{
var items = objects[i] as IEnumerable<string>;
int count = 0;
foreach (var itm in items)
{
dt.Rows[count][arr[i]] = itm;
count++;
}
}
}
return dt; // Converted Dynamic list to Datatable
}
return null;
}
public static DataTable DynamicToDT(List objects)
{
var data = objects.ToArray();
if (data.Count() == 0) return null;
var dt = new DataTable();
foreach (var key in ((IDictionary<string, object>)data[0]).Keys)
{
dt.Columns.Add(key);
}
foreach (var d in data)
{
dt.Rows.Add(((IDictionary<string, object>)d).Values.ToArray());
}
return dt;
}
// Obtem a lista dinamica via chamada API
List<dynamic> resultado = ExecutaQuery(sql);
// converte a lista dinamica com o resultado em JSON
string json = JsonConvert.SerializeObject(resultado);
// converte o json em datatable
DataTable dataTable = (DataTable)JsonConvert.DeserializeObject(json, (typeof(DataTable)));
I am trying to write a function which takes generic List/Enumerable and add DataRow to existing DataTable but only for Custom Columns.
public void AddGridRow<T>(IEnumerable<T> rowData, params String[] columnNames)
{
HashSet<String> columnsHashSet = new HashSet<String>(columnNames);
PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(typeof(T));
foreach (T item in rowData)
{
foreach (PropertyDescriptor prop in properties)
{
foreach (DataColumn column in _dataGridTable.Columns)
{
DataRow newRow = _dataGridTable.NewRow();
if (columnsHashSet.Contains(prop.Name))
{
newRow[prop.Name] = prop.GetValue(item) ?? DBNull.Value;
_dataGridTable.Rows.Add(newRow); // _dataGridTable is my existing DataTable
break;
}
}
}
}
}
Now the problem is it is not behaving correctly, lets say I have to add 1 row in 4 columns (columnNames), it is adding the 1 Row per columns. Also there are far many foreach loops.
How can I correct this and if possible optimize it.
You need to move the "_dataGridTable.Rows.Add(newRow);" line outside of the inner foreach loop:
foreach (T item in rowData)
{
foreach (PropertyDescriptor prop in properties)
{
DataRow newRow = _dataGridTable.NewRow();
foreach (DataColumn column in _dataGridTable.Columns)
{
if (columnsHashSet.Contains(prop.Name))
{
newRow[prop.Name] = prop.GetValue(item) ?? DBNull.Value;
break;
}
}
}
_dataGridTable.Rows.Add(newRow); // _dataGridTable is my existing DataTable
}
I am importing data from csv file, sometimes there are column headers and some times not the customer chooses custom columns(from multiple drop downs)
my problem is I am able to change the columns type and name but when I want to import data row into cloned table it just adds rows but no data with in those rows. If I rename the column to old values it works, let's say column 0 name is 0 if I change that to something else which I need to it won't fill the row below with data but If I change zero to zero again it will any idea:
here is my coding:
#region Manipulate headers
DataTable tblCloned = new DataTable();
tblCloned = tblDataTable.Clone();
int i = 0;
foreach (string item in lstRecord)
{
if (item != "Date")
{
var m = tblDataTable.Columns[i].DataType;
tblCloned.Columns[i].DataType = typeof(System.String);
tblCloned.Columns[i].ColumnName = item;
}
else if(item == "Date")
{
//get the proper date format
//FillDateFormatToColumn(tblCloned);
tblCloned.Columns[i].DataType = typeof(DateTime);
tblCloned.Columns[i].ColumnName = item;
}
i++;
}
tblCloned.AcceptChanges();
foreach (DataRow row in tblDataTable.Rows)
{
tblCloned.ImportRow(row);
}
tblCloned.AcceptChanges();
#endregion
in the second foreach loop when it calls to import data to cloned table it adds empty rows.
After couple of tries I came up with this solution which is working:
foreach (DataRow row in tblDataTable.Rows)
{
int x = 0;
DataRow dr = tblCloned.NewRow();
foreach (DataColumn dt in tblCloned.Columns)
{
dr[x] = row[x];
x++;
}
tblCloned.Rows.Add(dr);
//tblCloned.ImportRow(row);
}
but I will accept Scottie's answer because it is less code after all.
Instead of
foreach (DataRow row in tblDataTable.Rows)
{
tblCloned.ImportRow(row);
}
try
foreach (DataRow row in tblDataTable.Rows)
{
tblCloned.LoadDataRow(row.ItemArray, true);
}