I have the following object structure and trying to write to csv using csvhelper. but the filenames column in not getting added.
public class ClusterData
{
public IEnumerable<string> FileName { get; set; }
public int? ClusterNumber { get; set; }
public string TopTerm { get; set; }
}
using (var writer = new StreamWriter(#"C:\Clean.csv"))
{
var csv = new CsvWriter(writer);
csv.WriteHeader<ClusterData>();
foreach (var item in dataToCsv)
{
foreach (var filename in item.FileName)
{
csv.WriteField(filename);
csv.WriteField(item.ClusterNumber);
csv.WriteField(item.TopTerm);
csv.NextRecord();
}
}
writer.Flush();
}
how to achieve with this?i want the outer loop to be repeated once and inner loop to be repeated for each item in filename.
Thanks
Extract the desired data and then use the writer to send it to file
using (var writer = new StreamWriter(#"C:\Clean.csv")) {
var data = new List<ClusterData>();
//...assuming data is poulated
var dataToCsv = data.SelectMany(item => item.FileName.Select(filename => new {
FileName = filename,
ClusterNumber = item.ClusterNumber,
TopTerm = item.TopTerm
}));
var csv = new CsvWriter(writer);
csv.WriteRecords(dataToCsv);
}
A linq query is used to construct the desired object format for each file name in the data.
The data is then converted to CSV as it normally would using a CsvWriter
Related
My application reads .CSV file(which do not having a header in csv file) and converts into XML file.
For existing code wrote as
sr = new StreamReader(fs);
fs = null;
using (CsvReader csvReader = new CsvReader(sr))
{
sr = null;
csvReader.Configuration.HasHeaderRecord = hasHeaderRecord;
csvReader.Configuration.IgnoreBlankLines = false;
csvReader.Configuration.IgnoreReadingExceptions = true;
csvReader.Configuration.WillThrowOnMissingField = false;
csvReader.Configuration.TrimFields = true;
csvReader.Configuration.RegisterClassMap<Class1Map>();
FileRecords = csvReader.GetRecords<Class1>().ToList();
}
public class Class1Map : CsvClassMap<Class1>
{
public Class1Map()
{
Map(m => m.AccountId).Index(0);
Map(m => m.MeterId).Index(1);
.......
.......
}
}
But now for my new requirement, .csv file includes header and column names that are different compared to previous .csv. Somehow I have read the new CSV file and get values present in the csv file and mapped to class1.
Class1 properties are AccountId,MeterId etc.
But in new format the names are different now.
AccountId as AccountRef and MeterId as MeterSerial.
Can any one suggest how to map new file values of AccountRef,MeterSerial to class1 properties AccountId,MeterId
You could just add .Name() to your maps. Your first example with no header will use .Index() and your second example with a header will use .Name() to map the columns.
void Main()
{
var config1 = new CsvConfiguration(CultureInfo.InvariantCulture)
{
HasHeaderRecord = false
};
using (var reader = new StringReader("1,2\n3,4"))
using (var csv = new CsvReader(reader, config1))
{
csv.Context.RegisterClassMap<Class1Map>();
var records = csv.GetRecords<Class1>().Dump();
}
var config2 = new CsvConfiguration(CultureInfo.InvariantCulture)
{
HasHeaderRecord = true
};
using (var reader = new StringReader("MeterSerial,AccountRef\n4,5\n6,7"))
using (var csv = new CsvReader(reader, config2))
{
csv.Context.RegisterClassMap<Class1Map>();
var records = csv.GetRecords<Class1>().Dump();
}
}
public class Class1Map : ClassMap<Class1>
{
public Class1Map()
{
Map(m => m.AccountId).Index(0).Name("AccountRef");
Map(m => m.MeterId).Index(1).Name("MeterSerial");
}
}
public class Class1
{
public int AccountId { get; set; }
public int MeterId { get; set; }
}
Edit, thank you for the suggestion of using csvhelper, this is actually helping quite a lot.
What I have done is create a new method like so:
public static void AppendFile<T>(FileInfo fi, List<T> report)
{
var settings = new CsvConfiguration(new CultureInfo("en-GB"))
{
//Delimiter = ";"
};
using var stream = File.Open(fi.FullName, FileMode.Append);
using var writer = new StreamWriter(stream);
using (var csv = new CsvWriter(writer, settings))
{
csv.WriteRecords(report);
}
}
And gone through the example on the csvhelper site, creating a new class:
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
}
and then creating a new list:
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
}
and calling it like so:
AppendToFile.AppendFile(exportFile, records1);
This is working better than what I had before but, instead adding the new columns beside the last column, they are getting added at the bottom of the file.
For clarification,
what I'm trying to do:
what I'm getting:
As you'll be able to see, it's just being added as new rows rather than being separate columns, what do I need to change?
Another way you can accomplish your goal is to write each object to the row by hand using csvWriter.WriteHeader and csvWriter.WriteRecord.
void Main()
{
var settings = new CsvConfiguration(new CultureInfo("en-GB"))
{
Delimiter = ";"
};
var fooList = new List<Foo>()
{
new Foo { Id = 67, Name = "test1,test2"}
};
List<MyClass> records;
using (var reader = new StringReader("Author,Admin Owner\nChris Jones,\nJohn Thompson,\nBrian Oates,"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
records = csv.GetRecords<MyClass>().ToList();
}
using var csvWriter = new CsvWriter(Console.Out, settings);
csvWriter.WriteHeader<MyClass>();
csvWriter.WriteHeader<Foo>();
csvWriter.NextRecord();
var i = 0;
foreach (var record in records)
{
csvWriter.WriteRecord(record);
if (i < fooList.Count())
{
csvWriter.WriteRecord(fooList[i]);
}
csvWriter.NextRecord();
i++;
}
}
public class MyClass
{
public string Author { get; set; }
[Name("Admin Owner")]
public string AdminOwner { get; set; }
}
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
}
I managed to figure out a way that worked for me, might not be the most efficient but it does work.
public static void AppendFile(FileInfo fi, List<string> newColumns, DataTable newRows)
{
var settings = new CsvConfiguration(new CultureInfo("en-GB"))
{
Delimiter = ";"
};
var dt = new DataTable();
using (var reader = new StreamReader(fi.FullName))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
using (var dataReader = new CsvDataReader(csv))
{
dt.Load(dataReader);
foreach (var title in newColumns)
{
dt.Columns.Add(title);
}
dt.Rows.Clear();
foreach (DataRow row in newRows.Rows)
{
dt.Rows.Add(row.ItemArray);
}
}
}
using var streamWriter = new StreamWriter(fi.FullName);
using var csvWriter = new CsvWriter(streamWriter, settings);
// Write columns
foreach (DataColumn column in dt.Columns)
{
csvWriter.WriteField(column.ColumnName);
}
csvWriter.NextRecord();
// Write row values
foreach (DataRow row in dt.Rows)
{
for (var i = 0; i < dt.Columns.Count; i++)
{
csvWriter.WriteField(row[i]);
}
csvWriter.NextRecord();
}
}
I start by getting the contents of the csv file into a data table and then adding in the new columns that I need. I then clear all the rows in the datatable and add new ones in (the data that is removed is added back in via the newRows parameter) and then write the datatable to the csv file
I have this code to read a simple CSV file with the CsvHelper library
public class CSVData
{
public string Time { get; set; }
public double Value1 { get; set; }
public double Value2 { get; set; }
public double Value3 { get; set; }
}
using (var reader = new StreamReader("c:\\temp\\test.csv"))
using (var csv = new CsvReader(reader))
{
var records = csv.GetRecords<CSVData>();
}
and this is how th CSV data looks
"Time","Value1","Value2","Value3"
8/28/2019 4:32:09 PM,2.03,10229.73,10437.51821998
and I get this error when I try to read the records with
foreach (CSVData item in records)
CsvHelper.BadDataException: 'You can ignore bad data by setting BadDataFound to null.'
Your code actually worked fine for me as long as you use the US culture setting for double (using a . decimal point).
class Program
{
static void Main(string[] args)
{
Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US", false);
List<CSVData> records;
using (var reader = new StreamReader("test.csv"))
using (var csv = new CsvReader(reader))
{
records = csv.GetRecords<CSVData>().ToList();
}
foreach (var r in records)
{
Console.WriteLine($"Time:{r.Time} Value1:{r.Value1} Value2:{r.Value2} Value3:{r.Value3}");
}
}
}
This outputs:
Time: 8/28/2019 4:32:09 PM Value1:2.03 Value2:10229.73 Value3:10437.51821998
Try setting the culture info in the configuration. You may also need to set the delimiter.
using (var reader = new StreamReader("c:\\temp\\test.csv"))
using (var csv = new CsvReader(reader))
{
csv.Configuration.CultureInfo = new CultureInfo("en-US", false);
csv.Configuration.Delimiter = ",";
var records = csv.GetRecords<CSVData>();
}
You can try to set CultureInfo to CurrentCulture.
csv.Configuration.CultureInfo = CultureInfo.CurrentCulture;
I want to write a string into one Column in an .csv (Excel) file. My Problem is that the string is written into multiple Columns.
In this screenshot for example I have 20 Columns.
GetMetadataCompleteResponse resultValue = null;
string jsonData = null;
await Task.Run(() =>
{
byte[] rawData = Convert.FromBase64String(responseContent);
jsonData = CompressUtil.Unzip(rawData);
});
resultValue = JsonConvert.DeserializeObject<GetMetadataCompleteResponse>(jsonData);
foreach(string a in resultValue.Value.Values)
{
foreal += a;
}
await Log.Info("callWebservice for " + strUrl + ", Result: " + objErrorDetails.Code + ", " + foreal);
edit
I've noticed that the new Column starts after every ';'(semicolon). I probably can just replace it with something else.
I think you have 2 issues. The first one is how you write your CSV with simple string concatenation. With no escaping or double quote.
The Json will have commas , that will be separator in your CSV.
In order to produc e a valid CSV you should read the RFC 4180 and use a proper library to handle the Serialisation.
Here is an Minimal, Complete, and Verifiable example of writing a Json in a CSV column.
using CsvHelper;
using CsvHelper.Configuration;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
public class Program
{
public static void Main()
{
var input = new Foo
{
Label = "My Foo",
Bars = new List<Bar> {
new Bar{Label="Bar2"},
new Bar{Label="Bar1"},
new Bar{Label="Bar3"},
}
};
var json = JsonConvert.SerializeObject(input);
var myObject = new CsvObject
{
Label = "My CSV object",
FooString = json,
};
var result = "";
// Writing into a string instead of a file for debug purpuse.
using (var stream = new MemoryStream())
using (var reader = new StreamReader(stream))
using (var writer = new StreamWriter(stream))
using (var csv = new CsvWriter(writer))
{
csv.Configuration.RegisterClassMap<CsvObjectMap>();
csv.WriteHeader<CsvObject>();
csv.NextRecord();
csv.WriteRecord(myObject);
csv.NextRecord();
writer.Flush();
stream.Position = 0;
result = reader.ReadToEnd();
}
Console.WriteLine(result);
}
private sealed class CsvObjectMap : ClassMap<CsvObject>
{
public CsvObjectMap()
{
Map( m => m.FooString );
Map( m => m.Label );
}
}
public class CsvObject
{
public string Label { get; set; }
public string FooString { get; set; }
}
public class Foo
{
public string Label { get; set; }
public List<Bar> Bars { get; set; }
}
public class Bar
{
public string Label { get; set; }
}
}
Live demo : https://dotnetfiddle.net/SNqZX1
In this exemple I have used CsvHelper for CSV serialisation, and Json.NET for the Json serialisation. Note that Writing a CSV to a file is a more simlpe task that to a string like in this example
I am trying to use the CSVhelper plugin to read an uploaded CSV file. Here is my modelBinder class:
public class SurveyEmailListModelsModelBinder : DefaultModelBinder
{
public override object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
{
var csv = bindingContext.ValueProvider.GetValue(bindingContext.ModelName);
var file = ((csv.RawValue as HttpPostedFileBase[]) ?? Enumerable.Empty<HttpPostedFileBase>()).FirstOrDefault();
if (file == null || file.ContentLength < 1)
{
bindingContext.ModelState.AddModelError(
"",
"Please select a valid CSV file"
);
return null;
}
using (var reader = new StreamReader(file.InputStream))
using (var csvReader = new CsvReader(reader))
{
return csvReader.GetRecords<SurveyEmailListModels>().ToArray();
}
}
}
These are the objects I am trying to map to:
public class SurveyEmailListModels
{
[Key]
[CsvField(Ignore = true)]
public int SurveyEmailListId { get; set; }
[CsvField(Index = 0)]
public int ProgramId { get; set; }
[CsvField(Index = 1)]
public virtual SurveyProgramModels SurveyProgramModels { get; set; }
[CsvField(Index = 2)]
public string SurveyEmailAddress { get; set; }
[CsvField(Index = 3)]
public bool SurveyResponded { get; set; }
}
Inside the Visual Studio debugger I am getting an error:
base {"You must call read on the reader before accessing its data."} CsvHelper.CsvHelperException {CsvHelper.CsvReaderException}
Not used the plugin but the error message seems pretty clear. There must be a Read() function to call before accessing the results. Try changing your code to something like this:
using (var reader = new StreamReader(file.InputStream))
using (var csvReader = new CsvReader(reader))
{
// Use While(csvReader.Read()); if you want to read all the rows in the records)
csvReader.Read();
return csvReader.GetRecords<SurveyEmailListModels>().ToArray();
}
I had similar issue , but sorted out, when I tried the below code
void Main()
{
using (var reader = new StreamReader("path\\to\\file.csv"))
using (var csv = new CsvReader(reader,
System.Globalization.CultureInfo.CreateSpecificCulture("enUS")))
{
var records = csv.GetRecords<Foo>();
}
}
Please note below code wont work with latest versions of CSV Helper
using (var csvReader = new CsvReader(reader))