Develop Reference

CsvHelper dynamic column mapping

I am following this example to map custom column names to my class model:
CsvHelper Mapping by Name
In this particular part:
public FooMap()
{
Map(m => m.Id).Name("ColumnA");
Map(m => m.Name).Name("ColumnB");
}
Is it possible to use string as column name instead of hard-code it? Something like this --
public FooMap()
{
Map("Col1").Name("ColumnA");
Map("Col2").Name("ColumnB");
}
"Col1" and "Col2" are the property of my class model. I've tried to use reflection but it didn't work:
Map(x => typeof(MyClassModel).GetProperty("Col1")).Name("ColumnA");
Please let me know if what I am trying to achieve is possible. Some additional info -- the column mapping (source and destination) are both stored in a table.
Thanks!

This should allow you to use a string to map both the property name and the header name.
void Main()
{
var mapping = new Dictionary<string, string>
{
{"Id","FooId"},
{"Name","FooName"}
};
using (var reader = new StringReader("FooId,FooName\n1,Jordan"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
var fooMap = new DefaultClassMap<Foo>();
fooMap.Map(mapping);
csv.Context.RegisterClassMap(fooMap);
var records = csv.GetRecords<Foo>().ToList();
}
}
public static class CsvHelperExtensions
{
public static void Map<T>(this ClassMap<T> classMap, IDictionary<string, string> csvMappings)
{
foreach (var mapping in csvMappings)
{
var property = typeof(T).GetProperty(mapping.Key);
if (property == null)
{
throw new ArgumentException($"Class {typeof(T).Name} does not have a property named {mapping.Key}");
}
classMap.Map(typeof(T), property).Name(mapping.Value);
}
}
}
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
}

As an another approach, define an XML/JSON config file where you can define the columns you want to map. Write a parser that could parse your XML/JSON config and return the columns to be mapped dynamically. This approach allows you to map any no of columns on fly, without having to recompile your code.

CsvHelper - How to map by index the whole row to a model member

I'm working on a C# windows service that reads in a csv file into a List using CsvHelper along with it's class map by index functionality. I would like to store the original raw data row in each model.
I've tried using Map(m => m.Row).Index(-1); but that did not work. I also tried ConvertUsing, but I get a message that MemberMap does not contain a definition for 'ConvertUsing'.
The RegisterClassMap and csv.GetRecords functionality is doing a bulk read that doesn't give me an opportunity to capture the original raw data row.
Any help would be greatly appreciated. I need to create an email with the status (sending the data to a micro service) and the original raw data two, and would love to store it while CsvHelper is reading the file.

void Main()
{
var s = new StringBuilder();
s.Append("Id,Name\r\n");
s.Append("1,one\r\n");
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
};
using (var reader = new StringReader(s.ToString()))
using (var csv = new CsvReader(reader, config))
{
csv.Context.RegisterClassMap<FooMap>();
csv.GetRecords<Foo>().ToList().Dump();
}
}
private class Foo
{
public int Id { get; set; }
public string Name { get; set; }
public string RawRow { get; set; }
}
private class FooMap : ClassMap<Foo>
{
public FooMap()
{
Map(m => m.Id);
Map(m => m.Name);
Map(m => m.RawRow).Convert(args => args.Row.Parser.RawRecord);
}
}
.Dump() is a LINQPad method.

Apply CsvHelper custom converter to specific class map field(s)

Running CSVHelper 7.0.0 and trying to add a custom string convertor that can be applied to specific class map fields (do not want to applied globally to all fields of type string). Below are snippets on how I currently have my class map, custom convertor, and csv writter calls setup.
Class Map code snippet with custom convertor on NextReviewDate map field:
public sealed class MyCustomClassMap : ClassMap<MyCustomClass>
{
public MyCustomClassMap()
{
Map(m => m.ContentId).Index(0);
Map(m => m.Name).Index(1);
Map(m => m.ContentOwner).Index(2);
Map(m => m.ContentOwnerName).Index(3);
Map(m => m.CopyrightOwner).Index(4);
Map(m => m.CopyrightOwnerName).Index(5);
Map(m => m.NextReviewDate).Index(6).TypeConverter<DateTimeStringConverter>();
Map(m => m.ContentStatus).Index(7);
Map(m => m.UsageRights).Index(8);
Map(m => m.SchemaName).Index(9);
}
}
Custom string converter code snippet:
public class DateTimeStringConverter : StringConverter
{
public override object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
{
string formattedDateString = string.Empty;
if (DateTime.TryParse(text, out DateTime dateobj))
{
formattedDateString = dateobj.ToString("MM-dd-yyyy");
}
//throw new Exception("DateTimeStringConverter value: " + formattedDateString);
return formattedDateString;
}
}
Snippet of code of how I am registering my class map and write records:
csv.Configuration.RegisterClassMap<MyCustomClassMap>();
csv.WriteRecords(results);
To troubleshoot I added a throw exception in DateTimeStringConverter and appears it never gets called. Am I missing a piece? Right now the CSV is generating and includes the original NextReviewDate map field value without ever calling the custom convertor.
EDIT: based on #Self feedback changing custom string converter to the following resolved issue:
public class DateTimeStringConverter : DefaultTypeConverter
{
public override string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
{
string strVal = (string)value;
if (DateTime.TryParse(strVal, out DateTime dateobj))
{
strVal = dateobj.ToString("MM-dd-yyyy");
}
return strVal;
}
}

CSV Helper 26.1.0
First StringConverter offers only one method to overwrite object ConvertFromString(..).
The converstion to string is handled by nothing because it's suppose to be a string.
Here I supposse that your Type is DateTime and you got it in multiple Exotique format. If you have only one format you can change the default format for that type.
A simple demo class and it's mapping:
public class Test
{
public int Id { get; set; }
public DateTime DateTime { get; set; }
public DateTime Date { get; set; }
public DateTime Time { get; set; }
}
public sealed class TestMap : ClassMap<Test>
{
public TestMap()
{
AutoMap(CultureInfo.InvariantCulture);
Map(x => x.Date).TypeConverter(new DateStringConverter("MM - dd - yyyy"));
Map(x => x.Time).TypeConverter(new DateStringConverter("mm # hh # ss"));
}
}
I used a converter that inherit from ITypeConverter in order to have both ConvertFromString and ConvertToString.
With Customisable Format, culture, and style.
public class DateStringConverter : ITypeConverter
{
private readonly string _dateFormat;
private readonly CultureInfo _CultureInfo;
private readonly DateTimeStyles _DateTimeStyles;
public DateStringConverter(string dateFormat) :
this(dateFormat, CultureInfo.InvariantCulture, DateTimeStyles.None)
{ }
public DateStringConverter(string dateFormat, CultureInfo cultureInfo, DateTimeStyles dateTimeStyles)
{
_dateFormat = dateFormat;
_CultureInfo = cultureInfo;
_DateTimeStyles = dateTimeStyles;
}
public object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
{
string formattedDateString = string.Empty;
if (DateTime.TryParseExact(text, _dateFormat, _CultureInfo, _DateTimeStyles, out DateTime dateObj))
{
return dateObj;
}
return null;
}
public string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
{
if (value == null) return string.Empty;
if (DateTime.TryParse(value.ToString(), out DateTime dt))
return dt.ToString(_dateFormat);
else
return string.Empty;
}
}
Writing a CSV:
using (var writer = new StringWriter())
using (var csvWriter = new CsvWriter(writer, CultureInfo.InvariantCulture, true))
{
csvWriter.Context.RegisterClassMap<TestMap>();
csvWriter.WriteRecords(datas);
csvWriter.Flush();
csvTextOuput = writer.ToString();
}
Result:
Id,DateTime,Date,Time
1,04/14/2021 09:18:02,04 - 14 - 2021,18 # 09 # 02
2,04/15/2021 09:18:02,04 - 15 - 2021,18 # 09 # 02
3,04/16/2021 12:18:02,04 - 16 - 2021,18 # 12 # 02
Reading a CSV:
using (var reader = new StringReader(csvTextOuput))
using (var csvReader = new CsvReader(reader, CultureInfo.InvariantCulture, true))
{
csvReader.Context.RegisterClassMap<TestMap>();
ObjectFromCSV = csvReader.GetRecords<Test>().ToArray();
}
Result:
[
{
Date : 04/14/2021
DateTime : 04/14/2021
Id : 1
Time : 04/14/2021
},
{
Date : 04/15/2021
DateTime : 04/15/2021
Id : 2
Time : 04/14/2021
},
{
Date : 04/16/2021
DateTime : 04/16/2021
Id : 3
Time : 04/14/2021
}
]
Live demo https://dotnetfiddle.net/EMdhtn
CSV Helper 7
https://dotnetfiddle.net/5DgwxY
The only modification should be the absence of culture in the reader/writer ctor. And RegisterClassMap that moved from Configuration to Context
~new CsvReader(reader, CultureInfo.InvariantCulture, true))~ => new CsvReader(reader))
~csvWriter.Context.RegisterClassMap()~ => csvWriter.Configuration.RegisterClassMap();
Homogenous datetime format accros all property.
In case youhave the same format everywhere you those proposed solution:
CsvHelper changing how dates and times output
N.B:TypeConverterFactory or TypeConverterCache on older version.

CsvHelper - ClassMap by index for file with no header errors on read

I'm using csvhelper 18.0 and am trying to update some code that used 2.8.4.
I have a file that I'm trying to read that has no headers.
I've defined a ClassMap to map by index.
I've created the configuration of the CsvReader so HasHeaderRecord = false.
When I try to import this file, I get an error that states There is no header record to determine the index by name. I'm confused as to why an error is being thrown regarding the header record. The header record does not exist, which is why I am mapping with an index.
Would anyone know how I can read a headerless file and still map to a class?
Here is the class and mapping class:
public class TFile
{
public int Wn { get; set; }
public string Hiwn { get; set; }
public string Sync { get; set; }
}
public sealed class TFileMap : ClassMap<TFile>
{
public TFileMap()
{
Map(m => m.Wn).Index(0);
Map(m => m.Hiwn).Index(1);
Map(m => m.Sync).Index(2);
}
}
Here is the piece of code that throughs the error:
using (TextReader textReader = new StringReader(data))
{
var csvT = new CsvReader(textReader, CultureInfo.InvariantCulture);
csvT.Configuration.HasHeaderRecord = false;
csvT.Configuration.RegisterClassMap<TFileMap>();
csvT.Configuration.CultureInfo = new CultureInfo("en-AU");
// error occurs on this line
tData1 = csvT.GetRecords<TFile>().ToList();
}
Here is a small sample file:
37,1R,Y
38,1L,Y
39,2R,Y
40,2L,Y
Any help would be greatly appreciated. Thanks

I pulled CsvHelper version 18.0 and tried with your code and sample data and it worked just fine for me.
var data = #"37,1R,Y
38,1L,Y
39,2R,Y
40,2L,Y";
using (TextReader textReader = new StringReader(data))
{
var csvT = new CsvReader(textReader, CultureInfo.InvariantCulture);
csvT.Configuration.HasHeaderRecord = false;
csvT.Configuration.RegisterClassMap<TFileMap>();
csvT.Configuration.CultureInfo = new CultureInfo("en-AU");
var tData1 = csvT.GetRecords<TFile>().ToList();
tData1.Dump();
}

How to write only selected class fields into CSV with CsvHelper?

I use CsvHelper to read and write CSV files and it is great, yet I don't understand how to write only selected type fields.
Say we had:
using CsvHelper.Configuration;
namespace Project
{
public class DataView
{
[CsvField(Name = "N")]
public string ElementId { get; private set; }
[CsvField(Name = "Quantity")]
public double ResultQuantity { get; private set; }
public DataView(string id, double result)
{
ElementId = id;
ResultQuantity = result;
}
}
}
and we wanted to exclude "Quantity" CsvField from resulting CSV file that we currently generate via something like:
using (var myStream = saveFileDialog1.OpenFile())
{
using (var writer = new CsvWriter(new StreamWriter(myStream)))
{
writer.Configuration.Delimiter = '\t';
writer.WriteHeader(typeof(ResultView));
_researchResults.ForEach(writer.WriteRecord);
}
}
What could I use to dynamically exclude a type field from the CSV?
If it is necessary we could process the resulting file, yet I do not know how to remove an entire CSV column with CsvHelper.

I recently needed to achieve a similar result by determining what fields to include at runtime. This was my approach:
Create a mapping file to map which fields I need at runtime by passing in an enum into the class constructor
public sealed class MyClassMap : CsvClassMap<MyClass>
{
public MyClassMap(ClassType type)
{
switch (type)
{
case ClassType.TypeOdd
Map(m => m.Field1);
Map(m => m.Field3);
Map(m => m.Field5);
break;
case ClassType.TypeEven:
Map(m => m.Field2);
Map(m => m.Field4);
Map(m => m.Field6);
break;
case ClassType.TypeAll:
Map(m => m.Field1);
Map(m => m.Field2);
Map(m => m.Field3);
Map(m => m.Field4);
Map(m => m.Field5);
Map(m => m.Field6);
break;
}
}
}
Write out the records to using the mapping configuration
using (var memoryStream = new MemoryStream())
using (var streamWriter = new StreamWriter(memoryStream))
using (var csvWriter = new CsvWriter(streamWriter))
{
csvWriter.Configuration.RegisterClassMap(new MyClassMap(ClassType.TypeOdd));
csvWriter.WriteRecords(records);
streamWriter.Flush();
return memoryStream.ToArray();
}

Mark the field like this:
[CsvField( Ignore = true )]
public double ResultQuantity { get; private set; }
Update: Nevermind. I see you want to do this at runtime, rather than compile time. I'll leave this up as red flag for anyone else who might make the same mistake.

You can do this:
using (var myStream = saveFileDialog1.OpenFile())
{
using (var writer = new CsvWriter(new StreamWriter(myStream)))
{
writer.Configuration.AttributeMapping(typeof(DataView)); // Creates the CSV property mapping
writer.Configuration.Properties.RemoveAt(1); // Removes the property at the position 1
writer.Configuration.Delimiter = "\t";
writer.WriteHeader(typeof(DataView));
_researchResults.ForEach(writer.WriteRecord);
}
}
We are forcing the creation of the attribute mapping and then modifying it, removing the column dynamically.

I had a similar issue with my code and I fixed it by the following code.
you can do this:
var ignoreQuantity = true;
using (var myStream = saveFileDialog1.OpenFile())
{
using (var writer = new CsvWriter(new StreamWriter(myStream)))
{
var classMap = new DefaultClassMap<DataView>();
classMap.AutoMap();
classMap.Map(m => m.ResultQuantity).Ignore(ignoreQuantity)
writer.Configuration.RegisterClassMap(classMap);
writer.Configuration.Delimiter = "\t";
writer.WriteHeader(typeof(DataView));
_researchResults.ForEach(writer.WriteRecord);
}
}

I had to solve this also: I have a couple dozen record types with a common base class plus a common field that has to be ignored by all of them:
// Nothing special here
internal class MyClassMap<T> : ClassMap<T> where T : MyRecordBaseClass
{
public MyClassMap()
{
AutoMap();
Map( m => m.SOME_FIELD ).Ignore();
}
}
This part is generally well documented and not the dynamic part.
But one class needed special sauce by ignoring a different field dynamically, and though I could have created a separate map class, this didn't scale for what I expect will be a lot more of these, so I finally figured out how to do it properly:
...
// special processing for *one* record type
csvwriter.Configuration.RegisterClassMap<MyClassMap<ONE_RECORD_TYPE>>();
if (ShouldIgnore)
{
var map = csvwriter.Configuration.Maps.Find<ONE_RECORD_TYPE>();
map.Map( m => m.SOME_OTHER_FIELD ).Ignore();
}
...
This worked on CsvHelper versions 7.1.1 and 12.1.1.