Running CSVHelper 7.0.0 and trying to add a custom string convertor that can be applied to specific class map fields (do not want to applied globally to all fields of type string). Below are snippets on how I currently have my class map, custom convertor, and csv writter calls setup.
Class Map code snippet with custom convertor on NextReviewDate map field:
public sealed class MyCustomClassMap : ClassMap<MyCustomClass>
{
public MyCustomClassMap()
{
Map(m => m.ContentId).Index(0);
Map(m => m.Name).Index(1);
Map(m => m.ContentOwner).Index(2);
Map(m => m.ContentOwnerName).Index(3);
Map(m => m.CopyrightOwner).Index(4);
Map(m => m.CopyrightOwnerName).Index(5);
Map(m => m.NextReviewDate).Index(6).TypeConverter<DateTimeStringConverter>();
Map(m => m.ContentStatus).Index(7);
Map(m => m.UsageRights).Index(8);
Map(m => m.SchemaName).Index(9);
}
}
Custom string converter code snippet:
public class DateTimeStringConverter : StringConverter
{
public override object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
{
string formattedDateString = string.Empty;
if (DateTime.TryParse(text, out DateTime dateobj))
{
formattedDateString = dateobj.ToString("MM-dd-yyyy");
}
//throw new Exception("DateTimeStringConverter value: " + formattedDateString);
return formattedDateString;
}
}
Snippet of code of how I am registering my class map and write records:
csv.Configuration.RegisterClassMap<MyCustomClassMap>();
csv.WriteRecords(results);
To troubleshoot I added a throw exception in DateTimeStringConverter and appears it never gets called. Am I missing a piece? Right now the CSV is generating and includes the original NextReviewDate map field value without ever calling the custom convertor.
EDIT: based on #Self feedback changing custom string converter to the following resolved issue:
public class DateTimeStringConverter : DefaultTypeConverter
{
public override string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
{
string strVal = (string)value;
if (DateTime.TryParse(strVal, out DateTime dateobj))
{
strVal = dateobj.ToString("MM-dd-yyyy");
}
return strVal;
}
}
CSV Helper 26.1.0
First StringConverter offers only one method to overwrite object ConvertFromString(..).
The converstion to string is handled by nothing because it's suppose to be a string.
Here I supposse that your Type is DateTime and you got it in multiple Exotique format. If you have only one format you can change the default format for that type.
A simple demo class and it's mapping:
public class Test
{
public int Id { get; set; }
public DateTime DateTime { get; set; }
public DateTime Date { get; set; }
public DateTime Time { get; set; }
}
public sealed class TestMap : ClassMap<Test>
{
public TestMap()
{
AutoMap(CultureInfo.InvariantCulture);
Map(x => x.Date).TypeConverter(new DateStringConverter("MM - dd - yyyy"));
Map(x => x.Time).TypeConverter(new DateStringConverter("mm # hh # ss"));
}
}
I used a converter that inherit from ITypeConverter in order to have both ConvertFromString and ConvertToString.
With Customisable Format, culture, and style.
public class DateStringConverter : ITypeConverter
{
private readonly string _dateFormat;
private readonly CultureInfo _CultureInfo;
private readonly DateTimeStyles _DateTimeStyles;
public DateStringConverter(string dateFormat) :
this(dateFormat, CultureInfo.InvariantCulture, DateTimeStyles.None)
{ }
public DateStringConverter(string dateFormat, CultureInfo cultureInfo, DateTimeStyles dateTimeStyles)
{
_dateFormat = dateFormat;
_CultureInfo = cultureInfo;
_DateTimeStyles = dateTimeStyles;
}
public object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
{
string formattedDateString = string.Empty;
if (DateTime.TryParseExact(text, _dateFormat, _CultureInfo, _DateTimeStyles, out DateTime dateObj))
{
return dateObj;
}
return null;
}
public string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
{
if (value == null) return string.Empty;
if (DateTime.TryParse(value.ToString(), out DateTime dt))
return dt.ToString(_dateFormat);
else
return string.Empty;
}
}
Writing a CSV:
using (var writer = new StringWriter())
using (var csvWriter = new CsvWriter(writer, CultureInfo.InvariantCulture, true))
{
csvWriter.Context.RegisterClassMap<TestMap>();
csvWriter.WriteRecords(datas);
csvWriter.Flush();
csvTextOuput = writer.ToString();
}
Result:
Id,DateTime,Date,Time
1,04/14/2021 09:18:02,04 - 14 - 2021,18 # 09 # 02
2,04/15/2021 09:18:02,04 - 15 - 2021,18 # 09 # 02
3,04/16/2021 12:18:02,04 - 16 - 2021,18 # 12 # 02
Reading a CSV:
using (var reader = new StringReader(csvTextOuput))
using (var csvReader = new CsvReader(reader, CultureInfo.InvariantCulture, true))
{
csvReader.Context.RegisterClassMap<TestMap>();
ObjectFromCSV = csvReader.GetRecords<Test>().ToArray();
}
Result:
[
{
Date : 04/14/2021
DateTime : 04/14/2021
Id : 1
Time : 04/14/2021
},
{
Date : 04/15/2021
DateTime : 04/15/2021
Id : 2
Time : 04/14/2021
},
{
Date : 04/16/2021
DateTime : 04/16/2021
Id : 3
Time : 04/14/2021
}
]
Live demo https://dotnetfiddle.net/EMdhtn
CSV Helper 7
https://dotnetfiddle.net/5DgwxY
The only modification should be the absence of culture in the reader/writer ctor. And RegisterClassMap that moved from Configuration to Context
~new CsvReader(reader, CultureInfo.InvariantCulture, true))~ => new CsvReader(reader))
~csvWriter.Context.RegisterClassMap()~ => csvWriter.Configuration.RegisterClassMap();
Homogenous datetime format accros all property.
In case youhave the same format everywhere you those proposed solution:
CsvHelper changing how dates and times output
N.B:TypeConverterFactory or TypeConverterCache on older version.
Related
I am not able to set up proper DateTime formatting for the given csv format file. I tried different approaches but this one seems to me to be the closest to the truth. How can I set up this to make it work?
public class Parser
{
public static List<Order> ParseCsv()
{
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
Delimiter = ";",
HasHeaderRecord = true,
TrimOptions = TrimOptions.Trim,
MissingFieldFound = null
};
using (var reader = new StringReader("'Purchase Date'\r\n'2023-02-14T12:03:40Z'"))
using (var csv = new CsvReader(reader, config))
{
csv.Context.RegisterClassMap<PurchaseMap>();
return csv.GetRecords<Order>().ToList();
}
}
}
public class PurchaseMap : ClassMap<Order>
{
public PurchaseMap()
{
Map(m => m.PurchaseDate).Name("'Purchase Date'").TypeConverterOption.Format("yyyy-MM-ddTHH:mm:ss");
}
}
public class Order
{
public DateTime PurchaseDate { get; set; }
}
The error which I got:
CsvHelper.TypeConversion.TypeConverterException: 'The conversion
cannot be performed.
Text: ''2023-02-14T12:03:40Z''
MemberName: Purchase Date
MemberType: System.DateTime
TypeConverter: 'CsvHelper.TypeConversion.DateTimeConverter'
Adding to #Jaryn's answer. You can still use TypeConverterOption.Format. Having the specific format string is the key.
public class PurchaseMap : ClassMap<Order>
{
public PurchaseMap()
{
Map(m => m.PurchaseDate).Name("'Purchase Date'").TypeConverterOption.Format("\\'yyyy-MM-ddTHH:mm:ssZ\\'");
}
}
Your CSV string is not a usual one, i.e. the values are enclosed by single quote.
Not familiar with that library, I've tried setting Escape = '\'' in CsvConfiguration but it does not work.
By eliminating single quote and change this line your code works:
Map(m => m.PurchasDate).Name("Purchas Date").TypeConverterOption.DateTimeStyles(DateTimeStyles.AdjustToUniversal);
The answer is:
Map(m => m.PurchaseDate).Convert(s =>
DateTime.ParseExact(s.Row.Parser.Record[0], "\\'yyyy-MM-ddTHH:mm:ssZ\\'", null));
Convert gives the possibility to do the parsing. Records[0] says which parsing 'column' it tries to tackle.
I'm writing my ClassMap which works for my first basic fields (including those with column name not matching class member).
But I have 2 fields which need a particular work
1) I have a color stored as string. I need some code which convert the input to 2 values and store each one in a specific member.
2) I have an ID which match CSV item ID (that's the father or mother ID). But I need to convert it to the ID in my database (so I have to write some code to match CSV_ID to DB_ID).
Is it possible to add this custom logic with CSVHelper ?
Thanks for help.
Vincent
As it seems, CSVHelper also supports the same type converter injection during class mapping.
https://joshclose.github.io/CsvHelper/examples/configuration/class-maps/type-conversion
By combining this with the support for mapping by alternate names,
https://joshclose.github.io/CsvHelper/examples/configuration/class-maps/mapping-by-alternate-names
Having a csv file like this:
Id,Name,Color
1,OGUZ OZGUL,#f0f0f0
2,VINCENT,#80A0C0
3,OZGUL OGUZ,#00A000
it is possible to achieve what's needed as follows:
using System;
using System.Globalization;
using System.IO;
using CsvHelper;
using CsvHelper.Configuration;
using CsvHelper.TypeConversion;
using System.Linq;
namespace console
{
public class Program
{
public class Foo
{
// Represents the database Id
public int Id { get; set; }
public string Name { get; set; }
// Represents a three character color code, like #FFF
public string Color3 { get; set; }
// Represents a six character color code like #FFFFFF
public string Color6 { get; set; }
}
// OK, we are not converting between types here, but who cares?
// CSVHelper certainly doesn't.
public class IdConverter : DefaultTypeConverter
{
public override object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
{
string csvId = text;
int databaseId = Convert.ToInt32(text) + 10000;
return databaseId;
}
public override string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
{
return ((int)value - 10000).ToString();
}
}
// Again, we are changing the value as we wish, not the type.
public class Color3Converter : DefaultTypeConverter
{
public override object ConvertFromString(string text, IReaderRow row, MemberMapData memberMapData)
{
// format: #ffffff
return "#" + text[1] + text[3] + text[5];
}
public override string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)
{
// format: #fff
return "#" + ((string)value)[1] + "0" + ((string)value)[2] + "0" + ((string)value)[3] + "0";
}
}
// By combining a type converter and alternative name
// we achieve one CSV field value to be mapped to two properties
// of our class Foo
public sealed class FooMap : ClassMap<Foo>
{
public FooMap()
{
Map(m => m.Id).TypeConverter<IdConverter>();
Map(m => m.Name);
Map(m => m.Color3).TypeConverter<Color3Converter>().Name("Color");
Map(m => m.Color6).Name("Color");
}
}
static void Main(string[] args)
{
using (var reader = new StreamReader("data.csv"))
{
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
csv.Configuration.RegisterClassMap<FooMap>();
Foo[] records = csv.GetRecords<Foo>().ToArray();
foreach(Foo record in records)
{
Console.WriteLine
(
"Foo, Id: {0}, Name: {1}, Color3: {2}, Color6: {3}",
record.Id,
record.Name,
record.Color3,
record.Color6
);
}
}
}
}
}
}
The output of the program is:
Foo, Id: 10001, Name: OGUZ OZGUL, Color3: #fff, Color6: #f0f0f0
Foo, Id: 10002, Name: VINCENT, Color3: #8AC, Color6: #80A0C0
Foo, Id: 10003, Name: OZGUL OGUZ, Color3: #0A0, Color6: #00A000
I have a .json like this:
[
{
"number":"00000001",
"dt_doc":"2019-09-26T17:39.000Z",
"address":"complete address"
}
]
But I've got problem with the field dt_doc, this is my deserialization code...
I have this in the main:
public override void CreateNewOutputRows()
{
String jsonFileContent = File.ReadAllText(Variables.JsonFilePath);
JavaScriptSerializer js = new JavaScriptSerializer();
List<Testata> testata = js.Deserialize<List<Testata>>(jsonFileContent);
foreach(Testata test in testata)
{
Output0Buffer.AddRow();
Output0Buffer.number= test.number;
Output0Buffer.dtdoc = test.dt_doc;
Output0Buffer.address= test.address;
}
}
and in my class Testata.cs I have defined the field in this way:
public DateTime dt_doc { get; set; }
But I got an exception on this field, probably related to 8601 standard, is there any way to solve?
This is the exception:
Error: System.FormatException: 2019-09-26T17:39.000Z it's not a valid value for DateTime. ---> System.FormatException: String not recognized as valid DateTime value.
The error is because you are missing seconds in date
"dt_doc":"2019-09-26T17:39.000Z"
should be
"dt_doc":"2019-09-26T17:39.00.000Z"
If this is intentional then you can specify the format. I have tried this using Newtonsoft.Json
public class Testata
{
[JsonConverter(typeof(DateFormatConverter), "yyyy-MM-ddTHH:mm.fffZ")]
public DateTime dt_doc { get; set; }
}
public class DateFormatConverter : IsoDateTimeConverter
{
public DateFormatConverter(string format)
{
DateTimeFormat = format;
}
}
List<Testata> testata = JsonConvert.DeserializeObject<List<Testata>>(jsonString);
you could read it in your class as a string and then:
DateTime.ParseExact(test.dt_doc,"yyyy-MM-ddTHH:mm.fffZ");
I'm trying to enforce a use of an IsoDateTimeConverter on a specific property.
I register the routes and the Content Negotiator:
public class RouteMapper : IServiceRouteMapper
{
public void RegisterRoutes(IMapRoute mapRouteManager)
{
mapRouteManager.MapHttpRoute("SbApi", "default", "{controller}/{action}", new[] { "SbApi.Controllers" });
GlobalConfiguration.Configuration.Services.Replace(typeof(System.Net.Http.Formatting.IContentNegotiator), new JsonContentNegotiator());
}
}
public class JsonContentNegotiator : IContentNegotiator
{
public ContentNegotiationResult Negotiate(Type type, HttpRequestMessage request, IEnumerable<MediaTypeFormatter> formatters)
{
var formatter = new JsonMediaTypeFormatter();
formatter.UseDataContractJsonSerializer = false;
var isoDateConverter = formatter.SerializerSettings.Converters
.OfType<IsoDateTimeConverter>()
.FirstOrDefault();
if(isoDateConverter!=null){
formatter.SerializerSettings.Converters.Remove(isoDateConverter);
}
formatter.SerializerSettings.Converters.Add(new IsoDateTimeConverter() { DateTimeFormat = "dd-MM-yyyy" }); // custom date format, to see if the converter is used
var result = new ContentNegotiationResult(formatter, new MediaTypeHeaderValue("application/json"));
return result;
}
}
Attribute used on the property:
[JsonConverter(typeof(IsoDateTimeConverter))]
public System.DateTime EndDate { get; set; }
The output of the serializer:
"EndDate":"2016-01-01T00:00:00"
How do I make it use the IsoDateTimeConverter?
Edit:
I used a custom IContentNetogiator to enforce use of JSON over XML.
I used the custom datetime format just to see if the converted gets actually used.
What I really need is an ISO format (with the full timezone) but I'm not able to enforce it for some reason. I tried setting it up in the CustomDate, but to no avail.
public CustomDatetimeConverter()
{
base.DateTimeFormat = "yyyy'-'MM'-'dd'T'HH':'mm':'ss.FFFFFFFK"; // none of these work..
base.DateTimeFormat = "o";
base.Culture = System.Globalization.CultureInfo.InvariantCulture;
base.DateTimeStyles = System.Globalization.DateTimeStyles.AssumeLocal;
}
Edit 2:
Anyway, I got it working, sort of. Since nothing was working and I don't need the millisecond part of the time, I ended using the following format: "yyyy'-'MM'-'dd'T'HH':'mm':'ss.000000zzz" .
The "zzz" is used in place of "K" which for some reason doesn't work.
There are a couple of ways to solve this. Neither involves replacing the IContentNegotiator. The first (and easiest) way is to create a subclass of the IsoDateTimeConverter having your custom date format:
class CustomDateTimeConverter : IsoDateTimeConverter
{
public CustomDateTimeConverter()
{
base.DateTimeFormat = "dd-MM-yyyy";
}
}
Then change the [JsonConverter] attribute on your EndDate property:
[JsonConverter(typeof(CustomDateTimeConverter))]
public System.DateTime EndDate { get; set; }
Another way to do the same thing is to create a custom IContractResolver that will apply the IsoDateTimeConverter to your target property programmatically. Below is the code you would need for the resolver. Replace Your_Class with the name of the class containing the EndDate property.
class CustomResolver : DefaultContractResolver
{
protected override IList<JsonProperty> CreateProperties(Type type,
Newtonsoft.Json.MemberSerialization memberSerialization)
{
IList<JsonProperty> props = base.CreateProperties(type, memberSerialization);
if (type == typeof(Your_Class))
{
JsonProperty prop =
props.Where(p => p.PropertyName == "EndDate")
.FirstOrDefault();
if (prop != null)
{
prop.Converter =
new IsoDateTimeConverter { DateTimeFormat = "dd-MM-yyyy" };
}
}
return props;
}
}
To install this resolver in your project, add the following line to the Register method of your WebApiConfig class:
config.Formatters.JsonFormatter.SerializerSettings.ContractResolver =
new CustomResolver();
Is there a way to change how the DataContractJsonSerializer serializes dates?
Currently, it'll convert a date to something like:
{
"date": "/Date(1260597600000-0600)/"
}
I want to convert it into human readable date format.
I am building a RestApi using openrasta framework. Can i write OperationInterceptors which will at some stage before serialization/deserialization convert JSON datetime format to something which is human readable?Or is there any other way to do it?
Use DataContractJsonSerializer constructor to pass your serialization settings:
var s = new DataContractJsonSerializer(
typeof(YourTypeToSerialize),
new DataContractJsonSerializerSettings
{
DateTimeFormat = new DateTimeFormat("yyyy-MM-dd'T'HH:mm:ss")
}
);
Finally i have handled this issue as below(c#)
[DataMember]
public string Date { get; set; }
[IgnoreDataMember]
public DateTime? DateForInternalUse { get; set; }
[OnSerializing]
public void OnSerializing(StreamingContext context)
{
Date = (DateForInternalUse != null) ? ((DateTime)DateForInternalUse).ToString(DateTimeFormatForSerialization) : null;
}
[OnDeserialized]
public void OnDeserialized(StreamingContext context)
{
try
{
DateForInternalUse = !String.IsNullOrEmpty(Date) ? DateTime.ParseExact(Date, DateTimeFormats, null, DateTimeStyles.None) : (DateTime?)null;
}
catch (FormatException)
{
DateForInternalUse = null;
}
}
In this case we can specify the formats which we want to support which i have kept inside web.config
<add key="DateTimePattern" value="yyyy-MM-dd,yyyy-MM-dd hh:mm:ss zzz,yyyy-MM-dd hh:mm:ss" />
Let me know for further clarifications.