How to handle non-standard Json dates - c#

We've got a SQL table of data that is populated by clients for later processing by a background service. This works most of the time, but it appears that a certain subset of clients has been sending us an unexpected date format, which is resulting in JSON parsing errors. We're expecting dates of the format:
2018-01-20T13:42:43.1040000Z
but what we're getting from these clients is:
2018-01-20T3:42:43.1040000 p. m.Z
First prize would be to find a standard JSON parser or format string that knows how to convert these date strings properly. Second prize would be to write an update script for SQL that converts these strings into something usable. Otherwise, we're going to be writing scripts to parse these dates as strings and then massage them until they're valid, which is nasty.
Anyone know any standard parsers, or can suggest a date format string that will be able to interpret these weird date strings correctly?

You can set the DateFormatString of the JsonSerializerSettings object to customise the date formatting. However, as you also have a peculiar AM/PM designator, you also need to set the culture. You could create your own but for these purposes we can steal from the Catalan culture as that seems to match. Given that, we can use a format string like this (see here to understand what each part means):
yyyy-MM-dd'T'h:m:s.fffffff tt'Z'
So for example, this should work.
var catalan = System.Globalization.CultureInfo.CreateSpecificCulture("ca-FR");
JsonSerializerSettings settings = new JsonSerializerSettings
{
DateFormatString = "yyyy-MM-dd'T'h:m:s.fffffff tt'Z'",
Culture = catalan
};
var result = JsonConvert.DeserializeObject<YourObject>(json, settings);

Related

JSON: DateTimes deserialized format inconsistent [duplicate]

I've seen so many different standards for the JSON date format:
"\"\\/Date(1335205592410)\\/\"" .NET JavaScriptSerializer
"\"\\/Date(1335205592410-0500)\\/\"" .NET DataContractJsonSerializer
"2012-04-23T18:25:43.511Z" JavaScript built-in JSON object
"2012-04-21T18:25:43-05:00" ISO 8601
Which one is the right one? Or best? Is there any sort of standard on this?
JSON itself does not specify how dates should be represented, but JavaScript does.
You should use the format emitted by Date's toJSON method:
2012-04-23T18:25:43.511Z
Here's why:
It's human readable but also succinct
It sorts correctly
It includes fractional seconds, which can help re-establish chronology
It conforms to ISO 8601
ISO 8601 has been well-established internationally for more than a decade
ISO 8601 is endorsed by W3C, RFC3339, and XKCD
That being said, every date library ever written can understand "milliseconds since 1970". So for easy portability, ThiefMaster is right.
JSON does not know anything about dates. What .NET does is a non-standard hack/extension.
I would use a format that can be easily converted to a Date object in JavaScript, i.e. one that can be passed to new Date(...). The easiest and probably most portable format is the timestamp containing milliseconds since 1970.
There is no right format; The JSON specification does not specify a format for exchanging dates which is why there are so many different ways to do it.
The best format is arguably a date represented in ISO 8601 format (see Wikipedia); it is a well known and widely used format and can be handled across many different languages, making it very well suited for interoperability. If you have control over the generated json, for example, you provide data to other systems in json format, choosing 8601 as the date interchange format is a good choice.
If you do not have control over the generated json, for example, you are the consumer of json from several different existing systems, the best way of handling this is to have a date parsing utility function to handle the different formats expected.
When in doubt simply go to the javascript web console of a modern browser by pressing F12 (Ctrl+Shift+K in Firefox) and write the following:
new Date().toISOString()
Will output:
"2019-07-04T13:33:03.969Z"
Ta-da!!
From RFC 7493 (The I-JSON Message Format ):
I-JSON stands for either Internet JSON or Interoperable JSON, depending on who you ask.
Protocols often contain data items that are designed to contain
timestamps or time durations. It is RECOMMENDED that all such data
items be expressed as string values in ISO 8601 format, as specified
in RFC 3339, with the additional restrictions that uppercase rather
than lowercase letters be used, that the timezone be included not
defaulted, and that optional trailing seconds be included even when
their value is "00". It is also RECOMMENDED that all data items
containing time durations conform to the "duration" production in
Appendix A of RFC 3339, with the same additional restrictions.
Just for reference I've seen this format used:
Date.UTC(2017,2,22)
It works with JSONP which is supported by the $.getJSON() function. Not sure I would go so far as to recommend this approach... just throwing it out there as a possibility because people are doing it this way.
FWIW: Never use seconds since epoch in a communication protocol, nor milliseconds since epoch, because these are fraught with danger thanks to the randomized implementation of leap seconds (you have no idea whether sender and receiver both properly implement UTC leap seconds).
Kind of a pet hate, but many people believe that UTC is just the new name for GMT -- wrong! If your system does not implement leap seconds then you are using GMT (often called UTC despite being incorrect). If you do fully implement leap seconds you really are using UTC. Future leap seconds cannot be known; they get published by the IERS as necessary and require constant updates. If you are running a system that attempts to implement leap seconds but contains and out-of-date reference table (more common than you might think) then you have neither GMT, nor UTC, you have a wonky system pretending to be UTC.
These date counters are only compatible when expressed in a broken down format (y, m, d, etc). They are NEVER compatible in an epoch format. Keep that in mind.
"2014-01-01T23:28:56.782Z"
The date is represented in a standard and sortable format that represents a UTC time (indicated by the Z). ISO 8601 also supports time zones by replacing the Z with + or – value for the timezone offset:
"2014-02-01T09:28:56.321-10:00"
There are other variations of the timezone encoding in the ISO 8601 spec, but the –10:00 format is the only TZ format that current JSON parsers support. In general it’s best to use the UTC based format (Z) unless you have a specific need for figuring out the time zone in which the date was produced (possible only in server side generation).
NB:
var date = new Date();
console.log(date); // Wed Jan 01 2014 13:28:56 GMT-
1000 (Hawaiian Standard Time)
var json = JSON.stringify(date);
console.log(json); // "2014-01-01T23:28:56.782Z"
To tell you that's the preferred way even though JavaScript doesn't have a standard format for it
// JSON encoded date
var json = "\"2014-01-01T23:28:56.782Z\"";
var dateStr = JSON.parse(json);
console.log(dateStr); // 2014-01-01T23:28:56.782Z
JSON itself has no date format, it does not care how anyone stores dates. However, since this question is tagged with javascript, I assume you want to know how to store javascript dates in JSON. You can just pass in a date to the JSON.stringify method, and it will use Date.prototype.toJSON by default, which in turns uses Date.prototype.toISOString (MDN on Date.toJSON):
const json = JSON.stringify(new Date());
const parsed = JSON.parse(json); //2015-10-26T07:46:36.611Z
const date = new Date(parsed); // Back to date object
I also found it useful to use the reviver parameter of JSON.parse (MDN on JSON.parse) to automatically convert ISO strings back to javascript dates whenever I read JSON strings.
const isoDatePattern = new RegExp(/\d{4}-[01]\d-[0-3]\dT[0-2]\d:[0-5]\d:[0-5]\d\.\d+([+-][0-2]\d:[0-5]\d|Z)/);
const obj = {
a: 'foo',
b: new Date(1500000000000) // Fri Jul 14 2017, etc...
}
const json = JSON.stringify(obj);
// Convert back, use reviver function:
const parsed = JSON.parse(json, (key, value) => {
if (typeof value === 'string' && value.match(isoDatePattern)){
return new Date(value); // isostring, so cast to js date
}
return value; // leave any other value as-is
});
console.log(parsed.b); // // Fri Jul 14 2017, etc...
The prefered way is using 2018-04-23T18:25:43.511Z...
The picture below shows why this is the prefered way:
So as you see Date has a native Method toJSON, which return in this format and can be easily converted to Date again...
In Sharepoint 2013, getting data in JSON there is no format to convert date into date only format, because in that date should be in ISO format
yourDate.substring(0,10)
This may be helpful for you
I believe that the best format for universal interoperability is not the ISO-8601 string, but rather the format used by EJSON:
{ "myDateField": { "$date" : <ms-since-epoch> } }
As described here: https://docs.meteor.com/api/ejson.html
Benefits
Parsing performance: If you store dates as ISO-8601 strings, this is great if you are expecting a date value under that particular field, but if you have a system which must determine value types without context, you're parsing every string for a date format.
No Need for Date Validation: You need not worry about validation and verification of the date. Even if a string matches ISO-8601 format, it may not be a real date; this can never happen with an EJSON date.
Unambiguous Type Declaration: as far as generic data systems go, if you wanted to store an ISO string as a string in one case, and a real system date in another, generic systems adopting the ISO-8601 string format will not allow this, mechanically (without escape tricks or similar awful solutions).
Conclusion
I understand that a human-readable format (ISO-8601 string) is helpful and more convenient for 80% of use cases, and indeed no-one should ever be told not to store their dates as ISO-8601 strings if that's what their applications understand, but for a universally accepted transport format which should guarantee certain values to for sure be dates, how can we allow for ambiguity and need for so much validation?
it is work for me with parse Server
{
"ContractID": "203-17-DC0101-00003-10011",
"Supplier":"Sample Co., Ltd",
"Value":12345.80,
"Curency":"USD",
"StartDate": {
"__type": "Date",
"iso": "2017-08-22T06:11:00.000Z"
}
}
There is only one correct answer to this and most systems get it wrong. Number of milliseconds since epoch, aka a 64 bit integer. Time Zone is a UI concern and has no business in the app layer or db layer. Why does your db care what time zone something is, when you know it's going to store it as a 64 bit integer then do the transformation calculations.
Strip out the extraneous bits and just treat dates as numbers up to the UI. You can use simple arithmetic operators to do queries and logic.
The following code has worked for me. This code will print date in DD-MM-YYYY format.
DateValue=DateValue.substring(6,8)+"-"+DateValue.substring(4,6)+"-"+DateValue.substring(0,4);
else, you can also use:
DateValue=DateValue.substring(0,4)+"-"+DateValue.substring(4,6)+"-"+DateValue.substring(6,8);
I think that really depends on the use case. In many cases it might be more beneficial to use a proper object model (instead of rendering the date to a string), like so:
{
"person" :
{
"name" : {
"first": "Tom",
"middle": "M",
...
}
"dob" : {
"year": 2012,
"month": 4,
"day": 23,
"hour": 18,
"minute": 25,
"second": 43,
"timeZone": "America/New_York"
}
}
}
Admittedly this is more verbose than RFC 3339 but:
it's human readable as well
it implements a proper object model (as in OOP, as far as JSON permits it)
it supports time zones (not just the UTC offset at the given date and time)
it can support smaller units like milliseconds, nanoseconds, ... or simply fractional seconds
it doesn't require a separate parsing step (to parse the date-time string), the JSON parser will do everything for you
easy creation with any date-time framework or implementation in any language
can easily be extended to support other calendar scales (Hebrew, Chinese, Islamic ...) and eras (AD, BC, ...)
it's year 10000 safe ;-) (RFC 3339 isn't)
supports all-day dates and floating times (Javascript's Date.toJSON() doesn't)
I don't think that correct sorting (as noted by funroll for RFC 3339) is a feature that's really needed when serializing a date to JSON. Also that's only true for date-times having the same time zone offset.

Why date is displayed differently at server and locally?

I am reading data from an API file, which has this format
<DataPoint>
<Amount>38.361</Amount>
<Time>2014-01-02T12:00:00</Time>
</DataPoint>
when I get the time, and print it at my website at visual studio I get 02/01/2014.
However, If I upload it, I get 1/2/2014.
Why the same code produces different results when I use it at my pc, and when I upload it at the server?
And how I can fix that?
ps: I am programming in C# and I am using the object JArray to get the data if this is important
JArray a = JArray.Parse(text);
But it
That's because of the DateTime CULTURE info. British dates are arranged as dd/MM/yyyy and American dates are MM/dd/yyyy You could just use Datetime.ToString("dd/MM/yyyy") to convert it to datetime format you want to display.
Format in which numeric and datetime values are transformed into string depends on OS culture settings, when you don't set them explicitly in code. Looks like that's the case here.
If you need the same date/numeric format everywhere, no matter how user set's the OS preferences, you should provide IFormatProvider instance which will provide the formatting and override OS default one.
The most commonly used is CultureInfo.InvariantCulture:
var dateString = myDate.ToString(CultureInfo.InvariantCulture);

get the format of a string date in c#

I have a string date like 'Wednesday, May 15, 2013' when I Parse it o lost the original format, is there a way to know what was the original format date or get it before the final Parse?
No, you'll lose the original format, if will now be a different object type with no care for its original format prior to parsing.
Your best option would be to store the pre-parse format prior to parsing as a different variable.
However if you simply wish to format the date in that original format, see Farhad's answer.
You can use this for getting date in a fromat you wish.
String.Format("{0:D}", DateTime.Now); // Tuseday, May 21, 2013
The DateTime struct does not have a format, it stores all of those values on various properties. When you want to display it you specify which format to use.
Using a format specifier when displaying it will likely do what you want. This mdsn article provides some basic information on format specifiers. The only way you can use the exact format you show there is if you know it ahead of time and have a format specifier for it. If you know you'll want to display strings in that format throughout the program it will be easy, if you get many different formats and want to decide how to display your dt at runtime it will be fairly complicated. I'm sure you could write some code to figure out what it is, but once the DateTime is created it will have no notion of what format the string used to create it was in.

Datetime formats and JSON data in C#

I'm having an issue with date/time formats in ASP.NET/C#. I have my SQL Server database set up with a PostDate field set to a type of "datetime". But it's saving the date in a strange format. I added a new row through a form and I got this as the date/time string:
2012-09-28 14:56:48.910
When it gets parsed by JSON.NET it gets even stranger. I get:
2012-09-28T14:56:48.91
The date and time are obviously correct, but how do I set things so that I can parse the date into a human-friendly way? There isn't really any code to post because the date is being added when the row is inserted. I'd like to format this as "Sept. 28, 2012 2:56 pm". How do I do that? Do I need to format the string before or after it's parsed as JSON?
That's not a "strange" format at all. The second form is ISO-8601; the first is ISO-8601 without the T. Considering the strange formats you can get in JSON, it looks like you've been let off pretty lightly!
Serialization formats aren't meant to be user-friendly, particularly - they're meant to be machine-to-machine formats.
I would hope that JSON.NET would give you a DateTime after parsing; it should only be giving you the ISO-8601 format after you've converted back to JSON.
If you've got a DateTime that you want to format for user consumption, there are all kinds of options with standard and custom format strings. Don't forget that you should respect the culture of the user, as far as possible - so make sure you're taking appropriate steps to either set the thread's current culture to be the user's one, or that you're passing the culture explicitly to DateTime.ToString etc.
You can try it in C#:
.ToString("MMM d yyyy, h:mm tt")

DateTime problem when day <= 12

I've looked around a lot and short of writing a horrible chunk of code to manipulate the string, I'd like to ask if anyone knows a nice way of sorting it:
I have a bunch of date strings in cells that I'm pulling out such as:
03/05/2011
27/05/2011
31/05/2011
03/05/2011
09/05/2011
31/05/2011
etc.
While I'm reading any entires where the day can be construed as a month - i.e. entries 1, 4 and 5 above - it gets put in as a DateTime with the day and month swapped.
For example, 03/05/2011 gets read in as a DateTime "05/03/2011 00:00:00"
The others are all read and nicely provide me with a simple string of "27/05/2011".
I'm getting this info from Excel, using
((Excel.Range)worksheet.Cells[rowCount, 3]).Value.ToString()
If I try Value2 as with my other lines, it reads those odd dates as things like "40607" but again, will read the other dates normally.
If you use the DateTime.ParseExact function to convert a string to a DateTime object, you can specify the specific format used by your dates (which looks like "day/month/year") without having to do any string manipulation whatsoever.
Example:
var dateString = "03/05/2011";
var format = "dd/MM/yyyy";
var date = DateTime.ParseExact(dateString, format, CultureInfo.InvariantCulture);
More information on custom Date and Time format strings can be found here.
EDIT: Try using the DateTime.FromOADate method to convert the value returned by the Range.Value2 property to a DateTime object, e.g. something like this:
var dateTime = DateTime.FromOADate(((Excel.Range)worksheet.Cells[rowCount, 3]).Value2);
DateTime.ParseExact Method converts the specified string representation of a date and time to its DateTime equivalent using the specified format and culture-specific format information.
The format of the string representation must match the specified format exactly.
String dateString = "15/06/2008";
String format = "dd/MM/yyyy";
DateTime result =
DateTime.ParseExact(dateString, format, CultureInfo.InvariantCulture);
That sounds like a localization problem. Try setting your locale implicititly. For example in WPF application it's something like:
System.Threading.Thread.CurrentThread.CurrentCulture =
new System.Globalization.CultureInfo("en-US");
I have a bunch of date strings in cells that I'm pulling out such as:
No, you don't. You have a mix of strings that look like dates and dates that look like strings. This is an Excel issue, not a C# issue.
Not sure if you are creating the spreadsheet, or if you are getting it from somewhere else. But it the problem is that Excel attempt to parse text as it is entered in the cell. In this case, it is making some wrong decisions about the dates it finds.
If you enter a date like "03/05/2011", Excel will (incorrectly) parse it as March 5th, 2011, and store that as a numeric date code (40607). It then applies a date formatting to the cell (it uses m/d/yyyy on my machine).
If you enter a date like "31/05/2011", Excel can't parse it as a date, and it stores it as text.
To prove this, select the cells and go to Edit > Clear > Formats. All the "bad dates" will just show as numbers, all the rest will stay looking like dates.
You have a few choices:
Fix the data before its entered into Excel (prepend everything with a ' so its all entered as text, or make sure to create the spreadsheet on a machine that has the right date settings.)
Don't use the .Value.ToString() from Excel, just use .Text. This will ignore the bad parsing that Excel did, and should give you a consistent text value (from both types) that you can ParseExact with C#, per the other answers.
(2) is a lot easier, and if the spreadsheets already exist, may be your only choice.
The problem is because your Dates are being read as american culture or similar.
If you use the following you can specify the format you expect your dates to be in:use
DateTime result;
if(DateTime.TryParseExact("dd/MM/yyyy", out result))
{
// Got an English date
}

Categories

Resources