DynamoDB Can't Convert to System.DateTime - c#

I am using the C# Object Persistence model in my application, and am populating the table from an external source. I insert a row with the following date (in a String column): 2018-12-12T22:27:14.73Z. This is generated from a Timestamp using the following Go code:
&dynamodb.AttributeValue{S: aws.String(entity.TimeStamp.UTC().Format("2006-01-02T15:04:05.999Z"))}
However, the DynamoDB Object Persistence model chokes when trying to convert it to a System.DateTime, with the following error: System.InvalidOperationException: Unable to convert [2018-12-12T22:27:14.73Z] of type Amazon.DynamoDBv2.DocumentModel.Primitive to System.DateTime
If I let my service write a System.DateTime (using a POCO that contains a DateTime property), it looks something like this: 2018-12-19T07:45:36.431Z. What am I missing that prevents AWS from properly deserializing my dates? It looks like I'm writing them in the same format that Amazon is writing them?

Ok, I found the answer. DynamoDB expects the column to always have 3 digits of precision in the fractional part of the timestamp. When I formatted the time from Golang, I used "2006-01-02T15:04:05.999Z" as my format string, but this causes time.Format to truncate the trailing zeros (when they exist). Changing my time format string to always print the full precision, I was able to fix my issues.
That said, the documentation could be a little more clear about this precision requirement. Or the exception could be a little more explicit. Heres hoping this question/answer makes that exception searchable on Google!

Related

Failed to convert from a String to a TimeSpan error

I am attempting to post to my Database using a SqlCommand like so
queryLogResults.Parameters.Add("#executionTime", SqlDbType.Time).Value = executionTime;
This is the string I am passing in as executionTime
string performanceTime = _stopWatch.Elapsed.ToString("mm':'ss':'fff");
I am assuming it has something to do with the milleseconds because it works only sometimes, and if I post just minutes and seconds it works every time. Am I not using the right type of SqlDbType? In my Database ExecutionTime is a time(7) value.
If you're trying to set a parameter value that's of type SqlDbType.Time, then you shouldn't need to convert it to a string. Just use
TimeSpan performanceTime = _stopWatch.Elapsed;
queryLogResults.Parameters.Add("#executionTime", SqlDbType.Time).Value = performanceTime;
Looks to me like it's the single quotes in format string:
mm':'ss':'fff
Additionally, you need to include Hours as part of the string. Finally, milliseconds are traditionally separated with a decimal point, rather than a colon.
So a better format string looks like this:
HH:mm:ss.fff
Since ADO.Net wants to convert this string value back into a TimeSpan, it needs to be able to the parse the string, and those formatting differences are throwing it off.
Moreover, since ADO.Net will already use a TimeSpan value, I suggest changing the type of the method argument from string to TimeSpan. Then you can pass _stopWatch.Elapsed to the method without converting to a string.
This will also improve performance. Because of the complexity and variety of localization/culture options, converting Date and numeric values to and from strings is surprisingly expensive. You can save your computer some work by avoiding the two conversions in this code.

Serialization: DateTime.Now VS DateTime from database

Context: I'm serializing some data that includes a date of which I need to display milliseconds. I'm using a XmlSerializer to write an XML file. I'm taking some data from a database using a DataTable.
What I'm actually doing: I cast the date in this way
CDate(dr("MyDate")).ToUniversalTime()
But once serialized this does not includes milliseconds.
While trying some workarounds I've found out that serialize in the same way
DateTime.Now.ToUniversalTime()
does includes milliseconds.
Actual question: what should I do to include milliseconds while serializing dr("MyDate")?
Please note that answers in VB.NET or C# are both well accepted.
Have you checked, what CDate contains? For exmaple Definition of CDate shows that CDate contains only seconds, not milliseconds. When the original time in the database is stored in text format (with milliseconds) you should use DateTime.TryParseExact(...) to get the complete time value. Otherwise, when the database itself uses another DateTime-class, you should use this type directly to convert to datetime.

FormatException of DateTime.Parse. Check which part is wrong in my date string

I'd like to validate a date string in my TextBox.
If I have 2018-06-07 string in my TextBox, I can parse it with success with DateTime.TryParse or DateTime.Parse.
But if I have 2018-12-35 in my TextBox, the DateTime.Prase throws FormatException that is absolutely right.
How can I determine which part of DateTime is wrong. For Example how to determine if Day or Month is wrong.
DateTime.Parse tries a number of formats - some to do with the current culture, and some more invariant ones. It looks like it's including an attempt to parse with the ISO-8601 format of "yyyy-MM-dd" - which is valid for your first example, but not for your second. (There aren't 35 days in December.)
As it's trying to parse multiple formats, it doesn't necessarily make sense to isolate which part is "wrong" - different parts could be invalid for different formats.
How you should tackle this depends on where your data comes from. If it's meant to be machine-readable data, it's best to enforce a culture-invariant format (ideally ISO-8601) rather than using DateTime.Parse: specify the format you expect using DateTime.ParseExact or DateTime.TryParseExact. You still won't get information about which part is wrong, but it's at least easier to reason about. (That will also make it easier to know which calendar system is being used.)
If the data is from a user, ideally you'd present them with some form of calendar UI so they don't have to enter the text at all. At that point, only users who are entering the text directly would produce invalid input, and you may well view it as "okay" for them to get a blanket error message of "this value is invalid".
I don't believe .NET provides any indication of where the value was first seen to be invalid, even for a single value. Noda Time provides more information when parsing via the exception, but not in a machine-readable way. (The exception message provides diagnostic information, but that's probably not something to show the user.)
In short: you probably can't do exactly what you want to do here (without writing your own parser or validator) so it's best to try to work out alternative approaches. Writing a general purpose validator for this would be really hard.
If you only need to handle ISO-8601 dates, it's relatively straightforward:
Split the input by dashes. If there aren't three dashes, it's invalid
Parse each piece of the input as an integer. That can fail (on each piece separately)
Validate that the year is one you're happy to handle
Validate that the month is in the range 1-12
Validate that the day is valid for the month (use DateTime.DaysInMonth to help with that)
Each part of that is reasonably straightforward, but what you do with that information will be specific to your application. We don't really have enough context to help write the code without making significant assumptions.

Setting format of DateTime - how do I keep the value in DateTime format?

I have a DateTime being created in the format dd/MM/yyyy HH:mm:ss. I am writing code that interacts with a third-party SOAP library that requires a DateTime variable, in the format yyyy-MM-dd HH:mm:ss.
How do I change the way the information is stored in the DateTime variable, for the purpose of the call to the third-party SOAP library, i.e. no system-wide changes to dates?
I have investigated CultureInfo, which is mildly confusing and possibly too permanent a solution; the only time I need the DateTime changing is for an instance of this single call.
As an explanation, the library has a function GetOrders(DateTime startDate, DateTime endDate, TradingRoleCodeType roleType, OrderStatusCodeType statusType). When attempting to perform the function with DateTimes as created, it generates an error "Sorry, the end date was missing, invalid, or before the start date. must be in YYYY-MM-DD or YYYY-MM-DD HH:MI:SS format, and after the start date.". Given the format that is passed in as dd/MM/yyyy HH:mm:ss, I'd think this may be the problem.
I have a DateTime being created in the format dd/MM/yyyy HH:ii:ss
No, you do not. You have a DateTime. It has no format. It is a number - which is well documented, you know, in the documentation. The string form is never used in a stored DateTime, only when generating the string for presentation.
How do I change the way the information is stored in the DateTime
variable, for the purpose of the call to the third-party SOAP library,
i.e. no system-wide changes to dates?
You do not. I would suggest you talk to your SOAP library - and it is not SOAP btw., IIRC the format you give as example is not valid in SOAP. Yes, bad news. Someone wants Pseudo-Soap.
http://www.w3schools.com/schema/schema_dtypes_date.asp
describes all valid date, time and datetime formats and yours is NOT there.
You can change the default format on a thread level back and forth, so one solution is to set it before calls into the soap library. Another one is to have someone fix the SOAP layer to accept standard formats.
You can create a dummy date :
public class SomeClass
{
[XmlIgnore]
public DateTime SomeDate { get; set; }
[XmlElement("SomeDate")]
public string SomeDateString
{
get { return this.SomeDate.ToString("yyyy-MM-dd HH:mm:ss"); }
set { this.SomeDate = DateTime.Parse(value); }
}
}
Source : Force XmlSerializer to serialize DateTime as 'YYYY-MM-DD hh:mm:ss' --kbrimington
As it turns out, the problem - as some have pointed out - is not to do with the variable being a DateTime, nor its "format" which is not a "format", but is certainly the representation of the information in a method to be understood.
The basic issue with the information was a DateTime comparison between standard time and UTC time. The third-party library examined the DateTime as a UTC DateTime, which when at the right time of year to be caught with a difference in times can cause a problem comparing a DateTime; despite being presented as after the reference time to the user, the time is actually before the reference time when being calculated, meaning the comparison fails.
The main takeaway for this question is to interrogate the information being passed to functions, if you don't have access to third-party library code nor access to documentation with sufficient detail, and errors are occurring when interacting with said third-party code.
Particularly, test various use cases to determine what variable values cause a failure and which cause successful execution of code; identify a pattern, and then test specific use cases that confirm the pattern. From there, determine the actual error that is occurring and code to fix the issue.
In the case of DateTimes, where the code understands DateTimeKinds such as C#, remember to test the different DateTimeKinds to establish whether they can be a part of the problem; its not happened to me often, but it has happened (as evidenced by this question).
Finally, error codes don't help much, and can lead to poor questions and poor advice; trial and error appears to be the best in cases similar to this.
You don't need to change how it's stored, as already mentioned above.
You need to format is as a string according to ISO8601, which is what your SOAP service expects datetime parameter to be.
Check How to parse and generate DateTime objects in ISO 8601 format
and
Given a DateTime object, how do I get an ISO 8601 date in string format?

Prevent TestContext data from being automagically converted

I am parameterising my test cases by using data read from .csv files. One of the columns in the csv file has simple date values (as regular strings) in US format, eg mm/dd/yyyy. When the data is actually read and populated into a TestContext however, TestContext.DataRow["MyDateColumn"] actually returns a converted System.DateTime object, complete with a timestamp of 12:00:00 AM. I absolutely do not require or want this automatic conversion. How do I stop this from happening?
If the type of the MyDateColumn is set to datetime, then it would/should return a datetime object.
Try changing the type of MyDateColumn to be string and see if that does the trick.
UPDATE
Change dates in the CSV so that they are "mm/dd/yyyy" instead of mm/dd/yyyy.
DateTime.Parse(TestContext.DataRow["MyDateColumn"], CultureInfo.InvariantCulture).ToShortDateString()
I got solution for this :)
I just put ' before actual data. When I retrieve data, data comes as is. Before using it, I remove ' from data with substring method.
I remember using this technique on excel to make numbers appear as text. It worked for me.
I put double qoutes around the data. That gives back strings, without removing the quotes.
columnInt, columnString
1, "11.12.89"
2, "12.12.89"

Categories

Resources