Develop Reference

DateTime2 conversion randomly removes space between date and time

I have a small console application which is to import a CSV file into a database. It is in .NET CORE 3.1. The CSV file gets imported without any issues. The issue arises with trying to save the data to the table. The error being received is "String or binary data would be truncated. The statement has been terminated." This is received during the context.SaveChanges() call. To determine exactly what the error is, loaded up Profiler to see the offending statement. The offending piece of code was related to a field that holds a date and time. To start from the beginning and lead up to the issue.
The imported data is in a column and is represented as follows:
"20200404121655500"
"20200404121755500"
The import model property is defined as follows:
public string Date_And_Time { get; set; }
The data model property is defined as follows:
[Column(TypeName = "DATETIME2(7)")]
public DateTime? Date_And_Time { get; set; }
The conversion used to get the imported string to the data model field is as follows:
if (DateTime.TryParseExact(Date_And_Time.Trim()
.PadRight(21, '0')
.Substring(0, 21), "yyyyMMddHHmmssFFFFFFF", CultureInfo.InvariantCulture, DateTimeStyles.None, out DateTime newDateTime))
{ model.Date_And_Time = newDateTime; }
else { model.Date_And_Time = null; }
While debugging when the 2 different dates are processed then are set in the model.Date_And_Time as expected. The object expands producing all the correct information (year, day, hour, etc.). Prior to executing SaveChanges, inspecting the objects shows they both have correct DateTime objects. However upon executing that command throws the above SQL exception. When inspecting the merge command created by EF the dates are differently formatted. They are as follows:
'2020-04-0412:16:55.5000000' and '2020-04-04 12:17:55.5000000'
The first does not have a space between the date and time where as the second has a space. The first is what is causing the exception. When doing the SaveChanges there are on average 20 records being saved and some have the space and some do not there is no pattern that I can find. I have tried using the exact same date time and that also has some with and some without space.

After some trial and errors I switched from DateTime to DateTimeOffset. This was done on the property of the data model propriety, the type attribute of the same property and the TryParseException. No other changes and it ran with no exceptions. Needed to do another migration due to the change in attribute causing the table field tire change.
The only 3rd party package used was to import the CSV for and that went successfully.
The parametrized merge statement was seen in the profiler sorry I didn't City it better finding the answer.
I have no idea why the model with a C# data type of DateTime would produce a string although the parameter itself was designated as DateTime2 the value being passed in was in string format.
I don't understand what the difference is between the 2 data types and how they are handled that would cause this issue. I appreciate the comments and attempting to help.

DocumentDB Query to find created date

I want to retrieve the documents with only documentID and document Created DateTime details. I am able to get the DocumentID by following query but how can we get the Document created datetime.
select c.id from c
However, along with id I need to get the document created date also
Any help on this appreciated !

I am able to get the DocumentID by following query but how can we get
the Document created datetime.
From this official doc , the automatically generated properties of the document when it is created consists only of the following: _rid,_etag,_ts,_self,id.
_ts represents last updated timestamp of the resource, so if this document has not been updated, then this time is the createTime. You could query it via Select c.id,c._ts from c.
So, maybe you need to log createTime by yourself when you create documents so that you could query the createTime later.
Hope it helps you.
The id , _ts or other custom properties like DocumentReadingCreatedDate are all properties in the document which is not divided into system details or non-system details. Just some of the properties I mentioned above are the fields that cosmosdb will automatically generate for you.
So you just need to use sql : select c._ts from c to find the newest update time of this document. You could get a long type data such as 1520216339 ,then you could convert it into datetime format such as Sun Jan 18 22:16:56 CST 1970 in your program.
If your documents have been updated and you must get the create time of the document, you can't use _ts field. You could only use your custom field DocumentReadingCreatedDate.

The _ts is a Unix Timestamp value.
You need to convert it. The following C# sample will do it.
private static DateTime UnixTimeStampToDateTime(long unixTimeStamp)
{
System.DateTime dtDateTime = new System.DateTime(1970, 1, 1, 0, 0, 0, 0);
dtDateTime = dtDateTime.AddMilliseconds(unixTimeStamp);
return dtDateTime;
}
Or you can create a UFD that you can use in your query, The following is one that will work
function epochToDate (ts) {
return new Date(ts*1000);
}
You would use it as follows:
SELECT * FROM c WHERE udf.epochToDate(c._ts) == ....

Specflow step argument transformation on table cell contents with CreateInstance

Has anyone solved the riddle of how to apply SpecFlow Step Argument Transformations to cells in a table, in conjunction with the SpecFlow.Assist CreateInstance/CreateSet? (code combined here to save space)
Given a table like the following:
| Price | Zip | Effective Date |
| 10.00 | 90210 | in 2 days |
When the 'given' step executes
And the table data populates a poco
Then the effective date should be transformed into a DateTime with value of 2 days from today
[Given(#"a table like the following:")]
public void GivenATableLikeTheFollowing(Table table)
{
var temp = table.CreateInstance<Temp>();
}
internal class Temp
{
decimal Price { get; set; }
int Zip { get; set; }
DateTime EffectiveDate { get; set; }
}
[Binding]
public class Transforms
{
[StepArgumentTransformation(#"in (\d+) days?")]
public DateTime InXDaysTransform(int days)
{
return DateTime.Today.AddDays(days);
}
}
StepArgumentTransformation bindings apparently don't apply to table cell contents (since the step's argument is type Table), but somehow the SpecFlow.Assist CreateInstance/CreateSet will still transform cell data for basic types.
For example , if the Effective Date's contents are '11/13/2016' instead of 'in 2 days', the underlying poco's EffectiveDate property transforms to a DateTime just fine (or an int, decimal, etc).
I see some other solutions like applying a conversion within the step definition itself like here or creating a StepArgumentTransformation for the whole table, but... obvious cons. Update: this question is similar, but solutions also avoid mingling StepArgumentTransformation with CreateInstance/CreateSet.
There is also a section in the SpecFlow Assist Helpers docs about extending by registering value retrievers/comparers, but in my example, a DateTime set already exists. So, perhaps a custom DateTime type? It seems like perhaps there could be a check for StepArgumentTransformations on the known types, or something like that.
In the DateTime retriever, something like..
public virtual DateTime GetValue(string value)
{
var returnValue = DateTime.MinValue;
// check for StepArgumentTransformations here first?
DateTime.TryParse(value, out returnValue);
return returnValue;
}
Any ideas on what I am missing to get the StepArgumentTransformation to apply to the table cell contents when using table.CreateInstance? Or is one of the mentioned solutions the best/only way?

I have created a small prototype that can be used to reconfigure Assist to be able to pick up conversions with [StepArgumentTransformation] bindings.
My plan is to make a blog post about it, but until it is ready, maybe you can get out the essence from this gist. (I did it a year ago for SpecFlow v2.0, so some smaller adaptions might be necessary.)
https://gist.github.com/gasparnagy/a478e5b7ccb8f557a6dc

I don't think what you want is implemented currently, but theoretically I think it could be implemented. You can probably implement a new, enhanced DateTimeValueRetriever yourself which checks to see if the string is parseable as a datetime first and if not checks if any of the [StepArgumentTransformation] methods can parse it, and then replace the current DateTimeValueRetriever with your enhanced one. Then you could submit a pr offering your new version as an enhancement to the existing version, and see what the appetite is.

EF:6 - How to skip insertion of value if value has default value set in DB?

I'm fighting trough with Entity Framework 6 and MySQl Database
I got everything to work, however I'm confused with dates or not obligatory values.
In my database, in "Users" table I have column "RegistrationDate" which has default value of "CURRENT_TIMESTAMP"
what is mean that if value not provided at insertion time it will insert default value = date time of the server
I got my schema reverse engineered into C# and all perfectly works, however when I insert "User" without setting a date to "RegistrationDate" property, it insert into Database new date as "0001-01-01 00:00:00" and ignore "CURRENT_TIMESTAMP".
So im wondering how to set it to ignore "RegistrationDate" and do not insert anything into db if it wasn't specifically set to some date?

I have a guess that the SQL EF generates is setting the field value. Even if you don't set in code, EF doesn't know that the database has a default value, and doesn't know that he should ignore it.
This article, from 2011, says that there is a DatabaseGenerated attribute, which you could use like this:
[DatabaseGenerated(DatabaseGenerationOption.Computed)]
public DateTime RegistrationDate { get; set; }
So, EF now knows that it should retrieve the data when you query the database, but should rely on the database to set the value.
However, I don't know what it would do if you explicitly set the value. Maybe it will ignore it, which may be not what you really want.
I didn't test it, it's just a guess, but it's a nice solution in my opinion.
[Edit1] Some months ago, I saw this video, and the guy does something like this in his DbContext class (i believe you have it) at 49:12 (the video is in portuguese)(i have modified the code, but didn't test it):
//This method will be called for every change you do - performance may be a concern
public override int SaveChanges()
{
//Every entity that has a particular property
foreach (var entry in ChangeTracker.Entries().Where(entry => entry.Entity.GetType().GetProperty("YourDateField") != null))
{
if (entry.State == EntityState.Added)
{
var date = entry.Property("YourDateField");
//I guess that if it's 0001-01-01 00:00:00, you want it to be DateTime.Now, right?
//Of course you may want to verify if the value really is a DateTime - but for the sake of brevity, I wont.
if (date.CurrentValue == default(DateTime))
{
date.CurrentValue = DateTime.Now;
}
else //else what?
{
//Well, you don't really want to change this. It's the value you have set. But i'll leave it so you can see that the possibilities are infinite!
}
}
if (entry.State == EntryState.Modified)
{
//If it's modified, maybe you want to do the same thing.
//It's up to you, I would verify if the field has been set (with the default value cheking)
//and if it hasn't been set, I would add this:
date.IsModified = false;
//So EF would ignore it on the update SQL statement.
}
}
}

I think many of us have been caught out by default database values when dealing with EF - it doesn't take them into account (there are many questions on this - e.g. Entity Framework - default values doesn't set in sql server table )
I'd say if you haven't explicitly set a datetime and want it to be null, you'll need to do it in code.

Ignore DATETIME fields in MySqlDataAdapter.Update(DataTable)?

I have an app which worked fine with Sql Server. I have a DevExpress grid which shows just a record in carousel mode (not that this matters, I hope).
Now, I have changed the code to be database-agnostic and I'm testing MySql. When the user modified the record and accepted the changes, I was getting the following error:
Concurrency violation: the UpdateCommand affected 0 of the expected 1
records
After some research, I've come to the conclussion that the problem lies in DATETIME fields. I am using "Allow Zero Datetime=False; Convert Zero Datetime=True;" in my MySql connection string so I can convert default DATETIME values to .Net DateTime objects. The autogenerated UpdateCommand includes every field in the where clause, and I guess the comparison fails when the MySql DATETIMEs are set to the default value, as removing DATETIME fields the problem went away.
I have a Primary Key column, and the user isn't allowed to modify it, so what's the right way to issue a custom UpdateCommand so that there's only one field in the WHERE clause?
My current code for accepting changes:
Dim builder As DbCommandBuilder = m_Conn.CreateCommandBuilder(m_Adapter)
m_Adapter.Update(m_DataTable)
CreateCommandBuilder is an extension method on IDbConnection to create the correct an object with a correct implementatin of the DbCommandBuilder interface.

Your DBCommandBuilder should have a ConflictOption Property that needs to be set.
Presumably you want to set it to ConflictOption.OverwriteChanges.
I'm not sure if it works when you initialize the Adapter commands via the CommandBuilder Constructor but a
var builder = new MySqlCommandBuilder();
builder.ConflictOption = ConflictOption.OverwriteChanges;
builder.DataAdapter = m_Adapter;
should do.

Instead of using "Allow Zero Datetime=False; Convert Zero Datetime=True;" in your connection string (which FYI I'm not familiar with), I'd recommend using DateTime.Parse(value). You'll probably want to write a function so that you can easily handle nulls as well.
private DateTime getDateTimeField(string dbValue)
{
if (dbValue == null)
{
return new DateTime();
}
else {
return DateTime.Parse(dbValue);
}
}