The problem I am trying to solve is pretty simple: I need to make sure my timestamps are stored in the database as UTC. I use DateTime in my model objects, which maps to timespamp column in Postgres DB.
I'd like to do 2 things:
When a value is saved to the database, I want to check the DateTime.Kind field and throw up if it is not UTC. This will help me to capture errors if I try to feed a non-UTC time to a timestamp field. I do not want to do any transformations, just assert.
When loading from DB, I need those DateTime instances come out with DateTime.Kind set to UTC, so the rest of my application can recognize they are UTC and convert to local or whatever, as needed.
I only need to apply this logic to selected timestamp fields, not to all DateTime-s, because I have other DateTime values in my model, which need to be stored as they are.
I've tried this:
public class UtcTimestampMapper: SqlMapper.TypeHandler<DateTime>
{
public override void SetValue(IDbDataParameter parameter, DateTime dt)
{
if (dt.Kind == DateTimeKind.Utc)
parameter.Value = dt;
else
throw new InvalidOperationException("UTC timestamp expected");
}
public override DateTime Parse(object value)
{
return DateTime.SpecifyKind((DateTime)value, DateTimeKind.Utc);
}
}
This solution has 2 issues:
It applies to all DateTime values. With SetValue I can do the trick checking the column name in IDbDataParameter, maybe... But Parse method just gets a naked value, there is nothing I can do to determine where it goes.
The SetValue method won't be called at all, unless I remove the standard handler for DateTime. See this SO answer for details: Dapper TypeHandler.SetValue() not being called Again, if I replace the handler, I replace it globally, there is no way to choose where it applies.
So, is there a way to solve my problem with the facilities Dapper provides to customize mappings?
I'm using the extension below for MongoDB date fields because of MongoDB stores times in UTC by default.
public static class DateTimeExtensions {
public static DateTime AdjustUtcDiffByOffset(this DateTime time) {
return time.AddHours(DateTimeOffset.Now.Offset.Hours);
}
}
}
All of them cause different problems although I have tried a few ways like attributes or serialization methods in .NET on the application level. I have decided to use this extension in .NET for now but I think that this is not a complete and exact solution too.
Is there any solution for this problem on the database level without being dependent on programming language wtih an adjust or something else?
EDIT
I think that I should an explain more after comments below. I already know MongoDB stores times in UTC that linked in this post as you see above. This can be useful but I don't need any UTC time zone difference in my app and I don't want to deal in Presentation Layer for every programming language separately. And also, I don't even want only one extra row or function in the other layers because of move away than base logic or business.
Let the my architecture undertaking this. I'm pretty lazy, the life is really short and the birds are flying outside :) I don't want the different fields as like as string convertings unnecessarily. I need a datetime type in the db due to I'm doing many time calculation in the app.
I'm using .NET now and therefore MongoDB .NET driver. I have tried different serialization methods but it cause another problems in the Data Access architecture. In conclusion, I can use UTC in my another app but I don't want it now and I prefer the local time when I assign to the field. I have decided to use the encapsulation below for C# especially.
private DateTime _startTime;
public DateTime StartTime {
get => _startTime;
set => _startTime = new DateTime(value.Ticks, DateTimeKind.Utc);
}
I don't think it's possible from the db level. What I did, was to write custom setter for date properties which will force mongoDB to assume the time is already in UTC, thus avoid a conversion like below:
private DateTime _createdUTC;
public DateTime CreatedUtc
{
get
{
return _createdUTC;
}
set
{
_createdUTC = new DateTime(value.Ticks, DateTimeKind.Utc);
}
}
MongoDB always saves dates in UTC.
What you can do is indicate to the client that he must convert the date to the local date when retrieving data from the database.
You can register a serializer before instantiating the client:
BsonSerializer.RegisterSerializer(DateTimeSerializer.LocalInstance);
var client = new MongoClient(options) ...
I'm developing a WCF RESTful service with C#, .NET Framework 4.0 and Entity Framework Code First.
I have this class (that represents a table on database):
[DataContract]
public class PostLine
{
public int PostLineId { get; set; }
[DataMember]
public int? UserId { get; set; }
[DataMember]
public string Description { get; set; }
[DataMember]
public string DateUtc { get; set; }
public User Author { get; set; }
}
And I'm trying to do this:
DateTime fourDaysAgo = DateTime.Now.Date.AddDays(-4);
var postLines =
context.PostLines.Where(p => DateTime.Compare(DateTime.Parse(p.DateUtc), fourDaysAgo) > 0).Include("Author");
But I get the following error:
{System.NotSupportedException: LINQ to Entities doesn't recognize the method 'System.DateTime Parse(System.String)', which can not be converted into an expression of the repository.
I need that PostLine.DateUtc to be a string because I'm going to use it on my web service, and send it as JSON, so its better for me to store it as a string.
If I use DateTime type, I'm going to get something like this on JSON response:
{
"DateUtc": "/Date(1380924000000+0200)/",
"Description": "post_1",
"UserId": 1
}
Do you know how can I compare a string with a DateTime on a LINQ expression?
I think the best approach will be to split the property in two.
Entity Framework wants a DateTime property. That makes perfect sense.
For serialization you want a string property. That also makes perfect sense.
However, you're trying to use a single property for both, and that doesn't make sense, and isn't necessary.
[DataContract]
public class PostLine
{
...
public DateTime DateUtcAsDateTime { get; set; }
[DataMember, NotMapped]
public string DateUtcAsString {
get { return DateUtcAsDateTime.ToString(); }
set { DateUtcAsDateTime = DateTime.Parse(value); }
}
...
}
Now, DateUtcAsDateTime will be used by Entity Framework, and DateUtcAsString will be ignored by Entity Framework as it has a NotMapped attribute.
DateUtcAsString, on the other hand, is the only one of these properties that has a DataMember attribute, so should be the only one that gets serialized.
You can of course rename one of these properties back to DateUtc, if you want.
Update: as Matt Johnson points out, an improvement would be to specify the format in a way that always results in the exact same string. This ensures your strings don't change, just because your code gets moved to another server that happens to have different regional settings.
[DataMember, NotMapped]
public string DateUtcAsString {
get { return DateUtcAsDateTime.ToString("o"); }
set { DateUtcAsDateTime = DateTime.Parse(value, "o", null, DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal); }
}
Note that I am using DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal instead of the DateTimeStyles.RoundTripKind that he suggested, because the name DateUtc strongly suggests you always want UTC, never local time. And I'm not specifying any culture explicitly, as the "o" format is already independent of culture.
You could, if it's easier for your other code to handle, use "r" instead of "o" with the same benefits.
This is certainly an XY problem if I ever saw one. You're asking about comparing datetime as strings in Entity Framework, while it seems the real problem is that you don't like the default date format of the DataContractJsonSerializer that WCF uses by default.
Part of the problem is that you are mixing local and UTC. You're getting /Date(1380924000000+0200)/, which contains the local time zone offset of the server. This is because you started from DateTime.Now, which has a .Kind of DateTimeKind.Local.
If you instead used DateTime.UtcNow, it would have a .Kind of DateTimeKind.Utc, and would be serialized as /Date(1380924000000)/. And yes, the numerical portion of the format would be the same. Even when there is an offset specified, the number part is still related to UTC.
That's just one problem with this format. The other is that while DataContractJsonSerializer writes the local offset during serialization, it doesn't use it properly during deserialization. It just assumes that if any offset is provided, that the time should be local - even if the computer doing the deserialization has a completely different offset.
The better format to use in JSON is the ISO8601 format. For example, this UTC value would look like 2013-10-04T22:00:00.000Z. And while you can easily pass in a different date format to DataContractJsonSerializer if you use it directly, WCF doesn't easily expose this to you.
You could go down the route of changing the DataContractJsonSerializer settings via a custom WCF message formatter, such as described here, here and here, but it gets complicated very quickly. Be careful if you do this, and be sure to test thoroughly!
Another idea would be to write a custom WCF message formatter that uses JSON.Net instead of DataContractJsonSerializer. JSON.Net uses the ISO8601 format by default, so you would be set.
But honestly, the best solution is to not try to use WCF to build your REST endpoints. The date format issue is just the beginning. There are all sorts of other problems that can pop up along the way. Instead, use a modern framework that is designed for this purpose, such as ASP.Net WebAPI, or ServiceStack.
ASP.Net WebAPI uses JSON.Net, and ServiceStack has it's own JSON serializer called ServiceStack.Text. JSON.Net uses ISO8601 as it's default date format. If you use Service Stack, you'll need to set JsConfig.DateHandler = JsonDateHandler.ISO8601;
So in recap, you have two XY problems:
First, you chose WCF to build your rest endpoints, which was so problematic that the industry developed other solutions.
Second, you couldn't get DateTime to emit the correct format, so you decided to treat it as a string, which then you couldn't compare back to a DateTime in an EF query.
And yes, I realize I didn't answer your question of how to compare a DateTime input against a string property in Entity Framework, but I think now you can see why. If you really want to go down that road, you might find something useful in SqlFunctions or EntityFunctions - but even then, how are you going to build an efficient index on this column in your database? Querying will be really slow if you get a lot of data. I would advise against it.
I don't know how you are instantiating your DataContractJsonSerializer. If you are directly instantiating it...you can pass a DataContractJsonSerializerSettings, with a DateTimeFormat for setting how DateTime are serialized.
If you are using a behavior to instantiate your Serializer things are a little more complicated.
If you really want to go down the route of using a string for your class when transfering your data to your client, you should have a separate DTO class.
You can then use a library like AutoMapper to map your PostLine class to PostLineDto with an expression.
However the next problem you will face then is that your Expression<Func<PostLine,PostLineDto>> will contain Expression.MethodCall(DateTime.Parse) and you will have to inject an ExpressionVisitor that can convert...
p => p.DateUtc > DateTime.Parse("2013 Aug 1") - bool
into
p => p.DateUtc > new DateTime(1, Aug, 2013) - bool
Which is a real pain.
SQLCe doesn't support TimeSpan, so I need to save this in some other way. Is it possible to define somewhere a default conversion, like in the DbContext, or would I need to handle this manually in the repositories? I don't like to change my entity classes just because of this.
Btw, I only save TimeSpan of < 24h.
Example would be great if there are some neat tricks.
I know you say you don't want to modify your entities, but if you're using code first this would be pretty simple to do by modifying your entities :)
You could define a property that isn't mapped into the database, that uses another property as its backing store. In this example, TickCount would get saved in the database, but everything else could access Span which exposes TickCount as a TimeSpan struct.
public long TickCount { get; set; }
[NotMapped]
public TimeSpan Span {
get {
return new TimeSpan(TickCount);
}
set {
TickCount = value.Ticks;
}
}
Pretty old but in case anyone ever needs this, I have cloned and modified System.Data.SQLite in order to support TimeSpan properties (mapped to sqlite TIME columns)
You can find the source at https://github.com/arielflashner/System.Data.SQLite
Well, it seems this is basically NOT POSSIBLE.
A TimeSpan property must be marked NotMapped, or by fluent Ignore (then saved as per Steve's answer).
At least that's my understanding now.
EF does not support type mapping or type conversion..
https://stackoverflow.com/a/12053328
Btw, NHibernate maps TimeSpan to integer by default, when using SqLite. No conversions or tricks needed there. Not sure about other databases.
I'm trying to convert an existing (non-LINQ to SQL) class into a LINQ to SQL entity class which has an existing (db column) property like:
public string MyString
{
get { return myString; }
set { myString = FormatMyString(value); }
}
Is there a way to do this sort of processing on the value of entity class property before saving?
Should I use some sort of an entity-level saving event in which to do my formatting (if that will even work)?
I know LINQ to SQL provides validation and there are generated On...Changing() partial methods which provide access to the new value, by value (not by ref), but none of those methods seem to provide a way to actually modify/format the value while it is being set.
Thank you for your help.
What about using On...Changed()? It fires after the property value has changed. There you can check its value and update it using the FormatString.