Develop Reference

Dapper - how to apply a transformation to selected object fields?

The problem I am trying to solve is pretty simple: I need to make sure my timestamps are stored in the database as UTC. I use DateTime in my model objects, which maps to timespamp column in Postgres DB.
I'd like to do 2 things:
When a value is saved to the database, I want to check the DateTime.Kind field and throw up if it is not UTC. This will help me to capture errors if I try to feed a non-UTC time to a timestamp field. I do not want to do any transformations, just assert.
When loading from DB, I need those DateTime instances come out with DateTime.Kind set to UTC, so the rest of my application can recognize they are UTC and convert to local or whatever, as needed.
I only need to apply this logic to selected timestamp fields, not to all DateTime-s, because I have other DateTime values in my model, which need to be stored as they are.
I've tried this:
public class UtcTimestampMapper: SqlMapper.TypeHandler<DateTime>
{
public override void SetValue(IDbDataParameter parameter, DateTime dt)
{
if (dt.Kind == DateTimeKind.Utc)
parameter.Value = dt;
else
throw new InvalidOperationException("UTC timestamp expected");
}
public override DateTime Parse(object value)
{
return DateTime.SpecifyKind((DateTime)value, DateTimeKind.Utc);
}
}
This solution has 2 issues:
It applies to all DateTime values. With SetValue I can do the trick checking the column name in IDbDataParameter, maybe... But Parse method just gets a naked value, there is nothing I can do to determine where it goes.
The SetValue method won't be called at all, unless I remove the standard handler for DateTime. See this SO answer for details: Dapper TypeHandler.SetValue() not being called Again, if I replace the handler, I replace it globally, there is no way to choose where it applies.
So, is there a way to solve my problem with the facilities Dapper provides to customize mappings?

How can I turn UTC Timezone off for MongoDB

I'm using the extension below for MongoDB date fields because of MongoDB stores times in UTC by default.
public static class DateTimeExtensions {
public static DateTime AdjustUtcDiffByOffset(this DateTime time) {
return time.AddHours(DateTimeOffset.Now.Offset.Hours);
}
}
}
All of them cause different problems although I have tried a few ways like attributes or serialization methods in .NET on the application level. I have decided to use this extension in .NET for now but I think that this is not a complete and exact solution too.
Is there any solution for this problem on the database level without being dependent on programming language wtih an adjust or something else?
EDIT
I think that I should an explain more after comments below. I already know MongoDB stores times in UTC that linked in this post as you see above. This can be useful but I don't need any UTC time zone difference in my app and I don't want to deal in Presentation Layer for every programming language separately. And also, I don't even want only one extra row or function in the other layers because of move away than base logic or business.
Let the my architecture undertaking this. I'm pretty lazy, the life is really short and the birds are flying outside :) I don't want the different fields as like as string convertings unnecessarily. I need a datetime type in the db due to I'm doing many time calculation in the app.
I'm using .NET now and therefore MongoDB .NET driver. I have tried different serialization methods but it cause another problems in the Data Access architecture. In conclusion, I can use UTC in my another app but I don't want it now and I prefer the local time when I assign to the field. I have decided to use the encapsulation below for C# especially.
private DateTime _startTime;
public DateTime StartTime {
get => _startTime;
set => _startTime = new DateTime(value.Ticks, DateTimeKind.Utc);
}

I don't think it's possible from the db level. What I did, was to write custom setter for date properties which will force mongoDB to assume the time is already in UTC, thus avoid a conversion like below:
private DateTime _createdUTC;
public DateTime CreatedUtc
{
get
{
return _createdUTC;
}
set
{
_createdUTC = new DateTime(value.Ticks, DateTimeKind.Utc);
}
}

MongoDB always saves dates in UTC.
What you can do is indicate to the client that he must convert the date to the local date when retrieving data from the database.
You can register a serializer before instantiating the client:
BsonSerializer.RegisterSerializer(DateTimeSerializer.LocalInstance);
var client = new MongoClient(options) ...

Force OData DateTime format

Is it possible to force the way DateTimes are displayed in OData while keeping all functionnalities of IQueryable<> ?
Actually, I created a basic endpoint exposing data from my Database. One of the property is a DateTime and is displayed like that :
"MyDate": "2017-01-07T00:00:00",
However, I'd like to display it with another DateTime format, like dd/MM/yyyy
I tried this promising answer but my property has still the same undesired format when I call my endpoint with Postman.
How to force DateTime format in OData, while preserving OData filtering on this column and let the client decides to query, for example, all data with MyDate>= n days ?

However, I'd like to display it with another DateTime format, like dd/MM/yyyy
Then do not display it directly. The date format in odata is defined in the specs do your code knows what to expect and can then format it for public consumption IF it desires to show it. The whole concept of a standard is that people consuming it know what to expect. In particular - if you write your classes manually to consume them you do something wrong, the idea behind odata is that a code generator generates the client code, and it need to know waht to parse (into a javascript odata class).
Odata is NOT supposed to necessariyl be presented directly to the user. The date format is part of the standard. If you change it, it is not odata.
Oh, and another note: The datetime foramt you see there is actually a standard for which Javascript has methods and libraries to manipulate them. WHich is another reason you WANT this. Your custom format is something you have to write all the libraries for yourself.

I am adding my solution here as this was my first hit on Google.
I ran into the same issue, and tried the same solution with no success.
Note that this answer is based on an Asp.NET Core Web Api.
I based my answer on WebAPI OData Datetime serialization
but the way they added the Custom payload value converter doesn't seem applicable anymore.
You have to create a class derived from ODataPayloadValueConverter on which you override the ConvertToPayloadValue function:
namespace [YOUR_NAMESPACE_HERE];
using Microsoft.OData;
using Microsoft.OData.Edm;
public class CustomODataPayloadConverter : ODataPayloadValueConverter
{
public override object ConvertToPayloadValue(object value, IEdmTypeReference edmTypeReference)
{
if (edmTypeReference.PrimitiveKind() == EdmPrimitiveTypeKind.DateTimeOffset)
{
var dateTimeOffset = (DateTimeOffset)value;
return dateTimeOffset.ToString("dd/MM/yyyy");
}
return base.ConvertToPayloadValue(value, edmTypeReference);
}
}
You add the payload value converter in the Action<IServiceCollection> which is the 3rd parameter in ODataOptions.AddRouteComponents.
builder.AddOData(options =>
{
options.AddRouteComponents("api/v1", modelBuilder.GetEdmModel(), actions =>
{
actions.AddSingleton(typeof(ODataPayloadValueConverter), new CustomODataPayloadConverter());
});
});
I am accessing the ODataOptions inside IMVCBuilder.AddOData which in turn is an extension method.
using Microsoft.AspNetCore.OData;
Which is found in the 'Microsoft.AspNetCore.OData.Mvc' NuGet package.
My Postman response:
{
"#odata.context": "http://localhost:5267/api/v1/$metadata#ENTITY_NAME",
"value": [
{
"Id": "98aa1283-f17f-4d12-8e51-59a4ead2a23e",
"myDateTime": "28/09/2022"
}
]
}
where myDateTime is ofcourse, a DateTime.

Compare DateTime as a string into a LINQ to Entities doesn't recognize DateTime.parse(string)

I'm developing a WCF RESTful service with C#, .NET Framework 4.0 and Entity Framework Code First.
I have this class (that represents a table on database):
[DataContract]
public class PostLine
{
public int PostLineId { get; set; }
[DataMember]
public int? UserId { get; set; }
[DataMember]
public string Description { get; set; }
[DataMember]
public string DateUtc { get; set; }
public User Author { get; set; }
}
And I'm trying to do this:
DateTime fourDaysAgo = DateTime.Now.Date.AddDays(-4);
var postLines =
context.PostLines.Where(p => DateTime.Compare(DateTime.Parse(p.DateUtc), fourDaysAgo) > 0).Include("Author");
But I get the following error:
{System.NotSupportedException: LINQ to Entities doesn't recognize the method 'System.DateTime Parse(System.String)', which can not be converted into an expression of the repository.
I need that PostLine.DateUtc to be a string because I'm going to use it on my web service, and send it as JSON, so its better for me to store it as a string.
If I use DateTime type, I'm going to get something like this on JSON response:
{
"DateUtc": "/Date(1380924000000+0200)/",
"Description": "post_1",
"UserId": 1
}
Do you know how can I compare a string with a DateTime on a LINQ expression?

I think the best approach will be to split the property in two.
Entity Framework wants a DateTime property. That makes perfect sense.
For serialization you want a string property. That also makes perfect sense.
However, you're trying to use a single property for both, and that doesn't make sense, and isn't necessary.
[DataContract]
public class PostLine
{
...
public DateTime DateUtcAsDateTime { get; set; }
[DataMember, NotMapped]
public string DateUtcAsString {
get { return DateUtcAsDateTime.ToString(); }
set { DateUtcAsDateTime = DateTime.Parse(value); }
}
...
}
Now, DateUtcAsDateTime will be used by Entity Framework, and DateUtcAsString will be ignored by Entity Framework as it has a NotMapped attribute.
DateUtcAsString, on the other hand, is the only one of these properties that has a DataMember attribute, so should be the only one that gets serialized.
You can of course rename one of these properties back to DateUtc, if you want.
Update: as Matt Johnson points out, an improvement would be to specify the format in a way that always results in the exact same string. This ensures your strings don't change, just because your code gets moved to another server that happens to have different regional settings.
[DataMember, NotMapped]
public string DateUtcAsString {
get { return DateUtcAsDateTime.ToString("o"); }
set { DateUtcAsDateTime = DateTime.Parse(value, "o", null, DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal); }
}
Note that I am using DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal instead of the DateTimeStyles.RoundTripKind that he suggested, because the name DateUtc strongly suggests you always want UTC, never local time. And I'm not specifying any culture explicitly, as the "o" format is already independent of culture.
You could, if it's easier for your other code to handle, use "r" instead of "o" with the same benefits.

This is certainly an XY problem if I ever saw one. You're asking about comparing datetime as strings in Entity Framework, while it seems the real problem is that you don't like the default date format of the DataContractJsonSerializer that WCF uses by default.
Part of the problem is that you are mixing local and UTC. You're getting /Date(1380924000000+0200)/, which contains the local time zone offset of the server. This is because you started from DateTime.Now, which has a .Kind of DateTimeKind.Local.
If you instead used DateTime.UtcNow, it would have a .Kind of DateTimeKind.Utc, and would be serialized as /Date(1380924000000)/. And yes, the numerical portion of the format would be the same. Even when there is an offset specified, the number part is still related to UTC.
That's just one problem with this format. The other is that while DataContractJsonSerializer writes the local offset during serialization, it doesn't use it properly during deserialization. It just assumes that if any offset is provided, that the time should be local - even if the computer doing the deserialization has a completely different offset.
The better format to use in JSON is the ISO8601 format. For example, this UTC value would look like 2013-10-04T22:00:00.000Z. And while you can easily pass in a different date format to DataContractJsonSerializer if you use it directly, WCF doesn't easily expose this to you.
You could go down the route of changing the DataContractJsonSerializer settings via a custom WCF message formatter, such as described here, here and here, but it gets complicated very quickly. Be careful if you do this, and be sure to test thoroughly!
Another idea would be to write a custom WCF message formatter that uses JSON.Net instead of DataContractJsonSerializer. JSON.Net uses the ISO8601 format by default, so you would be set.
But honestly, the best solution is to not try to use WCF to build your REST endpoints. The date format issue is just the beginning. There are all sorts of other problems that can pop up along the way. Instead, use a modern framework that is designed for this purpose, such as ASP.Net WebAPI, or ServiceStack.
ASP.Net WebAPI uses JSON.Net, and ServiceStack has it's own JSON serializer called ServiceStack.Text. JSON.Net uses ISO8601 as it's default date format. If you use Service Stack, you'll need to set JsConfig.DateHandler = JsonDateHandler.ISO8601;
So in recap, you have two XY problems:
First, you chose WCF to build your rest endpoints, which was so problematic that the industry developed other solutions.
Second, you couldn't get DateTime to emit the correct format, so you decided to treat it as a string, which then you couldn't compare back to a DateTime in an EF query.
And yes, I realize I didn't answer your question of how to compare a DateTime input against a string property in Entity Framework, but I think now you can see why. If you really want to go down that road, you might find something useful in SqlFunctions or EntityFunctions - but even then, how are you going to build an efficient index on this column in your database? Querying will be really slow if you get a lot of data. I would advise against it.

I don't know how you are instantiating your DataContractJsonSerializer. If you are directly instantiating it...you can pass a DataContractJsonSerializerSettings, with a DateTimeFormat for setting how DateTime are serialized.
If you are using a behavior to instantiate your Serializer things are a little more complicated.

If you really want to go down the route of using a string for your class when transfering your data to your client, you should have a separate DTO class.
You can then use a library like AutoMapper to map your PostLine class to PostLineDto with an expression.
However the next problem you will face then is that your Expression<Func<PostLine,PostLineDto>> will contain Expression.MethodCall(DateTime.Parse) and you will have to inject an ExpressionVisitor that can convert...
p => p.DateUtc > DateTime.Parse("2013 Aug 1") - bool
into
p => p.DateUtc > new DateTime(1, Aug, 2013) - bool
Which is a real pain.

ASP.NET MVC UpdateModel() parameter conversion woes

A forewarning: I've relatively new to MVC and its paradigm and to some of its inner workings, but I'm pretty comfortable with it. This is my second ground-up MVC application and I'm a little stumped with how to solve a "problem" one of our testers found.
What the users get is an edit screen given an effective date for daily LIBOR rates that come from Treasury (percentages). The rates are always between 0 and 100 and consequently I've tried to constrain that range using a RangeAttribute in the metadata for one of my domain objects. I've specified the range like so:
[Required, DisplayName("Published Rate"), Range(typeof(decimal), "0", "100")]
public object PublishedRate { get; set; }
Notice that I'm passing in string values as the RangeAttribute does not have an overloaded constructor that takes decimals. This seem to work great until a user goes and enters something out of the ordinary, like:
"0.000000000000000000000000000000001"
This causes UpdateModel() to fail; the ModelState shows this error (three times for the same ModelState value, curiously):
The parameter conversion from type 'System.String' to type 'System.Decimal' failed.
Digging into the errors reveals the cause. The first line below is what's reported by the validation for the field. I found it curious that this did not bubble up to the model validation errors (i.e. did not show up in the summary validation list for the model):
"0.000000000000000000000000000000001 is not a valid value for Decimal."
"Value was either too large or too small for a Decimal."
System.Web.Mvc.ValueProviderResult.ConvertSimpleType() and System.ComponentModel.BaseNumberConverter.ConvertFrom() are throwing the exceptions.
A user is never going to enter a value such as this, but I wouldn't mind knowing if there are any mechanisms built in to MVC that could or will prevent this (server-side, that is). There doesn't seem to be an issue with numbers like the following, it only seems to break with ones that are very small.
"0.555555555555555555555555555555555"
At the end of the day I really only need 9 digits of precision. The database table column backing these values is a decimal(9,6). I know I could implement a custom model binder for my model and manually collect the values from the Request, but there's got to be something a little easier, such as a custom FilterAttribute or something that can correct the value before its attempted to be bound to the model, I'm just not sure what, and am looking for suggestions.
I seem to recall reading about some issues with trying to constrain decimal values using a RangeAttribute but I can't recall the issue. Perhaps you MVC gurus out there can shed some light on the situation.

You could use a Regex attribute to contain the decimal to a precision of 9. This would also allow you to add a custom message when the Regex fails, such as "Your value may have a maximum of 9 places after the decimal." or something similar. Also if you have client side validation enabled, the Regex will work in both client and server side validation.

So after a couple of hours of head banging I settled on the following custom model binder for decimals. It makes sure that all decimal values are parseable before binding them.
public class TreasuryIndexRateDecimalBinder : DefaultModelBinder
{
public override object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
{
var providerResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName);
if (providerResult != null)
{
decimal value;
if (!decimal.TryParse(providerResult.AttemptedValue, NumberStyles.Float, CultureInfo.CurrentCulture, out value))
{
// TODO: Decide whether to show an error
// bindingContext.ModelState.AddModelError(bindingContext.ModelName, "error message");
return 0m;
}
return Math.Round(value, 6);
}
return base.BindModel(controllerContext, bindingContext);
}
}
The binding is set up in Application_Start() to register it for all decimal values.
protected void Application_Start()
{
ModelBinders.Binders.Add(typeof(decimal), new TreasuryIndexRateDecimalBinder());
AreaRegistration.RegisterAllAreas();
RegisterRoutes(RouteTable.Routes);
}
Unless somebody comes along with a more interesting approach I think I'll stick with this.