Develop Reference

Entity Framework query using TimeZoneInfo cannot be translated to SQL Server query

I am working with time zones in my .NET application. I followed the best practices regarding time zones according to Microsoft (link below):
https://learn.microsoft.com/en-us/previous-versions/dotnet/articles/ms973825(v=msdn.10)
Here you can read that the best option is to store the time zone in the database with the time in that time zone and perform calculations transforming it to UTC. I was trying to filter the records based on the date using the following piece of code:
var result = ContextClassObject.Entity
.Where(e => TimeZoneInfo.ConvertTimeToUtc(e.Date) > DateTime.UtcNow)
.ToList();
I get the following error message:
System.InvalidOperationException: „The LINQ expression 'DbSet()
.Where(a => TimeZoneInfo.ConvertTimeToUtc(a.Date) > DateTime.UtcNow)' could not be translated. Additional information:
Translation of method 'System.TimeZoneInfo.ConvertTimeToUtc' failed. If this method can be mapped to your custom function, see https://go.microsoft.com/fwlink/?linkid=2132413 for more information. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'
I think it means that my query cannot be translated to a SQL query so I have to tell it directly how to translate it or materialize the data and then filter it in C#, which I don't want to do because it's gonna be slower than SQL Server could do it and premature materialization isn't a good thing to do.
Is there a way to make it work without mapping the function to a SQL query directly which would be quite complicated for a simple operation? Should I just store any DateTime in UTC, which is supposed to be a good method too and save myself the trouble?
Below is the example row of a database. The date is in the time zone, which id is stored right next to it:
I have tried storing it with DateTimeOffset like that:
Then the query below produces the same error:
TimeZoneInfo userTimeZone = TimeZoneInfo.FindSystemTimeZoneById("Alaskan Standard Time");
DateTime userDate = new DateTime(2022, 09, 27, 11, 58, 12);
DateTime dateInUTC = TimeZoneInfo.ConvertTimeToUtc(userDate, userTimeZone);
var result = await _dataContext.DateEntity
.Where(e => e.Date.Add(e.Date.Offset) > dateInUTC)
.ToListAsync();
I have tried different methods to add or subtract the offset and found out that only accessing the Offset property of the DateTimeOffset class cannot be translated. Using raw numbers works just fine.

How to add year to LINQ to entities query?

I am attempting to query a table in SQL using LINQ to entities and add a year to a Date column.
I have tried the following query below:
data = data.Where(x => x.DueDate.Value.AddYears(1) >= DateTime.Now);
When I do this, I get the following error message:
LINQ to Entities does not recognize the method 'System.DateTime AddYears(Int32)' method, and this method cannot be translated into a store expression.
I have done some research and it seems as if most people fix this issue by separating the query out and using a variable they then input into the expression.
I can't do this because I need to add the years to my data lambda expression and can't separate them out.
Is there is a simple way to fix this or is there a way to create a pseudo column that adds a year without actually creating an actual table column? I am trying to avoid having to create an entire new SQL column just for the purpose of adding a year to the date displayed.
Thanks for any help.

You can do the inverse and subtract 1 year from DateTime.Now and use that as the comparison value.
var yearAgo = DateTime.Now.AddYears(-1);
data = data.Where(x => x.DueDate >= yearAgo);
Side notes
The comparison value has a time component but based on the naming of the persisted value DueDate might not have a time component. You can remove the time component by using .Date (DateTime.Now.Date.AddYears(-1);) or .Today instead of .Now (DateTime.Today.AddYears(-1);).
If working in different time zones you will need a more robust solution. I would recommend reading Storing UTC Is Not a Silver Bullet by Jon Skeet.

How do I average DateTime values in SQL Azure with Entity Framework?

I have a SQL query that needs to average many datetime values server-side (in SQL Server). For example purposes, let's just consider it's a simple query like this on a table with millions of records:
SELECT
SomeField,
AVG(CAST(ADateTime AS decimal(18,8))) AS NumericRepresentation
FROM MyTable
GROUP BY SomeField
As shown, I can't simply take AVG(ADateTime) because SQL Server isn't happy with doing that, but converting it to a Decimal (and later converting it back to a DateTime) works well enough for me.
The obvious way to do something comparable with EntityFramework is to use .Select(tbl => tbl.ADateTime.Ticks).Average(), but this fails at runtime because DateTime.Ticks doesn't translate through Linq-to-Entities.
How should I best go about this? My main need is to average datetime values in some way. The temporary representation (decimals, ticks, etc) isn't terribly important as long as it can be translated back to a DateTime either in SQL or .NET code. What is important, though, is that the averaging is done in SQL Server (I have some fairly complex calculations with this over many records) and I can somehow have the translated DateTime in .NET (whether the translation happens in SQL Server or in .NET, I don't care).

In pure SQL you can do average on a date field with something like this:
-- the smallest date you could possibly have in your data
DECLARE #MinDate DATE = '1/1/1900'
SELECT
SomeField,
DATEADD(DAY, AVG(DATEDIFF(DAY, #MinDate, ADateTime)), #MinDate) as AvgDateTime
FROM MyTable
GROUP BY SomeField
Not sure yet how to translate this to LINQ :)
UPD: Here is the LINQ code:
private static void Test(IQueryable<SomeClass> data)
{
var minDate = DateTime.MinValue;
var avgMilliseconds = data.Select(x => x.SomeDateField.Subtract(minDate).TotalMilliseconds).Average();
var avgDate = minDate.AddMilliseconds(avgMilliseconds);
Console.WriteLine(avgMilliseconds);
Console.WriteLine(avgDate);
}

EF LINQ expressions can make use of SqlFunctions class to make sure the conversion happens correctly.
DateTime minDate = new DateTime(1900,1,1);
var avg = MyTable.Select(myrow => SQLFunctions.DateDiff("second",myrow.ADateTime,minDate).Average();
DateTime avgDate = minDate.AddSeconds(avg);
Previous answer, should be disregarded:
Use Convert.ToDouble. EntityFramework should be able to translate this LINQ to SQL is able to CONVERT(float,...) as long as your column is actually a DateTime and not DateTime2 or DateTimeOffset, but unfortunately Entity Framework is not able to recognize this construct.
.Select(tbl => Convert.ToDouble(tbl.ADateTime)).Average()
An alternate choice is to do it client side:
.Select(tbl => tbl.ADateTime).ToArray().Select(dt => dt.Ticks).Average()
though clearly that's not preferred if you're averaging millions of rows.

What will be captured by the C# Entity Framework linq query

When implementing specification pattern I came up with this code
public static class ItemSpecification
{
static public Expression<Func<Item, bool>> TodayOnlySpecification { get; } = item => item.ImplementationDate == DateTime.Today;
}
This expression will later on be used in EF linq filters for building queries. My question is, what date is actually used in the query. Is it the date that the static class is first initialized? Or the date that TodayOnlySpecification is referenced?
A concrete scenario would be
My application starts up today and I reference TodayOnlySpecification the first time for a linq query, it should use today's date for the eventual sql
My application kept running and I reference TodayOnlySpecification again tomorrow for another linq query, will today's date be used in the query or tomorrow's date be used?
Edit 1:
Testing with DateTime.Now and DateTime.Today yields very different results
DateTime.Now produces the following SQL query:
select ...
from ...
where [Extent1].[ImplementationDate] = (SysDateTime())
DateTime.Today produces the following SQL query:
select ...
from ...
where [Extent1].[ImplementationDate] = #p__linq__0',N'#p__linq__0 datetime2(7)',#p__linq__0='2019-06-11 00:00:00'`
This is just a guess, the fact that linq to entity is able to translate DateTime.Now to SysDateTime() means it is not using the value of DateTime.Now for translation. Instead, it is using the method to get the current time when generating SQL. This should also mean that DateTime.Today is captured as an method not a value.
Edit 2:
This then brings another question, if EF sometimes evaluates a method at the client side (e.g. DateTime.Today) and sometimes evaluates at the server side (e.g. DateTime.Now), which could be having a different timezone, how can one determine the exact behavior of EF queries?

What gets captured is
Expression.Call(
null, //Static call so no instance
methodOf(DateTime.Now),
new Type[0] //No generic arguments
);
In otherwords. Nothing gets captured.
I recommend you use the debugger to examine the IQueryable<T>.Expression to look at the tree.
EDIT:
To answer your question on how EntityFramework handles the above expression. It doesn't. It is upto the implementation of EntityFramework Provider to handle.
In the Case of EntityFramework.SqlServer, the code that handled this is located here
SqlFunctionCallHandler.HandleCanonicalFunctionCurrentDateTime line 1259

LINQ query maps dates incorrectly

My query looks like so:
using (var ctx = new PCLvsCompContext())
{
var brokerId = broker?.Id;
var symbolId = company?.Id;
var result = (from t in ctx.TradeHistoryJoineds
where t.TradeDate >= fromDate
&& t.TradeDate <= toDate
&& (brokerId == null || t.BrokerId == brokerId)
&& (symbolId == null || t.SymbolId == symbolId)
select t).OrderBy(x => x.TradeDate).ThenBy(x => x.BrokerName).ToList();
return result;
}
As an example, I run this query with dates like fromDate March-01-2017 toDate March-31-2017. I then captured the generated sql in SQL profiler that this query produces and ran it in SQL management studio. The output was as expected where for each weekday, each company has some trades. The query is based off of a view which casts all dates to "datetime" so that excel can parse them as dates correctly. However, when I put a breakpoint at "return result" and inspect the dates, all but 2 of the dates are March-1-2017. This is incorrect, the query result in SQL manager shows trades for almost every weekday in March (which is correct).
What is going on here? Why is Linq losing its mind?

Although based on the results I cannot see exactly how you would end up with those results, it is very common that you could be dealing with a DateTime timezone issue. I suspect that perhaps you saved your dates to the database using a DateTime object from say DateTime.Now instead of DateTime.UtcNow. So at that point in time and based on the machine it was called on it would be based on the timezone and datelight savings of that machine.
A DateTime object should not be used as it can relate to the region of the SQL database, the region of the server making this LINQ call and so the two regions could be on different timezones.
Instead you should always use DateTimeOffset.
If you cannot do that for some reason, then double-check your dates toDate and fromDate and do:
var utcToDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
var utcFromDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
Which gives something like this if it was run on 3rd April 2018 at 22:56.
2018-04-03T22:56:57.740Z
You would then also need to make sure when you save any date to the SQL backing store that you do ToUniversalTime() firstly. You can check your SQL tables directly with a normal SQL query and they should be stored in the database as the same UTC string format as above, and then the comparison should be obvious to whether it is valid.
However I would strongly recommend changing all your DateTime calls and gets to DateTimeOffset. The only time to use DateTime in majority of cases is for the final display to a user.

Thank you all for your suggestions. For those who are familiar with linq, EF and views this may seem like a stupid oversight, but I will post my shame for others in case this happens to them since the cause and the resulting behavior are not immediately obviously linked (as can be seen by all the suggestions, none of which actually point to the answer).
When you are querying a view using linq and Entity Framework, you apparently must have a column called 'Id', otherwise Entity Framework can't distinguish any of the rows from one another and simply makes some sort of duplication that I can't quite decipher (all the rows were unique based on their data even without an Id column, so I can't quite figure out why this is necessary).
So by adding an the TradeId with an 'Id' alias to the view, then Entity Framework seemed to return to sanity and map the data as expected.