When implementing specification pattern I came up with this code
public static class ItemSpecification
{
static public Expression<Func<Item, bool>> TodayOnlySpecification { get; } = item => item.ImplementationDate == DateTime.Today;
}
This expression will later on be used in EF linq filters for building queries. My question is, what date is actually used in the query. Is it the date that the static class is first initialized? Or the date that TodayOnlySpecification is referenced?
A concrete scenario would be
My application starts up today and I reference TodayOnlySpecification the first time for a linq query, it should use today's date for the eventual sql
My application kept running and I reference TodayOnlySpecification again tomorrow for another linq query, will today's date be used in the query or tomorrow's date be used?
Edit 1:
Testing with DateTime.Now and DateTime.Today yields very different results
DateTime.Now produces the following SQL query:
select ...
from ...
where [Extent1].[ImplementationDate] = (SysDateTime())
DateTime.Today produces the following SQL query:
select ...
from ...
where [Extent1].[ImplementationDate] = #p__linq__0',N'#p__linq__0 datetime2(7)',#p__linq__0='2019-06-11 00:00:00'`
This is just a guess, the fact that linq to entity is able to translate DateTime.Now to SysDateTime() means it is not using the value of DateTime.Now for translation. Instead, it is using the method to get the current time when generating SQL. This should also mean that DateTime.Today is captured as an method not a value.
Edit 2:
This then brings another question, if EF sometimes evaluates a method at the client side (e.g. DateTime.Today) and sometimes evaluates at the server side (e.g. DateTime.Now), which could be having a different timezone, how can one determine the exact behavior of EF queries?
What gets captured is
Expression.Call(
null, //Static call so no instance
methodOf(DateTime.Now),
new Type[0] //No generic arguments
);
In otherwords. Nothing gets captured.
I recommend you use the debugger to examine the IQueryable<T>.Expression to look at the tree.
EDIT:
To answer your question on how EntityFramework handles the above expression. It doesn't. It is upto the implementation of EntityFramework Provider to handle.
In the Case of EntityFramework.SqlServer, the code that handled this is located here
SqlFunctionCallHandler.HandleCanonicalFunctionCurrentDateTime line 1259
Related
I'm storing dates in SQL Server as varchar so they become strings in C# by Entity Framework.
Then I tried to convert then to date but the code throws an exception.
In SQL Server:
CREATE TABLE [dbo].[Achievements]
(
...
[DatePerformance] [VARCHAR](20) NULL,
)
The date is stored in this format : "25/04/2019"
In C#:
public static School_LPEntities db = new School_LPEntities();
I tried:
List<Achievements> res = db.Achievements.Where(x => DateTime.Parse(x.DatePerformance) <= DateTime.Today.Date).ToList();
and:
List<Achievements> res = db.Achievements.Where(x => DateTime.ParseExact(x.DatePerformance, "dd/MM/yyyy",
CultureInfo.InvariantCulture) <= DateTime.Today.Date).ToList();
The code throws:
System.NotSupportedException: 'LINQ to Entities does not recognize the method 'System.DateTime Parse(System.String)' method, and this method cannot be translated into a store expression.'
As others have said in the comments, the number of good reasons to store your dates as varchar's in SQL is zero.
You said:
so they become strings in C# by Entity Framework
Once you get the date in C# you can do this easily anyway with a .ToString().
But... as to your specific error, what it's telling you is that you can't use this method inside of an Entity Framework LINQ statement. It's best to think of LINQ statements as queries. Because, in fact, it needs to be able to convert your LINQ statement into an actual SQL query. You can't use DateTime.Parse() inside of a SQL query, so it throws this error.
If you really MUST store your dates as varchar's in the database (and again, this is a terrible idea) then I'm not seeing a really simple solution. Off hand, I would tell you to first, create your list, then loop through it:
List<Achievements> res = db.Achievements.ToList();
Then:
foreach (var achievement in res)
{
if (DateTime.Parse(achievement.DatePerformance) <= DateTime.Today.Date)
{
// do something here
}
}
But this is a really clunky way to work around the real solution: Just store your dates as dates!!!
My query looks like so:
using (var ctx = new PCLvsCompContext())
{
var brokerId = broker?.Id;
var symbolId = company?.Id;
var result = (from t in ctx.TradeHistoryJoineds
where t.TradeDate >= fromDate
&& t.TradeDate <= toDate
&& (brokerId == null || t.BrokerId == brokerId)
&& (symbolId == null || t.SymbolId == symbolId)
select t).OrderBy(x => x.TradeDate).ThenBy(x => x.BrokerName).ToList();
return result;
}
As an example, I run this query with dates like fromDate March-01-2017 toDate March-31-2017. I then captured the generated sql in SQL profiler that this query produces and ran it in SQL management studio. The output was as expected where for each weekday, each company has some trades. The query is based off of a view which casts all dates to "datetime" so that excel can parse them as dates correctly. However, when I put a breakpoint at "return result" and inspect the dates, all but 2 of the dates are March-1-2017. This is incorrect, the query result in SQL manager shows trades for almost every weekday in March (which is correct).
What is going on here? Why is Linq losing its mind?
Although based on the results I cannot see exactly how you would end up with those results, it is very common that you could be dealing with a DateTime timezone issue. I suspect that perhaps you saved your dates to the database using a DateTime object from say DateTime.Now instead of DateTime.UtcNow. So at that point in time and based on the machine it was called on it would be based on the timezone and datelight savings of that machine.
A DateTime object should not be used as it can relate to the region of the SQL database, the region of the server making this LINQ call and so the two regions could be on different timezones.
Instead you should always use DateTimeOffset.
If you cannot do that for some reason, then double-check your dates toDate and fromDate and do:
var utcToDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
var utcFromDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
Which gives something like this if it was run on 3rd April 2018 at 22:56.
2018-04-03T22:56:57.740Z
You would then also need to make sure when you save any date to the SQL backing store that you do ToUniversalTime() firstly. You can check your SQL tables directly with a normal SQL query and they should be stored in the database as the same UTC string format as above, and then the comparison should be obvious to whether it is valid.
However I would strongly recommend changing all your DateTime calls and gets to DateTimeOffset. The only time to use DateTime in majority of cases is for the final display to a user.
Thank you all for your suggestions. For those who are familiar with linq, EF and views this may seem like a stupid oversight, but I will post my shame for others in case this happens to them since the cause and the resulting behavior are not immediately obviously linked (as can be seen by all the suggestions, none of which actually point to the answer).
When you are querying a view using linq and Entity Framework, you apparently must have a column called 'Id', otherwise Entity Framework can't distinguish any of the rows from one another and simply makes some sort of duplication that I can't quite decipher (all the rows were unique based on their data even without an Id column, so I can't quite figure out why this is necessary).
So by adding an the TradeId with an 'Id' alias to the view, then Entity Framework seemed to return to sanity and map the data as expected.
I would like to know if it is possible to convert a DateTime object's timezone in a Linq Query. For example
var seats = await _seat.GetAll()
.Where(
DbFunctions.AddHours( x.StartTime, -1 ) >= x.CreationTime.ToUniversalTime());
In the above code, x.StartTime is in UTC timezone whereas x.Creation time is in local timezone. if I call ToUniversalTime() on x.CreationTime I get an exception.
Is it possible to do the conversion inside the Linq query?
I know it is possible to extract CreationTime before the query and convert it, but it would be great to know the possibility to convert it inside the Linq quert exists.
As #MartinLiversage explained in the columns the best solution would be as follows:
You can easily solve your problem by pulling all the rows to the client side and then perform the filtering using your predicate (putting aside the problems created by using "local time".) However, I assume that you want to filter the records on the database server. EF will have to translate your predicate to corresponding SQL and that limits what you can do in the Where clause. The DbFunctions is a way to declare certain simple computations that EF can translate to SQL but there is no way to perform time zone conversions. SQL Server has no concept of time zones.
And as #Martine mentioned the following could be taken into considertion:
SQL Server 2016 just added time zone support. However, there aren't EF DbFunctions for them yet, AFAIK. Still, probably not a good usage here.
This seems like an inconsistency, but I'm probably just missing something obvious. The base query is:
var events = db.tbl_special_events.Where(x => x.TimeStamp >= startDate);
The apparent inconsistency comes when I run the following code block:
int c1 = 0;
foreach (var e in events)
{
if (e.TimeStamp.DayOfWeek.ToString() == "Tuesday") c1++;
}
int c2 = events.Where(e => e.TimeStamp.DayOfWeek.ToString() == "Tuesday").Count();
After that runs, c1 is 1832, but c2 is 0. What am I missing?
I recreated this test and found that it may be directly related to the DateTime function.
The query that is generated:
exec sp_executesql N'SELECT [t0].[User_ID], [t0].[Email], [t0].[Password], [t0].[BrandID], [t0].[CustomerID], [t0].[Created], [t0].[EndDate], [t0].[AccountStatusID], [t0].[Verified], [t0].[ResetPasswordFailCount], [t0].[ResetPasswordLocked]
FROM [dbo].[User] AS [t0]
WHERE ((CONVERT(NVarChar,CONVERT(Int,(DATEPART(dw, [t0].[Created]) + (##DATEFIRST) + 6) % 7))) = #p0) AND ([t0].[BrandID] = #p1)',N'#p0 nvarchar(4000),#p1 int',#p0=N'Tuesday',#p1=3
Notice where #p0=N'Tuesday'
Keeping in mind that IQueryable and IEnumerable differ in that where IEnumerable represents an actual .net object, IQueryable converts your expression into an actual SQL statement used to query the database. So any values that you provide in that expression are actually sent to the database.
It is returning 0 results because there is no match. Reason being, the date conversion in SQL returns a 2 instead of 'Tuesday'. You can test this if you replace Tuesday with 2 in your LINQ WHERE clause, it'll actually work. This will work after enumerating it since the results will have been successfully mapped into a usable .net object where the DateTime.DayOfWeek conversion to "Tuesday" will work properly.
Your count is acting on an IQueryable<Event> while e is an enumerated instance. Therefore, some operations might not work as they cannot be translated into SQL [Edit: or will be translated in nonsensical SQL].
To ensure your Where clause works add an AsEnumerable() before it. This converts the IQueryable<Event> into an IEnumerable<Event> and tells the linq provider that it should stop generating SQL at this point.
So, this statement should provide the correct result:
int c2 = events.AsEnumerable()
.Where(e => e.TimeStamp.DayOfWeek.ToString() == "Tuesday")
.Count();
The actual code that cannot be converted causes the problem in SQL is e.TimeStamp.DayOfWeek.ToString().
Alternatively you can use the System.Data.Objects.SqlClient.SqlFunctions (doc here) class to hint the linq provider what it should be doing in SQL.
Edit
As #Servy pointed out this is a Linq to SQL question. However, the problem is quite common so I leave the answer and do not delete it.
On looking at the OP again, there could be another variable to the whole game ... lazy loading.
In the foreach loop the TimeStamp is lazy loaded. In the count query the provider tries to construct an SQL query, during this construction it might not be able to work with ToString (which is known to be problematic) and evaluate e.TimeStamp.DayOfWeek.ToString() to something different than "Tuesday".
The AsEnumerable() forces the provider to stop generating SQL, so that e.TimeStamp is lazy loaded again.
The only way kowing exactly what is going on, is to use tracing tool on the DB (e.g. SQL Server Profiler) to actually see the query (or queries) executed on the server.
Edit 2
Building on #Sinaesthetic's answer the reason why 0 is returned is that the Query tries to compare "4" to "Tuesday" which returns false and therefore the correct result is false.
Can be tested by executing
select ((CONVERT(NVarChar,CONVERT(Int,(DATEPART(dw, getdate()) + (##DATEFIRST) + 6) % 7))))
against the DB.
However, this also shows that it is up to the provider to decide what SQL it generates. Also it is up to the provider to decide what statements it supports and which statements it doesn't.
It also shows that using AsEnumerable to stop the SQL generation process can have an infulence on the semantic evaluation of a query.
I have a very strange problem with a LINQ to Entities query with EF1.
I have a method with a simple query:
public DateTime GetLastSuccessfulRun(string job)
{
var entities = GetEntities();
var query = from jr in entities.JOBRUNS
where jr.JOB_NAME == job && jr.JOB_INFO == "SUCCESS"
orderby jr.JOB_END descending
select jr.JOB_END;
var result = query.ToList().FirstOrDefault();
return result.HasValue ? result.Value : default(DateTime);
}
The method GetEntities returns an instance of a class that is derived from System.Data.Objects.ObjectContext and has automatically been created by the EF designer when I imported the schema of the database.
The query worked just fine for the last 15 or 16 months. And it still runs fine on our test system. In the live system however, there is a strange problem: Depending on the value of the parameter job, it returns the correct results or an empty result set, although there is data it should return.
Anyone ever had a strange case like that? Any ideas what could be the problem?
Some more info:
The database we query against is a Oracle 10g, we are using an enhanced version of the OracleEFProvider v0.2a.
The SQl statement that is returned by ToTraceString works just fine when executed directly via SQL Developer, even with the same parameter that is causing the problem in the LINQ query.
The following also returns the correct result:
entities.JOBRUNS.ToList().Where(x => x.JOB_NAME == job && x.JOB_INFO == "SUCCESS").Count();
The difference here is the call to ToList on the table before applying the where clause. This means two things:
The data is in the database and it is correct.
The problem seems to be the query including the where clause when executed by the EF Provider.
What really stuns me is, that this is a live system and the problem occurred without any changes to the database or the program. One call to that method returned the correct result and the next call five minutes later returned the wrong result. And since then, it only returns the wrong results.
Any hints, suggestions, ideas etc. are welcome, never mind, how far-fetched they seem! Please post them as answers, so I can vote on them, just for the fact for reading my lengthy question and bothering thinking about that strange problem... ;-)
First of all remove ObjectContext caching. Object context internally uses UnitOfWork and IdentityMap patterns. This can have big impact on queries.