Develop Reference

LINQ query maps dates incorrectly

My query looks like so:
using (var ctx = new PCLvsCompContext())
{
var brokerId = broker?.Id;
var symbolId = company?.Id;
var result = (from t in ctx.TradeHistoryJoineds
where t.TradeDate >= fromDate
&& t.TradeDate <= toDate
&& (brokerId == null || t.BrokerId == brokerId)
&& (symbolId == null || t.SymbolId == symbolId)
select t).OrderBy(x => x.TradeDate).ThenBy(x => x.BrokerName).ToList();
return result;
}
As an example, I run this query with dates like fromDate March-01-2017 toDate March-31-2017. I then captured the generated sql in SQL profiler that this query produces and ran it in SQL management studio. The output was as expected where for each weekday, each company has some trades. The query is based off of a view which casts all dates to "datetime" so that excel can parse them as dates correctly. However, when I put a breakpoint at "return result" and inspect the dates, all but 2 of the dates are March-1-2017. This is incorrect, the query result in SQL manager shows trades for almost every weekday in March (which is correct).
What is going on here? Why is Linq losing its mind?

Although based on the results I cannot see exactly how you would end up with those results, it is very common that you could be dealing with a DateTime timezone issue. I suspect that perhaps you saved your dates to the database using a DateTime object from say DateTime.Now instead of DateTime.UtcNow. So at that point in time and based on the machine it was called on it would be based on the timezone and datelight savings of that machine.
A DateTime object should not be used as it can relate to the region of the SQL database, the region of the server making this LINQ call and so the two regions could be on different timezones.
Instead you should always use DateTimeOffset.
If you cannot do that for some reason, then double-check your dates toDate and fromDate and do:
var utcToDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
var utcFromDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
Which gives something like this if it was run on 3rd April 2018 at 22:56.
2018-04-03T22:56:57.740Z
You would then also need to make sure when you save any date to the SQL backing store that you do ToUniversalTime() firstly. You can check your SQL tables directly with a normal SQL query and they should be stored in the database as the same UTC string format as above, and then the comparison should be obvious to whether it is valid.
However I would strongly recommend changing all your DateTime calls and gets to DateTimeOffset. The only time to use DateTime in majority of cases is for the final display to a user.

Thank you all for your suggestions. For those who are familiar with linq, EF and views this may seem like a stupid oversight, but I will post my shame for others in case this happens to them since the cause and the resulting behavior are not immediately obviously linked (as can be seen by all the suggestions, none of which actually point to the answer).
When you are querying a view using linq and Entity Framework, you apparently must have a column called 'Id', otherwise Entity Framework can't distinguish any of the rows from one another and simply makes some sort of duplication that I can't quite decipher (all the rows were unique based on their data even without an Id column, so I can't quite figure out why this is necessary).
So by adding an the TradeId with an 'Id' alias to the view, then Entity Framework seemed to return to sanity and map the data as expected.

Receiving Object reference not set to an instance of an object. on simple datetime retrieval from Raven

Not sure why its complaining about this error, but when comparing time stamps to DateTime.now I get a object reference error. Here is the simple code.
public class EmailTempKey
{
public string Id { get; set; }
public DateTime LastUpdated { get; set; }
public bool ShouldBeDisposed { get; set; }
}
//Linq Query to see if its been past 8 min
var emails = session.Query<EmailTempKey>().Where(x => DateTime.Now > x.LastUpdated.AddMinutes(8)).ToList();
If I remove the where clause it works, but obviously I need that. Thanks for any help

When you express a LINQ query, keep in mind that RavenDB has to translate that to Lucene before executing it. Therefore, you need to be thinking about how a particular field is being compared to the value you are passing in. You wrote:
Where(x => DateTime.Now > x.LastUpdated.AddMinutes(8))
Your LastUpdated field would need to be mutated for each record in order to resolve this query. This isn't something that Lucene or Raven can do. Just because you can express it in LINQ does not mean that it is valid for Raven. Instead, you could write:
Where(x => x.LastUpdated < DateTime.Now.AddMinutes(-8))
This is the algebraic equivalent to the query you specified, and should work, but there are still some problems.
You probably want an inclusive comparison, either <= or >=
The query will return items older than 8 minutes ago. I could be wrong, but I think you meant you wanted items newer than 8 minutes old, in which case you would flip the comparison:
Where(x => x.LastUpdated >= DateTime.Now.AddMinutes(-8))
Using DateTime.Now can be problematic. You are imposing the local time zone settings of the server onto your data. If your time zone uses daylight savings time, then twice a year you will get the wrong results from your query when your clocks transition. Even if you don't follow daylight savings time, you have a problem if you ever want to move your data somewhere else (like to the cloud, or an ISP). You should also read my post: The Case Against DateTime.Now. You should be storing your dates as UTC, or you can use DateTimeOffset - which works beautifully in RavenDB.
Perhaps you didn't realize, but RavenDB already keeps a Last-Modified value on every document in its metadata. You could use your own field if you want to, but why store it twice? You can query based on metadata with the Lucene syntax:
var results = session.Advanced.LuceneQuery<EmailTempKey>()
.WhereGreaterThan("#metadata.Last-Modified",
DateTime.UtcNow.AddMinutes(8));
If you want to stick with the LINQ syntax, you will need to define a static index instead.

I think this would fix it
Depending if the datetime LastUpdated may be null, choose one of the solutions below
May be null:
//Linq Query to see if its been past 8 min
var emails = session.Query<EmailTempKey>().Where(x => x.LastUpdated == DateTime.MinValue || (DateTime.Now > x.LastUpdated.AddMinutes(8)).ToList();
May not be null:
//Linq Query to see if its been past 8 min
var emails = session.Query<EmailTempKey>().Where(x => x.LastUpdated != DateTime.MinValue && (DateTime.Now > x.LastUpdated.AddMinutes(8)).ToList();

How to compare a string column(as DateTime) in LINQ?

I have a database with a ValidDate field - it's a string(we made a mistake, it should be a datetime, but we can't modify the database now.)
and now I want to compare this filed with a parameter(validDateStart) from the website:
priceList = priceList.Where(p => Convert.ToDateTime(p.ValidDate) >= Convert.ToDateTime(validDateStart));
var list = initPriceList.ToList();
But I get an error: The method ToDateTime is not implemented.
Can anybody give me some help? Thanks!

This is not supported in Linq to Entities (nor Linq to SQL to my knowledge). Remember that your query is executed on the database - where there is simply no equivalent for Convert.ToDateTime.
Any string parsing in your query would really just be a workaround - as a real solution make those columns not strings but datetime in the database and you would not have this problem in the first place.
A hacky workaround would be materializing all rows (you can use AsEnumerable() for that), then doing the parsing - this will have bad performance though but might work good enough if there are few rows:
var startDate = DateTime.Parse(validDateStart);
var list = priceList.AsEnumerable()
.Where(p => DateTime.Parse(p.ValidDate) >= startDate);
.ToList();
Edit:
With your example update it looks like you can just do string comparisons to do what you wanted - granted it's still a hack but would perform much better than materializing all rows. This is possible because your date format puts the most significant numbers first, then the less significant parts - it's year, then month, then day (should this not be the case and the day comes before the month in your example this solution will not work).
Assuming your input string validDateStart is in the same format as well you can just do:
var list = priceList.Where(p => p.ValidDate.CompareTo(validDateStart) >=0);
.ToList();
string comparison with String.CompareTo seems to be support both in Linq to Sql as well as Linq to Entities.

If all the records in your database always start with year, month and day (for example: the date format is yyyy-MM-dd HH:mm:ss or yyyy/MM/dd or yyyyMMdd) no matter if it has separators or not. The thing is that the values should has a format where it starts with year, month and day.
You can do the following:
1: Convert your filter value (website) to the same format as you have in your database:
// DateTime format in database: yyyy-MM-dd HH:mm:ss:ffffff
var from = filtro.CreationDateFrom?.ToString("yyyy-MM-dd");
var to = filtro.CreationDateTo?.AddDays(1).ToString("yyyy-MM-dd");
2: And write your query like this (using CompareTo method):
var query = (from x in ctx.TskTaskQueues
where x.CreationDatetime.CompareTo(from) >= 0
&& x.CreationDatetime.CompareTo(to) <= 0
select x);
It worked for me!
I'm not using LinqToEntities but I'm using LinqConnect (for Oracle) that is similar to LinqEntities.
If you use a format like this dd-MM-yyyy, it probably will not work.

LINQ to EF4 datetime comparison problems

I have a problem while comparing smalldatetime/datetime column value from DB(SQL Server 2008) with DateTime.Now in C#.
I know that datetime format in DB and on the server on which application is running are different, so I have done something like this to "cache" the results so date time comparison will be local and not be on server:
var inactiveAccounts = (from entry in ent.Accounts
where entry.Status == 0
select entry).ToArray();
var accountsFiltered = (from entry in inactiveAccounts
where entry.DeactivationDate < DateTime.Now
select entry).ToArray();
And at some pertiod of day I am not getting the right records.
I suspect it is due to where entry.DeactivationDate < DateTime.Now date comparison. And seems in EF it is tricky to work with datetime, please help me to identify the problem.

Don´t know if it solves your problem but i would create a local variable and set DateTime.Now on it. Then use the local variable in your linq query.
This is done, because DateTime.Now gets evaluated in your query each time the where clause is called. So for each entry in inactiveAccounts you are working against another DateTime.
Also if you want to compare Dates without the Time value, you should use DateTime.Today
var inactiveAccounts = (from entry in ent.Accounts
where entry.Status == 0
select entry).ToArray();
DateTime currentDateTime = DateTime.Now;
var accountsFiltered = (from entry in inactiveAccounts
where entry.DeactivationDate < currentDateTime
select entry).ToArray();

You can use this directly:
var inactiveAccounts = (from entry in ent.Accounts
where entry.Status == 0 && entry.DeactivationDate < DateTime.Now
select entry).ToArray();
Because DataTime.Now and DateTime.UtcNow are translated as canonical functions = they should not be evaluated on .NET side but translated toGETDATE() or GETUTCDATE() SQL function calls respectively.
The rest of your question is unanswerable because providing information like
And at some period of day I am not getting the right records.
You must put much bigger effort to analysis of the problem if you want to get answer including which periods cause problem, what timezones are used, etc. Generally we will not help you with that because we don't see your data. It is also not a problem of EF because in your case it happends completely in linq-to-object = plain .NET code.

Entity SQL compare datetime without milliseconds

I'm trying to fetch some records from MSSQL DB using EntityObject with EntitySQL query.
The field i'm using to filter is type of datetime.
The generated query projected without millisecond, to the SQL Server returns nothing.
When i'm adding the milliseconds, i get results.
How can i use the EntitySQL to get result without the milliseconds?
thanks.

It's not elegant, but this worked for me:
foos.SingleOrDefault(f => f.EntryDate.Year == bar.EntryDate.Year &&
f.EntryDate.Month == bar.EntryDate.Month &&
f.EntryDate.Day == bar.EntryDate.Day &&
f.EntryDate.Hour == bar.EntryDate.Hour &&
f.EntryDate.Minute == bar.EntryDate.Minute &&
f.EntryDate.Second == bar.EntryDate.Second);

One way to do it is to create a view of your data, where your datetime column is converted to smalldatetime (which does not have milliseconds).
Then add the view to your entity framework model, and read the data through the view.
Hope this helps
Shiraz

Workaround I found is on initial fetch store date as .ToBinary() then when you filter just do new DateTime(binaryValue) and compare with that.