I receive 48 files per day on a half hourly basis from market. These files have a start time as a property. I am using Entity Framework to view these files in a web application and as the files have a UK time but I am working with a european market the trading day begins the day before at 11pm and so I want to group these together based on the trading day.
In SQL I can accomplish this by:
select cast(DATEADD(hour, 1, START_TIME) as date), count(cast(START_TIME as date))
from imbalancecost
group by cast(DATEADD(hour, 1, START_TIME) as date)
I am trying to achieve a similar result in C# using the following attempt:
IEnumerable<IGrouping<DateTime, ImbalanceCost> imbalanceCost = db.ImbalanceCost.GroupBy(ic => ic.START_TIME).ToArray();
Is there any means of first adding the hour onto my grouping and then using only the date part of this new calculated value?
Is there any means of first adding the hour onto my grouping and then using only the date part of this new calculated value?
In LINQ to Entities (EF6) it's a matter of using respectively the canonical functions DbFunctions.AddHours and DbFunctions.TruncateTime:
db.ImbalanceCost.GroupBy(ic =>
DbFunctions.TruncateTime(DbFunctions.AddHours(ic.START_TIME, 1)).Value)
Note that .Value (or cast to DateTime) is to make the result DateTime rather than DateTime? returned by the canonical method in case ic.START_TIME is not nullable, hence you know the result is not nullable as well.
If I understand you correctly, you want to add an hour to the start time and group by the date part. That can be done as follows:
var imbalanceCost = db.ImbalanceCost
.Select(x => EntityFunctions.AddHours(x.START_TIME, 1))
.GroupBy(ic => ic.Value.Date)
.ToArray();
Related
I am attempting to query a table in SQL using LINQ to entities and add a year to a Date column.
I have tried the following query below:
data = data.Where(x => x.DueDate.Value.AddYears(1) >= DateTime.Now);
When I do this, I get the following error message:
LINQ to Entities does not recognize the method 'System.DateTime AddYears(Int32)' method, and this method cannot be translated into a store expression.
I have done some research and it seems as if most people fix this issue by separating the query out and using a variable they then input into the expression.
I can't do this because I need to add the years to my data lambda expression and can't separate them out.
Is there is a simple way to fix this or is there a way to create a pseudo column that adds a year without actually creating an actual table column? I am trying to avoid having to create an entire new SQL column just for the purpose of adding a year to the date displayed.
Thanks for any help.
You can do the inverse and subtract 1 year from DateTime.Now and use that as the comparison value.
var yearAgo = DateTime.Now.AddYears(-1);
data = data.Where(x => x.DueDate >= yearAgo);
Side notes
The comparison value has a time component but based on the naming of the persisted value DueDate might not have a time component. You can remove the time component by using .Date (DateTime.Now.Date.AddYears(-1);) or .Today instead of .Now (DateTime.Today.AddYears(-1);).
If working in different time zones you will need a more robust solution. I would recommend reading Storing UTC Is Not a Silver Bullet by Jon Skeet.
I need to build a report from some data in my database. It is a list of items representing the income per day in our systems. Our data is stored hourly so, a group by the date part of a datetime is used. Here the query:
dateDataQuery = query.GroupBy(o => new {o.Time.Month, o.Time.Year, o.Time.Day})
.OrderBy(o => o.Key.Year).ThenBy(o => o.Key.Month).ThenBy(o => o.Key.Day)
.Select(g => new StatData()
{
Day = g.Key.Day,
Month = g.Key.Month,
Income = g.Sum(o => o.Income),
});
This works correctly but the problem is that if we have a day with no transactions, there wouldn't be any record in the database for this day and so, the StatData object for this day would not appear in the result.
This is a problem because this data will be represented in a graph in our frontend and needs to be continuous. I mean, every date at a day resolution must appear in the result at least with an Income of 0.
So, is there any possibility to do this with Entity framework core and linq without having to go through all the returned data, and check if date exists manually?
My query looks like so:
using (var ctx = new PCLvsCompContext())
{
var brokerId = broker?.Id;
var symbolId = company?.Id;
var result = (from t in ctx.TradeHistoryJoineds
where t.TradeDate >= fromDate
&& t.TradeDate <= toDate
&& (brokerId == null || t.BrokerId == brokerId)
&& (symbolId == null || t.SymbolId == symbolId)
select t).OrderBy(x => x.TradeDate).ThenBy(x => x.BrokerName).ToList();
return result;
}
As an example, I run this query with dates like fromDate March-01-2017 toDate March-31-2017. I then captured the generated sql in SQL profiler that this query produces and ran it in SQL management studio. The output was as expected where for each weekday, each company has some trades. The query is based off of a view which casts all dates to "datetime" so that excel can parse them as dates correctly. However, when I put a breakpoint at "return result" and inspect the dates, all but 2 of the dates are March-1-2017. This is incorrect, the query result in SQL manager shows trades for almost every weekday in March (which is correct).
What is going on here? Why is Linq losing its mind?
Although based on the results I cannot see exactly how you would end up with those results, it is very common that you could be dealing with a DateTime timezone issue. I suspect that perhaps you saved your dates to the database using a DateTime object from say DateTime.Now instead of DateTime.UtcNow. So at that point in time and based on the machine it was called on it would be based on the timezone and datelight savings of that machine.
A DateTime object should not be used as it can relate to the region of the SQL database, the region of the server making this LINQ call and so the two regions could be on different timezones.
Instead you should always use DateTimeOffset.
If you cannot do that for some reason, then double-check your dates toDate and fromDate and do:
var utcToDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
var utcFromDate = toDate.ToUniversalTime().ToString("yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'");
Which gives something like this if it was run on 3rd April 2018 at 22:56.
2018-04-03T22:56:57.740Z
You would then also need to make sure when you save any date to the SQL backing store that you do ToUniversalTime() firstly. You can check your SQL tables directly with a normal SQL query and they should be stored in the database as the same UTC string format as above, and then the comparison should be obvious to whether it is valid.
However I would strongly recommend changing all your DateTime calls and gets to DateTimeOffset. The only time to use DateTime in majority of cases is for the final display to a user.
Thank you all for your suggestions. For those who are familiar with linq, EF and views this may seem like a stupid oversight, but I will post my shame for others in case this happens to them since the cause and the resulting behavior are not immediately obviously linked (as can be seen by all the suggestions, none of which actually point to the answer).
When you are querying a view using linq and Entity Framework, you apparently must have a column called 'Id', otherwise Entity Framework can't distinguish any of the rows from one another and simply makes some sort of duplication that I can't quite decipher (all the rows were unique based on their data even without an Id column, so I can't quite figure out why this is necessary).
So by adding an the TradeId with an 'Id' alias to the view, then Entity Framework seemed to return to sanity and map the data as expected.
I have a database with a ValidDate field - it's a string(we made a mistake, it should be a datetime, but we can't modify the database now.)
and now I want to compare this filed with a parameter(validDateStart) from the website:
priceList = priceList.Where(p => Convert.ToDateTime(p.ValidDate) >= Convert.ToDateTime(validDateStart));
var list = initPriceList.ToList();
But I get an error: The method ToDateTime is not implemented.
Can anybody give me some help? Thanks!
This is not supported in Linq to Entities (nor Linq to SQL to my knowledge). Remember that your query is executed on the database - where there is simply no equivalent for Convert.ToDateTime.
Any string parsing in your query would really just be a workaround - as a real solution make those columns not strings but datetime in the database and you would not have this problem in the first place.
A hacky workaround would be materializing all rows (you can use AsEnumerable() for that), then doing the parsing - this will have bad performance though but might work good enough if there are few rows:
var startDate = DateTime.Parse(validDateStart);
var list = priceList.AsEnumerable()
.Where(p => DateTime.Parse(p.ValidDate) >= startDate);
.ToList();
Edit:
With your example update it looks like you can just do string comparisons to do what you wanted - granted it's still a hack but would perform much better than materializing all rows. This is possible because your date format puts the most significant numbers first, then the less significant parts - it's year, then month, then day (should this not be the case and the day comes before the month in your example this solution will not work).
Assuming your input string validDateStart is in the same format as well you can just do:
var list = priceList.Where(p => p.ValidDate.CompareTo(validDateStart) >=0);
.ToList();
string comparison with String.CompareTo seems to be support both in Linq to Sql as well as Linq to Entities.
If all the records in your database always start with year, month and day (for example: the date format is yyyy-MM-dd HH:mm:ss or yyyy/MM/dd or yyyyMMdd) no matter if it has separators or not. The thing is that the values should has a format where it starts with year, month and day.
You can do the following:
1: Convert your filter value (website) to the same format as you have in your database:
// DateTime format in database: yyyy-MM-dd HH:mm:ss:ffffff
var from = filtro.CreationDateFrom?.ToString("yyyy-MM-dd");
var to = filtro.CreationDateTo?.AddDays(1).ToString("yyyy-MM-dd");
2: And write your query like this (using CompareTo method):
var query = (from x in ctx.TskTaskQueues
where x.CreationDatetime.CompareTo(from) >= 0
&& x.CreationDatetime.CompareTo(to) <= 0
select x);
It worked for me!
I'm not using LinqToEntities but I'm using LinqConnect (for Oracle) that is similar to LinqEntities.
If you use a format like this dd-MM-yyyy, it probably will not work.
I've a column in my table called Date, and I need to compare this date's WeekOfTheYear with DateTime.Now's WeekOfTheYear,
If I give like this,
var cal = CultureInfo.CurrentCulture.Calendar;
int week = cal.GetWeekOfYear(DateTime.Now, CalendarWeekRule.FirstDay, DayOfWeek.Sunday);
I am getting 26 here. The same way, in my Entity Framework, I need to compare this week's data, for this I tried like,
entities.WorkingDays.Where(a =>
cal.GetWeekOfYear(a.DATE,CalendarWeekRule.FirstDay,DayOfWeek.Sunday)
== cal.GetWeekOfYear(DateTime.Now, CalendarWeekRule.FirstDay,
DayOfWeek.Sunday)
when I run the query like this, am getting error like,
"LINQ to Entities does not recognize the method 'Int32 GetWeekOfYear
(System.DateTime, System.Globalization.CalendarWeekRule, System.DayOfWeek)'
method, and this method cannot be translated into a store expression."
How can I fetch the data for weekly basis here, can any one help me out here....thanks in advance
Call .ToList() first. Like this:
entities.WorkingDays.ToList().Where(a =>
cal.GetWeekOfYear(a.DATE,CalendarWeekRule.FirstDay,DayOfWeek.Sunday)
== cal.GetWeekOfYear(DateTime.Now, CalendarWeekRule.FirstDay,
DayOfWeek.Sunday)
See this post for duplicate issue. Basically, the data needs to be in memory before using your GetWeekOfYear functions.
As noted in the comments, this does bring the whole "WorkingDays" table into memory and therefore fetching more data than needed from the DB. Sometimes this is more preferable to using Stored Procedures and sometimes not, depending on the amount of data and other factors based on your application/database architecture.
You could probably use the day of year and divide it with 7 on both instances, and get a sufficient result?
Date.DayOfYear / 7