Develop Reference

Entity Framework Core: Group by using calculated field

I'm trying to write a query. What I have so far is shown below.
var x = from d in DbContext.StorageDetails
let state = (d.ReleaseDate.HasValue && d.ReleaseDate < date) || (d.TakeOrPayEndDate.HasValue && d.TakeOrPayEndDate < date) ?
StorageState.Closed :
(d.RailcarNumber == null || d.RailcarNumber == string.Empty) ?
d.TakeOrPayStartDate.HasValue ? StorageState.TakeOrPay : StorageState.Open :
(d.ArrivalDate.HasValue && d.ArrivalDate <= date) ? StorageState.Filled : StorageState.EnRoute
group d by new
{
d.CustomerId,
d.Customer.Name,
d.LocationId,
d.Location.City,
d.Location.State
} into g
select new
{
// ...
};
The part that's giving me trouble is that I want to include the calculated state value with each item. I don't want to group by this value but I want to be able to sum it.
// Note: This is after my group statement
select new
{
// state is a let variable and not part of x!
TotalOpen = g.Sum(x => x.state == StorageState.Open),
TotalClosed = g.Sum(x => x.state == StorageState.Closed),
// Etc.
};
Is there a way to do this? I don't seem able to select my own set of columns prior to group by. How can I insert this calculated field into each item?
Note: StorageState is an enum. I can just as easily cast it to an int in all of these expressions. I can figure that part out, but figuring it out is separate from my question here.

I don't know how you would write this as a query expression, nor if EF can translate that expression to sql. I would do something like the following;
var query = ....
// Select all the values you need
.Select(d => new {
d.CustomerId,
d.Customer.Name,
d.LocationId,
d.Location.City,
d.Location.State,
....
state = [insert expression]
})
// group by this key
.GroupBy(d => new {
d.CustomerId,
d.Name,
d.LocationId,
d.City,
d.State
},
// and define the final result set
(d, g) => new {
d.CustomerId,
d.Name,
d.LocationId,
d.City,
d.State,
TotalOpen = g.Sum(x => x.state == StorageState.Open ? 1 : 0),
TotalClosed = g.Sum(x => x.state == StorageState.Closed ? 1 : 0)
});
I think the sql EF will generate, will put the first select in the from clause, surrounded by your typical select / group by statement.

What about this:
TotalOpen = g.Sum(x => (x.ReleaseDate.HasValue && x.ReleaseDate < date) || (x.TakeOrPayEndDate.HasValue && x.TakeOrPayEndDate < date) ?
StorageState.Closed :
(x.RailcarNumber == null || x.RailcarNumber == string.Empty) ?
x.TakeOrPayStartDate.HasValue ? StorageState.TakeOrPay : StorageState.Open :
(x.ArrivalDate.HasValue && x.ArrivalDate <= date) ? StorageState.Filled : StorageState.EnRoute
== StorageState.Open? 1 : 0)
You would repeat the entire expression four times.

There seems to be two things happening here:
StorageDetails is queried from DbContext and somehow for every item(d) found, there's some execution that is being processed to determine the value of state...
state is determined for each d and then all the d's are grouped, and then after only few properties of d are selected...
This might be another approach:
var x = DbContext.StorageDetails
.GroupBy(d => new
{
d.CustomerId,
d.Name,
d.LocationId,
d.City,
d.State
}).Select(d => new
{
d.CustomerId,
d.Name,
d.LocationId,
d.City,
d.State,
d.state = getState(d)
}).ToList();
int getState(d)
{
return (d.ReleaseDate.HasValue && d.ReleaseDate < date) || (d.TakeOrPayEndDate.HasValue && d.TakeOrPayEndDate < date) ?
StorageState.Closed :
(d.RailcarNumber == null || d.RailcarNumber == string.Empty) ?
d.TakeOrPayStartDate.HasValue ? StorageState.TakeOrPay : StorageState.Open :
(d.ArrivalDate.HasValue && d.ArrivalDate <= date) ? StorageState.Filled : StorageState.EnRoute;
}
var TotalOpen = x.Count(x => x.state == StorageState.Open),
var TotalClosed = x.Count(x => x.state == StorageState.Closed)
Basically you first get your data from database group it. After grouping it, you then select specific properties and introduce one property called state... But state is calculated, using another function which takes a row from database return a value from enum StorageState like you have done on the question. Then return the result as list.
Only after you have your list will it be easier to check which of the rows returned contains an open or closed...
Now you can do normal looping and loop through your x and calculate TotalClosed and TotalClosed.

Joining table to a list using Entity Framework

I have the following Entity Framework function that it joining a table to a list. Each item in serviceSuburbList contains two ints, ServiceId and SuburbId.
public List<SearchResults> GetSearchResultsList(List<ServiceSuburbPair> serviceSuburbList)
{
var srtList = new List<SearchResults>();
srtList = DataContext.Set<SearchResults>()
.AsEnumerable()
.Where(x => serviceSuburbList.Any(m => m.ServiceId == x.ServiceId &&
m.SuburbId == x.SuburbId))
.ToList();
return srtList;
}
Obviously that AsEnumerable is killing my performance. I'm unsure of another way to do this. Basically, I have my SearchResults table and I want to find records that match serviceSuburbList.

If serviceSuburbList's length is not big, you can make several Unions:
var table = DataContext.Set<SearchResults>();
IQuerable<SearchResults> query = null;
foreach(var y in serviceSuburbList)
{
var temp = table.Where(x => x.ServiceId == y.ServiceId && x.SuburbId == y.SuburbId);
query = query == null ? temp : query.Union(temp);
}
var srtList = query.ToList();
Another solution - to use Z.EntityFramework.Plus.EF6 library:
var srtList = serviceSuburbList.Select(y =>
ctx.Customer.DeferredFirstOrDefault(
x => x.ServiceId == y.ServiceId && x.SuburbId == y.SuburbId
).FutureValue()
).ToList().Select(x => x.Value).Where(x => x != null).ToList();
//all queries together as a batch will be sent to database
//when first time .Value property will be requested

Using linq to find MIN(date) within linq query

I am using linq to extract data. This data contains a date and some other values. The thing i that these dates can occur more then once, because the dates can have the same value but a different timestamp. I want to extract the anonymous type with the earliest timestamp. How can i do this in linq ?
this is my code:
var result = (from a in UnitOfWork.ActivityLessonParticipantService.Query
.Where(a => a.ActivityLesson.Activity.Id == activityId)
.Where(a => a.ActivityLesson.From >= startDate && (a.ActivityLesson.To == startDate || a.ActivityLesson.To <= endDate)).OrderBy(d => d.ActivityLesson.From)
where !a.ActivityLesson.IsDeleted && !a.ActivityLesson.Activity.IsDeleted && a.Appeared == true
select new
{
CPR = a.User.UserName,
FullName = a.User.FullName,
ActivityFromDate = a.ActivityLesson.From,
}).OrderBy(c => c.CPR).ToList();
thanks

You can GroupBy the Date property of DateTime and then order this group by DateTime and use First to pick only the first record/object:
var query = from a in UnitOfWork.ActivityLessonParticipantService.Query
where a.ActivityLesson.Activity.Id == activityId
&& a.ActivityLesson.From >= startDate
&& (a.ActivityLesson.To == startDate || a.ActivityLesson.To <= endDate)
&& !a.ActivityLesson.IsDeleted
&& !a.ActivityLesson.Activity.IsDeleted
&& a.Appeared
select a;
var firstByDate = query.GroupBy(a => a.ActivityLesson.From.Date)
.Select(grp => grp.OrderBy(a => a.ActivityLesson.From).First())
.OrderBy(a => a.User.UserName)
.Select(a => new
{
CPR = a.User.UserName,
FullName = a.User.FullName,
ActivityFromDate = a.ActivityLesson.From,
}).ToList();
Due to LINQ's deferred execution this is actually a single query that gets executed at the final ToList. I'm mixing query and method syntax because i prefer method syntax when it comes to GroupBy but it's a matter of taste.

Code First Entity Framework Linq Statement Returning Missing Records

I have the following query:
orderMessageEmail.MessageChain = DbContext.Current
.Messages
.Where(c =>
c.OrderId == orderMessageEmail.OrderId &&
c.IsPublic &&
c.MessageId != orderMessageEmail.MessageId &&
c.MessageId < orderMessageEmail.MessageId
)
.Select(c => new OrderMessageChain()
{
CreateDateTime = c.CreateDateTime,
MessageId = c.MessageId,
Message = c.MessageData,
UserFirstName = c.User.FirstName,
UserLastName = c.User.LastName,
CustomerFirstName = c.CustomerAccountPerson.FirstName,
CustomerLastName = c.CustomerAccountPerson.LastName,
SentFrom = c.SentFrom
})
.OrderByDescending(c => c.MessageId)
.Take(10)
.ToList();
The problem I'm running into is that whenever c.User is null it doesn't return ANY record for the OrderMessageChain
I'd like to just have it return UserFirstName and UserLastName as empty strings instead of completely eliminating that OrderMessageChain from the list.
Does this make sense?
One more thing..
Simply testing this query:
var t = DbContext.Current
.Messages
.Include("User")
.Where(c =>
c.MessageId < 138120 &&
c.OrderId == 170496 &&
c.IsPublic)
.ToList();
When I manually execute the same query in the DB i'm shown 3 records, however t is showing zero.
I always thought Include worked as a Left Join - is that not the case?
One more thing...
OK so i think i'm starting to realize what is going on here..
I didn't realize this, but it appears that the DBA setup the DB field UserId on the Message Table to NOT be nullable, however, in the case when there isn't a user record the UserId field contains a 0 (zero) value, instead of null... argh...
I think this is leading to code first to believe it should perform an inner join instead of a left join per here
So i'm not quite sure how to fix this.. is there anyway I can force code first to somehow perform a left join on that navigation property instead of an inner?

You should be able to select the data you need, convert it to an IEnumerable and do the mapping in-memory, something like (the untested);
orderMessageEmail.MessageChain = DbContext.Current
.Messages
.Where(c =>
c.OrderId == orderMessageEmail.OrderId &&
c.IsPublic &&
c.MessageId != orderMessageEmail.MessageId &&
c.MessageId < orderMessageEmail.MessageId
)
.OrderByDescending(c => c.MessageId)
.Take(10)
.Select(c => new {
c, u = c.User, cap = c.CustomerAccountPerson
}).
.AsEnumerable()
.Select(c => new OrderMessageChain()
{
CreateDateTime = c.c.CreateDateTime,
MessageId = c.c.MessageId,
Message = c.c.MessageData,
UserFirstName = c.u == null ? "" : c.u.FirstName,
UserLastName = c.u == null ? "" : c.u.LastName,
CustomerFirstName = c.cap == null ? "" : c.cap.FirstName,
CustomerLastName = c.cap == null ? "" : c.cap.LastName,
SentFrom = c.c.SentFrom
})
.ToList();

Convert SQL statement to Linq / Lambda Query

Im using MEF and executing a task which needs to be grouped with aggregate functions only when it returns more than 1 record. I need both the Max of the start hour and the min of the end hour grouped into a single record like my sql would result in on the restuled task
var ohs = await Bl.UoW.Repositories.OperatingHours
.FindInDataSourceAsync(oh => ((oh.ProductId == productTypeId
&& oh.StateId == state)
|| (oh.StateId == complianceHours.State)));
Here is the SQL that gets me basiccally what I need when more than 1 record returned
SELECT
StateId,
MAX(ComplianceHourStart),
MIN(ComplianceHourEnd)
FROM
OperatingHours
GROUP BY
StateId
HAVING
StateId = 'CA'
So when more than 1 I can filter it further but not sure how to achieve max and min?
if (ohs != null && ohs.Count() > 1)
{
//
ohs = ohs.GroupBy(x => x.State).Max(x => x.ComplianceHourStart?...
}
Thanks

From your SQL, this should be close:
var result = context.OperatingHours
.GroupBy(oh => oh.StateId)
.Select(oh => new {StateId = oh.Key,
MaxStart = oh.Max(x => x.ComplianceHourStart),
MinEnd = oh.Min(x => x.ComplianceHourEnd)});
...although I'm not sure why you are grouping when you are restricting the state id column (group key). The following should also suffice:
var result = context.OperatingHours
.Where(oh => oh.StateId == 'CA')
.Select(oh => new {MaxStart = oh.Max(x => x.ComplianceHourStart),
MinEnd = oh.Min(x => x.ComplianceHourEnd)});

Something like this should do it:
ohs = ohs.GroupBy(x => x.State)
.Select(g => new
{
//You need to make a choice on StateId, here... First one?
StateId = g.First().StateId,
MaxComplianceHourStart = g.Max(o => o.ComplianceHourStart),
MinComplianceHourEnd = g.Min(o => o.ComplianceHourEnd)
});