I'm using .NET Core 3.1, which has some differences to pre .NET Core 3.0 around Linq queries from what I can gather: https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-3.0/breaking-changes#linq-queries-are-no-longer-evaluated-on-the-client.
There seems to be a lot of information out there for earlier versions of .Net core that don't quite work for me.
All join tables and entities should be configured correctly for automatic creation, so I won't add them all unless someone needs to see them.
In light of that link I also need to be careful that I don't download the entire notes database, which could be millions of lines eventually. Some advice on tracking or notracking, firstordefaultasync etc might be helpful to if it makes a difference.
#Rena helped me with this earlier to get this example working for a single many to many query, thanks heaps Rena:
public async Task<List<Note>> GetAllNotesForNoteTypeAsync(string notetype)
{
var noteData = await context.Note
.Include(n => n.NoteNoteTypeJoins)
.ThenInclude(t => t.NoteType)
.ToListAsync();
var noteDataWithTypes = noteData.Where(i => i.NoteNoteTypeJoins.Any(x => x.NoteType.Type.ToString() == notetype))
.ToList();
return noteDataWithTypes;
}
I have another query I need, which goes one level deeper. These two attempts below obviously don't work but they explain what I am trying to do, trying to filter notes by notetype group text, which is four tables removed via two many to many relationships:
public async Task<List<Note>> GetAllNotesForNoteTypeGroupAsync(string notetypegroup)
{
var noteData = await context.Note
.Include(n => n.NoteNoteTypeJoins)
.ThenInclude(t => t.NoteType)
.ThenInclude(gt => gt.NoteTypeNoteTypeGroupJoins)
.ThenInclude(g => g.NoteTypeGroup)
.Where(g => g.NoteTypeGroup.Group == notetypegroup)
.ToListAsync();
return noteData;
}
or:
public async Task<List<Note>> GetAllNotesForNoteTypeGroupAsync(string notetypegroup)
{
var noteData = await context.Note
.Select(note => new
{
mytypejoin = note.NoteNoteTypeJoins
.Select(notetypejoin => new
{
mynotetype = notetypejoin.NoteType
.Select(notetype => new
{
mynotetype = notetype.NoteTypeNoteTypeGroupJoins
.Select(notetypegroupjoins => new
{
mytypegroup = notetypegroupjoins
.Where(i => i.NoteTypeNoteTypeGroupJoins
.Any(x => x.NoteTypeGroup.Group.ToString() == notetypegroup)
}),
}),
}),
});
return noteData;
Any help would be greatly appreciated, I'm fairly new to this, Thank you!
Here is the answer compliments of Chris:
var noteData = await context.Note.Where(n =>
n.NoteNoteTypeJoins.Any(ntj =>
ntj.NoteType.NoteTypeNoteTypeGroupJoins.Any(ntg => ntg.NoteTypeGroup.Group == notetypegroup)))
.ToListAsync();
You'd need to use Any recursively to dig into the lower relationships, but it does get ugly.
var noteData = await context.Note.Where(n =>
n.NoteTypeJoins.Any(ntj =>
ntj.NoteType.Any(nt =>
nt.NoteTypeNoteTypeGroupJoins.Any(ntntgj =>
ntntgj.NoteTypeGroup.Group == notetypegroup)))))
.ToListAsync();
You don't need all the Include and ThenInlude, as EF will automatically issue the necessary joins in order to make the query in the first place, since it includes all these relationships.
It's also important to note (and will become more problematic with deeper level queries like this), that it's not going to just return NoteTypes, for example, that are in this group. It's going to return Notes (since that's the base entity you're querying) that have any NoteTypes in this group, with all the related NoteTypes, whether they're in the group or not. You just won't have notes where there are zero associated note types in this group.
If you're looking to filter the actual relationships, you'll have to explicitly load. See: https://learn.microsoft.com/en-us/ef/core/querying/related-data#querying-related-entities
I'm looking for suggestions on how to write a query. For each Goal, I want to select the first Task (sorted by Task.Sequence), in addition to any tasks with ShowAlways == true. (My actual query is more complex, but this query demonstrates the limitations I'm running into.)
I tried something like this:
var tasks = (from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
let nextTaskId = g.Tasks.OrderBy(tt => tt.Sequence).Select(tt => tt.Id).DefaultIfEmpty(-1).FirstOrDefault()
where t.ShowAlways || t.Id == nextTaskId
select new CalendarTask
{
// Member assignment
}).ToList();
But this query appears to be too complex.
System.InvalidOperationException: 'Processing of the LINQ expression 'OrderBy<Task, int>(
source: MaterializeCollectionNavigation(Navigation: Goal.Tasks(< Tasks > k__BackingField, DbSet<Task>) Collection ToDependent Task Inverse: Goal, Where<Task>(
source: NavigationExpansionExpression
Source: Where<Task>(
source: DbSet<Task>,
predicate: (t0) => Property<Nullable<int>>((Unhandled parameter: ti0).Outer.Inner, "Id") == Property<Nullable<int>>(t0, "GoalId"))
PendingSelector: (t0) => NavigationTreeExpression
Value: EntityReferenceTask
Expression: t0
,
predicate: (i) => Property<Nullable<int>>(NavigationTreeExpression
Value: EntityReferenceGoal
Expression: (Unhandled parameter: ti0).Outer.Inner, "Id") == Property<Nullable<int>>(i, "GoalId"))),
keySelector: (tt) => tt.Sequence)' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.'
The problem is the line let nextTaskId =.... If I comment out that, there is no error. (But I don't get what I'm after.)
I'll readily admit that I don't understand the details of the error message. About the only other way I can think of to approach this is return all the Tasks and then sort and filter them on the client. But my preference is not to retrieve data I don't need.
Can anyone see any other ways to approach this query?
Note: I'm using the very latest version of Visual Studio and .NET.
UPDATE:
I tried a different, but less efficient approach to this query.
var tasks = (DbContext.Areas
.Where(a => a.UserId == UserManager.GetUserId(User) && !a.OnHold)
.SelectMany(a => a.Goals)
.Where(g => !g.OnHold)
.Select(g => g.Tasks.Where(tt => !tt.OnHold && !tt.Completed).OrderBy(tt => tt.Sequence).FirstOrDefault()))
.Union(DbContext.Areas
.Where(a => a.UserId == UserManager.GetUserId(User) && !a.OnHold)
.SelectMany(a => a.Goals)
.Where(g => !g.OnHold)
.Select(g => g.Tasks.Where(tt => !tt.OnHold && !tt.Completed && (tt.DueDate.HasValue || tt.AlwaysShow)).OrderBy(tt => tt.Sequence).FirstOrDefault()))
.Distinct()
.Select(t => new CalendarTask
{
Id = t.Id,
Title = t.Title,
Goal = t.Goal.Title,
CssClass = t.Goal.Area.CssClass,
DueDate = t.DueDate,
Completed = t.Completed
});
But this also produced an error:
System.InvalidOperationException: 'Processing of the LINQ expression 'Where<Task>(
source: MaterializeCollectionNavigation(Navigation: Goal.Tasks (<Tasks>k__BackingField, DbSet<Task>) Collection ToDependent Task Inverse: Goal, Where<Task>(
source: NavigationExpansionExpression
Source: Where<Task>(
source: DbSet<Task>,
predicate: (t) => Property<Nullable<int>>((Unhandled parameter: ti).Inner, "Id") == Property<Nullable<int>>(t, "GoalId"))
PendingSelector: (t) => NavigationTreeExpression
Value: EntityReferenceTask
Expression: t
,
predicate: (i) => Property<Nullable<int>>(NavigationTreeExpression
Value: EntityReferenceGoal
Expression: (Unhandled parameter: ti).Inner, "Id") == Property<Nullable<int>>(i, "GoalId"))),
predicate: (tt) => !(tt.OnHold) && !(tt.Completed))' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.'
This is a good example for the need of full reproducible example. When trying to reproduce the issue with similar entity models, I was either getting a different error about DefaulIfEmpty(-1) (apparently not supported, don't forget to remove it - the SQL query will work correctly w/o it) or no error when removing it.
Then I noticed a small deeply hidden difference in your error messages compared to mine, which led me to the cause of the problem:
MaterializeCollectionNavigation(Navigation: Goal.Tasks (<Tasks>k__BackingField, DbSet<Task>)
specifically the DbSet<Task> at the end (in my case it was ICollection<Task>). I realized that you used DbSet<T> type for collection navigation property rather than the usual ICollection<T>, IEnumerable<T>, List<T> etc., e.g.
public class Goal
{
// ...
public DbSet<Task> Tasks { get; set; }
}
Simply don't do that. DbSet<T> is a special EF Core class, supposed to be used only from DbContext to represent db table, view or raw SQL query result set. And more importantly, DbSets are the only real EF Core query roots, so it's not surprising that such usage confuses the EF Core query translator.
So change it to some of the supported interfaces/classes (for instance, ICollection<Task>) and the original problem will be solved.
Then removing the DefaultIfEmpty(-1) will allow successfully translating the first query in question.
I don't have EF Core up and running, but are you able to split it up like this?
var allTasks = DbContext.Areas
.SelectMany(a => a.Goals)
.SelectMany(a => a.Tasks);
var always = allTasks.Where(t => t.ShowAlways);
var next = allTasks
.OrderBy(tt => tt.Sequence)
.Take(1);
var result = always
.Concat(next)
.Select(t => new
{
// Member assignment
})
.ToList();
Edit: Sorry, I'm not great with query syntax, maybe this does what you need?
var allGoals = DbContext.Areas
.SelectMany(a => a.Goals);
var allTasks = DbContext.Areas
.SelectMany(a => a.Goals)
.SelectMany(a => a.Tasks);
var always = allGoals
.SelectMany(a => a.Tasks)
.Where(t => t.ShowAlways);
var nextTasks = allGoals
.SelectMany(g => g.Tasks.OrderBy(tt => tt.Sequence).Take(1));
var result = always
.Concat(nextTasks)
.Select(t => new
{
// Member assignment
})
.ToList();
I would recommend you start by breaking up this query into individual parts. Try iterating through the Goals in a foreach with your Task logic inside. Add each new CalendarTask to a List that you defined ahead of time.
Overall breaking this logic up and experimenting a bit will probably lead you to some insight with the limitations of Entity Framework Core.
I think we might separate the query into two steps. First, query each goals and get the min Sequence task and store them(maybe with a anonymous type like {NextTaskId,Goal}). Then, we query the temp data and get the result. For example
Areas.SelectMany(x=>x.Goals)
.Select(g=>new {
NextTaskId=g.Tasks.OrderBy(t=>t.Sequence).FirstOrDefault()?.Id,
Tasks=g.Tasks.Where(t=>t.ShowAlways)
})
.SelectMany(a=>a.Tasks,(a,task)=>new {
NextTaskId = a.NextTaskId,
Task = task
});
I tried to create the linq request but I'm not sure about the result
var tasks = ( from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
join oneTask in (from t in DbContext.Tasks
group t by t.Id into gt
select new {
Id = gt.Key,
Sequence = gt.Min(t => t.Sequence)
}) on new { t.Id, t.Sequence } equals new { oneTask.Id,oneTask.Sequence }
select new {Area = a, Goal = g, Task = t})
.Union(
from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
where t.ShowAlways
select new {Area = a, Goal = g, Task = t});
I currently don't have EF Core, but do you really need to compare this much?
Wouldn't querying the tasks be sufficient?
If there is a navigation property or foreign key defined I could imaging using something like this:
Tasks.Where(task => task.Sequence == Tasks.Where(t => t.GoalIdentity == task.GoalIdentity).Min(t => t.Sequence) || task.ShowAlways);
I'm trying to get some content by taxonomies, that are a part of that content/ Here's what I have:
var taxonomyTerms = _taxonomyService.GetTerms(taxonomy.Id).Where(t => termsToSearch.Contains(t.Name)).ToList();
var listOfTermsIds= taxonomyTerms.Select(x => x.Id).ToList();
//works well until here, I have my list of terms ids
var originalContentItems = _cm
.Query<GenericContentPart, GenericContentRecord>()
.Join<TermsPartRecord>().Where(l => !l.Terms.Select(t => t.TermRecord.Id).Except(listOfTermsIds).Any()).List();
//this returns no records
I've managed to do this with a foreach, but I want to do the same with an expression. The problem is that last bit of code doesn't return me any records.
Any help?
I found the problem:
contentItems = _cm
.Query<GenericContentPart, GenericContentRecord>()
.Join<TermsPartRecord>().ForPart<TermsPart>().List()
.Where(l => !listOfTermsIds.Except(l.Terms.Select(t => t.TermRecord.Id).ToList()).Any());
Thanks.
I have some issues wrapping my head around the .include in LINQ. I'd appreciate a lot if someone could show me the query to achieve this:
I have two entities, User and Validation (which is a serie of validation codes) . They are related in EF5 as 1:*. I'm trying to get a User object and eager load the Validations collection.
At this moment, I'm doing this (cx being the Context inside a using block):
var user = cx.Users
.Where(u => u.UserGuid.Equals(UserGuid))
.FirstOrDefault();
if (user == null)
{
return 0;
}
var validation = cx.Validations
.Where(v => v.Code.Equals(Code))
.Where(v => v.UserId.Equals(user.UserId))
.FirstOrDefault();
But how do I achieve this in only one query so that then I can use
user.Validations.FirstOrDefault();
without getting an error and by only getting the Validations that satisfy the Validation.Code == Code test?
Thanks for your help (I know I'm sounding confused but that's because I am).
Did you try something like this:
var validation = cx.Validations.Where(v=>v.Code==Code &&
cx.Users.Any(u=>u.UserID==v.UserID &&
u.UserGuid==UserGuid)
).FirstOrDefault();
You might need to include the related records as well by doing:
var user = cx.Users
.Where(u => u.UserGuid.Equals(UserGuid))
.Include(u => u.Validations)
.FirstOrDefault();
but then you should be able to select the requested validaiton with
user.Validations
.FirstOrDefault(v => v.Code.Equals(Code));
I can get 1..0 data by RIA Services fine.
var sFiltered = this.ObjectContext.Sequences.Include("Schedules").Include("Events")
.Include("Events.EventFrames")
.Include("Events.EventRules")
.Include("Events.EventFrames.EventFramePlugins")
.Include("Events.EventFrames.EventFramePlugins.EventFramePluginParameters")
.Include("Events.EventFrames.EventFramePlugins.EventFramePluginContentItems")
.Where(s => s.ID == schedule.SequenceID).FirstOrDefault();
So the code above works great.
The problem is that I want to get data by ClientContentItemID [EventFramePlugins] from [ClientContentItemElements]
Please have a look at the image below. But what I don't like to do is to use 1 extra request from the WPF client to get this data. So the idea is use 1 request to get ALL data I need.
Thank you!!!
I think the following query would return the desired result. it finally selects an anonymous type (ClientContentItemElementId, Sequence) which you can change to get the appropriate result. However, I did not test the generated sql to see if this approach is acceptable at all.
sequences.Include.....Where(s => s.ID == schedule.SequenceID).SelectMany(s => s.Events).SelectMany(e => e.EventFrames).SelectMany(ef => ef.EventFramePlugins)
.SelectMany(efp => efp.EventFramePluginContents).SelectMany(efpc => efpc.ClientContentItems).
SelectMany(cci => cci.ClientContentItemElemts).Where(ccie => ccie.ClientContentItemElementId == myValue).
Select(
ccie =>
new
{
ccie,
ccie.ClientContentItem.EventFramePluginContentItem.EventFramePlugin.EventFrame.Event.
Sequence
});