Ef core: Sequence contains no element when doing MaxAsync

Ef core: Sequence contains no element when doing MaxAsync - c#

I'm using ef core in my asp core API project.
I have to find the highest order index.
Example:
Data table: Id, ForeignId, OrderIndex
So I'm doing:
var highestOrderIndex = await _context
.ExampleDbSet
.Where(x =>
x.ForeignId == foreignId)
.MaxAsync(x =>
x.OrderIndex);
The problem is when the example db set is containing 0 elements. This will throw an exception: Sequence contains no element.
Is there an elegant way to do this? Because I don't want to get all the elements from the database. And it should be async.
Thanks

Actually there is quite elegant (and more performant compared to the suggested in the other answer because it's executing just a single database query) way by utilizing the fact that aggregate methods like Min, Max throw Sequence contains no element exception only when used with non nullable overloads, but nullable overloads simply return null instead.
So all you need is to promote the non nullable property type to the corresponding nullable type. For instance, if the OrderIndex type is int, the only change to your query could be
.MaxAsync(x => (int?)x.OrderIndex);
Note that this will also change the type of the receiving variable highestOrderIndex to int?. You can check for null and react accordingly, or you can simply combine the aggregate function call with ?? operator and provide some default value, for instance
var highestOrderIndex = (await _context.ExampleDbSet
.Where(x => x.ForeignId == foreignId)
.MaxAsync(x => (int?)x.OrderIndex)) ?? -1; // or whatever "magic" number works for you

Doing an AnyAsync and then a MaxAsync will result in two separate database calls. You can condense it into one by making sure the sequence contains a "default" minimum value. This is a useful trick anywhere you use the Linq Max/Min methods, not just in database code:
context.ExampleDbSet
.Where(w => w.ForeignId == foreignId)
.Select(s => s.OrderIndex)
.Concat(new[] { 0 })
.MaxAsync();

Another way with a bit better perfomance than the MaxAsync, if the default value is the one you want to get, if there are no results:
var highestOrderIndex = await _context.ExampleDbSet
.Where(x => x.ForeignId == foreignId)
.OrderByDescending(x => x.OrderIndex)
.Select(x => x.OrderIndex)
.FirstOrDefaultAsync();
TOP is faster than the aggregate functions, look at the execution plan in your SQL Server.

You can find if any records exist and if they do, then to find the max. Something like this:
var query = _context.ExampleDbSet
.Where(x => x.ForeignId == foreignId);
var itemsExist = await query.AnyAsync();
int maxOrderIndex = 0;
if(itemsExist)
{
maxOrderIndex = await query.MaxAsync(x => x.OrderIndex);
}
Here you won't have to retrieve all of the items from the database, only check if a record exists which is much much faster and you can also keep the method async.

You can use DefaultIfEmpty and Select before MaxAsync.
var highestOrderIndex = await _context
.ExampleDbSet
.Where(x =>
x.ForeignId == foreignId)
.Select(x => x.OrderIndex)
.DefaultIfEmpty() // default is 0 if OrderIndex is int or long
// .DefaultIfEmpty(-1) // default is -1
.MaxAsync();

Related

Workaround for DefaultIfEmpty

I have something like this:
public enum CoolEnum
{
Yes = 0,
No = 1,
Perhaps = 2,
Maybe = 3,
Perchance = 4,
}
and I want to do the following:
CoolEnum? enum = await this.context
.MyTable
.Where(x => x.Id == id)
.Select(x => x.CoolEnum)
.DefaultIfEmpty((CoolEnum?)null)
.FirstAsync();
but I get the error
Processing of the LINQ expression 'DefaultIfEmpty<Nullable<CoolEnum>> (...) failed.
This may indicate either a bug or a limitation in EF Core.
As I have seen here, it seems it's a known low priority issue.
Then I thought I could do the following:
CoolEnum enum = await this.context
.MyTable
.Where(x => x.Id == id)
.Select(x => x.CoolEnum)
.FirstAsync();
CoolEnum? nullableEnum = enum == default ? null : enum;
But this will change all my existing enums with the default value to null, and that's not what I want.
Which clean workaround can I use? I see the following options:
Return the whole object of MyTable. But if it's big, this is a waste of resources.
Define a "null" default element in my Enum.
Split the query in 2 queries, one to see if there are elements and the other to get the element.
Neither of them seems clean. Any better idea?

You can select the nullable value (by using the regular C# cast to the corresponding nullable type) and then FirstOrDefault{Async}:
.Select(x => (CoolEnum?)x.CoolEnum)
.FirstOrDefaultAsync();

Problem with LINQ query: Select first task from each goal

I'm looking for suggestions on how to write a query. For each Goal, I want to select the first Task (sorted by Task.Sequence), in addition to any tasks with ShowAlways == true. (My actual query is more complex, but this query demonstrates the limitations I'm running into.)
I tried something like this:
var tasks = (from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
let nextTaskId = g.Tasks.OrderBy(tt => tt.Sequence).Select(tt => tt.Id).DefaultIfEmpty(-1).FirstOrDefault()
where t.ShowAlways || t.Id == nextTaskId
select new CalendarTask
{
// Member assignment
}).ToList();
But this query appears to be too complex.
System.InvalidOperationException: 'Processing of the LINQ expression 'OrderBy<Task, int>(
source: MaterializeCollectionNavigation(Navigation: Goal.Tasks(< Tasks > k__BackingField, DbSet<Task>) Collection ToDependent Task Inverse: Goal, Where<Task>(
source: NavigationExpansionExpression
Source: Where<Task>(
source: DbSet<Task>,
predicate: (t0) => Property<Nullable<int>>((Unhandled parameter: ti0).Outer.Inner, "Id") == Property<Nullable<int>>(t0, "GoalId"))
PendingSelector: (t0) => NavigationTreeExpression
Value: EntityReferenceTask
Expression: t0
,
predicate: (i) => Property<Nullable<int>>(NavigationTreeExpression
Value: EntityReferenceGoal
Expression: (Unhandled parameter: ti0).Outer.Inner, "Id") == Property<Nullable<int>>(i, "GoalId"))),
keySelector: (tt) => tt.Sequence)' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.'
The problem is the line let nextTaskId =.... If I comment out that, there is no error. (But I don't get what I'm after.)
I'll readily admit that I don't understand the details of the error message. About the only other way I can think of to approach this is return all the Tasks and then sort and filter them on the client. But my preference is not to retrieve data I don't need.
Can anyone see any other ways to approach this query?
Note: I'm using the very latest version of Visual Studio and .NET.
UPDATE:
I tried a different, but less efficient approach to this query.
var tasks = (DbContext.Areas
.Where(a => a.UserId == UserManager.GetUserId(User) && !a.OnHold)
.SelectMany(a => a.Goals)
.Where(g => !g.OnHold)
.Select(g => g.Tasks.Where(tt => !tt.OnHold && !tt.Completed).OrderBy(tt => tt.Sequence).FirstOrDefault()))
.Union(DbContext.Areas
.Where(a => a.UserId == UserManager.GetUserId(User) && !a.OnHold)
.SelectMany(a => a.Goals)
.Where(g => !g.OnHold)
.Select(g => g.Tasks.Where(tt => !tt.OnHold && !tt.Completed && (tt.DueDate.HasValue || tt.AlwaysShow)).OrderBy(tt => tt.Sequence).FirstOrDefault()))
.Distinct()
.Select(t => new CalendarTask
{
Id = t.Id,
Title = t.Title,
Goal = t.Goal.Title,
CssClass = t.Goal.Area.CssClass,
DueDate = t.DueDate,
Completed = t.Completed
});
But this also produced an error:
System.InvalidOperationException: 'Processing of the LINQ expression 'Where<Task>(
source: MaterializeCollectionNavigation(Navigation: Goal.Tasks (<Tasks>k__BackingField, DbSet<Task>) Collection ToDependent Task Inverse: Goal, Where<Task>(
source: NavigationExpansionExpression
Source: Where<Task>(
source: DbSet<Task>,
predicate: (t) => Property<Nullable<int>>((Unhandled parameter: ti).Inner, "Id") == Property<Nullable<int>>(t, "GoalId"))
PendingSelector: (t) => NavigationTreeExpression
Value: EntityReferenceTask
Expression: t
,
predicate: (i) => Property<Nullable<int>>(NavigationTreeExpression
Value: EntityReferenceGoal
Expression: (Unhandled parameter: ti).Inner, "Id") == Property<Nullable<int>>(i, "GoalId"))),
predicate: (tt) => !(tt.OnHold) && !(tt.Completed))' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.'

This is a good example for the need of full reproducible example. When trying to reproduce the issue with similar entity models, I was either getting a different error about DefaulIfEmpty(-1) (apparently not supported, don't forget to remove it - the SQL query will work correctly w/o it) or no error when removing it.
Then I noticed a small deeply hidden difference in your error messages compared to mine, which led me to the cause of the problem:
MaterializeCollectionNavigation(Navigation: Goal.Tasks (<Tasks>k__BackingField, DbSet<Task>)
specifically the DbSet<Task> at the end (in my case it was ICollection<Task>). I realized that you used DbSet<T> type for collection navigation property rather than the usual ICollection<T>, IEnumerable<T>, List<T> etc., e.g.
public class Goal
{
// ...
public DbSet<Task> Tasks { get; set; }
}
Simply don't do that. DbSet<T> is a special EF Core class, supposed to be used only from DbContext to represent db table, view or raw SQL query result set. And more importantly, DbSets are the only real EF Core query roots, so it's not surprising that such usage confuses the EF Core query translator.
So change it to some of the supported interfaces/classes (for instance, ICollection<Task>) and the original problem will be solved.
Then removing the DefaultIfEmpty(-1) will allow successfully translating the first query in question.

I don't have EF Core up and running, but are you able to split it up like this?
var allTasks = DbContext.Areas
.SelectMany(a => a.Goals)
.SelectMany(a => a.Tasks);
var always = allTasks.Where(t => t.ShowAlways);
var next = allTasks
.OrderBy(tt => tt.Sequence)
.Take(1);
var result = always
.Concat(next)
.Select(t => new
{
// Member assignment
})
.ToList();
Edit: Sorry, I'm not great with query syntax, maybe this does what you need?
var allGoals = DbContext.Areas
.SelectMany(a => a.Goals);
var allTasks = DbContext.Areas
.SelectMany(a => a.Goals)
.SelectMany(a => a.Tasks);
var always = allGoals
.SelectMany(a => a.Tasks)
.Where(t => t.ShowAlways);
var nextTasks = allGoals
.SelectMany(g => g.Tasks.OrderBy(tt => tt.Sequence).Take(1));
var result = always
.Concat(nextTasks)
.Select(t => new
{
// Member assignment
})
.ToList();

I would recommend you start by breaking up this query into individual parts. Try iterating through the Goals in a foreach with your Task logic inside. Add each new CalendarTask to a List that you defined ahead of time.
Overall breaking this logic up and experimenting a bit will probably lead you to some insight with the limitations of Entity Framework Core.

I think we might separate the query into two steps. First, query each goals and get the min Sequence task and store them(maybe with a anonymous type like {NextTaskId,Goal}). Then, we query the temp data and get the result. For example
Areas.SelectMany(x=>x.Goals)
.Select(g=>new {
NextTaskId=g.Tasks.OrderBy(t=>t.Sequence).FirstOrDefault()?.Id,
Tasks=g.Tasks.Where(t=>t.ShowAlways)
})
.SelectMany(a=>a.Tasks,(a,task)=>new {
NextTaskId = a.NextTaskId,
Task = task
});

I tried to create the linq request but I'm not sure about the result
var tasks = ( from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
join oneTask in (from t in DbContext.Tasks
group t by t.Id into gt
select new {
Id = gt.Key,
Sequence = gt.Min(t => t.Sequence)
}) on new { t.Id, t.Sequence } equals new { oneTask.Id,oneTask.Sequence }
select new {Area = a, Goal = g, Task = t})
.Union(
from a in DbContext.Areas
from g in a.Goals
from t in g.Tasks
where t.ShowAlways
select new {Area = a, Goal = g, Task = t});

I currently don't have EF Core, but do you really need to compare this much?
Wouldn't querying the tasks be sufficient?
If there is a navigation property or foreign key defined I could imaging using something like this:
Tasks.Where(task => task.Sequence == Tasks.Where(t => t.GoalIdentity == task.GoalIdentity).Min(t => t.Sequence) || task.ShowAlways);

EntityFramework: OrderBy() numeric property when mapped column type is textual

return JsonConvert.SerializeObject(new Entities()
.Student_Master
.Where(k => k.Student_Location == Location && k.Student_Course == Program)
.OrderBy(i => i.Student_Batch)
.Select(i => i.Student_Batch)
.Distinct()
.ToList());
Output:
[23,24,28,25,30,26,27,29]
require Output
[23,24,25,26,27,28,29,30]
I tried with OrderBy(i => i.Student_Batch) but in database Student_Batch datatype is string so not sorting correctly
I tried like following
var data=new Entities().Student_Master.Where(k => k.Student_Location == Location && k.Student_Course == Program).OrderBy(i => i.Student_Batch).Select(i => i.Student_Batch).Distinct().ToList();
foreach(var obj in data)
{
//converted string to int then store in array
}
Is there any easy way?

Okay so since the problem is with sorting. You have few options and i will show 2 of them. First is that you can use Array.Sort() which is pretty common:
string[] values = new Entities()
.Student_Master
.Where(k => k.Student_Location == Location && k.Student_Course == Program).Select(i => i.Student_Batch)
.Distinct().ToArray();
Array.Sort(values); // all you need.
Second common way is to create custom comparer and use it inside OrderBy :
public class MeComparer : IComparer<string> {
public int Compare(string stringA, string stringB) {
// your compare logic goes here...
// eg. return int.Parse(stringA) - int.Parse(stringB)
}
}
// and use it like
return JsonConvert.SerializeObject(new Entities()
.Student_Master
.Where(k => k.Student_Location == Location && k.Student_Course == Program)
.Select(i => i.Student_Batch)
.Distinct()
.ToList()
.OrderBy(i => i.Student_Batch, new MeComparer()) // <-- HERE
);

.Distinct() removes any .OrderBy() clause, because by definition, Distinct() (or DISTINCT in SQL) returns an un-ordered set of distinct values. You need to chain your .OrderBy() call after the .Distinct() call.
Having your values as strings does pose a problem when you want to sort them by their numeric value. If you can't change the database schema, you can use this method to project the values to integers, and then do .Distinct() and .OrderBy().
Finally, you should properly dispose your Entities object after you use it, to close the database connection, preferably by enclosing it in a using directive.

As Linq2Entities does not support conversion from string to int (neither using int.Parse not Convert.ToInt32) you have to convert your IQueryAble to IEnumerable using AsEnumerable. Of course this is performance-wise nightmare, however as SQL has no way on making integer-operations on strings by on-the-fly-conversion this is what you need.
Btw.: Using ToArrayor ToList will also enumerate the collection and put it into the memory.

Why do the 2 LINQ queries get evaluated differently

Query 1
var resulty = db.QIS
.Where(w=>w.QSA.Any(a => a.QSID != w.QSID))
.Select(s => s.QSID).ToList();
Query 2
var resultz = db.QIS
.Where(w=>w.QSA.Where(h => h.QSID == w.QSID).Select(s => s.QSID).FirstOrDefault() != w.QSID)
.Select(s => s.QSID).ToList();
Table QIS and QSA are related Many:Many. The idea here is to find how many QIS.ID's are not Found in QSA where QIS.QID == QSA.QID.
Query 1 returns 0
Query 2 on the other hand gives me what I expected to see ( the list off all non matching QIS records.)
Why will the any not work? - i found myself running into the same situation a couple of times now in seperate scenarios... thanks for any help / thoughts.
PS: Prefer method syntax.

In the filtering in your second version, will only be true if the inner Where clause returns no elements, so that FirstOrDefault() returns null or 0 (depending on if the type is nullable or not).
w=>w.QSA.Where(h => h.QSID == w.QSID)
.Select(s => s.QSID).FirstOrDefault() != w.QSID
Which is equivalent to (now assuming QSID is a non nullable numeric type, if it is nullable, use null instead of zero):
w=>w.QSA.Where(h => h.QSID == w.QSID)
.Select(s => s.QSID).FirstOrDefault() == 0
which can be rewritten to:
w=>w.QSA.Where(h => h.QSID == w.QSID).FirstOrDefault() == null
which can be rewritten to:
w=>!w.QSA.Any(h => h.QSID == w.QSID)
which is nearly the same as your initial version, but not exactly. You still want to check for equivalence inside the Any() filter, but then negate the result.

Chained selects with where at the end

Given following structure: a person has functions. Each function has roles. Each roles has features. Now I would like to figure out with linq if a given person has a certain feature, but I am doing something wrong with this query. As a result I always get the count of the functions (but I'd like to get the count of the features):
var count = person.Functions
.Select(fu => fu.Roles
.Select(r => r.Features
.Where(f => f.FeatureId == 99999)))
.Count();
What am I doing wrong here? According to this query I expect either 0 (hasn't got the feature) or 1.

var query = from function in person.Functions
from role in function.Roles
from feature in role.Features
where feature.FeatureId == 99999
select feature;
var count = query.Count();
or
var count = person.Functions
.SelectMany(function => function.Roles)
.SelectMany(role => role.Features)
.Count(feature => feature.FeatureId == 99999);
If you don't need the exact count but just want to know if the person has the feature or not, use Any instead of Count.

var count = person.Functions
.SelectMany(p => p.Roles)
.SelectMany(r => r.Features)
.Where(f => f.FeatureId == 99999)
.Count();
I'm not really sure, but I think you want the total number of Features with teh given Id. You would want to use SelectMany.

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

Ef core: Sequence contains no element when doing MaxAsync - c#

You can use DefaultIfEmpty and Select before MaxAsync. var highestOrderIndex = await _context .ExampleDbSet .Where(x => x.ForeignId == foreignId) .Select(x => x.OrderIndex) .DefaultIfEmpty() // default is 0 if OrderIndex is int or long // .DefaultIfEmpty(-1) // default is -1 .MaxAsync();

Related

Workaround for DefaultIfEmpty

Problem with LINQ query: Select first task from each goal

EntityFramework: OrderBy() numeric property when mapped column type is textual

Why do the 2 LINQ queries get evaluated differently

Chained selects with where at the end

Categories

Resources