Develop Reference

LINQ statement is not translatable

I have the following code containing LINQ statements:
public async Task<HashSet<long>> GetMembersRecursive(IEnumerable<long> groupIds)
{
var containsGroupId = InExpression<Group>("Id", groupIds);
var containsParentId = InExpression<RecursiveGroupModel>("ParentId", groupIds);
var groupIdsArray = groupIds as long[] ?? groupIds.ToArray();
return new HashSet<long>(await MyContext
.Groups
.Where(containsGroupId)
.Select(a => new
{
Members = MyContext
.ViewWithRecursiveGroups
.Where(containsParentId)
.SelectMany(c => c.Group.Members)
.Union(a.Members)
.Where(b => !b.User.IsActive)
})
.SelectMany(a => a.Members.Select(b => b.MemberId))
.Distinct()
.ToListAsync());
}
private static Expression<Func<T, bool>> InExpression<T>(string propertyName, IEnumerable<long> array)
{
var p = Expression.Parameter(typeof(T), "x");
var contains = typeof(Enumerable).GetMethods(BindingFlags.Static | BindingFlags.Public)
.Single(x => x.Name == "Contains" && x.GetParameters().Length == 2)
.MakeGenericMethod(typeof(long));
var property = Expression.PropertyOrField(p, propertyName);
var body = Expression.Call(
contains
, Expression.Constant(array)
, property
);
return Expression.Lambda<Func<T, bool>>(body, p);
}
The error I receive is:
Microsoft.EntityFrameworkCore: Processing of the LINQ expression 'DbSet<RecursiveGroupModel>
.Where(b => __groupIdsArray_1
.Contains(b.ParentId))
.SelectMany(c => c.Group.GroupMembers)
.Union((MaterializeCollectionNavigation(
navigation: Navigation: Group.GroupMembers,
subquery: (NavigationExpansionExpression
Source: DbSet<GroupMember>
.Where(l0 => EF.Property<Nullable<long>>(l, "Id") != null && EF.Property<Nullable<long>>(l, "Id") == EF.Property<Nullable<long>>(l0, "GroupId1"))
PendingSelector: l0 => (NavigationTreeExpression
Value: (EntityReference: GroupMember)
Expression: l0)
)
.Where(i => EF.Property<Nullable<long>>((NavigationTreeExpression
Value: (EntityReference: Group)
Expression: l), "Id") != null && EF.Property<Nullable<long>>((NavigationTreeExpression
Value: (EntityReference: Group)
Expression: l), "Id") == EF.Property<Nullable<long>>(i, "GroupId1"))))' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.
The view:
CREATE VIEW [dbo].[View_WithRecursiveGroups] AS
WITH RecursiveGroups (GroupId, ParentId) AS
(
SELECT Id, ParentId
FROM Group
WHERE ParentId IS NOT NULL
UNION ALL
SELECT Group.Id, t.ParentId
FROM GroupTree t
JOIN Group ON t.GroupId = Group.ParentId
)
SELECT * FROM RecursiveGroups
Apologies in advance if some variable names don't match up- I had to sanitize before posting.
I understand that it cannot convert code to SQL and so it's asking me to enumerate early or rewrite so that it's translatable. I have tired rearranging the query and breaking it up into smaller queries but the SelectMany on the recursive view seems to not be possible to convert to SQL.
Is there a way to get this working in-database? Or am I going about this completely the wrong way?

As an alternative, you can use raw sql query. In Entity Framework Code, we need to define a POCO class and a DbSet for that class. In your case you will need to define some YourClass:
public DbQuery<YourClass> YourClasses { get; set; }
and code to execute:
var result = context.YourClasses.FromSql("YOURSQL_SCRIPT").ToList();
var asyncresult = await context.YourClasses.FromSql("YOURSQL_SCRIPT").ToListAsync();

Yeah, welcome to the wonderfull world of EfCore 3.1 where all you can do is "Hello world".
Your query has various "problems" because EfCore does not really do LINQ processing except for super easy cases.
.Union(a.Members)
Can not be translated to run server side and client side processing is not enabled. Your only choises are:
Force server execution for both parts (using AsEnumerable) then Union on the client. That only works if you do not use that as part of a larger statement (i.e. intersect) otherwise it is "pull all the data to the client" time and that is not good.
At the current point in time I can only advice you to throw out EfCore and use EntityFramework which - as per framework 3.1 - is again available. Or use Entity Framework Classic which is a port that runs on netstandard 2.0 and has global query filters (which are THE ONE feature of EfCore I like). At last this is what I am currently getting to because - well - "better but without any features and not working" is not cutting it for me.
Whether or not EfCore will be extended (they seem not to see it as a fix) to handle anything except the most basic LINQ statements (and sometimes even not those) is unknown at this point - a lot of the changes in 3.1 are quite discouraging.
You MAY be able to move it into views etc. - but you may find out quite fast that EfCore has even more limitations and maintaining all the views gets quite tendious, too. I run into serious problems with the fact that I can not put any condition in front of any projection even in the most simple cases. And even simple bugs get commented on "we do not feel comfortable changing the pipeline, please wait for version 5 in november". Example? https://github.com/dotnet/efcore/issues/15279.

give that if you want to convert this view to Linq...
CREATE VIEW [dbo].[View_WithRecursiveGroups] AS
WITH RecursiveGroups (GroupId, ParentId) AS
(
SELECT Id, ParentId
FROM Group
WHERE ParentId IS NOT NULL
UNION ALL
SELECT Group.Id, t.ParentId
FROM GroupTree t
JOIN Group ON t.GroupId = Group.ParentId
)
var data1 = db.Group.where(x=>x.ParentId != nul)
.Select(x=>new {x.Id, x.ParentId})
.Tolist()
var data2 = (from g in db.Groups
join gt in db.GroupTree on g.ParentId equals gt.GroupId
select new { d.Id, ParentId })
.ToList();
create a class reprocenting the data and have the query return as List of known type and
just union the two lists.
linqpad is a very useful tool for learn how to create the linq which give you the sql you want.

LINQ Query OrderByDescending not recognizing lambda expression

I'm fairly new to LINQ, but I think I'm getting the hang of it.
I'm trying to group a select statement, then order it descending. I've got pretty far just looking through other questions, but when I try to attach the OrderByDescending() function, lambda expressions I try to add are not recognized by intellisense.
var test = db.UserPokemons.Where(x => x.PkmnDataId == pokemon.Id && x.StepsToHatch == 0)
.GroupBy(n => n.User1.Country).OrderByDescending();
So, for example, .OrderbyDescending(x => x.User1.Country) does not contain a definition for User1.
Is this to do with the ordering of my statements? I think it's because I'm calling GroupBy before OrderBy, but I can't wrap my head around how to fix it.
How can I order my groups?
Oh! Nearly forgot - I only want the top 3 countries, so is there an easy way to restrict that as well?
How to use GroupBy and OrderByDescending in the same LINQ function
How to select only the top/first 3 groups
Thanks for any help!

I think this is what you want:
var test = db.UserPokemons.Where(x => x.PkmnDataId == pokemon.Id && x.StepsToHatch == 0).
GroupBy(n => n.User1.Country).Select(x=>x.First()).OrderByDescending().Take(3);

var countriesInOrder = db.UserPokemons.Where(x => x.PkmnDataId == pokemon.Id && x.StepsToHatch == 0).
GroupBy(n => n.User1.Country).OrderByDescending(x => x.Key).Take(3).ToArray();
Got it - the lambda expression needs to go to the x.Key after grouping.
(Thanks jitender!)

Why Reverse() cannot be converted into SQL?

My application is running under ASP.NET 4.0, which uses BLToolkti as ORM tool.
I have some queryable expression:
var q = db.GetTable<T>()
.Where(tb=>tb.TeamId==MyTeamId && tb.Season==MySeasonId)
.OrderByDescending(tb=>tb.Id)
.Take(20)
.Reverse()
Attempt to convert q.ToList() causes the following error:
Sequence 'Table(TeamBudget).Where(tb => ((tb.TeamId ==
value(VfmElita.DataLogicLayer.Teams.Team+TeamBudget+<>c__DisplayClass78).teamId)
AndAlso (tb.Season ==
value(VfmElita.DataLogicLayer.Teams.Team+TeamBudget+<>c__DisplayClass78).season))).OrderByDescending(tb
=> Convert(tb.Id)).Take(20).Reverse()' cannot be converted to SQL.
If I remove ".Reverse()" from the queryable object everything works fine.
What is the reason why queryable object with .Reverse() cannot be converted into SQL? Is that BLToolkit limitation? Is there any solution workaround for that?
Thank you!

It's pretty clear what the other LINQ methods convert to (where, order by, top(20)), but what would Reverse() convert to? I can't think of an SQL statement I've seen that mimics that behavior, and when you're querying the database your LINQ statement must ultimately resolve to valid SQL.
This may not be what you're going for, but one option would be to execute the query first using ToList(), then apply Reverse():
var q = db.GetTable<T>()
.Where(tb => tb.TeamId == MyTeamId && tb.Season == MySeasonId)
.OrderByDescending(tb => tb.Id)
.Take(20)
.ToList()
.Reverse();
Alternatively, you could get the count and skip that many records first, although this could be inaccurate if the number of records change between calls. Plus it's two queries instead of just one.
var totalRecords = db.GetTable<T>()
.Count(tb => tb.TeamId == MyTeamId && tb.Season == MySeasonId);
var q = db.GetTable<T>()
.Where(tb => tb.TeamId == MyTeamId && tb.Season == MySeasonId)
.Order(tb => tb.Id)
.Skip(totalRecords)
.Take(20);

Need help to use Expression Tree to build a LINQ query

I need to build an epression tree for a LINQ query that looks something like this:
collection.OrderBy(e => ((MyObject)e["PropertyIndex"]).dictionary.Where(k => k.Key == "keyName").Select(k => k.Value).Single());
I looked at this link that explains how to chain OrderBy methods. I don't know how do I add Where inside OrderBy using Expression Tree.
Update:
I need to sort data in memory dynamically. So the linq query could look something like this:
collection.OrederBy(field1).ThenByDescending(field2).ThenBy(field3)
I know only at runtime how many fields I need to sort by.
Any of fieldX can be a complex object. The type of the object of course will be known at runtime. One of the object has a structure like you see in the LINQ query above. It has a dictionary and I have to sort for a specific key. In my case dictionary contains localized data like:
{{"en-US", "word (en)"}, {"es-ES", "word (es)"} , ....}
I need to sort by specific language.

It appears your query is doing this:
from k in collection
where k.Key == "keyName"
orderby ((MyObject)k)["PropertyIndex"]
select k.Value
and you could add more where clauses like this:
from k in collection
where k.Key == "keyName"
&& k.OtherProperty == OtherValue
orderby ((MyObject)k)["PropertyIndex"]
select k.Value
EDIT: With the clarified requirements, I'd recommend you first do all your where clauses (no need to sort data you'll just ignore), then apply all the .OrderBy(). If you can make them lambdas, that's much easier than the link you suggested (pun intended):
.OrderBy( e => e.Property1 ).OrderBy( e => e.Property2 )
If you'd like to "dynamically" form these, do something like this:
var query = (from k in collection select k);
query = query.Where( e => e.Property1 == "value" );
var orderedQuery = query.OrderBy( e => e.Property1 );
orderedQuery = query.Orderby( e => e.Property2 );
var result = orderedQuery.Select( e => e.Value ).Single();
Sprinkle some conditions around these things, and you'll be golden. Note that query is of type IQueriable<T> and orderedQuery is of type IOrderedQueriable<T>, which is why you can't (without casting) reuse the same var.

You just need to apply first order field by OrderBy and all other fields by ThenBy. Of cource you have to use temporarry variable of type IOrderedEnumerable.
If you need to add some filters, then you have to add Where BEFORE any Order.
If there is many possible order options and you don't want to hardcode them, then you can use Dynamic LinQ and specify order fields as strings.

Issue using Where method in LINQ

Consider this line of code:
List<SIDB_TransactionInformation> transaction = SIDB.SIDB_TransactionInformations
.Where(k => k.iscurrent == true & k.objectid == SIDB.func_GetObjectID("dbo.SIDB_Module")).ToList();
List<SIDB_Module> module = SIDB.SIDB_Modules
.Where(k => k.moduleid == transaction
.Where(j => j.transactionid == k.moduleid)
.SingleOrDefault().transactionid).ToList();
I do have 2 invocation of where method in different collection. First i distinct my list via iscurrent and objectid after that I do have other invocation of where method (for SIDB_Modules) to distinct the list via moduleid where in the the values refer to the transactionid of my previous list. Now i have an error message like this Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains() operator.
sorry i'm new in lambda expression. need help badly

I think this is what you're looking for
List<SIDB_Module> module = SIDB
.SIDB_Modules
.Where(k => transaction.Any(j => j.transactionid == k.moduleid))
.ToList();
Make a list of SIDB_Modules where there is a transaction whose transactionid is equal to the moduleid. LINQ to Sql might have an issue with Any, I don't remember, if it does you can rewrite it with an extra step like this
var transactionIds = transaction.Select(j => j.transactionid);
List<SIDB_Module> module = SIDB
.SIDB_Modules
.Where(k => transactionIds.Contains(k.moduleid))
.ToList();
If performance is an issue you might consider going with the second method and putting transactionIds into something that implements ISet<T> and has a constant time lookup.

Well, it looks like you're trying to do a join between SIDB_TransactionInformations and SIDB.SIDB_Modules. If so, try
var objectID = SIDB.func_GetObjectID("dbo.SIDB_Module");
List<SIDB_Module> modules = (from module in SIDB.SIDB_Modules
join transaction in SIDB.SIDB_TransactionInformations on module.moduleid equals transaction.transactionid
where transaction.iscurrent && transaction.objectid == objectID
select module).ToList();