I am trying to build a sane query in EF Core that returns a collection of things that are, in turn derived from a collection of things. Basically in raw SQL one would do a JOIN.
Its in ASP.NET Core so the initial collection is the list of Roles on the SecurityPrincipal object:
var roles = User.FindAll(ClaimTypes.Role).Select(r=>r.Value);
These roles are then mapped to Groups in our Database, so I can look those up:
var groupsQuery = dbContext.Groups.Where(g=>roles.Any(r=>r==g.GroupName));
var groups = await groupsQuery.ToListAsync();
This query is quite happy and returns a collection of groups as expected. The groups however have access to another resource, which is what I really want and because its a many to many relationship there is a bridging table.
This is me trying to query the AssetGroup joining table so I can get all the Assets referenced by all the Groups that map to a Role on the SecurityPrincipal.
var assetGroupsQuery = dbContext.AssetsGroups.Where(ag => groupsQuery.Any(ag => ag.Id == a.GroupId));
var assetGroups = await assetGroupsQuery.ToListAsync();
When I perform the second query I get a lot of spam in my output window:
The LINQ expression 'where ([ag].Id == [ag].GroupId)' could not be translated and will be evaluated locally.
The LINQ expression 'Any()' could not be translated and will be evaluated locally.
The LINQ expression 'where {from Group g in __groups_0 where ([ag].Id == [ag].GroupId) select [ag] => Any()}' could not be translated and will be evaluated locally.
The LINQ expression 'where ([ag].Id == [ag].GroupId)' could not be translated and will be evaluated locally.
The LINQ expression 'Any()' could not be translated and will be evaluated locally.
Any clues on how one should phrase a nested query like this so EF Core can compose a single SQL query properly?
In general avoid using Any or any LINQ operator other than Contains on in memory collection like your roles (which according to the code should be of type IEnumerable<string>).
In other words, instead of
.Where(g => roles.Any(r => r == g.GroupName))
use the functionally equivalent
.Where(g => roles.Contains(g.GroupName))
The later is guaranteed to be translated to SQL IN, while the former isn't.
Interestingly and at the same time misleading is that EF Core tries to be smart and translate the former the same way as Contains, and succeeds when the containing query is executed, but not when used as part of another query.
It could be considered a current EF Core implementation defect. But the workaround/solution is (as mentioned in the beginning) to not rely on it and always use Contains.
Related
In linq is there a difference between:
EFDbContext _db = new EFDbContext();
1)_db.UserQuizes
.Where(uq => uq.UserId == currentUserId && uq.QuizId == quizId)
.Include(qz => qz.Quiz.VerbalQuizes.Select(q => q.Question)).First()
2)_db.UserQuizes
.Include(qz => qz.Quiz.VerbalQuizes.Select(q => q.Question))
.Where(uq => uq.UserId == currentUserId && uq.QuizId == quizId).First()
3)_db.UserQuizes
.Include(qz => qz.Quiz.VerbalQuizes.Select(q => q.Question))
First(uq => uq.UserId == currentUserId && uq.QuizId == quizId)
Notice that first query uses include after where and second before where, but result is the same. Also how to see actual sql query? In this particular case perfomance is my main goal, can i improve the query? i need to change two properties : UserQuizes property, and UserQuizes-> VerbalQuizes-> Question property.
Would it be better to split up it two queries or use it like as it is
Ordering of instructions like you've shown often won't make a difference in EF or LINQ to SQL. The query builder turns your entire LINQ statement into an abstract logical representation, and then another pass converts the logical structure into a SQL statement. So the WHERE predicates are all going to end up in the same place. The predicates inside a First() just get pushed over to the WHERE clause. The Include statements also get accumulated and projected to JOINs to include the extra columns needed to produce the included entity.
So the short answer is that EF will usually produce the most logical SQL statement regardless of the order in which you constructed your LINQ statement. If you need to tune it further, you should look at a stored procedure where you can hand-craft the SQL.
I am using CosmosDB and what to load filtered data. I want to do the filtering on the database side but I had some problems, which I think I managed to resolve but not sure why one way works but not another.
I have a list of CarPositions
// I fill this list from cosmosdb data
var carPosition = new List<CarPositions>()
My goal is to only get the cars if I have their positions in the carPosition list
I tried to do something like this
var iterator = CarContainer
.GetItemLinqQueryable<Car>(true)
.Where(x => carPosition.Any(cp => cp.Id == x.Id))
.ToFeedIterator();
Which throws an exception; "Input is not of type IDocumentQuery"
From what I understood from looking online is that CosmosDb provider does not support ANY linq when translating the query to SQL
This then lead me to try doing the same thing but with using Select and Contain
1)
var iterator = CarContainer
.GetItemLinqQueryable<Car>(true)
.Where(x => carPosition.Select(cp => cp.Id).ToList().Contain(x.Id))
.ToFeedIterator();
Again causing the same exception
I then tried this which works but to me this is doing the exact same thing
2)
var carPosIds = carPosition.Select(cp => cp.Id).ToList();
var iterator = CarContainer
.GetItemLinqQueryable<Car>(true)
.Where(x => carPosIds.Contain(x.Id))
.ToFeedIterator();
Can someone explain why the second one works but not the first? Also why the Any linq did not work?
Because the linq statement needs to be translated into a sql query which will run on the database server. The Contains statement inside the Where statement is supported because the SDK has implemented a translation using the sql IN clause (presumably).
While your Any query seems the same, just imagine it being a different expression and how difficult (or nearly impossible) it would become to translate it into an SQL query as the expression grows more complex. E.g. .Any(cp => cp.Id * 2 - x.Id / 2 == x.Id).
If this would be pure C# code running in memory instead of being translated by the SDK into a query for Cosmos all above methods would work (some better than others).
I know that lots of question related to this error, but I can not find out a way to convert my query to meet my query. My error: 'LINQ to Entities does not recognize the method 'Boolean TryGetValue(Int32, System.Collections.Generic.List1[System.Nullable1[System.Int32]] ByRef)' method, and this method cannot be translated into a store expression.'
My mind is melting down!
var groupedKeyAndValueOfProjectIdAndZoneIds = groupedProjectDelegationByProjectId.ToDictionary(keySelector: x => x.ProjectId, elementSelector: x => x.ZoneIds);
...
var data = projects
.Select(p => new Project
{
Id = p.Id,
ProjectName = p.Name,
Zones = p.Zones.Where(z =>
(zoneIds.Contains(z.Id) || (groupedKeyAndValueOfProjectIdAndZoneIds.TryGetValue(p.Id, out outValue) ? outValue.Contains(z.Id) : false)))
...
Given that groupedKeyAndValueOfProjectAndZones is Dictionary<int, List<int>>.
Please help me.
The problem you are having is that you are trying to mix two sources of data together. Underneath the hood LINQ to Entities wants to take the expression you are expressing in LINQ and translate it into a SQL query. In other words when you are writing a select in LINQ you are getting 1:1 mapping in SQL. When you throw a dictionary of data the way groupedKeyAndValueOfProjectAndZones is into the mix LINQ to Entities doesn't know how to represent this as its an in memory data source that has no SQL equivalent to run.
To fix this you need to either move the data contained in groupedKeyAndValueOfProjectAndZones into the database and query it from there or you need to provide the filtering you are doing post the LINQ to Entities query
My current database solution includes three tables called Establishment, Feature, and a linking many-to-many table called EstablishmentFeature (since an establishment can have many features, and a feature can exists across multiple establishments).
I need to generate a query that returns establishments that meet only certain criteria, namely, which establishments have X features based on a collection of featureId's being passed in. The establishment must have ALL features that are being search, i.e.. AND not OR condition.
I got the SQL to achieve the desired result, but I am pulling my hair out trying to work out the LINQ (lambra prefereably) equivalent. The T-SQL is:
SELECT e.[EstablishmentId], e.[Name], e.[Description]
FROM Establishment e
INNER JOIN EstablishmentFeature ef
ON e.[EstablishmentId] = ef.[EstablishmentId]
WHERE ef.[FeatureId] in ('20', '15', '72')
GROUP BY e.[EstablishmentId], e.[Name], e.[Description]
HAVING COUNT(*) = 3
I tried to use Linqer to convert the SQL but Linqer crashes when it attempts the conversion. I tried reinstalling Linqer, but it crashes without fail when trying to compile the LINQ syntax. (Simpler conversions work though). Also tried to work out the LINQ equivalent using LinqPad, but I just ended up chasing my tail...
Is this something I will have to use PredicateBuilder for? Somewhat exhausted, I don't want to go through the PredicateBuilder learning curve if there is a simple solution that is escaping me.
I'd try this (For all given ids there is any (= at least one) feature that has this given id):
var establishments = context.Establishments
.Where(e => featureIds.All(fid => e.Features.Any(f => f.FeatureId == fid)))
.ToList();
(featureIds is an IEnumerable<int> with the Ids being searched for.)
I am trying to implement some search functionality within our app and have a situation where a User can select multiple Topics from a list and we want to return all activities that match at least one of the selected Topics. Each Activity can have 0-to-many Topics.
I can write a straight SQL query that gives me the results I want like so:
SELECT *
FROM dbo.ACTIVITY_VERSION av
WHERE ACTIVITY_VERSION_ID IN (
SELECT ACTIVITY_VERSION_ID
FROM dbo.ACTIVITY_TOPIC at
WHERE at.TOPIC_ID IN (3,4)
)
What I can't figure out is how to write a LINQ query (we are using Linq to Sql) that returns the same results.
I've tried:
activities.Where(x => criteria.TopicIds.Intersect(x.TopicIds).Any());
this works if activities is a list of in memory objects (i.e. a Linq to Objects query), but I get an error if I try to use the same code in a query that hits the database. The error I receive is:
Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
I believe that this means that Linq to Sql doesn't know how to translate either Intersect or Any (or possibly both). If that is the case, I understand why it isn't working, but I still don't know how to make it do what I want it to and my Google-fu has not provided me with anything that works.
Haven't tested it. But this is how you ll go about it.
List<int> IDs = new List<int>();
IDs.Add(3);
IDs.Add(4);
var ACTIVITY_VERSION_IDs = ACTIVITY_TOPIC
.Where(AT => IDs.Contains(AT.TOPIC_ID))
.Select(AT=>AT.ACTIVITY_VERSION_ID)
var results = ACTIVITY_VERSION
.Where(AV => ACTIVITY_VERSION_IDs.Contains(AV.ACTIVITY_VERSION_ID))