Develop Reference

uses First/FirstOrDefault/Last/LastOrDefault operation without OrderBy and filter which may lead to unpredictable results

I have a linq query which gave me the warning but it still works. I want to get rid of the warning.
uses First/FirstOrDefault/Last/LastOrDefault operation without OrderBy and filter which may lead to unpredictable results.
The linq query is
var list = (from u in _db.user
join r in _db.resource on u.userId equals r.userId
join t in _db.team on u.bossId equals t.bossId
where r.pid == pid
select new MyDto
{
pid = pid,
userId = u.userId,
teamId = t.teamId,
name = t.name
}).GroupBy(d => d.userId).Select(x => x.First()).OrderBy(y => y.userId).ToList();
I use EntityFramework Core 2.1
UPDATE:
I changed the code by the comments.
var list = (from u in _db.user
join r in _db.resource on u.userId equals r.userId
join t in _db.team on u.bossId equals t.bossId
where r.pid == pid
select new MyDto
{
pid = pid,
userId = u.userId,
teamId = t.teamId,
name = t.name
})
.GroupBy(d => d.userId)
.Select(x => x.OrderBy(y => y.userId)
.First())
.ToList();
Then there is a different warning.
The LINQ expression 'GroupBy([user].userId, new MyDto() {pid =
Convert(_8_locals1_pid_2, Int16), userId = [user].UserId, .....) could
not be translated and will be evaluated locally.

We have this expression
.Select(x => x.First())
Which record will be first for that expression? There's no way to know, because at this point the OrderBy() clause which follows hasn't processed yet. You could get different results each time you run the same query on the same data, depending on what order the records were returned from the database. The results are not predictable, exactly as the error message said.
But surely the database will return them in the same order each time? No, you can't assume that. The order of results in an SQL query is not defined unless there is an ORDER BY clause with the query. Most of the time you'll get primary key ordering (which does not have to match insert order!), but there are lots of things that can change this: matching a different index, JOIN to a table with a different order or different index, parallel execution with another query on the same table + round robin index walking, and much more.
To fix this, you must call OrderBy() before you can call First().
Looking a little deeper, this is not even part of the SQL. This work is happening on your client. That's not good, because any indexes on the table are no longer available. It should be possible to do all this work on the database server, but selecting the first record of a group may mean you need a lateral join/APPLY or row_number() windowing function, which are hard to reproduce with EF. To completely remove all warnings, you may have to write a raw SQL statement:
select userId, teamId, name, pid
from (
select u.userId, t.teamId, t.name, r.pid, row_number() over (order by u.userId) rn
from User u
inner join resource r on r.userId = u.userId
inner join team t on t.bossId = u.bossId
where r.pid = #pid
) d
where d.rn = 1
Looking around, it is possible to use row_number() in EF, but at this point I personally find the SQL much easier to work with. My view is ORMs don't help for these more complicated queries, because you still have to know the SQL you want, and you also have to know the intricacies of the ORM in order to build it. In other words, the tool that was supposed to make your job easier made it harder instead.

Can I have a where clause that works on a grandchild in a LINQ query?

I have three tables that are each connected with a primary key and foreign key such as TestId and UserTestId etc.
Exam > Test > UserTest
It's my understanding I can use LINQ to get the data from these like this:
var exams = _examsRepository
.GetAll()
.Where(q => q.SubjectId == subjectId)
.Include(q => q.Tests.Select(t => t.UserTests))
.ToList();
This will select all the exams, Tests for the exams and UserTests for those Tests where SubjectId == subjectID
Is there any possible way I could further limit this so that it only showed the data for when the UserTests had a UserId of 123?
If the answer is no then should I rewrite this LINQ to first go to the _userTestsRepository and then work in the other direction up instead of down?

In this question they offer a solution which does what you want but in my opinion feels a little like a hack.
Alternatively you could step away from include statements and do a manual join, you can apply as many filters on sub tables as you like that way.
Could look a lil like this :
var data = (from ex in context.exams
join t in context.tests on ex.Id equals test.ExamID
join ut in context.userTests on t.Id equals ut.TestId
where ut.UserId = 123
select new {ex, ut}).ToList();

How to write an linq statement to get the last of a group of records

I have 2 SQL statements that basically do the same thing, that is, retrieve the last record from a table based on a datetime field for a group of records. I am using the data-first Entity Framework model. How would I write either of these SQL statements using LINQ Lambda functions?
ie,
var u = db.AccessCodeUsage.Where(...).GroupBy(...)
rather than
var u = from a in db.AccessCodeUsage
where ...
group by ...
SQL Statements:
SELECT *
FROM AccessCodeUsage a
WHERE NOT EXISTS (SELECT 1
FROM AccessCodeUsage
WHERE LocationId = a.LocationId
AND Timestamp > a.Timestamp)
SELECT a.*
FROM AccessCodeUsage a
WHERE a.Timestamp =
(SELECT MAX(Timestamp)
FROM AccessCodeUsage
WHERE a.LocationId = LocationId
AND a.AccessCode = AccessCode
GROUP By LocationId, AccessCode)

If you need to have the method-call form, but are finding it tricky to work out, then use the other syntax first:
from a in db.AccessCodeUsage
orderby a.TimeStamp descending
group a by a.LocationId into grp
from g in grp select g.First();
Then convert to method calls by taking each clause one at a time:
db.AccessCodeUsage
.OrderByDescending(a => a.TimeStamp)
.GroupBy(a => a.LocationId)
.Select(g => g.First());
From which I can workout the second without bothering to write out the linq-syntax form first:
db.AccessCodeUsage
.OrderByDescending(a => a.TimeStamp)
.GroupBy(a => new {a.LocationId, a.AccessCode})
.Select(g => g.First());
(Except it doesn't include what may be a bug, in that if timestamps aren't guaranteed unique, the SQL given in the question could include some extra inappropriate results).
I can't check on the SQL produced right now, but it should hopefully be equivalent in results (if not necessarily matching). There's cases where grouping doesn't translate to SQL well, but I certainly don't think this would be one.

I ended up using the following which corresponds to the first SQL statement.
// Retrieve only the latest (greatest value in timestamp field) record for each Access Code
var last = AccessCodeUsages.Where(u1 => !AccessCodeUsages
.Any(u2 => u2.LocationId == u1.LocationId &&
u2.AccessCode == u1.AccessCode &&
u2.Timestamp > u1.Timestamp));

is there a better way to write this frankenstein LINQ query that searches for values in a child table and orders them by relevance?

I have a table of Users and a one to many UserSkills table. I need to be able to search for users based on skills. This query takes a list of desired skills and searches for users who have those skills. I want to sort the users based on the number of desired skills they posses. So if a users only has 1 of 3 desired skills he will be further down the list than the user who has 3 of 3 desired skills.
I start with my comma separated list of skill IDs that are being searched for:
List<short> searchedSkillsRaw = skills.Value.Split(',').Select(i => short.Parse(i)).ToList();
I then filter out only the types of users that are searchable:
List<User> users = (from u in db.Users
where
u.Verified == true &&
u.Level > 0 &&
u.Type == 1 &&
(u.UserDetail.City == city.SelectedValue || u.UserDetail.City == null)
select u).ToList();
and then comes the crazy part:
var fUsers = from u in users
select new
{
u.Id,
u.FirstName,
u.LastName,
u.UserName,
UserPhone = u.UserDetail.Phone,
UserSkills = (from uskills in u.UserSkills
join skillsJoin in configSkills on uskills.SkillId equals skillsJoin.ValueIdInt into tempSkills
from skillsJoin in tempSkills.DefaultIfEmpty()
where uskills.UserId == u.Id
select new
{
SkillId = uskills.SkillId,
SkillName = skillsJoin.Name,
SkillNameFound = searchedSkillsRaw.Contains(uskills.SkillId)
}),
UserSkillsFound = (from uskills in u.UserSkills
where uskills.UserId == u.Id && searchedSkillsRaw.Contains(uskills.SkillId)
select uskills.UserId).Count()
} into userResults
where userResults.UserSkillsFound > 0
orderby userResults.UserSkillsFound descending
select userResults;
and this works! But it seems super bloated and inefficient to me. Especially the secondary part that counts the number of skills found.
Thanks for any advice you can give.
--r

I think that should do the trick:
(from u in users
where u.UserSkills.Any(skill => searchedSkillsRaw.Contains(skill.SkillId))
select new
{
u.Id,
u.FirstName,
u.LastName,
u.UserName,
UserPhone = u.UserDetail.Phone,
UserSkills = u.UserSkills,
UserSkillsFound = u.UserSkills.Where(skill => searchedSkillsRaw.Contains(skill.SkillId)).Count()
} into userResults
orderby userResults.UserSkillsFound descending
select userResult).ToList();
However, since this is a query that gets executed on SQL server I strongly recommend to remove the 'ToList()' call from the first query. Because that actually causes LINQ to run two separate queries on the SQL server. You should change it to IQueryable instead. The power of LINQ is to construct queries in several steps without having to actually execute it in between. So 'ToList' should be called only at the end when the entire query has been constructed. In fact what you currently do is running the second query in memory rather than on the database server.
In regards to your UserSkills one-to-many relation you do not need to do an explicity join in LINQ. You can just access the collection property instead.
Let me know if you need more explanation.
Michael

Why not just let people do, say, fUsers.UserSkills.Count()? It would reduce the amount of data retrieved from the server in the first place.
Alternatively, you could create a View that has a calculated field in it and then map that to a type. Would push the query for count down into the DB.

How to do a subquery in LINQ?

Here's an example of the query I'm trying to convert to LINQ:
SELECT *
FROM Users
WHERE Users.lastname LIKE '%fra%'
AND Users.Id IN (
SELECT UserId
FROM CompanyRolesToUsers
WHERE CompanyRoleId in (2,3,4) )
There is a FK relationship between CompanyRolesToUsers and Users, but it's a many to many relationship and CompanyRolesToUsers is the junction table.
We already have most of our site built, and we already have most of the filtering working by building Expressions using a PredicateExtensions class.
The code for the straightforward filters looks something like this:
if (!string.IsNullOrEmpty(TextBoxLastName.Text))
{
predicateAnd = predicateAnd.And(c => c.LastName.Contains(
TextBoxLastName.Text.Trim()));
}
e.Result = context.Users.Where(predicateAnd);
I'm trying to add a predicate for a subselect in another table. (CompanyRolesToUsers)
What I'd like to be able to add is something that does this:
int[] selectedRoles = GetSelectedRoles();
if( selectedRoles.Length > 0 )
{
//somehow only select the userid from here ???:
var subquery = from u in CompanyRolesToUsers
where u.RoleID in selectedRoles
select u.UserId;
//somehow transform this into an Expression ???:
var subExpression = Expression.Invoke(subquery);
//and add it on to the existing expressions ???:
predicateAnd = predicateAnd.And(subExpression);
}
Is there any way to do this? It's frustrating because I can write the stored procedure easily, but I'm new to this LINQ thing and I have a deadline. I haven't been able to find an example that matches up, but I'm sure it's there somewhere.

Here's a subquery for you!
List<int> IdsToFind = new List<int>() {2, 3, 4};
db.Users
.Where(u => SqlMethods.Like(u.LastName, "%fra%"))
.Where(u =>
db.CompanyRolesToUsers
.Where(crtu => IdsToFind.Contains(crtu.CompanyRoleId))
.Select(crtu => crtu.UserId)
.Contains(u.Id)
)
Regarding this portion of the question:
predicateAnd = predicateAnd.And(c => c.LastName.Contains(
TextBoxLastName.Text.Trim()));
I strongly recommend extracting the string from the textbox before authoring the query.
string searchString = TextBoxLastName.Text.Trim();
predicateAnd = predicateAnd.And(c => c.LastName.Contains( searchString));
You want to maintain good control over what gets sent to the database. In the original code, one possible reading is that an untrimmed string gets sent into the database for trimming - which is not good work for the database to be doing.

There is no subquery needed with this statement, which is better written as
select u.*
from Users u, CompanyRolesToUsers c
where u.Id = c.UserId --join just specified here, perfectly fine
and u.lastname like '%fra%'
and c.CompanyRoleId in (2,3,4)
or
select u.*
from Users u inner join CompanyRolesToUsers c
on u.Id = c.UserId --explicit "join" statement, no diff from above, just preference
where u.lastname like '%fra%'
and c.CompanyRoleId in (2,3,4)
That being said, in LINQ it would be
from u in Users
from c in CompanyRolesToUsers
where u.Id == c.UserId &&
u.LastName.Contains("fra") &&
selectedRoles.Contains(c.CompanyRoleId)
select u
or
from u in Users
join c in CompanyRolesToUsers
on u.Id equals c.UserId
where u.LastName.Contains("fra") &&
selectedRoles.Contains(c.CompanyRoleId)
select u
Which again, are both respectable ways to represent this. I prefer the explicit "join" syntax in both cases myself, but there it is...

This is how I've been doing subqueries in LINQ, I think this should get what you want. You can replace the explicit CompanyRoleId == 2... with another subquery for the different roles you want or join it as well.
from u in Users
join c in (
from crt in CompanyRolesToUsers
where CompanyRoleId == 2
|| CompanyRoleId == 3
|| CompanyRoleId == 4) on u.UserId equals c.UserId
where u.lastname.Contains("fra")
select u;

You could do something like this for your case - (syntax may be a bit off). Also look at this link
subQuery = (from crtu in CompanyRolesToUsers where crtu.RoleId==2 || crtu.RoleId==3 select crtu.UserId).ToArrayList();
finalQuery = from u in Users where u.LastName.Contains('fra') && subQuery.Contains(u.Id) select u;

Ok, here's a basic join query that gets the correct records:
int[] selectedRolesArr = GetSelectedRoles();
if( selectedRolesArr != null && selectedRolesArr.Length > 0 )
{
//this join version requires the use of distinct to prevent muliple records
//being returned for users with more than one company role.
IQueryable retVal = (from u in context.Users
join c in context.CompanyRolesToUsers
on u.Id equals c.UserId
where u.LastName.Contains( "fra" ) &&
selectedRolesArr.Contains( c.CompanyRoleId )
select u).Distinct();
}
But here's the code that most easily integrates with the algorithm that we already had in place:
int[] selectedRolesArr = GetSelectedRoles();
if ( useAnd )
{
predicateAnd = predicateAnd.And( u => (from c in context.CompanyRolesToUsers
where selectedRolesArr.Contains(c.CompanyRoleId)
select c.UserId).Contains(u.Id));
}
else
{
predicateOr = predicateOr.Or( u => (from c in context.CompanyRolesToUsers
where selectedRolesArr.Contains(c.CompanyRoleId)
select c.UserId).Contains(u.Id) );
}
which is thanks to a poster at the LINQtoSQL forum

Here's a version of the SQL that returns the correct records:
select distinct u.*
from Users u, CompanyRolesToUsers c
where u.Id = c.UserId --join just specified here, perfectly fine
and u.firstname like '%amy%'
and c.CompanyRoleId in (2,3,4)
Also, note that (2,3,4) is a list selected from a checkbox list by the web app user, and I forgot to mention that I just hardcoded that for simplicity. Really it's an array of CompanyRoleId values, so it could be (1) or (2,5) or (1,2,3,4,6,7,99).
Also the other thing that I should specify more clearly, is that the PredicateExtensions are used to dynamically add predicate clauses to the Where for the query, depending on which form fields the web app user has filled in. So the tricky part for me is how to transform the working query into a LINQ Expression that I can attach to the dynamic list of expressions.
I'll give some of the sample LINQ queries a shot and see if I can integrate them with our code, and then get post my results. Thanks!
marcel