Develop Reference

.NET Core 3 InvalidOperationException on OrderBy with dynamic field name

I'm migrating an existing web API from .NET Core 2 o 3 version.
After several problems, I manage to make it work, with the exception of Dynamic OrderBy by column name.
This is my code, that worked great with .net core 2:
public async Task<IEnumerable<Clientes_view>> GetClientes(int bActivos, int nRegistroInic, int nRegistros, string sOrdenar,
int nSentido, string sFiltro, int nTipo = -1, int idCliente = -1)
{
var clientes = this.context.Set<Clientes_view>()
.Where(e => e.RazonFantasia.Contains(sFiltro) || e.RazonFantasia.Contains(sFiltro)
|| e.Cuit.Contains(sFiltro) || e.Mail.StartsWith(sFiltro) || string.IsNullOrEmpty(sFiltro))
.Where(e => (e.Activo && bActivos == 1) || bActivos == -1 || (!e.Activo && bActivos == 0))
.Where(e => e.IdTipoCliente == nTipo || nTipo == -1)
.Where(e => e.IdCliente == idCliente || idCliente == -1);
if (!string.IsNullOrEmpty(sOrdenar))
{
var propertyInfo = this.context.Set<Clientes_view>().First().GetType().GetProperty(sOrdenar,
BindingFlags.IgnoreCase | BindingFlags.Public | BindingFlags.Instance);
if (propertyInfo != null) if (nSentido == -1) clientes = clientes.OrderByDescending(e => propertyInfo.GetValue(e, null));
else clientes = clientes.OrderBy(e => propertyInfo.GetValue(e, null));
}
clientes = clientes.Skip(nRegistroInic).Take(nRegistros);
return await clientes.ToListAsync();
}
And the error I'm getting is the following:
System.InvalidOperationException: The LINQ expression 'DbSet
.Where(c => True)
.Where(c => c.Activo && True || False || False)
.Where(c => True)
.Where(c => True)
.OrderBy(c => __propertyInfo_3.GetValue(
obj: c,
index: null))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync().
Any thoughts?
Thanks!

You need to actually generate the member access expression, all you've done was used reflection to get the value of some object, and provided that as the expression. That will not work, the query provider will not be able to translate that.
You need to do something like this:
if (!String.IsNullOrEmpty(sOrdenar))
{
var type = typeof(Clientes_view);
var prop = type.GetProperty(sOrdenar);
if (prop != null)
{
var param = Expression.Parameter(type);
var expr = Expression.Lambda<Func<Clientes_view, object>>(
Expression.Convert(Expression.Property(param, prop), typeof(object)),
param
);
if (nSentido == -1)
clientes = clientes.OrderByDescending(expr);
else
clientes = clientes.OrderBy(expr);
}
}

Your problem is that you are using reflection inside of order by, while probably you should use sorting string.
One of the options
Install-Package System.Linq.Dynamic
using System.Linq.Dynamic;
then you can sort
query.OrderBy("item.item_id DESC")
Other option without any library in case you dont have many sort options would be:
switch(sOrdenar){
case "Field1"
clientes = nSentido == -1 ? clientes.OrderBy(entity=> entity.Field1) : clientes.OrderByDescending(entity=> entity.Field1);
break;
case "OtherField"
clientes = nSentido == -1 ? clientes.OrderBy(entity=> entity.OtherField) : clientes.OrderByDescending(entity=> entity.OtherField);
break;
}
Personally I prefer second option better, because then I can be sure that user is able to sort only on allowed fields otherwise you can have performance issues in case you have large tables and users start sorting on wrong fields (Never trust your users :) ).

EF Core attempts to translate as much of your query to a server-side query (i.e. SQL) as possible. In versions before 3.0, any code that could not be converted was silently run on the client - however, this can cause massive and often unintuitive performance issues, so from 3.0 the decision was made that if any query code cannot be translated, an exception would immediately be thrown.
Reference: https://learn.microsoft.com/en-us/ef/core/querying/client-eval#previous-versions
The end result is that you either need to rearchitect your code to separate the parts that can and can't be run on the server, or alternatively force everything to be run on the client. The referenced document explains how to achieve the latter, but note that doing so will likely have significant performance impact.
In your case, the stuff inside the if (!string.IsNullOrEmpty(sOrdenar)) block is what is causing the problem. You should be aware that this implies that whenever that block has been executed, the paging that follows it (Skip and Take) has not been executed on the server, always the client - so if you've ever had performance problems with this method, now you know why!

It's pretty obvious that calling properties via reflection can't be automatically translated into SQL query.
The only ways it could have worked before was either that this branch was never taken, or the whole query was processed by your application instead of on the database side.
To fix this, do as the error message suggests: break the query into DB and application parts, e.g.
if (!string.IsNullOrEmpty(sOrdenar))
{
IEnumerable<Clientes_view> list = await clientes.AsAsyncEnumerable();
list = list.Where(.....); //here you may use everything you like
return list;
}
If you are searching for a way to generate the OrderBy part dynamically on the server side, take a look at this answer; apparently it's written for classic EF, but should probably work in EF Core with minor adjustments.

LINQ statement using one of two parameters

I'm working on a LINQ statement. I have a table of cities where the records have either a countryId or a stateId. I'd like to just write the one statement and have the where clause check to see which of the two parameters is null and then select on the one that is not.
Here's what I'm working with:
public List<City> Cities(int? countryTypeId, int? stateTypeId)
{
if (countryTypeId == null && stateTypeId == null)
return null;
return _db.City
.Where(x => x.StateTypeId == stateTypeId
&& x.CountryTypeId == countryTypeId)
.OrderBy(x => x.Description)
.ToDTOs();
}
I'm pretty new to LINQ, and I know this code isn't right, just adding it for context.

If the State and Country ids are all >0 you simply can do this, no need to check for null
.Where(x => x.StateTypeId == stateTypeId.GetValueOrDefault()
&& x.CountryTypeId == countryTypeId.GetValueOrDefault())
Else you need to add the condition if those nullable inputs have value or not, as mentioned in the comment
Edit: after seeing some comments, if you are looking for list of cities based on either of the parameters, then you should be using || not && in your where condition
Where(x => (stateTypeId.HasValue && stateTypeId.Value == x.StateTypeId)
|| (countryTypeId.HasValue && countryTypeId.Value == x.CountryTypeId))
Note the order matters, this code will first check if stateTypeId has value and if it has it'll match only the cities with that stateTypeId

_db.City.Where(c => c.CountryTypeId?.Equals(countryTypeId) ?? false
| c.StateTypeId?.Equals(stateTypeId) ?? false);
Using null conditional operators - when a type Id is null use the null coalescing operator to return false and fail the match - otherwise check for equality and return matching.
Note you cannot short circuit the OR operator here!
I'm not sure if this is the case, but if one of the input parameters was always null and the entries were guaranteed to always have one property null, the following would be a cool solution:
_db.City.Where(c => (c.CountryTypeId ?? c.StateTypeId) == (countryTypeId ?? stateTypeId))

My DBA has sufficiently beaten it into my head that ignoring parameters in a query (ex: WHERE Field = #PARAM or #PARAM IS NULL) can result in very bad things. As a result, I would encourage you to conditionally add only the parameters that you absolutely need. Fortunately, given that you are working with just two possible parameters, this is trivial.
Start with the base of your query, and then add to it.
var queryBase = _db.City.OrderBy(x => x.Description);
if (countryTypeId.HasValue)
{
queryBase = queryBase.Where(x => x.CountryTypeId == countryTypeId);
}
if (stateTypeId.HasValue)
{
queryBase = queryBase.Where(x => x.StateTypeId == stateTypeId);
}
return queryBase.ToDTOs(); // or .ToList() for a more universal outcome
Add whatever logic you may need if parameters are mutually exclusive, one supercedes the other, etc.

Where function on list with multiple conditions returns errors

I'm trying to filter a list based on a few criteria and the .Where() function gives me an error in 2 parts of the same method.
if (string.IsNullOrWhiteSpace(champs))
{
data = dal.GetVueTache().Where(t =>
t.ProjetDescription64.ToLower().Contains(filtre.ToLower())
// *This Line || t.ProjetDescription256.ToLower().Contains(filtre.ToLower())
|| t.Description256.ToLower().Contains(filtre.ToLower())
||t.ResponsableNomCourt.ToLower().Contains(filtre.ToLower())
|| t.PrioriteDesc.ToLower().Contains(filtre.ToLower())
).ToList();
}
If I use any of the previous conditions except the one on the nullable field alone I get a perfectly filtered list on that criteria, if I add an OR "||" then I get a System.NullReferenceException on the first criteria.
I also have a similar issue in another part of the same method
else
{
data = dal.GetVueTache().Where(t =>
t.GetType().GetProperty(champs).GetValue(t).ToString().ToLower().Contains(filtre.ToLower())
).ToList();
}
This one filters my list based on the criteria "filtre" on the property "champs". It works on every property but the second one, which is a nullable one. I understand that this is where the issue comes from, but I can't find a way to test if the property is null before evaluating it and work around this from inside the .Where() Method.
Any advice will be greatly appreciated!!
Edit :
Thanks to Ivan Stoev for his solution!
The correct syntax for testing the null value in the first case is:
|| (t.ProjetDescription256 != null && t.ProjetDescription256.ToLower().Contains(filtre.ToLower()))
In the second case:
(t.GetType().GetProperty(champs).GetValue(t) != null && t.GetType().GetProperty(champs).GetValue(t).ToString().ToLower().Contains(filtre.ToLower()))

Just do a null check either the old way:
|| (t.ProjetDescription256 != null && t.ProjetDescription256.ToLower().Contains(filtre.ToLower()))
or the C# 6 way (utilizing the null conditional operator):
|| t.ProjetDescription256?.ToLower().Contains(filtre.ToLower()) == true
Btw, you can greatly simplify similar checks and avoid such errors by writing a simple custom extension methods like this:
public static class StringExtensions
{
public static bool ContainsIgnoreCase(this string source, string target)
{
return source == null ? target == null : target != null && source.IndexOf(target, StringComparison.CurrentCultureIgnoreCase) >= 0;
}
}
so your snippet becomes simply:
data = dal.GetVueTache().Where(
t => t.ProjetDescription64.ContainsIgnoreCase(filtre)
|| t.ProjetDescription256.ContainsIgnoreCase(filtre)
|| t.Description256.ContainsIgnoreCase(filtre)
|| t.ResponsableNomCourt.ContainsIgnoreCase(filtre)
|| t.PrioriteDesc.ContainsIgnoreCase(filtre)
).ToList();

I can't figure out where this NullReferenceException is coming from

EDIT Upon further reflection I decided to amend my answer below to reduce the possibility of misinterpretation of the problem.
I am getting a NullReferenceException on this IEnumarable LINQ query:
designations.ForEach(desg =>
desg.ModifiedBy.UserName = userNames.SingleOrDefault(name =>
desg.ModifiedBy != null && name.Id == desg.ModifiedBy.Id) != null ?
userNames.SingleOrDefault(name => name.Id == desg.ModifiedBy.Id).UserName :
"Not Available");
My best guess at the moment is that it comes from the following statement on the third line of the code above:
desg.ModifiedBy != null && name.Id == desg.ModifiedBy.Id
and the runtime is throwing a NullReferenceException on desg.ModifiedBy.Id. However, I "know" that c# truth-ness evaluations short-circuit (see, e.g., || Operator). Here, if desg.ModifiedBy == null then we should never get to evaluate name.Id == desg.ModifiedBy.Id. Nonetheless, this seems to be happening. Is this possible?

I think you are getting null at value assignation
desg.ModifiedBy.UserName = userNames.SingleOrDefault(name =>

linq where clause and count result in null exception

The code below works unless p.School.SchoolName turns out to be null, in which case it results in a NullReferenceException.
if (ExistingUsers.Where(p => p.StudentID == item.StaffID &&
p.School.SchoolName == item.SchoolID).Count() > 0)
{
// Do stuff.
}
ExistingUsers is a list of users:
public List<User> ExistingUsers;
Here is the relevant portion of the stacktrace:
System.NullReferenceException: Object reference not set to an instance of an object.
at System.Linq.Enumerable.WhereListIterator1.MoveNext()
at System.Linq.Enumerable.Count[TSource](IEnumerable1 source)
How should I handle this where clause?
Thanks very much in advance.

I suspect p.School is null, not SchoolName. Simply add a null check before accessing SchoolName. Also, use Any() to check if there are any results instead of Count() > 0 unless you're really in need of the count. This performs better since not all items are iterated if any exist.
var result = ExistingUsers.Where(p => p.StudentID == item.StaffID
&& p.School != null
&& p.School.SchoolName == item.SchoolID)
.Any();
if (result) { /* do something */ }

For all database nullable columns, we should either add null check or do simple comparision a == b instead of a.ToLower() == b.ToLower() or similar string operations.
My observation as below:
As they get iterated through Enumerable of LINQ Query for comparision against with input string/value, any null value (of database column) and operations on it would raise exception, but Enumerable becomes NULL, though query is not null.

In the case where you want to get the null value (all the student, with school or not) Use left join.
There are a good example on MSDN

If I remember correctly (not at my developer PC at the moment and can't check with Reflector), using the == operator results in calling the instance implementation string.Equals(string), not the static implementation String.Equals(string, string).
Assuming that your problem is due to SchoolName being null, as you suggest, try this:
if (ExistingUsers.Where(
p => p.StudentID == item.StaffID
&& String.Equals( p.School.SchoolName, item.SchoolID)).Count() > 0)
{
// Do stuff.
}
Of course, comments by other answers count as well:
Using Any() instead of Count() > 0 will generally perform better
If p.School is the null, you'll need an extra check
Hope this helps.