I have an endpoint which should list all Candidatures filtered by search term, so what I do is to create a method which accepts candidature entity and searchTerm as params. Then I pass that method to Where clause, but the problem is that I got NullReferenceException because navigation properties are nulls. If I put statement inside Where clause instead of the method then it doesn't throw Exception. The question is how to fix this, but I want to keep the external method because there will be a lot more logic, but I need to have access to all navigation properties i.e. they should be populated.
if (!string.IsNullOrEmpty(searchTerm))
{
query = query.Where(c => FilterBySearchTerm(c, searchTerm));
}
var result = await query.Select(c => new CandidaturesResponseModel()
{
Id = c.Id,
Name = c.PrimaryMember.FullName, // that's filled
}).ToListAsync();
private bool FilterBySearchTerm(Candidature c, string searchTerm)
{
return c.PrimaryMember.FirstName.Contains(searchTerm); // here is the exception because PrimaryMember navigation property is null. So I want this to be filled.
}
The issue is that you're materializing the query by using your FilterBySearchTerm method. EF cannot translate random methods to SQL, so it has to go ahead and run the query, get the results back and then apply your Where. EF would actually throw an exception in the past, but EF Core handles this silently.
Anyways, once the query is run, you're done. Your filtering is happening in-memory, and at that point, without an Include, your related entities are not there to work with. Long and short, you'll need to build your filter in place (rather than using a separate method) in order for EF to be able to translate that to SQL.
An alternative approach which may serve you better is pass a queryable to your FilterBySearchTerm method. For example, instead of doing:
query = query.Where(c => FilterBySearchTerm(c, searchTerm));
Do
query = FilterBySearchTerm(query, searchTerm);
Then, inside FilterBySearchTerm, you can directly apply Where clauses to the passed in query. That allows you to build an actual query that EF can understand, while also encapsulating the logic.
Just use Include method to add PrimaryMember
query = query.Include(x=> x.PrimaryMember).Where(c => FilterBySearchTerm(c, searchTerm));
Related
I'm having an issue in Linq and have an object like this one:
{"Id": "",
"tDate": "2021-02-08T13:15:59.5419262Z",
"innerObject": [
{
"innerObjectId": "",
"tDate": "2021-02-08T13:15:59.528643Z"
}
],
...};
This is connected to an api, and when it's called I want to query this object and return all the attributes in ObjectDTO while changing the tDate in case there's a date in the innerObject (I know that condition works, so no need to worry with that):
this.query = cosmosDbProvider.GetContainer().GetItemLinqQueryable<ObjectDTO>().AsQueryable()
.Where(t => someconditions)
.Select(o => { o.tDate = o.innerOject.Where(t => t.innerObjectId == input.innerObjectId).Select(t => t.tDate).ToArray()[0] ?? o.tDate;
return o; });
However, I can't seem to make this work. The errors I'm getting are 'A lambda expression with a statement body cannot be converted to an expression tree' and 'An expression tree may not contain an assignment operator'.
The other option I've tried is manually assigning each variable of ObjectDTO, but that looks awful.
Does anyone have any ideas?
I would try to use a "foreach" in that case with the "where" results.
Also if you want to use LINQ you may check this answer.
Is nice to remember that if you are working with an IQueryable you should know that the "ToList" method would trigger the Entity Framework query in the database before going to your "ForEach". In that case, try to have a look at this LINQ to SQL syntax
There are two enumerations you need to be aware of...one is the IEnumerable which is a concreate representation of the data and is in memory for access. There other is IQueryable which is not a hard representation, but a set of possibilities for what the data will look like and is sent to the database and not in memory.
But the method Where returns an IQueryable which is fine and can be chained with other IQueryable returns, but the Select is asking to change the shape of existing data (as IEnumerable).
Put in a ToList() (which will make the IQueryable(s) to be executed and return a concrete List<T> of objects) after the Where( ... ); and that will provide a valid data set to the Select operate on.
I have a query similar to this that I am running against our database:
var result = await _context.MyEntities
.Select(x => new SubEntityDto { Id = x.SubEntity.Id })
.Distinct()
.ToListAsync();
The entity SubEntity can be the same entity set on multiple MyEntity.
Since I want a list of SubEntityDto with no duplicates, I run .Distinct() on the resulting IQueryable<SubEntityDto>. This is the SubEntityDto class:
public class SubEntityDto
{
public int Id { get; set; }
}
And this works, but I don't know how it manages to make the list distinct when working with the DTOs. Doesn't .Distinct() use the .Equals method in this scenario? And doesn't .Equals default to reference equality, which checks whether two instances are the same instance?
If I load the list from the database, and then do Distinct(), it doesn't work anymore, like this:
var subEntities = await _context.MyEntities
.Select(x => new SubEntityDto { Id = x.SubEntity.Id })
.ToListAsync();
var distinctSubEntities = subEntities.Distinct().ToList(); // Not distinct.
I'm thinking that it somehow manages to do this when creating the SQL for the SQL query, but can anyone tell me what is happening? I'm puzzled about the fact that it seems to keep track of the entites after they're mapped to DTOs.
As mentioned in the comments to your question, when you create your query it gets translated into SQL query by your provider. So in the first case it is Queryable.Distinct(), however, in the second case you have already run your query against your database by calling ToListAsync() and result of it already translated to DTO's in memory (objects). Therefore Enumerable.Distinct() is run.
You can also use your debugger to see the created query
You can see the created query by your provider by putting a breakpoint and then investigating the contents of the variable DebugView→Query→Text Visualizer will show you the result.
finalProducts = context.Products.OrderBy(p => p.Name.LevenshteinDistance(query))
.Skip(from)
.Take(36)
.ToList<Product>();
I use method LevenshteinDistance() to find match with query, but it shows error
LINQ to Entities does not recognize the method 'Int32 LevenshteinDistance(System.String, System.String)' method, and this method cannot be translated into a store expression.
What should I do to sort products from database using this method?
You will need to order the in-memory list:
var products = context.Products.ToList();
var finalProducts = products.OrderBy(p => p.Name.LevenshteinDistance(query)).Skip(from).Take(36).ToList<Product>();
...or re-write the logic of your method in a stored procedure and execute this one. LINQ-to-Entities will obviously not be able to translate your custom C# method into T-SQL.
You are 'trying' to create a query to Database, using a method most possibly written by yourself. However, the problem is that framework does not know how to convert that method to a logical query. You need to order the result after you get the results from the database.
As others have commented, the problem is that you are trying to run your own method on the database. You can't do this, as the database doesn't know about your method.
It's important to understand how Linq2Enties queries work. Until you enumerate the query, nothing is actually done. It's only when you enumerate the query (say by calling an extension method like ToList(), ToArray() or any of several OrAbc() methods) that the database accesses the data and returns some results to the code. At that point, any further code runs independently of the database, and so you can call your own method.
So, what you can do is this...
finalProducts = context.Products
.ToList<Product>() // Enumerates
.OrderBy(p => p.Name.LevenshteinDistance(query))
.Skip(from)
.Take(36)
.ToList<Product>();
That will work. However, you need to be aware of the consequences of doing this, as the database will return every product, which may take some time if you have a lot. You would have to try it and see if this is a problem for you.
If performance is an issue, then you would probably have to implement the distance function as a stored procedure, but that's a lot more work. Try it this way first.
I have the following method in a data access class which uses entity framework:
public static IEnumerable<entityType> GetWhere(Func<entityType, bool> wherePredicate)
{
using (DataEntities db = new DataEntities())
{
var query = (wherePredicate != null)
? db.Set<entityType>().Where(wherePredicate).ToList()
: db.Set<entityType>().ToList();
return query;
}
}
This works fine when I use the entities across all layers... however I am trying to move to using a DTO class and I would like to do something like the following:
public static IEnumerable<EntityTypeDTO> GetWhere(Func<EntityTypeDTO, bool> wherePredicate)
{
//call a method here which will convert Func<EntityTypeDTO,bool> to
// Func<EntityType,bool>
using (DataEntities db = new DataEntities())
{
var query = new List<EntityType>();
if (wherePredicate == null)
{
query = db.Set<EntityType>().ToList();
}
else
{
query = (wherePredicate != null)
? db.Set<EntityType>().Where(wherePredicate).AsQueryable<EntityType>().ToList()
: db.Set<EntityType>().ToList();
}
List<EntityTypeDTO> result = new List<EntityTypeDTO>();
foreach(EntityType item in query)
{
result.Add(item.ToDTO());
}
return result;
}
}
Essentially I want a method which will convert Func to Func.
I think I have to break down the Func into an expression tree and then rebuild it somehow in the entityType?
I want to do this to allow the Presentation Layer to just pass the Expression queries?
Am I missing something basic or is there an easier design pattern that can pass a query from a DTO to a data access class without knowing the details of the query?
I have tried making the DTO inherit from the entity which doesn't seem to work either?
If there is a better design pattern that I am missing I would love a pointer and I can investigate from there...
Firstly I would suggest that you put a querying layer of your own in front of Entity Framework rather than allowing any arbitrary Func to be passed in because it will be very easy in the future to pass a Func that Entity Framework can not translate into a SQL statement (it can only translate some expressions - the basics are fine but if your expression calls a C# method, for example, then Entity Framework will probably fail).
So your search layer could have classes that you build up as criteria (eg. a "ContainsName" search class or a "ProductHasId" class) that are then translated into expressions in your search layer. This separates your app entirely from the ORM, which means that ORM details (like the entities or like the limitations of what Funcs can and can't be translated) don't leak out. There's lots out there that's been written about this some of arrangement.
One final note, though, if you are working close to the ORM layer, Entity Framework is very clever and you could probably get a long way without trying to translate your Func<dto, bool> to a Func<entity, bool>. For example, in the below code, accessing "context.Products" returns a "DbSet" and calling Select on it returns an IQueryable and calling Where on that also returns an IQueryable. Entity Framework will translate all of that into a single SQL statement so it won't pull all other Products into memory and then filter the ID on that memory set, it will actually perform the filtering in SQL even though the filter is operating on a projected type (which is equivalent to the DTO in your case) and not the Entity Framework entity -
var results = context.Products
.Select(p => new { ID = p.ProductID, Name = p.ProductName })
.Where(p => p.ID < 10)
.ToList();
The SQL executed is:
SELECT
[Extent1].[ProductID] AS [ProductID],
[Extent1].[ProductName] AS [ProductName]
FROM [dbo].[Products] AS [Extent1]
WHERE [Extent1].[ProductID] < 10
So, if you changed your code to get something like..
return context.Products
.Map<Product, ProductDTO()>()
.Where(productDtoWherePredicate)
.ToList();
.. then you might be just fine with the Funcs that you already have. I presume that you already have some sort of mapping functions to get from EF entities to DTOs (but if not then you might want to look into AutoMapper to help you out - which has support for "projections", which are basically IQueryable maps).
I am going to put this up as an answer.Thanks to Dan for the quick answer. Looking at what you are saying I can write a query/filter set of classes. for example, take the following code:
GetProducts().GetProductsInCategory().GetProductsWithinPriceRange(minPrice, maxPrice);
This code would run like so: Get Products would get all products in the table and the remaining functions would filter the results. if all queries run like this it may put a significant load on the Data Access Layer/ DB Server Connections... not sure.
or
An Alternate I will work on also is:
If each function creates a Linq expression, I could combine them like this: How do I combine multiple linq queries into one results set?
this may allow me to do this in a manner where I can return the filtered results set from the database.
Either way I am marking this as answered. I will update when I have more details.
I have a LINQ to SQL class, we'll call it Test, and I want to be able to access properties with LINQ queries but I get the famed "No Supported Translation to SQL" runtime error. I'm interested in the conceptual problem. Here is my simplified class:
public class Test
{
public int ID {get; set;} // Stored in Database
public int NonForeignKeyValue {get; set;} // Stored in Database
}
Here is sort of an example of what I'm trying to accomplish, but I don't want the overhead of always explicitly writing the join in LINQ:
var db = (new DataContext()).GetTable<Test>();
var q = (from t in db.GetTable<Test>()
join o in db.GetTable<OtherTable>() on o.ID equals t.ID
where t.OtherStuff
select t)
I'd like to be able to add a property to Test that tells me if there are any rows in OtherTable that could be joined with Test:
public bool IsInOtherTable
{
get
{
return (new DataContext())
.GetTable<OtherTabke>()
.Any(x => x.NonForeignKeyValue == this.NonForeignKeyValue));
}
}
Ultimately this is what I want my code to look like, but it errors. I basically want to return all entries that contain some database computed value:
using (DataContext db = new DataContext())
{
var q = db.GetTable<Test>()
.Where(x => x.IsInOtherTable && x.OtherStuff); //Error
}
I'm basically trying to save myself from writing this code every single time I want to check if Test has certain information in another table. I'm not that interested in the exact problem I described, I'm more interested in how to conceptually add the join part to the SQL and still use LINQ. I'm guessing I use Linq.Expression, but I really don't know and I'm not aware of how to do it.
As an aside, I could just write the actual SQL, as its not that complicated, but I'd like to know how to get around this and still use LINQ.
Edit: I tried this property, but I get the same error. Its more complicated that just changing the return type to Expression...
public System.Linq.Expressions.Expression<Func<Article3, bool>> Exists
{
get
{
using (DataContext db = new DataContext())
{
return i => db.GetTable<OtherTable>()
.Any(x => x.NonForeignKeyValue == i.NonForeignKeyValue));
}
}
}
Each time the linq generator is to translate a code into a query, it has to process an expression tree.
In your examples, you are not passing around expression but rather - properties, delegates, i.e. stuff which the expression visitor is unable to "step into".
In general, try to rethink your conditions so that instead of bool you have Expression<T, bool> etc.
http://netpl.blogspot.com/2008/02/linq-to-object-vs-linq-to-sql-and.html
Firstly, I think you may be overestimating "the overhead of always explicitly writing the join in LINQ". It's an extra line of code which has the advantage of being relatively self-documenting as to just what you are doing (always a nice thing), and any other approach is going to be turned first into SQL and then into a query plan that will be at least as expensive to execute, possibly more expensive (SQLServer is good a joins!)
Still, there are two alternatives I can thinkof.
One is to have an EntityRef property on the class that defines this relationship with the other table. You can then test if it is null in your query (or EntitySet if it's on the other side of a one-to-many relationship).
The other is to define a SQL function that returns a bit result indicating whether an id refers to a row that does or doesn't relate to the other table.
Then define a protected method on your DataContext-derived class that matches the signature in C# terms, and has a Function attribute mapping it to that SQL function. (Since this isn't something that you can give a sensible non-db-using version of in the C# version, you can implement the C# function by calling ExecuteMethodCall).
Then you can use that method instead of the join.
Still, this is likely less self-explanatory in the code and at risk of being less efficient than just keeping the join.