I have a LINQ query running with multiple joins and I want to pass it around as an IQueryable<T> and apply additional filters in other methods.
The problem is that I can't work out how to pass around a var data type and keep it strongly typed, and if I try to put it in my own class (EG: .Select((a,b) => new MyClass(a,b))) I get errors when I try to add later Where clauses because my class has no translations into SQL. Is there any way I can do one of the following:
Make my class map to SQL?
Make the var data-type implement an interface (So I can pass it round as though it's that)?
Something I haven't though of that'll solve my issue?
Example:
public void Main()
{
using (DBDataContext context = new DBDataContext())
{
var result = context.TableAs.Join(
context.TableBs,
a => a.BID,
b => b.ID,
(a,b) => new {A = a, B = b}
);
result = addNeedValue(result, 4);
}
}
private ???? addNeedValue(???? result, int value)
{
return result.Where(r => r.A.Value == value);
}
PS: I know in my example I can flatten out the function easily, but in the real thing it'd be an absolute mess if I tried.
All you have to do is de-anonymize your type that you're constructing as a result of the query - new { A = a, B = b }
Just create a class with those properties and name it appropriately. Then your query will have a type of IQueryable<your type>
Like so:
public class MyClass
{
public int A { get; set; }
public int B { get; set; }
}
public void Main()
{
using (DBDataContext context = new DBDataContext())
{
var result = context.TableAs.Join(
context.TableBs,
a => a.BID,
b => b.ID,
(a,b) => new MyClass {A = a, B = b}
);
result = addNeedValue(result, 4);
}
}
private IQueryable<MyClass> addNeedValue(IQueryable<MyClass> result, int value)
{
return result.Where(r => r.A.Value == value);
}
Here's two different approaches; the first applies the filter before doing the Join, since the joined queries don't have to just be basic tables. The second approach applies the filter after the join, using an intermediary projection (and changing it to return the predicate, rather than applying it internally).
This has been tested successfully on .NET 3.5 and .NET 4; note that in 3.5 (SP1) the Expression.Invoke (for the second example) won't work on EF, but is fine for LINQ-to-SQL.
If you want to run the example, I've used Northwind (just because that is what I had locally):
using System;
using System.Linq;
using System.Linq.Expressions;
using ConsoleApplication1; // my data-context's namespace
static class Program
{
public static void Main()
{
using (var context = new TestDataContext())
{
context.Log = Console.Out; // to check it has worked
IQueryable<Order> lhs = context.Orders;
IQueryable<Order_Detail> rhs = context.Order_Details;
// how ever many predicates etc here
rhs = addBeforeJoin(rhs, 4);
var result = lhs.Join(rhs,
a => a.OrderID,
b => b.OrderID,
(a, b) => new { A = a, B = b }
);
// or add after
result = result.Where(row => row.B, addAfterJoin(100));
Console.WriteLine(result.Count());
}
}
private static IQueryable<Order_Detail> addBeforeJoin(IQueryable<Order_Detail> query, int value)
{
return query.Where(r => r.Quantity >= value);
}
private static Expression<Func<Order_Detail, bool>> addAfterJoin(int value)
{
return r => r.Quantity <= value;
}
private static IQueryable<TSource> Where<TSource, TProjection>(
this IQueryable<TSource> source,
Expression<Func<TSource, TProjection>> selector,
Expression<Func<TProjection, bool>> predicate)
{
return source.Where(
Expression.Lambda<Func<TSource, bool>>(
Expression.Invoke(predicate, selector.Body),
selector.Parameters));
}
}
You are returning what's known as an 'anonymous type' from your join - it's not a specific database type, but is instead just an object containing A and B. Because the type is anonymous, I don't think you're going to be able to write a function that returns that type - or a collection of that type.
If it were a type that SQL / LINQ-to-SQL knew about then you'd probably be able to do this.
One way to get LINQ-to-SQL to recognize the type might be to create a stored procedure which performs the join and selects the relevant columns. If you add that stored procedure to your DBML file, LINQ-to-SQL will understand the type of each 'row' in the result. You should then be able to have your function return an IQueryable<ResultSetRow>.
I definitely don't recommend it, but you can probably use dynamic here, if you're on .NET 4.0.
Related
I'm building my own reflection functions for certain types of searches.
The problem is that I want to search a group of IDs within a list of IDs and filter my search/select query to have only these specific objects.
This is the same as using "IN()" in Linq-Entity framework. But I can't use a.objid.
return query.Where(a => ObjectsToFind.Contains(a.objid));
However, "a.objid" is causing errors because I use T Template.
So a is "T a" instead of "MyTable a" so that I can call it's "objid" property.
I know there is a way to do this with parameter expressions. However, I can't figure it out.
Here's what I tried to replace that above line with:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, string contains)
{
var ObjectsToFind = new List<int>(); // I want to search IN() this function that gets filled in here.
ObjectsToFind = FillObjectsToFind(); // just the object id integers I want to filter
var parameter = Expression.Parameter(typeof(T), "type");
var propertyExpression = Expression.Property(parameter, "objid"); // I look for this
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(int) }); // int type
var vars = Expression.Variable(List<int>,); // I can't use "constant", but what do I use here?
var containsExpression = Expression.Call(propertyExpression, method, vars);
return query.Where(Expression.Lambda<Func<T, bool>>(containsExpression, parameter));
}
Replacing "T" with the actual table entity, causes a lot of problems, so I decided to keep the T.
else if (s.ST == "function")
{ // search func
myTable a;
DC2 = MyUtility.WhereFunctionContains(DC2, a => a.objid, s.S);
// s.S is my search query string.
// s.ST means I'm searching for functions (from another associated table)
// I don't understand how a.objid gets called, I wanted to use Template/Reflections.
}
Here's how I call Zev's function.
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, Expression<Func<T, int>> converter, string contains)
{
FunctionsClass fc = new FunctionsClass();
var ObjectsToFind = new List<int>();
ObjectsToFind = fc.SearchContainFunction(contains); // I grabbed my list of IDs to search
return query.Where(t => ObjectsToFind.Contains(converter(t)));
}
If I understand you correctly, you have a number of queries on different types:
IQueryable<Person> personsQry = ...
IQueryable<Sale> salesQry = ...
IQueryable<Order> ordersQry = ...
and you have a method that generates a List<int>, called FillObjectsToFind:
public List<int> FillObjectsToFind()
{
//code here
}
You want to be able to limit each of the above queries to only have the id in the returned list. As an added bonus, this should be an extension method, so you can call it like this:
var personsFiltered = personsQry.WhereFunctionContains(...);
var salesFiltered = salesQry.WhereFunctionContains(...);
var ordersFiltered = ordersQry.WhereFunctionContains(...);
The problem is each query is of a separate type, and you would prefer to write one method that covers all of them.
The first part of the solution is to define a generic method:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query)
{
//code here
}
but there is still a problem: the only type we know of is type T which is not a real type, but a placeholder for actual types. Since these types could be anything -- string, System.Random, SqlConnection, an ASP.NET Label, a WPF TextBlock -- there is no way of knowing how to compare each object to a List of ints.
The most straightforward solution is to define an interface:
interface IHasObjID
{
int ObjID {get;set;}
}
Then each type should implement this interface:
class Person : IHasObjID
{
int objID;
int ObjID
{
get {return objID;}
set {objID = value;}
}
}
//implement sales and orders similarly
Once that is done, you can define a constraint on the types allowed by the method. Now that the type definitely has an ObjID property, we can query on that:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query) where T : IHasObjID
{
var intsToFind = FillObjectsToFind();
return query.Where(t => intsToFind.Contains(t.ObjID));
}
This is what King King was telling you in this comment.
I propose that when calling this function, you also pass in how to get at the integer from the type:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, Expression<Func<T,int>> converter)
{
var intsToFind = FillObjectsToFind();
return query.Where(t => intsToFind.Contains(converter(t)));
}
However, I haven't tested this code, and since we are working with Entity Framework and expressions I suspect there is still an issue: An expression cannot be "called" within an expression.
I wanted to suggest the above, but it doesn't compile with the following error -- 'converter' is a 'variable' but used like a 'method'.
After all that, the solution is straightforward, using Join:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, Expression<Func<T,int>> converter)
{
var ints = new List<int>() { 1, 2, 3, 4, 5 };
return query.Join(ints,converter,i=>i,(t,i) => t);
}
to be called like this:
var filteredPersons = query.WhereFunctionContains(p => p.PersonID);
If this is only used with a single type MyTable:
public static IQueryable<MyTable> WhereFunctionContains(this IQueryable<MyTable> query)
{
var ints = new List<int>() { 1, 2, 3, 4, 5 };
return query.Join(ints, mt=>mt.objid, i=>i, (t,i) => t);
}
Some links from the C# Programming Guide (in order of the answer):
Extension methods
Generics
Generic methods
Generic constraints
Interfaces
LINQ inner joins
Also, see here for a nice overview of LINQ operators, such as Select, Where, Count and ToList.
I may have function calls like this :
foo(**new {x.ID,x.Name}**);
and LINQ:
(IQueryable<SomeTableName>).where(x=>x.ID>1).select(x=>**new {x.ID,x.Name}**);
Is it possible to replace the "new {x.ID,x.Name}" part with a function, expression or variable, so I can change from time to time, in only 1 place?
Thank you.
I know I can make an Expression, which be used inside LINQ only
public static Func<SomeTableName, Object> Select_RS_Food = x => new { x.ID,x.Name };
but I also want to use for normal anonymous object creating. Like:
foo(CreateMyObject(x));
Update
Current:
return new { myTable.ID, myTable.Name};
//And
db.SomeTable.Select (x=> new { myTable.ID, myTable.Name));
Expect:
return SomeMagicCode;
//And
db.SomeTable.Select (x=> SomeMagicCode);
You could define a helper class like this to allow you to specify only some of the generic type parameters when creating a Func<T, R>:
public static class Funk<T>
{
public static Func<T, R> Make<R>(Func<T, R> func)
{
return func;
}
}
And use it like this:
var selector = Funk<SomeTableName>.Make(x => new {x.ID, x.Name});
var result = db.SomeTable.Where(x => x.ID>1).Select(selector);
This takes advantage of type inference to determine the return type, so you only have to define the input type (SomeTableName). The var keyword makes the selector variable implicitly typed, you won't have to specify Func<SomeTableName, _AnonymousType1_>. You can now reuse selector in multiple queries.
You will not be able to save this as property or field of a class, however, because implicit typing is not supported in type members. If you want to be able to reuse this selection function, you must use a named type or perhaps dynamic—but I wouldn't expect this to work with any current ORM.
For the other lambda expressions (x => x.ID == 1 and x => x.ID > 1), there are no anonymous types involved so you can simply do this:
public static Func<SomeTableName, bool> IDEquals1 = x => x.ID == 1;
public static Func<SomeTableName, bool> IDGreaterThan1 = x => x.ID == 1;
But by the way, if your using an ORM like entity framework, you probably need to use an Expression instead. Maybe something kind of like this:
public static Expression<Func<T, bool>> IDEquals1 = x => x.ID == 1;
Or even this:
public interface IHasID
{
int ID { get; }
}
public SomeTableName : IHasID { ... }
public static Expression<Func<T, bool>> IDEquals1<T>() where T : IHasID
{
return x => x.ID == 1;
}
I have the following query:
// Type T is constrained to a class that contains "ID" property
// propertiesToQuery is a list constructed based on type T
var set = AppContext.Set<T>();
var result = set.SelectMany(x => propertiesToQuery.Select(p => new { x.ID, Value = x.GetType().GetProperty(p.Name).GetValue(x) })
.Where(p => p.Value != null)
.Select(p => new SearchIndexItem
{
Key = p.Value.ToString(),
Url = Url.Action("Edit", type.Name, new { p.ID }),
Type = type
}));
Now because linq to entities does not allow to use PropertyInfo in queries, I need to run a ToList() on the set in order to execute the query on the db first and then perform the desired SelectMany().
This queries more than it needs to from the database which will be a problem when there's a lot of data (queried columns are of type string, others can be blobs and these are the ones I don't want to pull the data from)
So the question is how can I limit the columns queried from the db based on a list constructed at runtime?
I was trying to create an expression tree, and pass it to the Select() method on the set, but the problem was with creating the anonymous type, which can be different depenging on type T.
Your observation here:
the problem was with creating the anonymous type, which can be different depenging on type T
is accurate; it is hugely problematic to construct the result columns at runtime. The only simply way to do that is to make it look like you are populating a projection of a type that has all the members, for example the equivalent of:
// where our list is "Foo", "Bar":
x => new SomeType {
Foo = x.Foo,
Bar = x.Bar
// but other SomeType properties exist, but are't mapped
}
The obvious contender would be the entity type, so you are partially mapping a set of Customer rows to Customer objects - but most ORMs won't let you do that: if the projection is an entity type they want the entire type (i.e. x => x). You might be able to create a second version of the entity type that is a regular POCO/DTO but isn't part of the entity model, i.e.
Customer x => new CustomerDto {
Foo = x.Foo,
Bar = x.Bar
}
which you can do as part of Expression.MemberInit at runtime. Example:
class Foo
{
public string A { get; set; }
public int B { get; set; }
public DateTime C { get; set; }
}
class FooDto
{
public string A { get; set; }
public int B { get; set; }
public DateTime C { get; set; }
}
class Program
{
static void Main(string[] args)
{
var data = new[] { new Foo { A = "a", B = 1, C = DateTime.Now}}
.AsQueryable();
var mapped = PartialMap<Foo, FooDto>(data, "A", "C").ToList();
}
static IQueryable<TTo> PartialMap<TFrom, TTo>(
IQueryable<TFrom> source, params string[] members)
{
var p = Expression.Parameter(typeof(TFrom));
var body = Expression.MemberInit(Expression.New(typeof(TTo)),
from member in members
select (MemberBinding)Expression.Bind(
typeof(TTo).GetMember(member).Single(),
Expression.PropertyOrField(p, member))
);
return source.Select(Expression.Lambda<Func<TFrom, TTo>>(body, p));
}
}
in the output, A and C have values, but B does not.
How can I dynamically sort an Entity Framework query based on a value provided by client?
I have 2 user inputs: one that is value to filter the project by and the other is the way to order the results - either by date, state, priority or type.
The data is displayed in a Grid.
I have one query like this:
Expression<Func<Issue, object>> sortExpression = IssuesConversionsFilter.Convert(sortBy);
Requests = query.Where(i => i.Project.ProjectID == projectId && i.Project.Enabled)
.OrderByDescending(sortExpression)
.Skip(( currentPage - 1 )*take)
.Take(take)
IssuesConversionFilter is a static class with a cache keyed by an enum and with a value that is an Expression<Func<Issue, object>>:
internal static class IssuesConversionsFilter
{
readonly static IDictionary<IssuesSortBy, Expression<Func<Issue, object>>> Cache = new Dictionary<IssuesSortBy, Expression<Func<Issue, object>>>();
static IssuesConversionsFilter() {
Cache.Add(IssuesSortBy.Date, i => i.CreatedDate);
Cache.Add(IssuesSortBy.Priority, i => i.Priority);
Cache.Add(IssuesSortBy.Type, i => i.Type);
Cache.Add(IssuesSortBy.State, i => i.State);
}
public static Expression<Func<Issue, object>> Convert(IssuesSortBy sortBy) {
if(Cache.ContainsKey(sortBy) == false)
throw new InvalidOperationException();
return Cache[sortBy];
}
}
The problem that the return type has to be an expression that returns an object and LINQ to Entities only seems to support primitive types. With LING-to-Objects this works fine.
How can I get this to work?
Instead of trying to get the helper class to be able to return a generic expression that can be used in the OrderBy(Descending) method call (which you've found is difficult when the data types aren't all the same), it will be much easier to just perform the call to OrderBy(Descending) in the helper method. For example:
internal static class IssuesConversionsFilter
{
public static IOrderedQueryable<Issue> Convert(IQueryable<Issue> query, IssuesSortBy sortBy)
{
switch (sortBy)
{
case IssuesSortBy.Date:
return query.OrderByDescending(i => i.CreatedDate);
case IssuesSortBy.Priority:
return query.OrderByDescending(i => i.Priority);
case IssuesSortBy.Type:
return query.OrderByDescending(i => i.Type);
case IssuesSortBy.State:
return query.OrderByDescending(i => i.State);
default:
throw new ArgumentOutOfRangeException("sortBy");
}
}
}
Then you can use that method like this:
var orderedQuery = IssuesConversionsFilter.Convert(unOrderedQuery, IssuesSortBy.Date);
You could also change the signature of the Convert method to include this:
public static IOrderedQueryable<Issue> Convert(this IQueryable<Issue> query, IssuesSortBy sortBy)
and then you'd be able to use it as an extension method. That way, you'll maintain the fluid calling style from your example:
var Requests = query.Where(i => i.Project.ProjectID == projectId && i.Project.Enabled)
.Convert(sortBy)
.Skip(( currentPage - 1 )*take)
.Take(take)
I have a method:
public static void GetObjects()
{
using(MyContext context = new MyContext())
{
var objects = context.Bars.Where(b => b.Prop1 != null)
.Select(b => new MyObject{Prop = b.Prop1, Name = b.Name})
.ToList();
foreach(var object in objects)
{
// do something with the object
}
}
}
I refactored the method to make it more general so that I can pass in a Func so that I can specify the where statement and what property from the Bars table gets assigned to MyObject.Prop like this:
public static void GetObjectsV2(Func<Bar, bool> whereFunc, Func<Bar, string> selectPropFunc)
{
using(MyContext context = new MyContext())
{
var objects = context.Bars.Where(whereFunc)
.Select(b => new MyObject{Prop = selectPropFunc(b), Name = b.Name})
.ToList();
foreach(var object in objects)
{
// do something with the object
}
}
}
GetObjectsV2 seems to run much slower than GetObjects. Are there any reasons this would affect performance, and if so, are there any ways around this while still keeping the function flexible?
The reason it is running slower is because you are passing in a Func<Bar, bool> which forces the context to retrive ALL Bars and then run the Func on the returned result set. A way to make this run better is to pass in Expression<Func<Bar, bool>>
Putting that all together will result in the following:
public static void GetObjectsV2(Expression<Func<Bar, bool>> whereFunc, Expression<Func<Bar, string>> selectPropFunc)
{
using(MyContext context = new MyContext())
{
var objects = context.Bars.Where(whereFunc)
.Select(selectPropFunc)
.ToList();
foreach(var object in objects)
{
// do something with the object
}
}
}
As I discovered in my own question, .Where(o => whereFunc(o)) is not the same as .Where(whereFunc) in the Entity Framework.
The first one, .Where(Expression<Func<Bar, bool>>) works like any other linq call, simply appending the expression to the expression tree.
In the second case, .Where(Func<Bar, bool>>), it will compile and evaluate the linq call (which so far is just context.Bars) before applying the whereFunc predicate.
So, to answer your question, the second one is much slower because it is pulling the entire Bars table into memory before doing anything with it. Using .Where(o => whereFunc(o)) instead should fix that
(or, as Mark suggests, change the type of whereFunc to Expression<Func<Bar, bool>>, which Func<Bar, bool> is implicitly convertible to)