given a list of ids, I can query all relevant rows by:
context.Table.Where(q => listOfIds.Contains(q.Id));
But how do you achieve the same functionality when the Table has a composite key?
This is a nasty problem for which I don't know any elegant solution.
Suppose you have these key combinations, and you only want to select the marked ones (*).
Id1 Id2
--- ---
1 2 *
1 3
1 6
2 2 *
2 3 *
... (many more)
How to do this is a way that Entity Framework is happy? Let's look at some possible solutions and see if they're any good.
Solution 1: Join (or Contains) with pairs
The best solution would be to create a list of the pairs you want, for instance Tuples, (List<Tuple<int,int>>) and join the database data with this list:
from entity in db.Table // db is a DbContext
join pair in Tuples on new { entity.Id1, entity.Id2 }
equals new { Id1 = pair.Item1, Id2 = pair.Item2 }
select entity
In LINQ to objects this would be perfect, but, too bad, EF will throw an exception like
Unable to create a constant value of type 'System.Tuple`2 (...) Only primitive types or enumeration types are supported in this context.
which is a rather clumsy way to tell you that it can't translate this statement into SQL, because Tuples is not a list of primitive values (like int or string). For the same reason a similar statement using Contains (or any other LINQ statement) would fail.
Solution 2: In-memory
Of course we could turn the problem into simple LINQ to objects like so:
from entity in db.Table.AsEnumerable() // fetch db.Table into memory first
join pair Tuples on new { entity.Id1, entity.Id2 }
equals new { Id1 = pair.Item1, Id2 = pair.Item2 }
select entity
Needless to say that this is not a good solution. db.Table could contain millions of records.
Solution 3: Two Contains statements (incorrect)
So let's offer EF two lists of primitive values, [1,2] for Id1 and [2,3] for Id2. We don't want to use join, so let's use Contains:
from entity in db.Table
where ids1.Contains(entity.Id1) && ids2.Contains(entity.Id2)
select entity
But now the results also contains entity {1,3}! Well, of course, this entity perfectly matches the two predicates. But let's keep in mind that we're getting closer. In stead of pulling millions of entities into memory, we now only get four of them.
Solution 4: One Contains with computed values
Solution 3 failed because the two separate Contains statements don't only filter the combinations of their values. What if we create a list of combinations first and try to match these combinations? We know from solution 1 that this list should contain primitive values. For instance:
var computed = ids1.Zip(ids2, (i1,i2) => i1 * i2); // [2,6]
and the LINQ statement:
from entity in db.Table
where computed.Contains(entity.Id1 * entity.Id2)
select entity
There are some problems with this approach. First, you'll see that this also returns entity {1,6}. The combination function (a*b) does not produce values that uniquely identify a pair in the database. Now we could create a list of strings like ["Id1=1,Id2=2","Id1=2,Id2=3]" and do
from entity in db.Table
where computed.Contains("Id1=" + entity.Id1 + "," + "Id2=" + entity.Id2)
select entity
(This would work in EF6, not in earlier versions).
This is getting pretty messy. But a more important problem is that this solution is not sargable, which means: it bypasses any database indexes on Id1 and Id2 that could have been used otherwise. This will perform very very poorly.
Solution 5: Best of 2 and 3
So the most viable solution I can think of is a combination of Contains and a join in memory: First do the contains statement as in solution 3. Remember, it got us very close to what we wanted. Then refine the query result by joining the result as an in-memory list:
var rawSelection = from entity in db.Table
where ids1.Contains(entity.Id1) && ids2.Contains(entity.Id2)
select entity;
var refined = from entity in rawSelection.AsEnumerable()
join pair in Tuples on new { entity.Id1, entity.Id2 }
equals new { Id1 = pair.Item1, Id2 = pair.Item2 }
select entity;
It's not elegant, messy all the same maybe, but so far it's the only scalable1 solution to this problem I found, and applied in my own code.
Solution 6: Build a query with OR clauses
Using a Predicate builder like Linqkit or alternatives, you can build a query that contains an OR clause for each element in the list of combinations. This could be a viable option for really short lists. With a couple of hundreds of elements, the query will start performing very poorly. So I don't consider this a good solution unless you can be 100% sure that there will always be a small number of elements. One elaboration of this option can be found here.
Solution 7: Unions
There's also a solution using UNIONs that I posted later here.
1As far as the Contains statement is scalable: Scalable Contains method for LINQ against a SQL backend
Solution for Entity Framework Core with SQL Server
🎉 NEW! QueryableValues EF6 Edition has arrived!
The following solution makes use of QueryableValues. This is a library that I wrote to primarily solve the problem of query plan cache pollution in SQL Server caused by queries that compose local values using the Contains LINQ method. It also allows you to compose values of complex types in your queries in a performant way, which will achieve what's being asked in this question.
First you will need to install and set up the library, after doing that you can use any of the following patterns that will allow you to query your entities using a composite key:
// Required to make the AsQueryableValues method available on the DbContext.
using BlazarTech.QueryableValues;
// Local data that will be used to query by the composite key
// of the fictitious OrderProduct table.
var values = new[]
{
new { OrderId = 1, ProductId = 10 },
new { OrderId = 2, ProductId = 20 },
new { OrderId = 3, ProductId = 30 }
};
// Optional helper variable (needed by the second example due to CS0854)
var queryableValues = dbContext.AsQueryableValues(values);
// Example 1 - Using a Join (preferred).
var example1Results = dbContext
.OrderProduct
.Join(
queryableValues,
e => new { e.OrderId, e.ProductId },
v => new { v.OrderId, v.ProductId },
(e, v) => e
)
.ToList();
// Example 2 - Using Any (similar behavior as Contains).
var example2Results = dbContext
.OrderProduct
.Where(e => queryableValues
.Where(v =>
v.OrderId == e.OrderId &&
v.ProductId == e.ProductId
)
.Any()
)
.ToList();
Useful Links
Nuget Package
GitHub Repository
Benchmarks
QueryableValues is distributed under the MIT license.
You can use Union for each composite primary key:
var compositeKeys = new List<CK>
{
new CK { id1 = 1, id2 = 2 },
new CK { id1 = 1, id2 = 3 },
new CK { id1 = 2, id2 = 4 }
};
IQuerable<CK> query = null;
foreach(var ck in compositeKeys)
{
var temp = context.Table.Where(x => x.id1 == ck.id1 && x.id2 == ck.id2);
query = query == null ? temp : query.Union(temp);
}
var result = query.ToList();
You can create a collection of strings with both keys like this (I am assuming that your keys are int type):
var id1id2Strings = listOfIds.Select(p => p.Id1+ "-" + p.Id2);
Then you can just use "Contains" on your db:
using (dbEntities context = new dbEntities())
{
var rec = await context.Table1.Where(entity => id1id2Strings .Contains(entity.Id1+ "-" + entity.Id2));
return rec.ToList();
}
You need a set of objects representing the keys you want to query.
class Key
{
int Id1 {get;set;}
int Id2 {get;set;}
If you have two lists and you simply check that each value appears in their respective list then you are getting the cartesian product of the lists - which is likely not what you want. Instead you need to query the specific combinations required
List<Key> keys = // get keys;
context.Table.Where(q => keys.Any(k => k.Id1 == q.Id1 && k.Id2 == q.Id2));
I'm not completely sure that this is valid use of Entity Framework; you may have issues with sending the Key type to the database. If that happens then you can be creative:
var composites = keys.Select(k => p1 * k.Id1 + p2 * k.Id2).ToList();
context.Table.Where(q => composites.Contains(p1 * q.Id1 + p2 * q.Id2));
You can create an isomorphic function (prime numbers are good for this), something like a hashcode, which you can use to compare the pair of values. As long as the multiplicative factors are co-prime this pattern will be isomorphic (one-to-one) - i.e. the result of p1*Id1 + p2*Id2 will uniquely identify the values of Id1 and Id2 as long as the prime numbers are correctly chosen.
But then you end up in a situation where you're implementing complex concepts and someone is going to have to support this. Probably better to write a stored procedure which takes the valid key objects.
Ran into this issue as well and needed a solution that both did not perform a table scan and also provided exact matches.
This can be achieved by combining Solution 3 and Solution 4 from Gert Arnold's Answer
var firstIds = results.Select(r => r.FirstId);
var secondIds = results.Select(r => r.SecondId);
var compositeIds = results.Select(r => $"{r.FirstId}:{r.SecondId}");
var query = from e in dbContext.Table
//first check the indexes to avoid a table scan
where firstIds.Contains(e.FirstId) && secondIds.Contains(e.SecondId))
//then compare the compositeId for an exact match
//ToString() must be called unless using EF Core 5+
where compositeIds.Contains(e.FirstId.ToString() + ":" + e.SecondId.ToString()))
select e;
var entities = await query.ToListAsync();
For EF Core I use a slightly modified version of the bucketized IN method by EricEJ to map composite keys as tuples. It performs pretty well for small sets of data.
Sample usage
List<(int Id, int Id2)> listOfIds = ...
context.Table.In(listOfIds, q => q.Id, q => q.Id2);
Implementation
public static IQueryable<TQuery> In<TKey1, TKey2, TQuery>(
this IQueryable<TQuery> queryable,
IEnumerable<(TKey1, TKey2)> values,
Expression<Func<TQuery, TKey1>> key1Selector,
Expression<Func<TQuery, TKey2>> key2Selector)
{
if (values is null)
{
throw new ArgumentNullException(nameof(values));
}
if (key1Selector is null)
{
throw new ArgumentNullException(nameof(key1Selector));
}
if (key2Selector is null)
{
throw new ArgumentNullException(nameof(key2Selector));
}
if (!values.Any())
{
return queryable.Take(0);
}
var distinctValues = Bucketize(values);
if (distinctValues.Length > 1024)
{
throw new ArgumentException("Too many parameters for SQL Server, reduce the number of parameters", nameof(values));
}
var predicates = distinctValues
.Select(v =>
{
// Create an expression that captures the variable so EF can turn this into a parameterized SQL query
Expression<Func<TKey1>> value1AsExpression = () => v.Item1;
Expression<Func<TKey2>> value2AsExpression = () => v.Item2;
var firstEqual = Expression.Equal(key1Selector.Body, value1AsExpression.Body);
var visitor = new ReplaceParameterVisitor(key2Selector.Parameters[0], key1Selector.Parameters[0]);
var secondEqual = Expression.Equal(visitor.Visit(key2Selector.Body), value2AsExpression.Body);
return Expression.AndAlso(firstEqual, secondEqual);
})
.ToList();
while (predicates.Count > 1)
{
predicates = PairWise(predicates).Select(p => Expression.OrElse(p.Item1, p.Item2)).ToList();
}
var body = predicates.Single();
var clause = Expression.Lambda<Func<TQuery, bool>>(body, key1Selector.Parameters[0]);
return queryable.Where(clause);
}
class ReplaceParameterVisitor : ExpressionVisitor
{
private ParameterExpression _oldParameter;
private ParameterExpression _newParameter;
public ReplaceParameterVisitor(ParameterExpression oldParameter, ParameterExpression newParameter)
{
_oldParameter = oldParameter;
_newParameter = newParameter;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (ReferenceEquals(node, _oldParameter))
return _newParameter;
return base.VisitParameter(node);
}
}
/// <summary>
/// Break a list of items tuples of pairs.
/// </summary>
private static IEnumerable<(T, T)> PairWise<T>(this IEnumerable<T> source)
{
var sourceEnumerator = source.GetEnumerator();
while (sourceEnumerator.MoveNext())
{
var a = sourceEnumerator.Current;
sourceEnumerator.MoveNext();
var b = sourceEnumerator.Current;
yield return (a, b);
}
}
private static TKey[] Bucketize<TKey>(IEnumerable<TKey> values)
{
var distinctValueList = values.Distinct().ToList();
// Calculate bucket size as 1,2,4,8,16,32,64,...
var bucket = 1;
while (distinctValueList.Count > bucket)
{
bucket *= 2;
}
// Fill all slots.
var lastValue = distinctValueList.Last();
for (var index = distinctValueList.Count; index < bucket; index++)
{
distinctValueList.Add(lastValue);
}
var distinctValues = distinctValueList.ToArray();
return distinctValues;
}
In the absence of a general solution, I think there are two things to consider:
Avoid multi-column primary keys (will make unit testing easier too).
But if you have to, chances are that one of them will reduce the
query result size to O(n) where n is the size of the ideal query
result. From here, its Solution 5 from Gerd Arnold above.
For example, the problem leading me to this question was querying order lines, where the key is order id + order line number + order type, and the source had the order type being implicit. That is, the order type was a constant, order ID would reduce the query set to order lines of relevant orders, and there would usually be 5 or less of these per order.
To rephrase: If you have a composite key, changes are that one of them have very few duplicates. Apply Solution 5 from above with that.
I tried this solution and it worked with me and the output query was perfect without any parameters
using LinqKit; // nuget
var customField_Ids = customFields?.Select(t => new CustomFieldKey { Id = t.Id, TicketId = t.TicketId }).ToList();
var uniqueIds1 = customField_Ids.Select(cf => cf.Id).Distinct().ToList();
var uniqueIds2 = customField_Ids.Select(cf => cf.TicketId).Distinct().ToList();
var predicate = PredicateBuilder.New<CustomFieldKey>(false); //LinqKit
var lambdas = new List<Expression<Func<CustomFieldKey, bool>>>();
foreach (var cfKey in customField_Ids)
{
var id = uniqueIds1.Where(uid => uid == cfKey.Id).Take(1).ToList();
var ticketId = uniqueIds2.Where(uid => uid == cfKey.TicketId).Take(1).ToList();
lambdas.Add(t => id.Contains(t.Id) && ticketId.Contains(t.TicketId));
}
predicate = AggregateExtensions.AggregateBalanced(lambdas.ToArray(), (expr1, expr2) =>
{
var invokedExpr = Expression.Invoke(expr2, expr1.Parameters.Cast<Expression>());
return Expression.Lambda<Func<CustomFieldKey, bool>>
(Expression.OrElse(expr1.Body, invokedExpr), expr1.Parameters);
});
var modifiedCustomField_Ids = repository.GetTable<CustomFieldLocal>()
.Select(cf => new CustomFieldKey() { Id = cf.Id, TicketId = cf.TicketId }).Where(predicate).ToArray();
I ended up writing a helper for this problem that relies on System.Linq.Dynamic.Core;
Its a lot of code and don't have time to refactor at the moment but input / suggestions appreciated.
public static IQueryable<TEntity> WhereIsOneOf<TEntity, TSource>(this IQueryable<TEntity> dbSet,
IEnumerable<TSource> source,
Expression<Func<TEntity, TSource,bool>> predicate) where TEntity : class
{
var (where, pDict) = GetEntityPredicate(predicate, source);
return dbSet.Where(where, pDict);
(string WhereStr, IDictionary<string, object> paramDict) GetEntityPredicate(Expression<Func<TEntity, TSource, bool>> func, IEnumerable<TSource> source)
{
var firstP = func.Parameters[0];
var binaryExpressions = RecurseBinaryExpressions((BinaryExpression)func.Body);
var i = 0;
var paramDict = new Dictionary<string, object>();
var res = new List<string>();
foreach (var sourceItem in source)
{
var innerRes = new List<string>();
foreach (var bExp in binaryExpressions)
{
var emp = ToEMemberPredicate(firstP, bExp);
var val = emp.GetKeyValue(sourceItem);
var pName = $"#{i++}";
paramDict.Add(pName, val);
var str = $"{emp.EntityMemberName} {emp.SQLOperator} {pName}";
innerRes.Add(str);
}
res.Add( "(" + string.Join(" and ", innerRes) + ")");
}
var sRes = string.Join(" || ", res);
return (sRes, paramDict);
}
EMemberPredicate ToEMemberPredicate(ParameterExpression firstP, BinaryExpression bExp)
{
var lMember = (MemberExpression)bExp.Left;
var rMember = (MemberExpression)bExp.Right;
var entityMember = lMember.Expression == firstP ? lMember : rMember;
var keyMember = entityMember == lMember ? rMember : lMember;
return new EMemberPredicate(entityMember, keyMember, bExp.NodeType);
}
List<BinaryExpression> RecurseBinaryExpressions(BinaryExpression e, List<BinaryExpression> runningList = null)
{
if (runningList == null) runningList = new List<BinaryExpression>();
if (e.Left is BinaryExpression lbe)
{
var additions = RecurseBinaryExpressions(lbe);
runningList.AddRange(additions);
}
if (e.Right is BinaryExpression rbe)
{
var additions = RecurseBinaryExpressions(rbe);
runningList.AddRange(additions);
}
if (e.Left is MemberExpression && e.Right is MemberExpression)
{
runningList.Add(e);
}
return runningList;
}
}
Helper class:
public class EMemberPredicate
{
public readonly MemberExpression EntityMember;
public readonly MemberExpression KeyMember;
public readonly PropertyInfo KeyMemberPropInfo;
public readonly string EntityMemberName;
public readonly string SQLOperator;
public EMemberPredicate(MemberExpression entityMember, MemberExpression keyMember, ExpressionType eType)
{
EntityMember = entityMember;
KeyMember = keyMember;
KeyMemberPropInfo = (PropertyInfo)keyMember.Member;
EntityMemberName = entityMember.Member.Name;
SQLOperator = BinaryExpressionToMSSQLOperator(eType);
}
public object GetKeyValue(object o)
{
return KeyMemberPropInfo.GetValue(o, null);
}
private string BinaryExpressionToMSSQLOperator(ExpressionType eType)
{
switch (eType)
{
case ExpressionType.Equal:
return "==";
case ExpressionType.GreaterThan:
return ">";
case ExpressionType.GreaterThanOrEqual:
return ">=";
case ExpressionType.LessThan:
return "<";
case ExpressionType.LessThanOrEqual:
return "<=";
case ExpressionType.NotEqual:
return "<>";
default:
throw new ArgumentException($"{eType} is not a handled Expression Type.");
}
}
}
Use Like so:
// This can be a Tuple or whatever.. If Tuple, then y below would be .Item1, etc.
// This data structure is up to you but is what I use.
[FromBody] List<CustomerAddressPk> cKeys
var res = await dbCtx.CustomerAddress
.WhereIsOneOf(cKeys, (x, y) => y.CustomerId == x.CustomerId
&& x.AddressId == y.AddressId)
.ToListAsync();
Hope this helps others.
in Case of composite key you can use another idlist and add a condition for that in your code
context.Table.Where(q => listOfIds.Contains(q.Id) && listOfIds2.Contains(q.Id2));
or you can use one another trick create a list of your keys by adding them
listofid.add(id+id1+......)
context.Table.Where(q => listOfIds.Contains(q.Id+q.id1+.......));
I tried this on EF Core 5.0.3 with the Postgres provider.
context.Table
.Select(entity => new
{
Entity = entity,
CompositeKey = entity.Id1 + entity.Id2,
})
.Where(x => compositeKeys.Contains(x.CompositeKey))
.Select(x => x.Entity);
This produced SQL like:
SELECT *
FROM table AS t
WHERE t.Id1 + t.Id2 IN (#__compositeKeys_0)),
Caveats
this should only be used where the combination of Id1 and Id2 will always produce a unique result (e.g., they're both UUIDs)
this cannot use indexes, though you could save the composite key to the db with an index
I have a method in my data layer in which I pass in a set of search parameters, dynamically build up the required 'where' clause using the PredicateBuilder class from LINQKit and then execute the LINQ query to return a list of objects.
In some cases the search criteria may contain a category name which doesn't exist in the Library table, I then need to join to this table to perform the necessary query. The reason it's been done this way is because there are potentially hundreds of categories per book and for optimisation reasons I only want the join to be performed if actually required.
Now my question is, is it possible to dynamically append this join to the LINQ query?
After a few attempts at trying to get this to work I've unfortunately had to resort to the 'cut-and-paste' pattern which I dislike immensely but needed something that worked for it to get 'out-the-door'.
The code below is an extract with variables renamed of what I've currently got (i.e. this isn't really a library application!):
public IEnumerable<ILibrarySearchResultsDTO> SearchLibrary(ISearchLibrary searchValues)
{
var whereStatement = PredicateBuilder.True<Library>();
bool categorySearch = false;
if (!string.IsNullOrEmpty(searchValues.AuthorFirstName))
whereStatement = whereStatement.And(q => q.AuthorFirstName == searchValues.AuthorFirstName);
if (!string.IsNullOrEmpty(searchValues.AuthorLastName))
whereStatement = whereStatement.And(q => q.AuthorLastName == searchValues.AuthorLastName);
if (!string.IsNullOrEmpty(searchValues.CategoryName))
categorySearch = true;
var libraryObjectSet = Context.CreateObjectSet<Library>();
LibraryObjectSet.MergeOption = MergeOption.NoTracking;
var categoriesObjectSet = Context.CreateObjectSet<Categories>();
categoriesObjectSet.MergeOption = MergeOption.NoTracking;
if (!categorySearch)
{
var query = from lib in libraryObjectSet
.Where(whereStatement)
.Take(ConfigHelper.MaxQueryRecords)
.AsExpandable()
select new LibrarySearchResultsDTO()
{
BookName = lib.BookName,
AuthorFirstName = lib.AuthorFirstName,
AuthorLastName = lib.AuthorLastName,
ISBN = lib.ISBN
};
}
else
{
var query = from lib in LibraryObjectSet
.Where(whereStatement)
.Take(ConfigHelper.MaxQueryRecords)
.AsExpandable()
join categories_LKP in categoriesObjectSet on new { CategoryID = lib.CategoryID, CategoryName = searchValues.CategoryName }
equals new { CategoryID = categories_LKP.CategoryID, CategoryName = categories_LKP.CategoryName }
select new LibrarySearchResultsDTO()
{
BookName = lib.BookName,
AuthorFirstName = lib.AuthorFirstName,
AuthorLastName = lib.AuthorLastName,
ISBN = lib.ISBN
};
}
return query.ToList();
}
I've had to create the sample code in Notepad++ and because it's a contrived example I haven't been able to check if it compiles. Should do though (I hope!).
If there is a navigation property from Library to Category you can just dynamically add another predicate:
if (!string.IsNullOrEmpty(searchValues.CategoryName))
{
whereStatement = whereStatement
.And(q => q.Categories
.Any(c => c.CategoryName == searchValues.CategoryName));
}
If the navigation property is not present, you can still use a join in the predicate without having to duplicate the whole query. But it could be a good reason to add a navigation property.
You may use Reflection API like a following generic function...which compiles a dynamic query with a unknown type...
IQueryable<T> getQuery<T>(T myTableEntity, string[] arrayOfQueryTerms, Expression<Func<T, bool>> predicate)
{
var fieldOrProperty = getMemberInfo(predicate);
}
MemberInfo getmemberInfo<T>(Expression<Func<T,bool> expr)
{
var memberExpr = expr as MemberExpression;
if (memberExpr != null) return memberExpr.Member;
throw new ArgumentException();
}
var q = getQuery<FooTable>(foo, new[]{"Bar","Baz"}, x=>x.FieldName);
I would like to return a set of entities who has and ID that is contained in a list or array of IDs using LINQ and Data Services. I know how to this using LinqToEF but I am at a loss how to this with Data Services or using OData query conventions for that matter.
My thought is that I would do something like:
int[] intArray = {321456, 321355, 218994, 189232};
var query = (from data in context.Entity
where intArray.contains(data.ID)
select data);
Is there any way to accomplish using Data Services / OData? I know I could probably hack it with a Service Operation but I would prefer not to do that.
Cheers.
Currently OData (the underlying protocol) doesn't support the Contains operation. So that's why the client library does not translate the above query.
People are basically using two ways to overcome this limitation:
1) Use service operations as you noted.
2) Construct a where clause dynamically which uses simple comparisons to compare the value to each item from the array. So if the array contains 1, 2, 3, the where would be data.ID == 1 || data.ID == 2 || data.ID == 3
The #2 solution is nice because it's a client side only change. The downside is, that it only works for small arrays. If the array contains too many items the expression gets too long and that leads to all kinds of troubles.
The #1 solution doesn't have the size problem, but you need to provide the operation on the server.
Here is my realization of WhereIn() Method, to filter IQueryable collection by a set of selected entities:
public static IQueryable<T> WhereIn<T,TProp>(this IQueryable<T> source, Expression<Func<T,TProp>> memberExpr, IEnumerable<TProp> values) where T : class
{
Expression predicate = null;
ParameterExpression param = Expression.Parameter(typeof(T), "t");
bool IsFirst = true;
// Create a comparison for each value eg:
// IN: t => t.Id == 1 | t.Id == 2
MemberExpression me = (MemberExpression) memberExpr.Body;
foreach (TProp val in values)
{
ConstantExpression ce = Expression.Constant(val);
Expression comparison = Expression.Equal(me, ce);
if (IsFirst)
{
predicate = comparison;
IsFirst = false;
}
else
{
predicate = Expression.Or(predicate, comparison);
}
}
return predicate != null
? source.Where(Expression.Lambda<Func<T, bool>>(predicate, param)).AsQueryable<T>()
: source;
}
And calling of this method looks like:
IQueryable<Product> q = context.Products.ToList();
var SelectedProducts = new List<Product>
{
new Product{Id=23},
new Product{Id=56}
};
...
// Collecting set of product id's
var selectedProductsIds = SelectedProducts.Select(p => p.Id).ToList();
// Filtering products
q = q.WhereIn(c => c.Product.Id, selectedProductsIds);
Thank you men you really helped me :) :)
I did it like Vitek Karas said.
1) Download the Dynamic query library
Check this link
No need to read it just download the Dynamic query library
2)Check the project named DynamicQuery. In it you will find a class named Dynamic.cs . Copy It to your project
3)Generate your project( If you are using silverlight an error that say ReaderWriterLock is not found will appear. Don't be affraid. Just comment or delete the lines that make errors( there is just 6 or 7 lines that make errors) )
4) All done you just need now to write your query
Example: ordersContext.CLIENTS.Where(" NUMCLI > 200 || NUMCLI < 20");
All done. If you have to use the 'Contains' method you just to write a method that iterate over your array and return the string that your request will use.
private string MyFilter()
{ string st = "";
foreach(var element in myTab)
{
st = st + "ThePropertyInTheTable =" + element + "||";
}
return st;
}
I hope you understand me and that i helped someone :)
EDIT 1
I apologize but after reading the 2 suggested articles I still don't understand what I should use. I understand that using IQueryable is not preferred for various reasons but does that eliminate IEnumerable as well? Is a DataTable really my best option?
In short, I guess, what is the preferred Return type?
I have the following simple LINQ query that I want to abstract out into a DAL. What is the type of var and therefore what type should my method be?
ConnectDBDataContext context = new ConnectDBDataContext();
var lName = textEdit1.Text;
var searchByPersonLName = from c in context.tblPersons
where c.LastName == lName
orderby c.LastName
select new { c.FirstName,c.LastName,c.PersonID};
dataGridView1.DataSource = searchByPersonLName;
When I hover over it in VS it says IQueryable<T> but when I put in a breakpoint and run it it seems to call itself IEnumerable. Which is correct and how should I declare my method?
Like this -->
public static DataTable SearchPerson(string SearhParam)
{
ConnectDBDataContext context = new ConnectDBDataContext();
var persons = (from person in context.tblPersons
orderby person.LastName
select new { person.PersonID, person.LastName, person.FirstName, person.SSN });
var filteredPersonsList = persons.Where(p => p.LastName == SearhParam).ToList();
if (filteredPersonsList.Count == 0)
filteredPersonsList = persons.Where(p => p.LastName.StartsWith(SearhParam)).ToList();
var dataTable = filteredPersonsList.CopyLinqToDataTable();
return dataTable;
}
If I use IQueryable<T> what is <T> or how do I know that and what would I return?
Thanks!
For Reference the CopyToDataTable() is below.
public static DataTable CopyLinqToDataTable<T>(this IEnumerable<T> source)
{
return new ObjectShredder<T>().Shred(source, null, null);
}
public static DataTable CopyLinqToDataTable<T>(this IEnumerable<T> source,
DataTable table, LoadOption? options)
{
return new ObjectShredder<T>().Shred(source, table, options);
}
First off, IQueryable implements IEnumerable, so that is why you are potentially seeing both. See here for more details
Generally, I would recommend your DAL return your actually objects whenever possible.
I would read this blog for guidelines on how to, and how not to do what you are suggesting. Short answer, don't return IQueryable.
EDIT:
Example:
internal static File[] GetAllFilesByUserID(int userID)
{
var db = GetDataContext();
return (from files in db.Files where files.OwnerUserID == userID select files).ToArray();
}
What he means is to map your data to the object you are wanting the DAL to return.
In answer to your first question "var" is really just short for variable, and the type is what ever type is defined in the assignment.
var myvariable = string.empty;
In this example the type is that of a string.
var myreader = new StringReader();
While in this example the type is that of a StringReader.
As for your second question of "what is ". T is a generic type.
For an example of where your dal would be returning an actual object:
public Product GetProduct(int ProductID)
{
var product = from p in db.MyTable
where p.productID == ProductID
select new product { name = p.name, pricepoint = p.pricepoint, qty = p.quantity };
return product;
}
I've got a list of People that are returned from an external app and I'm creating an exclusion list in my local app to give me the option of manually removing people from the list.
I have a composite key which I have created that is common to both and I want to find an efficient way of removing people from my List using my List
e.g
class Person
{
prop string compositeKey { get; set; }
}
class Exclusions
{
prop string compositeKey { get; set; }
}
List<Person> people = GetFromDB;
List<Exclusions> exclusions = GetFromOtherDB;
List<Person> filteredResults = People - exclustions using the composite key as a comparer
I thought LINQ was the ideal way of doing this but after trying joins, extension methods, using yields, etc. I'm still having trouble.
If this were SQL I would use a not in (?,?,?) query.
Have a look at the Except method, which you use like this:
var resultingList =
listOfOriginalItems.Except(listOfItemsToLeaveOut, equalityComparer)
You'll want to use the overload I've linked to, which lets you specify a custom IEqualityComparer. That way you can specify how items match based on your composite key. (If you've already overridden Equals, though, you shouldn't need the IEqualityComparer.)
Edit: Since it appears you're using two different types of classes, here's another way that might be simpler. Assuming a List<Person> called persons and a List<Exclusion> called exclusions:
var exclusionKeys =
exclusions.Select(x => x.compositeKey);
var resultingPersons =
persons.Where(x => !exclusionKeys.Contains(x.compositeKey));
In other words: Select from exclusions just the keys, then pick from persons all the Person objects that don't have any of those keys.
I would just use the FindAll method on the List class. i.e.:
List<Person> filteredResults =
people.FindAll(p => return !exclusions.Contains(p));
Not sure if the syntax will exactly match your objects, but I think you can see where I'm going with this.
Many thanks for this guys.
I mangaged to get this down to one line:
var results = from p in People
where !(from e in exclusions
select e.CompositeKey).Contains(p.CompositeKey)
select p;
Thanks again everyone.
var thisList = new List<string>{ "a", "b", "c" };
var otherList = new List<string> {"a", "b"};
var theOnesThatDontMatch = thisList
.Where(item=> otherList.All(otherItem=> item != otherItem))
.ToList();
var theOnesThatDoMatch = thisList
.Where(item=> otherList.Any(otherItem=> item == otherItem))
.ToList();
Console.WriteLine("don't match: {0}", string.Join(",", theOnesThatDontMatch));
Console.WriteLine("do match: {0}", string.Join(",", theOnesThatDoMatch));
//Output:
//don't match: c
//do match: a,b
Adapt the list types and lambdas accordingly, and you can filter out anything.
https://dotnetfiddle.net/6bMCvN
You can use the "Except" extension method (see http://msdn.microsoft.com/en-us/library/bb337804.aspx)
In your code
var difference = people.Except(exclusions);
I couldn't figure out how to do this in pure MS LINQ, so I wrote my own extension method to do it:
public static bool In<T>(this T objToCheck, params T[] values)
{
if (values == null || values.Length == 0)
{
return false; //early out
}
else
{
foreach (T t in values)
{
if (t.Equals(objToCheck))
return true; //RETURN found!
}
return false; //nothing found
}
}
I would do something like this but i bet there is a simpler way. i think the sql from linqtosql would use a select from person Where NOT EXIST(select from your exclusion list)
static class Program
{
public class Person
{
public string Key { get; set; }
public Person(string key)
{
Key = key;
}
}
public class NotPerson
{
public string Key { get; set; }
public NotPerson(string key)
{
Key = key;
}
}
static void Main()
{
List<Person> persons = new List<Person>()
{
new Person ("1"),
new Person ("2"),
new Person ("3"),
new Person ("4")
};
List<NotPerson> notpersons = new List<NotPerson>()
{
new NotPerson ("3"),
new NotPerson ("4")
};
var filteredResults = from n in persons
where !notpersons.Any(y => n.Key == y.Key)
select n;
foreach (var item in filteredResults)
{
Console.WriteLine(item.Key);
}
}
}
This LINQ below will generate the SQL for a left outer join and then take all of the results that don't find a match in your exclusion list.
List<Person> filteredResults =from p in people
join e in exclusions on p.compositeKey equals e.compositeKey into temp
from t in temp.DefaultIfEmpty()
where t.compositeKey == null
select p
let me know if it works!
var result = Data.Where(x =>
{
bool condition = true;
double accord = (double)x[Table.Columns.IndexOf(FiltercomboBox.Text)];
return condition && accord >= double.Parse(FilterLowertextBox.Text) && accord <= double.Parse(FilterUppertextBox.Text);
});