Develop Reference

Efficiently make iterative LINQ to SQL queries

I am trying to implement the following algorithm using LINQ-to-SQL:
Given a list of strings L, return every row R in the DB for which every string in L is a substring of one of the column values in R.
The question is how do I do this iteratively for every string in L? I don't know how I can slickly put it all into one Linq-To-SQL statement. Note that I have no problem writing code along the lines of:
field1.contains(...) || field2.contains(...) || ...
as there are not that many columns.
E.g., if the input is
["Charlie", "Doctor", "Kor"]
we would output all rows that have a field with "Charlie" as a substring, a field with "Doctor" as a substring, and a field with "Kor" as a substring.
One approach I thought of was to make separate SQL queries for each input value and to take the intersection of all of those.
Another approach is to pick just one of the strings from the input, make a SQL query on that, convert it to a list, and filter out the rest of the strings one at a time using just LINQ in C#.
Any thoughts on an optimal way to do this?

I would try All extension method (EF6 supports it, not sure about LINQ to SQL):
List<string> values = new List<string> { "Charlie", "Doctor", "Kor" };
var query = db.Table
.Where(r => values.All(v => r.Field1.Contains(v) || r.Field2.Contains(v) || ...));
Update: Well, the assumption was wrong - as mentioned in the comments, unfortunately LINQ to SQL does not support the above construct (shame on them).
As usual in such cases, I would build dynamically a corresponding predicate expression.
In this particular case we need something like this (for N fields and M values):
r => (r.Field1.Contains(value1) || r.Field2.Contains(value1) ... || r.FieldN.Contains(value1))
&& (r.Field1.Contains(value2) || r.Field2.Contains(value2) ... || r.FieldN.Contains(value2))
...
&& (r.Field1.Contains(valueM) || r.Field2.Contains(valueM) ... || r.FieldN.Contains(valueM));
And here is a custom extension method which does that:
public static class QueryableExtensions
{
public static IQueryable<T> WhereContainsAll<T>(
this IQueryable<T> source,
IEnumerable<string> values,
params Expression<Func<T, string>>[] members)
{
var parameter = Expression.Parameter(typeof(T), "r");
var body = values
.Select(value => members
.Select(member => (Expression)Expression.Call(
Expression.MakeMemberAccess(parameter, ((MemberExpression)member.Body).Member),
"Contains", Type.EmptyTypes, Expression.Constant(value)))
.Aggregate(Expression.OrElse))
.Aggregate(Expression.AndAlso);
var predicate = Expression.Lambda<Func<T, bool>>(body, parameter);
return source.Where(predicate);
}
}
and the sample usage would be
List<string> values = new List<string> { "Charlie", "Doctor", "Kor" };
var query = db.Table.WhereContainsAll(values,
r => r.Field1, r => r.Field2, r => r.Field3, ...);
which should lead to a single SQL query which IMO should be optimal because the heavy work will be done by the database engine. Of course the query most likely will cause full table scan, but the same will happen even with single Contains (SQL LIKE) criteria.

Try this (I made an example using Lists):
var dbValues = new List<string> {"hello", "how", "are", "you"};
var substrings = new List<string> {"ello", "re"};
var result = dbValues.Where(i => substrings.Any(l => i.Contains(l))).ToList();
Result will contain {"hello","are"}
Example with database:
using (var db = new MyDatabase())
{
var substrings = new List<string> { "ello", "re" };
var result = db.MyTable.Where(i => substrings.Any(l => i.Value.Contains(l))).ToList();
}

LINQ - Where Clause using Contains

I have a LINQ statement that I need to do a "contains" with, but also need some sort of loop.
The format of the data is as follows:
x.Product_Name = "product[x], product[y], product[z]"
As user selects multiple items from a list to search on.
I need to find anything within Product_Name that was selected from the user.
var names = JsonConvert.DeserializeObject<IEnumerable<string>>
(criteria.value).ToArray();
This line gets the items a user selected from the list and stores them in an array.
query = query.Where(x => names.contains(x.Product_Name))
Doesn't work because Product_Name is a flattened out version of products, so I can't do this.
What I need is something like the following:
foreach (string s in names)
{
projectsQuery = projectsQuery.Where(x => x.Product_Name.Contains(s));
}
But when the SQL is created for the above, it uses an AND conditional instead of an OR conditional. I need to find any instances where string s is contained within the Product_Name.

You can achieve it by creating Expression tree manually. Although it is kind of hard to manage code.
var containsMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var xParameter = Expression.Parameter(typeof(x), "x");
var searchexpression = new stack<expression>();
foreach (string s in names)
{
var containsmethodexp = expression.call(Expression.Property(xParameter, "Product_Name", containsMethod, expression.constant(s));
if (searchexpression.count == 0)
{
searchexpression.push(containsmethodexp);
}
else
{
searchexpression.push(expression.orelse(containsmethodexp, searchexpression.pop()));
}
}
var finalResult = projectsQuery.Where(Expression.Lambda<Func<x, bool>>(searchexpression.pop(), new ParameterExpression[] { xParameter }));
here x is your Entity Name

Not able to Select after using Expression Tree in Group By

I have been referring this post to group by using expression tree. Here is my code:
String[] fields = { "DepartmentID", "SkillID" };
var groupLambda = GroupByExpression<Person>(fields);
var query = dbContext.People.GroupBy(groupLambda.Compile());
var queryResult = query.ToList();
Here is the method GroupByExpression which uses solution given in aforesaid post (Thanks Daniel!):
public static Expression<Func<TItem, object>> GroupByExpression<TItem>(string[] propertyNames)
{
var properties = propertyNames.Select(name => typeof(TItem).GetProperty(name)).ToArray();
var propertyTypes = properties.Select(p => p.PropertyType).ToArray();
var tupleTypeDefinition = typeof(Tuple).Assembly.GetType("System.Tuple`" + properties.Length);
var tupleType = tupleTypeDefinition.MakeGenericType(propertyTypes);
var constructor = tupleType.GetConstructor(propertyTypes);
var param = Expression.Parameter(typeof(TItem), "x");
var body = Expression.New(constructor, properties.Select(p => Expression.Property(param, p)));
var expr = Expression.Lambda<Func<TItem, object>>(body, param);
return expr;
}
I want to be able to identify fields in the group by keys with strong names in select part like query.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
How do I do this?

Now... I won't give you the solution to the question you asked, but I'll try to help you :-)
If you want to do dynamic queries, you should probably use DynamicLinq
With DynamicLinq you can do things like:
IQueryable query = context.YourTable;
var groups = query.GroupBy("new (Field1, Field2)");
I'm rereading your question...
I want to be able to identify fields in the group by keys with strong names in select part like query.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
You can't. GroupBy in general will return a IGrouping<TKey, TSource>. TKey is dynamic (because you build it based on strings), so you can't "extract" it and pass it to the compiler, so you can't do the select with strong names.
There is a single exception: if you know the types and numbers of the GroupBy TKey then something can be done. So, you gave us:
String[] fields = { "DepartmentID", "SkillID" };
If you always have two int then you can cast your query with:
.Cast<IGrouping<Tuple<int, int>, Person>>()
.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
Note that, as I've written in a comment, your GroupBy will be executed client-side, and everything after the GroupBy will be executed client-side (where client-side == where your program is vs sql-side == where your sql server is)!
DynamicLinq will solve the problem of executing the query sql-side instead of client-side, but won't solve the problem of strong vs weak naming (after a DynamicLinq you can: A) use .Cast<>() method or B) return a dynamic object/IEnumerable<dynamic>)

The syntax you're using new { x.Key.DepartmentID, x.Key.SkillID } constructs an anonymous class at compile time. If you want to create an anonymous class at runtime, see here. However, that won't allow you to "identify fields in the group by keys with strong names". If you want to construct an anonymous class at runtime, but be able to use those names at compile time, I'm afraid that's impossible.

Implementing a "like" operator multiple times in one Linq to Entities query

We have a list of strings and we need to filter our results by that list. Example would be find all students who have SSNs that start with 465, 496, or 497 (plus x more)
List<string> list = GetPossibleStartsWithValues();
var qry = from student in entities.Students.WhereStartsWith(x=>x.SSN, list)
select new SearchedStudent
{
Name = student.Name,
SSN = student.SSN,
...
}
The code provided here is close to what we need, but we can't figure out how to impliment the StartsWith that we need using the Expression Class.

Well, you could try this:
public static IQueryable<T> WhereStartsWith<T>(this IQueryable<T> source,
Expression<Func<T, string>> projection,
List<T> list)
{
return source.Where(x => list.Any(y => projection(x).StartsWith(y)));
}
That may well not work, but it would be worth trying before you go into anything more complicated.
EDIT: As you say, the above won't compile - you basically need to build an expression tree representing the bit within the Where clause. Oops. However, before you start doing that, it would be worth seeing whether it'll work in the end. Try this:
List<string> list = GetPossibleStartsWithValues();
var qry = from student in entities.Students
.Where(student => list.Any(y => student.SSN.StartsWith(y)))
select new SearchedStudent
{
Name = student.Name,
SSN = student.SSN,
...
}
If that doesn't work, then making a more general method won't be any use :(

How about using a compound statement such as
var qry = from student in entities.Students.Where(
s => list.Where( x => s.StartsWith(x)).Count() != 0 )

Conditional where with or criteria linqtosql

I've figured out how to do conditional queries with linq to sql and I've also figured out how to OR where clauses. Unfortunately I can't figure out how to do both at once. I can do a conditional where clause something like:
var ResultsFromProfiles = from AllPeeps in SearchDC.aspnet_Users
select AllPeeps;
if (SearchFirstNameBox.Checked)
{
ResultsFromProfiles = ResultsFromProfiles.Where(p => p.tblUserProfile.FirstName.Contains(SearchTerm));
}
if (SearchLastNameBox.Checked)
{
ResultsFromProfiles = ResultsFromProfiles.Where(p => p.tblUserProfile.LastName.Contains(SearchTerm));
}
This will get me any profiles where the first name AND the last name contain the search term.
Or I could do:
var ResultsFromProfiles = from p in SearchDC.aspnet_Users
where p.tblUserProfile.LastName.Contains(SearchTerm) ||
p.tblUserProfile.FirstName.Contains(SearchTerm)
select p;
This would get me any profiles where the first name OR the last name contains the search term.
I have a bunch of checkboxes where the user can specify which fields they want to search for teh search term, so I want to be able to build a query that will conditionally add them as in the first code snippet above, but add them as an OR so they work like the second snippet. That way it will search for any matches anywhere in the specified fields.
Any tips?

Yeah, use PredicateBuilder. It lets you build up dynamic queries with And or Or semantics. It's free and stable- I use it all over the place.

One way to do this is to manipulate the LINQ expression tree for your query. In your case, you'd need to build a lambda expression and substitute it in for your call to Where. Here is a working example based on a list, but the code to manipulate the expression tree is the same regardless of the query provider.
List<User> Users = new List<User>();
Users.Add(new User() { FirstName = "John", LastName = "Smith" });
Users.Add(new User() { FirstName = "Jane", LastName = "Smith" });
string Query = "John";
var Queryable = Users.AsQueryable();
var Results = (from u in Queryable
select u);
//initial method call... the lambda u => false is a place-holder that is about to be replaced
MethodCallExpression WhereExpression = (MethodCallExpression)Results.Where(u => false).Expression;
//define search options
Expression<Func<User, string, bool>> FilterLastName = (u, query) => u.LastName.Contains(query);
Expression<Func<User, string, bool>> FilterFirstName = (u, query) => u.FirstName.Contains(query);
//build a lambda based on selected search options... tie the u parameter to UserParameter and the query parameter to our Query constant
ParameterExpression UserParameter = Expression.Parameter(typeof(User), "u");
Expression Predicate = Expression.Constant(false); //for simplicity, since we're or-ing, we'll start off with false || ...
//if (condition for filtering by last name)
{
Predicate = Expression.Or(Predicate, Expression.Invoke(FilterLastName, UserParameter, Expression.Constant(Query)));
}
//if (condition for filtering by first name)
{
Predicate = Expression.Or(Predicate, Expression.Invoke(FilterFirstName, UserParameter, Expression.Constant(Query)));
}
//final method call... lambda u => false is the second parameter, and is replaced with a new lambda based on the predicate we just constructed
WhereExpression = Expression.Call(WhereExpression.Object, WhereExpression.Method, WhereExpression.Arguments[0], Expression.Lambda(Predicate, UserParameter));
//get a new IQueryable for our new expression
Results = Results.Provider.CreateQuery<User>(WhereExpression);
//enumerate results as usual
foreach (User u in Results)
{
Console.WriteLine("{0} {1}", u.FirstName, u.LastName);
}
Working with expression trees can typically be simplified by using the visitor pattern, but I've omitted that so you could more clearly see the work that has to be done.

Here's a suggestion.
I haven't tried compiling and running it, and LinqToSQL is full of surprises, so no guarantees :-)
var ResultsFromProfiles = from AllPeeps in SearchDC.aspnet_Users select AllPeeps;
IEnumerable<AspNet_User> total = new AspNew_User[0];
if (SearchFirstNameBox.Checked)
{
total = total.Concat(ResultsFromProfiles.Where(p => p.tblUserProfile.FirstName.Contains(SearchTerm));}
}
if (SearchLastNameBox.Checked)
{
total = total.Concat(ResultsFromProfiles.Where(p => p.tblUserProfile.LastName.Contains(SearchTerm));
}
total = total.Distinct();