Develop Reference

Generate dynamic LINQ expression based on array

I am using LINQ exprssion to query customers and filter them by state names. I have following query which works fine till the time I have 4 items in my statesArray.
public void GetCustomersForSelectedStates(string[] statesArray)
{
var customers = _repo.GetAllCustomers();
var filteredCustomers = from CUST in customers
join ST in States on CT.Tag_Id equals ST.Id
where CUST.ID == customer.ID && (ST.Name == statesArray[0] ||ST.Name ==statesArray[1] || ST.Name== statesArray[2]||ST.Name =statesArray[3])
//Do something with customers
}
I want following exprssion to be created dynamically:
(ST.Name == statesArray[0] ||ST.Name ==statesArray[1] ||
ST.Name== statesArray[2]||ST.Name =statesArray[3])
For example , create the dynamicQuery like below
var dynamicQuery = "(";
var dynamicQuery = "(";
for (int i = 0; i < statesArray.Count(); i++)
{
dynamicQuery += "ST.Name =="+statesArray[0];
if(i==statesArray.Count())
dynamicQuery+=")"
}
and then use it something like following,
//Psuedo code
var customers = _repo.GetAllCustomers();
var filteredCustomers = from CUST in customers
join ST in States on CT.Tag_Id equals ST.Id
where CUST.ID == customer.ID && Expression(dynamicQuery)

To do that via dynamic expressions basically means building a tree of:
(x.Foo == val0 || x.Foo == val1 || x.Foo == val2)
You can do that like this:
static Expression<Func<T, bool>> Where<T, TVal>(Expression<Func<T, TVal>> selector,
IEnumerable<TVal> values)
{
Expression result = null;
foreach (var val in values)
{
var match = Expression.Equal(
selector.Body,
Expression.Constant(val, typeof(TVal)));
result = result == null ? match : Expression.OrElse(result, match);
}
if (result == null) return x => true; // always match if no inputs
return Expression.Lambda<Func<T, bool>>(result, selector.Parameters);
}
with example usage:
string[] names = { "a", "c" };
var predicate = Where<Customer, string>(c => c.Name, names);
You can then use this predicate in the IQueryable<T>.Where extension method.
To combine it in your case, first do your regular LINQ:
var customers = _repo.GetAllCustomers();
var filteredCustomers = from CUST in customers
join ST in States on CT.Tag_Id equals ST.Id
where CUST.ID == customer.ID;
Now as a separate step apply the extra filter:
customers = customers.Where(predicate);
What this does is accept an input lambda of the form c => c.Name, then reuses the c.Name body for each c.Name == {val}, and reuses the c parameter as the parameter for the lambda we're creating. Each value from values becomes a typed constant. The Expression.Equal gives us the == test. The OrElse is the ||, noting that if result is null, this is the first item, so we just use the match expression itself.

You can add another join
var customers = _repo.GetAllCustomers();
var filteredCustomers = from CUST in customers
join ST in States on CT.Tag_Id equals ST.Id
join SA in statesArray on ST.Name equals SA // No need dynamic expression now
where CUST.ID == customer.ID
//Do something with customers

Improve the time complexity of current Linq queries

I have the following lists:
RakeSnapshots, ProductMovements
Aim is to process the both and get the count of elements that match a condition, as follows:
Consider RakeSnapshots with StatusCode == "Dumping"
Consider ProductMovement with Status == "InProgress"
Fetch the count of all elements both lists, which meet the condition RakeSnapshots.RakeCode equal to ProductMovements.ProductCode
Following are my current options:
// Code 1:
var resultCount = ProductMovements.Where(x => RakeSnapshots
.Where(r => r.StatusCode == "Dumping")
.Any(y => y.RakeCode == x.ProductCode &&
x.Status == "InProgress"))
.Count();
// Code 2:
var productMovementsInprogress = ProductMovements.Where(x => x.Status == "InProgress");
var rakeSnapShotsDumping = RakeSnapshots.Where(r => r.StatusCode == "Dumping");
var resultCount = productMovementsInprogress.Zip(rakeSnapShotsDumping,(x,y) => (y.RakeCode == x.ProductCode) ? true : false)
.Where(x => x).Count();
Challenge is both the codes are O(n^2) complexity, is there a way to improve it, this will hurt if the data is very large

You can use an inner join to do this:
var dumpingRakeSnapshots = rakeSnapshots.Where(r => r.StatusCode == "Dumping");
var inProgressProductMovements = productMovements.Where(p => p.Status == "InProgress");
var matches =
from r in dumpingRakeSnapshots
join p in inProgressProductMovements on r.RakeCode equals p.ProductCode
select r;
int count = matches.Count(); // Here's the answer.
Note that (as Ivan Stoev points out) this only works if RakeCode is the primary key of RakeSnapshots.
If it is not, you will have to use a grouped join.
Here's the Linq query syntax version that you should use in that case, but note that this is exactly the same as Ivan's answer (only in Linq query form):
var matches =
from r in dumpingRakeSnapshots
join p in inProgressProductMovements on r.RakeCode equals p.ProductCode into gj
select gj;
For completeness, here's a compilable console app that demonstrates the different results you'll get if RakeCode and ProductCode are not primary keys:
using System;
using System.Collections.Generic;
using System.Linq;
namespace ConsoleApp1
{
class RakeSnapshot
{
public string StatusCode;
public string RakeCode;
}
class ProductMovement
{
public string Status;
public string ProductCode;
}
sealed class Program
{
void run()
{
var rakeSnapshots = new List<RakeSnapshot>
{
new RakeSnapshot {StatusCode = "Dumping", RakeCode = "1"},
new RakeSnapshot {StatusCode = "Dumping", RakeCode = "1"},
new RakeSnapshot {StatusCode = "Dumping", RakeCode = "2"}
};
var productMovements = new List<ProductMovement>
{
new ProductMovement {Status = "InProgress", ProductCode = "1"},
new ProductMovement {Status = "InProgress", ProductCode = "2"},
new ProductMovement {Status = "InProgress", ProductCode = "2"}
};
var dumpingRakeSnapshots = rakeSnapshots.Where(r => r.StatusCode == "Dumping");
var inProgressProductMovements = productMovements.Where(p => p.Status == "InProgress");
// Inner join.
var matches1 =
from r in dumpingRakeSnapshots
join p in inProgressProductMovements on r.RakeCode equals p.ProductCode
select r;
Console.WriteLine(matches1.Count());
// Grouped join.
var matches2 =
from r in dumpingRakeSnapshots
join p in inProgressProductMovements on r.RakeCode equals p.ProductCode into gj
select gj;
Console.WriteLine(matches2.Count());
// OP's code.
var resultCount =
productMovements
.Count(x => rakeSnapshots
.Where(r => r.StatusCode == "Dumping")
.Any(y => y.RakeCode == x.ProductCode && x.Status == "InProgress"));
Console.WriteLine(resultCount);
}
static void Main(string[] args)
{
new Program().run();
}
}
}

Sounds like Group Join which (as well as Join) is the most efficient LINQ way of correlating two sets:
var resultCount = ProductMovements.Where(p => p.Status == "InProgress")
.GroupJoin(RakeSnapshots.Where(r => r.StatusCode == "Dumping"),
p => p.ProductCode, r => r.RakeCode, (p, match) => match)
.Count(match => match.Any());
The time complexity of the above is O(N+M).

Normally, with an O(N^2), you'd look to create an intermediate 'search' data structure which speeds up the lookup. Something like a hash table for O(1) access, or a sorted list for O(log N) access.
Technically, you have two different lists, so the actual order would be O(P.R), where P is the number of product movements, and R is the number of rake snapshots.
In your case, this is your original code;
var resultCount = ProductMovements
.Where(x => RakeSnapshots
.Where(r => r.StatusCode == "Dumping")
.Any(y => y.RakeCode == x.ProductCode &&
x.Status == "InProgress"))
.Count();
Is O(P.R) because for each P, the inner where clause is looping through every R. I'd look to creating a Dictionary<T> or HashSet<T>, then transforming your code to something like
var rakeSnapshotSummary = ... magic happens here ...;
var resultCount = ProductMovements
.Where(x => rakeSnapshotSummary[x.ProductCode] == true)
.Count();
In this way, creating the snapshot is O(R), lookup into the data structure is O(1), and creating the result is O(P), for a much healthier O(P+R). I thing that's is as good as it can be.
So my suggestion for your indexing routine would be something like;
var rakeSnapshotSummary = new HashSet<string>(RakeSnapshots
.Where(r => r.StatusCode == "Dumping")
.Select(r => r.RakeCode));
This creates a HashSet<string> which will have O(1) time complexity for testing existance of a rake code. Then your final line looks like
var resultCount = ProductMovements
.Where(x => x.Status == "InProgress" && rakeSnapshotSummary.Contains(x.ProductCode))
.Count();
So overall, O(P+R) or, roughly, O(2N) => O(N).

Getting an array of dates (yearly and absolute)

I have the following 2 functions + 1 in progress:
public static IEnumerable<LockedDate> GetAllByEmployee(int employeeID)
{
var v = LockedDates.GetAll();
return from p in v where p.EmployeeID == employeeID select p;
}
public static IEnumerable<LockedDate> GetAllByCompany()
{
var v = LockedDates.GetAll();
return from p in v where p.EmployeeID == null select p;
}
public static List<LockedDate> GetAllForEmployee(int employeeID)
{
var empList = GetAllByEmployee(employeeID);
var compList = GetAllByCompany();
//What do???
return from p in empList and from q in compList where p and q are not duplicate . toList()
}
These use LINQ-SQL.
A LockedDate has a Date and an IsYearly bool. If IsYearly is true then the year should not be considered. Time should never be considered.
I now need a function that will include all the LockedDates that the employee has and the company ones into one list, without any duplicates. So if IsYearly && dd/mm are == or !IsYearly && dd/mm/yyyy are == then there is a duplicate. What might be an efficient, non-naive way of doing this?
Thanks

var yearly = source.Where(p => p.IsYearly)
.GroupBy(p => new { Month = p.Date.Month, Day = p.Date.Day })
.Select(g => g.First());
var nonYearly = source.Where(p => !p.IsYearly)
.GroupBy(p => p.Date.Date)
.Select(g => g.First());
return yearly.Union(nonYearly).ToList();
source could be done easily by Union method:
var source = GetAllByEmployee(employeeID).Union(GetAllByCompany());

cant you do this
var v = LockedDates.GetAll();
return from p in v where p.EmployeeID == null || p.EmployeeID == employeeID select p;

Dynamically filtering a column/property in to an EF 4.1 query using C#

I'm trying to create a generic "search engine" in C# using linq. I have a simple search engine that functions and look like the following.
var query = "joh smi";
var searchTerms = query.Split(new char[] { ' ' });
var numberOfTerms = searchTerms.Length;
var matches = from p in this.context.People
from t in searchTerms
where p.FirstName.Contains(t) ||
p.LastName.Contains(t)
group p by p into g
where g.Count() == numberOfTerms
select g.Key;
I want it to be more generic so I can call it like this:
var matches = Search<Person>(dataset, query, p => p.FirstName, p => p.LastName);
I've gotten as far as the following, but it fails with a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." System.NotSupportedException.
static IEnumerable<T> Find<T>(IQueryable<T> items, string query,
params Func<T, string>[] properties)
{
var terms = query.Split(' ');
var numberOfParts = terms.Length;
foreach (var prop in properties)
{
var transformed = items.SelectMany(item => terms,
(item, term) => new { term, item });
// crashes due to this method call
var filtered = transformed.Where(p => prop(p.item).Contains(p.term));
items = filtered.Select(p => p.item);
}
return from i in items
group i by i into g
where g.Count() == numberOfParts
select g.Key;
}
I'm certain it's doable, there just has to be a way to compile i => i.FirstName to an Expression<Func<T, bool>>, but that's where my LINQ expertise ends. Does anyone have any ideas?

You should use a Predicate Builder to construct your Or query, something like:
var predicate = PredicateBuilder.False<T>();
foreach (var prop in properties)
{
Func<T, string> currentProp = prop;
predicate = predicate.Or (p => currentProp(p.item).Contains(p.term));
}
var result = items.Where(predicate );

Look into using a Specification Pattern. Check out this blog. Specifically, look at the spec pattern he developed. This is a similar thought to #Variant where you can build a dynamic specification and pass it to your context or repository.

It turns out the content of the queries just needed to be 'Expanded'. I used a library I found here to expand the expressions. I think that allows Linq to Entities to translate it in to sql. You'll notice Expand gets called over and over again; I think all of them are necessary. It works, anyway. Code to follow:
using System.Linq.Expressions;
public static class SearchEngine<T>
{
class IandT<T>
{
public string Term { get; set; }
public T Item { get; set; }
}
public static IEnumerable<T> Find(
IQueryable<T> items,
string query,
params Expression<Func<T, string>>[] properties)
{
var terms = query.Split(new char[] { ' ' },
StringSplitOptions.RemoveEmptyEntries);
var numberOfParts = terms.Length;
Expression<Func<IandT<T>, bool>> falseCond = a => false;
Func<Expression<Func<IandT<T>, bool>>,
Expression<Func<IandT<T>, bool>>,
Expression<Func<IandT<T>, bool>>> combineOr =
(f, g) => (b) => f.Expand(b) || g.Expand(b);
var criteria = falseCond;
foreach (var prop in properties)
{
var currentprop = prop;
Expression<Func<IandT<T>, bool>> current = c =>
currentprop.Expand(c.Item).IndexOf(c.Term) != -1;
criteria = combineOr(criteria.Expand(), current.Expand());
}
return from p in items.ToExpandable()
from t in terms
where criteria.Expand(new IandT<T> { Item = p, Term = t })
group p by p into g
where g.Count() == numberOfParts
select g.Key;
}
}
It can be called via the following code:
var matches = Search<Person>(dataset, query, p => p.FirstName, p => p.LastName);

Dynamic "Not" by parameter in LINQ (Or any other code for that matter)

Okay,
This may be really simple or it may not even be possible, or I am just having a brain freeze :)
Here is an example of what I am trying to do:
public void SomeMethod(bool include)
{
using (AccountDataContext db = AccountContextFactory.CreateContext())
{
if (include)
{
var query = from a in db.FundingTypes where a.FundingTypeId == 1 select a;
}
else
{
var query = from a in db.FundingTypes where a.FundingTypeId != 1 select a;
}
}
}
I would like to dynamically change the != and = without having to write an entire new query. The query that I am using in real life is very large and I don't like code duplication.
Thought or Ideas?
Thanks

That seems perfectly straightforward.
var predicate = include ?
(Func<int, bool>) x=>x == 1 :
(Func<int, bool>) x=>x != 1 ;
var query = from a in db.FundingTypes where predicate(a.FundingTypeId) select a;

How about this:
var query =
from a in db.FundingTypes
where (a.FundingTypeId == 1) == include
select a;
Joe

How about this:
public void SomeMethod(bool include, Func<int, bool> query)
{
using (AccountDataContext db = AccountContextFactory.CreateContext())
{
var query = from a in db.FundingTypes where query(a.FundingTypeId) select a;
}
}

Try this:
SomeMethod(bool include)
{
using (AccountDataContext db = AccountContextFactory.CreateContext())
{
var query = from a in db.FundingTypes where !include ^ (a.FundingTypeId == 1) select a;
}
}
Edit simplified the logic with an XOR

Try this:
var predicate = include
? (Func<FundingType, bool>) f => f.FundingTypeId == 1
: (Func<FundingType, bool>) f => f.FundingTypeId != 1
return (from a in db.FundingTypes select a).Where(predicate);