Develop Reference

Filtering IEnumerable by collection (Linq)

I want to filter an IEnumerable object by a specific property of whatever object it is collecting. I want the option to filter by one or more property value but how many values (and what values) to filter by is only known at runtime.
Ok, so to give an example, the collected objects could be the following struct:
public struct Person
{
public string Name { get; set; }
public string Profession{ get; set; }
}
This struct could then be used by the following list, which I have populated with some arbitrary values:
List<Person> people= new List<Person>;
people.Add(new Person(){Name = "Mickey", Profession="Tinker"};
people.Add(new Person(){Name = "Donald", Profession="Tailor"};
people.Add(new Person(){Name = "Goofy", Profession="Soldier"};
people.Add(new Person(){Name = "Pluto", Profession="Spy"};
This is then put into an IEnumerable (all of them are transferred to it first)
var wantedPeople = from n in this.people select n;
So say a user was only interested in the "Tailor" and "Spy" professions, and via some sort of gui trickery the following collection was created:
List<string> wantedProfessions = new List<string>();
wantedProfessions.Add("Tailor");
wantedProfessions.Add("Spy");
Now what Linq statement can I use to filer my wantedPeople so it only includes the tailor and spy entries?
I know I could use a where clause but I don't know how to tailor it to get what I want (and doing the following is not what I want as it only works with the wantedProfessions collection above (e.g. this collection will change at runtime):
wantedPeople = from n in wantedPeople
where n.Profession == wantedProffessions[0] || n.Profession == wantedProffessions[1]
select n;

If you want to check any wanted profession from given list:
wantedPeople = from n in wantedPeople
where wantedProffessions.Contains(n.Profession)
select n;
Or you can build query with lambda syntax by applying filters one by one:
var query = people.AsEnumerable();
if (!String.IsNullOrEmpty(name))
query = query.Where(p => p.Name == name);
if (wantedProfessions.Any())
query = query.Where(p => wantedProfessions.Contains(p.Profession));
If you wanted to create more complex filters, like some name, and several professions, you can use Specification pattern. Specification can be defined by this simple interface:
public interface ISpecification<T>
{
bool Satisfied(T entity);
}
It just checks whether given entity (person) satisfies specification. Specification also look very simple:
public class PersonNameSpecification : ISpecification<Person>
{
private string _name;
public PersonNameSpecification(string name)
{
_name = name;
}
public bool Satisfied(Person person)
{
return person.Name == _name;
}
}
Profession specification:
public class PersonProfessionSpecification : ISpecification<Person>
{
private string[] _professions;
public PersonProfessionSpecification(params string[] professions)
{
_professions = professions;
}
public bool Satisfied(Person person)
{
return _professions.Contains(person.Profession);
}
}
You can create specifications which implement boolean logic, like OrSpecification or AndSpecification:
public class AndSpecification<T> : ISpecification<T>
{
private ISpecification<T> _specA;
private ISpecification<T> _specB;
public AndSpecification(ISpecification<T> specA, ISpecification<T> specB)
{
_specA = specA;
_specB = specB;
}
public bool Satisfied(T entity)
{
return _specA.Satisfied(entity) && _specB.Satisfied(entity);
}
}
public static class SpecificationExtensions
{
public static ISpecification<T> And<T>(
this ISpecification<T> specA, ISpecification<T> specB)
{
return new AndSpecification<T>(specA, specB);
}
}
Now you can create complex specification which describes people you want to get:
var professionSpec = new PersonProfessionSpecification("Tailor", "Spy");
var nameSpec = new PersonNameSpecification("Pluto");
var spec = professionSpec.And(nameSpec);
And get required people:
var result = people.Where(spec.Satisfied);

Sergey B's Solution is the correct one for your example.
Assuming that you weren't using a collection that had the Contains() method, you could also do the following:
var wantedPeople = from n in people
from p in wantedProffessions
where n.Profession.Equals(p)
select n;

C#: how to define an extension method as "with" in F#?

F# has a convenient feature "with", example:
type Product = { Name:string; Price:int };;
let p = { Name="Test"; Price=42; };;
let p2 = { p with Name="Test2" };;
F# created keyword "with" as the record types are by default immutable.
Now, is it possible to define a similar extension in C#?
seems it's a bit tricky, as in C# i'm not sure how to convert a string
Name="Test2"
to a delegate or expression?

public static T With<T, U>(this T obj, Expression<Func<T, U>> property, U value)
where T : ICloneable {
if (obj == null)
throw new ArgumentNullException("obj");
if (property == null)
throw new ArgumentNullException("property");
var memExpr = property.Body as MemberExpression;
if (memExpr == null || !(memExpr.Member is PropertyInfo))
throw new ArgumentException("Must refer to a property", "property");
var copy = (T)obj.Clone();
var propInfo = (PropertyInfo)memExpr.Member;
propInfo.SetValue(copy, value, null);
return copy;
}
public class Foo : ICloneable {
public int Id { get; set; }
public string Bar { get; set; }
object ICloneable.Clone() {
return new Foo { Id = this.Id, Bar = this.Bar };
}
}
public static void Test() {
var foo = new Foo { Id = 1, Bar = "blah" };
var newFoo = foo.With(x => x.Bar, "boo-ya");
Console.WriteLine(newFoo.Bar); //boo-ya
}
Or, using a copy constructor:
public class Foo {
public Foo(Foo other) {
this.Id = other.Id;
this.Bar = other.Bar;
}
public Foo() { }
public int Id { get; set; }
public string Bar { get; set; }
}
public static void Test() {
var foo = new Foo { Id = 1, Bar = "blah" };
var newFoo = new Foo(foo) { Bar = "boo-ya" };
Console.WriteLine(newFoo.Bar);
}
And a slight variation on George's excellent suggestion, that allows for multiple assignments:
public static T With<T>(this T obj, params Action<T>[] assignments)
where T : ICloneable {
if (obj == null)
throw new ArgumentNullException("obj");
if (assignments == null)
throw new ArgumentNullException("assignments");
var copy = (T)obj.Clone();
foreach (var a in assignments) {
a(copy);
}
return copy;
}
public static void Test() {
var foo = new Foo { Id = 1, Bar = "blah" };
var newFoo = foo.With(x => x.Id = 2, x => x.Bar = "boo-ya");
Console.WriteLine(newFoo.Bar);
}
I would probably use the second one since (1) any general purpose solution is going to be unnecessarily slow and convoluted; (2) it has the closest syntax to what you want (and the syntax does what you expect); (3) F# copy-and-update expressions are implemented similarly.

Maybe something like this:
void Main()
{
var NewProduct = ExistingProduct.With(P => P.Name = "Test2");
}
// Define other methods and classes here
public static class Extensions
{
public T With<T>(this T Instance, Action<T> Act) where T : ICloneable
{
var Result = Instance.Clone();
Act(Result);
return Result;
}
}

As an alternative to lambda function, you can use parameters with default values. The only minor issue is that you have to pick some default value that means do not change this parameter (for reference types), but null should be a safe choice:
class Product {
public string Name { get; private set; }
public int Price { get; private set; }
public Product(string name, int price) {
Name = name; Price = price;
}
// Creates a new product using the current values and changing
// the values of the specified arguments to a new value
public Product With(string name = null, int? price = null) {
return new Product(name ?? Name, price ?? Price);
}
}
// Then you can write:
var prod2 = prod1.With(name = "New product");
You have to define the method yourself, but that's always the case (unless you're going to use reflection, which less efficient). I think the syntax is reasonably nice too. If you want to make it as nice as in F#, then you'll have to use F# :-)

There is no native ability to do this in C# short of an extension method, but at what cost? a and b are reference types and any suggestion that b is based ("with") on a causes immediate confusion as to how many objects we are working with. Is there only one? Is b a copy of a ? Does b point to a ?
C# is not F#.
Please see a previous SO question of mine as answered by Eric Lippert:
"Amongst my rules of thumb for writing clear code is: put all side effects in statements; non-statement expressions should have no side effects."
More fluent C# / .NET

generic GetById for complex PK

I am looking a way to create Generic GetById that get params object[] as parameter, knows to find the key/s field/s and know to find the relevant entity.
In the way to find a solution I thought on a generic method that returns the PK fields definition and a generic method that can return the entity based on fields.
I am looking for something I can use in table with one or more fields as primary key.
EDIT
one or more fields as primary key example =
table Customers have (CompanyId, CustomerName, Address, CreateDate).
The primary key of Customers are CompanyId are CustomerName.
I am looking for generic GetById that will know to handle also those such of tables.

You can't get "generic" approach if you don't know how many members is in the key and what types do they have. I modified my solution for single key to multiple keys but as you can see it is not generic - it uses order in which keys are defined:
// Base repository class for entity with any complex key
public abstract class RepositoryBase<TEntity> where TEntity : class
{
private readonly string _entitySetName;
private readonly string[] _keyNames;
protected ObjectContext Context { get; private set; }
protected ObjectSet<TEntity> ObjectSet { get; private set; }
protected RepositoryBase(ObjectContext context)
{
if (context == null)
{
throw new ArgumentNullException("context");
}
Context = context;
ObjectSet = context.CreateObjectSet<TEntity>();
// Get entity set for current entity type
var entitySet = ObjectSet.EntitySet;
// Build full name of entity set for current entity type
_entitySetName = context.DefaultContainerName + "." + entitySet.Name;
// Get name of the entity's key properties
_keyNames = entitySet.ElementType.KeyMembers.Select(k => k.Name).ToArray();
}
public virtual TEntity GetByKey(params object[] keys)
{
if (keys.Length != _keyNames.Length)
{
throw new ArgumentException("Invalid number of key members");
}
// Merge key names and values by its order in array
var keyPairs = _keyNames.Zip(keys, (keyName, keyValue) =>
new KeyValuePair<string, object>(keyName, keyValue));
// Build entity key
var entityKey = new EntityKey(_entitySetName, keyPairs);
// Query first current state manager and if entity is not found query database!!!
return (TEntity)Context.GetObjectByKey(entityKey);
}
// Rest of repository implementation
}

I don't know how useful this would be because it is generic but you could do this:
public TEntity GetById<TEntity>(params Expression<Func<TEntity, bool>>[] keys) where TEntity : class
{
if (keys == null)
return default(TEntity);
var table = context.CreateObjectSet<TEntity>();
IQueryable<TEntity> query = null;
foreach (var item in keys)
{
if (query == null)
query = table.Where(item);
else
query = query.Where(item);
}
return query.FirstOrDefault();
}
and then you could call it like this:
var result = this.GetById<MyEntity>(a => a.EntityProperty1 == 2, a => a.EntityProperty2 == DateTime.Now);
Disclaimer: this really isn't a GetByid, it's really a "let me give you a couple parameters and give me the first entity that matches". But that being said, it uses generics and it will return an entity if there is a match and you search based on primary keys.

I think you can't implement such thing because you won't be able to join between each passed value with the appropriate key field.
I'd suggest using custom method for each entity:
Assuming Code and Name are keys in Person table:
public IEnumerable<Person> ReadById(int code, string name)
{
using (var entities = new Entities())
return entities.Persons.Where(p => p.Code = code && p.Name = name);
}

Ok this is my second stab at it. I think this would work for you.
public static class QueryExtensions
{
public static Customer GetByKey(this IQueryable<Customer> query, int customerId,string customerName)
{
return query.FirstOrDefault(a => a.CustomerId == customerId && a.CustomerName == customerName);
}
}
So the beauty behind this extension method is you can now do this:
Customer customer = Db.Customers.GetByKey(1,"myname");
You obviously have to do this for every type, but probably worth it if you need it :)

I think the set up is a bit but I do think creating a reusable pattern will pay off in the long run. I just wrote this up and haven't tested, but I based off a searching pattern I use a lot.
Required Interfaces:
public interface IKeyContainer<T>
{
Expression<Func<T, bool>> GetKey();
}
public interface IGetService<T>
{
T GetByKey(IKeyContainer<T> key);
}
Example Entities:
public class Foo
{
public int Id { get; set; }
}
public class ComplexFoo
{
public int Key1 { get; set; }
public int Key2 { get; set; }
}
Implementation Example:
public class FooKeyContainer : IKeyContainer<Foo>
{
private readonly int _id;
public FooKeyContainer(int id)
{
_id = id;
}
public Expression<Func<Foo, bool>> GetKey()
{
Expression<Func<Foo, bool>> key = x => x.Id == _id;
return key;
}
}
public class ComplexFooKeyContainer : IKeyContainer<ComplexFoo>
{
private readonly int _id;
private readonly int _id2;
public ComplexFooKeyContainer(int id, int id2)
{
_id = id;
_id2 = id2;
}
public Expression<Func<ComplexFoo, bool>> GetKey()
{
Expression<Func<ComplexFoo, bool>> key = x => x.Key1 == _id && x.Key2 == _id2;
return key;
}
}
public class ComplexFooService : IGetService<ComplexFoo>
{
public ComplexFoo GetByKey(IKeyContainer<ComplexFoo> key)
{
var entities = new List<ComplexFoo>();
return entities.Where(key.GetKey()).FirstOrDefault();
}
}
Usage:
var complexFoo = ComplexFooService.GetByKey(new ComplexFooKeyContainer(1, 2));

IEnumerable.Except() and a custom comparer

I'm having troubles with the Except() method.
Instead of returning the difference, it returns the original set.
I've tried implementing the IEquatable and IEqualityComparer in the Account class.
I've also tried creating a separate IEqualityComparer class for Account.
When the Except() method is called from main, it doesn't seem to call my custom Equals() method, but when I tried Count(), it did call the custom GetHashCode() method!
I'm sure I made a trivial mistake somewhere and I hope a fresh pair of eyes can help me.
main:
IEnumerable<Account> everyPartnerID =
from partner in dataContext.Partners
select new Account { IDPartner = partner.ID, Name = partner.Name };
IEnumerable<Account> hasAccountPartnerID =
from partner in dataContext.Partners
from account in dataContext.Accounts
where
!partner.ID.Equals(Guid.Empty) &&
account.IDPartner.Equals(partner.ID) &&
account.Username.Equals("Special")
select new Account { IDPartner = partner.ID, Name = partner.Name };
IEnumerable<Account> noAccountPartnerID =
everyPartnerID.Except(
hasAccountPartnerID,
new LambdaComparer<Account>((x, y) => x.IDPartner.Equals(y.IDPartner)));
Account:
public class Account : IEquatable<Account>
{
public Guid IDPartner{ get; set; }
public string Name{ get; set; }
/* #region IEquatable<Account> Members
public bool Equals(Account other)
{
return this.IDPartner.Equals(other.IDPartner);
}
#endregion*/
}
LambdaComparer:
public class LambdaComparer<T> : IEqualityComparer<T>
{
private readonly Func<T, T, bool> _lambdaComparer;
private readonly Func<T, int> _lambdaHash;
public LambdaComparer(Func<T, T, bool> lambdaComparer) :
this(lambdaComparer, o => o.GetHashCode())
{
}
public LambdaComparer(Func<T, T, bool> lambdaComparer, Func<T, int> lambdaHash)
{
if (lambdaComparer == null)
throw new ArgumentNullException("lambdaComparer");
if (lambdaHash == null)
throw new ArgumentNullException("lambdaHash");
_lambdaComparer = lambdaComparer;
_lambdaHash = lambdaHash;
}
public bool Equals(T x, T y)
{
return _lambdaComparer(x, y);
}
public int GetHashCode(T obj)
{
return _lambdaHash(obj);
}
}

Basically your LambdaComparer class is broken when you pass in just a single function, because it uses the "identity hash code" provider if you don't provide anything else. The hash code is used by Except, and that's what's causing the problem.
Three options here:
Implement your own ExceptBy method and then preferably contribute it to MoreLINQ which contains that sort of thing.
Use a different implementation of IEqualityComparer<T>. I have a ProjectionEqualityComparer class you can use in MiscUtil - or you can use the code as posted in another question.
Pass a lambda expression into your LambdaComparer code to use for the hash:
new LambdaComparer<Account>((x, y) => x.IDPartner.Equals(y.IDPartner)),
x => x.IDPartner.GetHashCode());

You could also quickly fix your LambdaComparer to work when only the equality parameters are supplied like this:
public LambdaComparer(Func<T, T, bool> lambdaComparer) :
this(lambdaComparer, o => 1)
{
}

Look here, how to use and implementing IEqualityComparer in way with linq.Except and beyond.
https://www.dreamincode.net/forums/topic/352582-linq-by-example-3-methods-using-iequalitycomparer/
public class Department {
public string Code { get; set; }
public string Name { get; set; }
}
public class DepartmentComparer : IEqualityComparer {
// equal if their Codes are equal
public bool Equals(Department x, Department y) {
// reference the same objects?
if (Object.ReferenceEquals(x, y)) return true;
// is either null?
if (Object.ReferenceEquals(x, null) || Object.ReferenceEquals(y, null))
return false;
return x.Code == y.Code;
}
public int GetHashCode(Department dept) {
// If Equals() returns true for a pair of objects
// then GetHashCode() must return the same value for these objects.
// if null default to 0
if (Object.ReferenceEquals(dept, null)) return 0;
return dept.Code.GetHashCode();
}
}
IEnumerable<Department> deptExcept = departments.Except(departments2,
new DepartmentComparer());
foreach (Department dept in deptExcept) {
Console.WriteLine("{0} {1}", dept.Code, dept.Name);
}
// departments not in departments2: AC, Accounts.

IMO, this answer above is the simplest solution compared to other solutions for this problem. I tweaked it such that I use the same logic for the Object class's Equals() and GetHasCode(). The benefit is that this solution is completely transparent to the client linq expression.
public class Ericsson4GCell
{
public string CellName { get; set; }
public string OtherDependantProperty { get; set; }
public override bool Equals(Object y)
{
var rhsCell = y as Ericsson4GCell;
// reference the same objects?
if (Object.ReferenceEquals(this, rhsCell)) return true;
// is either null?
if (Object.ReferenceEquals(this, null) || Object.ReferenceEquals(rhsCell, null))
return false;
return this.CellName == rhsCell.CellName;
}
public override int GetHashCode()
{
// If Equals() returns true for a pair of objects
// then GetHashCode() must return the same value for these objects.
// if null default to 0
if (Object.ReferenceEquals(this, null)) return 0;
return this.CellName.GetHashCode();
}
}

Is it possible to cache a value evaluated in a lambda expression?

In the ContainsIngredients method in the following code, is it possible to cache the p.Ingredients value instead of explicitly referencing it several times? This is a fairly trivial example that I just cooked up for illustrative purposes, but the code I'm working on references values deep inside p eg. p.InnerObject.ExpensiveMethod().Value
edit:
I'm using the PredicateBuilder from http://www.albahari.com/nutshell/predicatebuilder.html
public class IngredientBag
{
private readonly Dictionary<string, string> _ingredients = new Dictionary<string, string>();
public void Add(string type, string name)
{
_ingredients.Add(type, name);
}
public string Get(string type)
{
return _ingredients[type];
}
public bool Contains(string type)
{
return _ingredients.ContainsKey(type);
}
}
public class Potion
{
public IngredientBag Ingredients { get; private set;}
public string Name {get; private set;}
public Potion(string name) : this(name, null)
{
}
public Potion(string name, IngredientBag ingredients)
{
Name = name;
Ingredients = ingredients;
}
public static Expression<Func<Potion, bool>>
ContainsIngredients(string ingredientType, params string[] ingredients)
{
var predicate = PredicateBuilder.False<Potion>();
// Here, I'm accessing p.Ingredients several times in one
// expression. Is there any way to cache this value and
// reference the cached value in the expression?
foreach (var ingredient in ingredients)
{
var temp = ingredient;
predicate = predicate.Or (
p => p.Ingredients != null &&
p.Ingredients.Contains(ingredientType) &&
p.Ingredients.Get(ingredientType).Contains(temp));
}
return predicate;
}
}
[STAThread]
static void Main()
{
var potions = new List<Potion>
{
new Potion("Invisibility", new IngredientBag()),
new Potion("Bonus"),
new Potion("Speed", new IngredientBag()),
new Potion("Strength", new IngredientBag()),
new Potion("Dummy Potion")
};
potions[0].Ingredients.Add("solid", "Eye of Newt");
potions[0].Ingredients.Add("liquid", "Gall of Peacock");
potions[0].Ingredients.Add("gas", "Breath of Spider");
potions[2].Ingredients.Add("solid", "Hair of Toad");
potions[2].Ingredients.Add("gas", "Peacock's anguish");
potions[3].Ingredients.Add("liquid", "Peacock Sweat");
potions[3].Ingredients.Add("gas", "Newt's aura");
var predicate = Potion.ContainsIngredients("solid", "Newt", "Toad")
.Or(Potion.ContainsIngredients("gas", "Spider", "Scorpion"));
foreach (var result in
from p in potions
where(predicate).Compile()(p)
select p)
{
Console.WriteLine(result.Name);
}
}

Have you considered Memoization?
The basic idea is this; if you have an expensive function call, there is a function which will calculate the expensive value on first call, but return a cached version thereafter. The function looks like this;
static Func<T> Remember<T>(Func<T> GetExpensiveValue)
{
bool isCached= false;
T cachedResult = default(T);
return () =>
{
if (!isCached)
{
cachedResult = GetExpensiveValue();
isCached = true;
}
return cachedResult;
};
}
This means you can write this;
// here's something that takes ages to calculate
Func<string> MyExpensiveMethod = () =>
{
System.Threading.Thread.Sleep(5000);
return "that took ages!";
};
// and heres a function call that only calculates it the once.
Func<string> CachedMethod = Remember(() => MyExpensiveMethod());
// only the first line takes five seconds;
// the second and third calls are instant.
Console.WriteLine(CachedMethod());
Console.WriteLine(CachedMethod());
Console.WriteLine(CachedMethod());
As a general strategy, it might help.

Can't you simply write your boolean expression in a separate static function which you call from your lambda - passing p.Ingredients as a parameter...
private static bool IsIngredientPresent(IngredientBag i, string ingredientType, string ingredient)
{
return i != null && i.Contains(ingredientType) && i.Get(ingredientType).Contains(ingredient);
}
public static Expression<Func<Potion, bool>>
ContainsIngredients(string ingredientType, params string[] ingredients)
{
var predicate = PredicateBuilder.False<Potion>();
// Here, I'm accessing p.Ingredients several times in one
// expression. Is there any way to cache this value and
// reference the cached value in the expression?
foreach (var ingredient in ingredients)
{
var temp = ingredient;
predicate = predicate.Or(
p => IsIngredientPresent(p.Ingredients, ingredientType, temp));
}
return predicate;
}

Well, in this case, if you can't use Memoization, you're rather restricted since you can really only use the stack as your cache: You've got no way to declare a new variable at the scope you'll need. All I can think of (and I'm not claiming it will be pretty) that will do what you want but retain the composability you need would be something like...
private static bool TestWith<T>(T cached, Func<T, bool> predicate)
{
return predicate(cached);
}
public static Expression<Func<Potion, bool>>
ContainsIngredients(string ingredientType, params string[] ingredients)
{
var predicate = PredicateBuilder.False<Potion>();
// Here, I'm accessing p.Ingredients several times in one
// expression. Is there any way to cache this value and
// reference the cached value in the expression?
foreach (var ingredient in ingredients)
{
var temp = ingredient;
predicate = predicate.Or (
p => TestWith(p.Ingredients,
i => i != null &&
i.Contains(ingredientType) &&
i.Get(ingredientType).Contains(temp));
}
return predicate;
}
You could combine together the results from multiple TestWith calls into a more complex boolean expression where required - caching the appropriate expensive value with each call - or you can nest them within the lambdas passed as the second parameter to deal with your complex deep hierarchies.
It would be quite hard to read code though and since you might be introducing a bunch more stack transitions with all the TestWith calls, whether it improves performance would depend on just how expensive your ExpensiveCall() was.
As a note, there won't be any inlining in the original example as suggested by another answer since the expression compiler doesn't do that level of optimisation as far as I know.

I would say no in this case. I assume that the compiler can figure out that it uses the p.Ingredients variable 3 times and will keep the variable closeby on the stack or the registers or whatever it uses.

Turbulent Intellect has the exactly right answer.
I just want to advise that you can strip some of the nulls and exceptions out of the types you are using to make it friendlier to use them.
public class IngredientBag
{
private Dictionary<string, string> _ingredients =
new Dictionary<string, string>();
public void Add(string type, string name)
{
_ingredients[type] = name;
}
public string Get(string type)
{
return _ingredients.ContainsKey(type) ? _ingredients[type] : null;
}
public bool Has(string type, string name)
{
return name == null ? false : this.Get(type) == name;
}
}
public Potion(string name) : this(name, new IngredientBag()) { }
Then, if you have the query parameters in this structure...
Dictionary<string, List<string>> ingredients;
You can write the query like this.
from p in Potions
where ingredients.Any(i => i.Value.Any(v => p.IngredientBag.Has(i.Key, v))
select p;
PS, why readonly?