I am Looking for a cleaner way to perform the following operation:
Filter out distinct values from a dictionary of type Dictionary<T,IEnumerable<T>> , based on the uniqueness of the Value (i.e. unique by one of the attributes of T in the IEnumerable<T>).
We can ignore the key on the dictionary. Can someone suggest a good way of achieving the above ?
Jurgen Camilleri has the piece for 'SelectMany().Distinct()'. For the comparer, we use the following comparer all the time to compare based on a property:
Usage
//this uses 'Product' for 'T'. If you want to just use 'T' you'd have to constrain it
var distinctValues = dictionary.SelectMany((entry) => entry.Value)
.Distinct(new LambdaEqualityComparer<Product>(p=>p.ProductName));
Code
public class LambdaEqualityComparer<T> : IEqualityComparer<T>
{
private Func<T,object> _action;
public LambdaEqualityComparer(Func<T, object> action)
{
_action = action;
}
public bool Equals(T x, T y)
{
var areEqual = baseCheck(x, y) ?? baseCheck(getObj(x), getObj(y));
if (areEqual != null)
{
return areEqual.Value;
}
return _action(x).Equals(_action(y));
}
public int GetHashCode(T obj)
{
return _action(obj).GetHashCode();
}
/// <summary>
/// True = return true
/// False = return false
/// null = continue evaluating
/// </summary>
/// <param name="x"></param>
/// <param name="y"></param>
/// <returns></returns>
private bool? baseCheck(object x, object y)
{
if (x == null && y == null)
{
return true;
}
else if (x == null || y == null)
{
return false;
}
return null;
}
private object getObj(T t)
{
try
{
return _action(t);
}
catch (NullReferenceException)
{
}
return null;
}
}
This should work for your case:
IEnumerable<T> allValues = dictionary.SelectMany((entry) => entry.Value);
IEnumerable<T> distinctValues = allValues.Distinct(comparer);
Make sure that you create a comparer class which implements IEqualityComparer<T> so that Distinct() can differentiate between instances of T (unless T is a class which already has a comparer in the .NET Framework such as System.String).
I wrote a method that will merge dictionaries removing duplicates. Perhaps you could use the guts of this to perform something similar.
public static Dictionary<TKey, List<TValue>> MergeDictionaries<TKey, TValue>(this IEnumerable<Dictionary<TKey, List<TValue>>> dictionaries)
{
return dictionaries.SelectMany(dict => dict)
.ToLookup(pair => pair.Key, pair => pair.Value)
.ToDictionary(group => group.Key, group => group.SelectMany(value => value).Distinct().ToList());
}
Maybe, just this section:
.ToLookup(pair => pair.Key, pair => pair.Value)
.ToDictionary(group => group.Key, group => group.SelectMany(value => value).Distinct().ToList());
I have a function called "CreateCriteriaExpression" that takes a json string and creates a linq expression from it.
This method is called by another called "GetByCriteria", which calls the "CreateCriteriaExpression" method and then executes that expression against an entity framework context.
For all of my entity framework objects, the "GetByCriteria" method is identical except for it's types. So I am trying to convert it to use generics instead of hard coded types.
When the "GetByCriteria" method gets to the point that it has to call the "CreateCriteriaExpression" method, I am having it use a factory class to determine the appropriate class/method to use. Then in the "linq expression" class, the linq expression for a particular type is created and returned.
The problem I am having is that the linq expression has to be created for a specific type, but the return value is of the generic type and it won't automatically convert between the two, even though the one is a parent of the other (covariance).
Is there any way I can make this work?
Some example code:
The "GetByCriteria" method:
/// <summary>
/// Gets a <see cref="System.Collections.Generic.List"/> of <see cref="TEntity"/>
/// objects that match the passed JSON string.
/// </summary>
/// <param name="myCriteria">A list of JSON strings containing a key/value pair of "parameterNames" and "parameterValues".</param>
/// <param name="myMatchMethod">Defines which matching method to use when finding matches on the <paramref name="myCriteria"/>.</param>
/// <returns>
/// A <see cref="System.Collections.Generic.List"/> of <see cref="TEntity"/>
/// objects.
/// </returns>
/// <seealso cref="TEntity"/>
///
/// <seealso cref="Common.MultipleCriteriaMatchMethod"/>
/// <remarks>
/// This method takes a <see cref="System.Collections.Generic.List"/> of JSON strings, and a <see cref="Common.MultipleCriteriaMatchMethod"/> and returns a
/// <see cref="System.Collections.Generic.List"/> of all matching
/// <see cref="TEntity"/> objects from the back-end database. The <paramref name="myMatchMethod"/> is used to determine how to match when multiple <paramref name="myCriteria"/> are passed. You can require that any results must match on ALL the passed JSON criteria, or on ANY of the passed criteria. This is essentially an "AND" versus and "OR" comparison.
/// </remarks>
[ContractVerification(true)]
public static List<TEntity> GetByCriteria<TContext, TEntity>(List<string> myCriteria, Common.MultipleCriteriaMatchMethod myMatchMethod)
where TContext : System.Data.Objects.ObjectContext, new()
where TEntity : System.Data.Objects.DataClasses.EntityObject
{
// Setup Contracts
Contract.Requires(myCriteria != null);
TContext db = new TContext();
// Intialize return variable
List<TEntity> result = null;
// Initialize working variables
// Set the predicates to True by default (for "AND" matches)
var predicate = PredicateBuilder.True<TEntity>();
var customPropertiesPredicate = PredicateBuilder.True<TEntity>();
// Set the predicates to Falase by default (for "OR" matches)
if (myMatchMethod == Common.MultipleCriteriaMatchMethod.MatchOnAny)
{
predicate = PredicateBuilder.False<TEntity>();
customPropertiesPredicate = PredicateBuilder.False<TEntity>();
}
// Loop over each Criteria object in the passed list of criteria
foreach (string x in myCriteria)
{
// Set the Criteria to local scope (sometimes there are scope problems with LINQ)
string item = x;
if (item != null)
{
JsonLinqParser parser = JsonLinqParserFactory.GetParser(typeof(TEntity));
// If the designated MultipleCriteriaMatchMethod is "MatchOnAll" then use "AND" statements
if (myMatchMethod == Common.MultipleCriteriaMatchMethod.MatchOnAll)
{
predicate = predicate.Expand().And<TEntity>(parser.CreateCriteriaExpression<TEntity>(item).Expand());
customPropertiesPredicate = customPropertiesPredicate.Expand().And<TEntity>(parser.CreateCriteriaExpressionForCustomProperties<TEntity>(item).Expand());
}
// If the designated MultipleCriteriaMatchMethod is "MatchOnAny" then use "OR" statements
else if (myMatchMethod == Common.MultipleCriteriaMatchMethod.MatchOnAny)
{
predicate = predicate.Expand().Or<TEntity>(parser.CreateCriteriaExpression<TEntity>(item).Expand());
customPropertiesPredicate = customPropertiesPredicate.Expand().Or<TEntity>(parser.CreateCriteriaExpressionForCustomProperties<TEntity>(item).Expand());
}
}
}
// Set a temporary var to hold the results
List<TEntity> qry = null;
// Set some Contract Assumptions to waive Static Contract warnings on build
Contract.Assume(predicate != null);
Contract.Assume(customPropertiesPredicate != null);
// Run the query against the backend database
qry = db.CreateObjectSet<TEntity>().AsExpandable<TEntity>().Where<TEntity>(predicate).ToList<TEntity>();
//qry = db.CreateObjectSet<TEntity>().Where(predicate).ToList<TEntity>();
// Run the query for custom properties against the resultset obtained from the database
qry = qry.Where<TEntity>(customPropertiesPredicate.Compile()).ToList<TEntity>();
// Verify that there are results
if (qry != null && qry.Count != 0)
{
result = qry;
}
// Return the results
return result;
}
The JsonLinqParser class (does not build):
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using LinqKit;
using Newtonsoft.Json.Linq;
namespace DAL
{
internal class JsonLinqParser_Paser : JsonLinqParser
{
internal override System.Linq.Expressions.Expression<Func<TEntity, bool>> CreateCriteriaExpression<TEntity>(string myCriteria)
{
var predicate = PredicateBuilder.True<BestAvailableFIP>();
JObject o = JObject.Parse(myCriteria);
// bmp
decimal _bmp;
if (o["bmp"] != null && decimal.TryParse((string)o["bmp"], out _bmp))
{
predicate = predicate.And<BestAvailableFIP>(x => x.bmp == _bmp);
}
// COUNTY
if (!string.IsNullOrWhiteSpace((string)o["COUNTY"]))
{
string _myStringValue = (string)o["COUNTY"];
predicate = predicate.And<BestAvailableFIP>(x => x.COUNTY.Contains(_myStringValue));
}
// emp
decimal _emp;
if (o["emp"] != null && decimal.TryParse((string)o["emp"], out _emp))
{
predicate = predicate.And<BestAvailableFIP>(x => x.emp == _emp);
}
// FIPSCO_STR
if (!string.IsNullOrWhiteSpace((string)o["FIPSCO_STR"]))
{
string _myStringValue = (string)o["FIPSCO_STR"];
predicate = predicate.And<BestAvailableFIP>(x => x.FIPSCO_STR.Contains(_myStringValue));
}
// FIPSCODE
double _FIPSCODE;
if (o["FIPSCODE"] != null && double.TryParse((string)o["FIPSCODE"], out _FIPSCODE))
{
predicate = predicate.And<BestAvailableFIP>(x => x.FIPSCODE == _FIPSCODE);
}
// FROMDESC
if (!string.IsNullOrWhiteSpace((string)o["FROMDESC"]))
{
string _myStringValue = (string)o["FROMDESC"];
predicate = predicate.And<BestAvailableFIP>(x => x.FROMDESC.Contains(_myStringValue));
}
// LANEMI
decimal _LANEMI;
if (o["LANEMI"] != null && decimal.TryParse((string)o["LANEMI"], out _LANEMI))
{
predicate = predicate.And<BestAvailableFIP>(x => x.LANEMI == _LANEMI);
}
// MPO_ABBV
if (!string.IsNullOrWhiteSpace((string)o["MPO_ABBV"]))
{
string _myStringValue = (string)o["MPO_ABBV"];
predicate = predicate.And<BestAvailableFIP>(x => x.MPO_ABBV.Contains(_myStringValue));
}
// owner
if (!string.IsNullOrWhiteSpace((string)o["owner"]))
{
string _myStringValue = (string)o["owner"];
predicate = predicate.And<BestAvailableFIP>(x => x.owner.Contains(_myStringValue));
}
// PASER
decimal _PASER;
if (o["PASER"] != null && decimal.TryParse((string)o["PASER"], out _PASER))
{
predicate = predicate.And<BestAvailableFIP>(x => x.PASER == _PASER);
}
// PASER_GROUP
if (!string.IsNullOrWhiteSpace((string)o["PASER_GROUP"]))
{
string _myStringValue = (string)o["PASER_GROUP"];
predicate = predicate.And<BestAvailableFIP>(x => x.PASER_GROUP.Contains(_myStringValue));
}
// pr
decimal _pr;
if (o["pr"] != null && decimal.TryParse((string)o["pr"], out _pr))
{
predicate = predicate.And<BestAvailableFIP>(x => x.pr == _pr);
}
// RDNAME
if (!string.IsNullOrWhiteSpace((string)o["RDNAME"]))
{
string _myStringValue = (string)o["RDNAME"];
predicate = predicate.And<BestAvailableFIP>(x => x.RDNAME.Contains(_myStringValue));
}
// SPDR_ABBV
if (!string.IsNullOrWhiteSpace((string)o["SPDR_ABBV"]))
{
string _myStringValue = (string)o["SPDR_ABBV"];
predicate = predicate.And<BestAvailableFIP>(x => x.SPDR_ABBV.Contains(_myStringValue));
}
// TODESC
if (!string.IsNullOrWhiteSpace((string)o["TODESC"]))
{
string _myStringValue = (string)o["TODESC"];
predicate = predicate.And<BestAvailableFIP>(x => x.TODESC.Contains(_myStringValue));
}
// TYPE
if (!string.IsNullOrWhiteSpace((string)o["TYPE"]))
{
string _myStringValue = (string)o["TYPE"];
predicate = predicate.And<BestAvailableFIP>(x => x.TYPE.Contains(_myStringValue));
}
return predicate;
}
internal override System.Linq.Expressions.Expression<Func<TEntity, bool>> CreateCriteriaExpressionForCustomProperties<TEntity>(string myCriteria)
{
var predicate = PredicateBuilder.True<TEntity>();
return predicate;
}
}
}
The JsonLinqParser base class:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Linq.Expressions;
namespace DAL
{
abstract class JsonLinqParser
{
abstract internal Expression<Func<TEntity, bool>> CreateCriteriaExpression<TEntity>(string myCriteria)
where TEntity : System.Data.Objects.DataClasses.EntityObject;
abstract internal Expression<Func<TEntity, bool>> CreateCriteriaExpressionForCustomProperties<TEntity>(string myCriteria)
where TEntity : System.Data.Objects.DataClasses.EntityObject;
}
}
The factory class:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace DAL
{
internal static class JsonLinqParserFactory
{
internal static JsonLinqParser GetParser(Type type)
{
switch (type.Name)
{
case "BestAvailableFIP":
return new JsonLinqParser_Paser();
default:
//if we reach this point then we failed to find a matching type. Throw
//an exception.
throw new Exception("Failed to find a matching JsonLinqParser in JsonLinqParserFactory.GetParser() - Unknown Type: " + type.Name);
}
}
}
}
The problem is that JsonLinqParser_Paser's signatures are generic, type-agnostic, and your implementation is its specialization for concrete BestAvailableFIP type. This is not a covariance issue, it's just type incompatibility (at compiler level).
The solution is to make JsonLinqParser to be generic type (not having generic methods) - or even an interface, then make JsonLinqParser_Paser to implement JsonLinqParser<BestAvailableFIP>. We'll then have everything matching.
The IJsonLinqParser interface:
interface IJsonLinqParser<TEntity>
where TEntity : System.Data.Objects.DataClasses.EntityObject
{
Expression<Func<TEntity, bool>> CreateCriteriaExpression(string myCriteria);
Expression<Func<TEntity, bool>> CreateCriteriaExpressionForCustomProperties(string myCriteria)
}
The implementation - signatures for JsonLinqParser_Paser:
internal class JsonLinqParser_Paser : IJsonLinqParser<BestAvailableFIP>
{
public Expression<Func<BestAvailableFIP, bool>> CreateCriteriaExpression(string myCriteria)
{
// implementation as yours
}
public Expression<Func<BestAvailableFIP, bool>> CreateCriteriaExpressionForCustomProperties(string myCriteria)
{
// implementation as yours
}
}
The factory needs to return IJsonLinqParser<TEntity>, what is not a problem as we know TEntity there:
internal static class JsonLinqParserFactory
{
internal static IJsonLinqParser<TEntity> GetParser<TEntity>()
where TEntity : System.Data.Objects.DataClasses.EntityObject
{
switch (typeof(TEntity).Name)
{
case "BestAvailableFIP":
return (IJsonLinqParser<TEntity>) new JsonLinqParser_Paser();
default:
//if we reach this point then we failed to find a matching type. Throw
//an exception.
throw new Exception("Failed to find a matching JsonLinqParser in JsonLinqParserFactory.GetParser() - Unknown Type: " + typeof(TEntity).Name);
}
}
}
And finally in GetByCriteria you can have:
IJsonLinqParser<TEntity> parser = JsonLinqParserFactory.GetParser<TEntity>();
No need for <TEntity> in parser method calls now, as parser is already TEntity-specific.
Hope this helps.
By the way, your factory infrastructure can be easily replaced by good IoC tool.
I know I can call linq's Except and specify a custom IEqualityComparer, but implementing a new Comparer class for each data type seems like an overkill for this purpose. Can I use a lambda expression to provide the equality function, like when I use Where or other LINQ functions?
If I can't, is there an alternative?
For any one still looking; here's another way of implementing a custom lambda comparer.
public class LambdaComparer<T> : IEqualityComparer<T>
{
private readonly Func<T, T, bool> _expression;
public LambdaComparer(Func<T, T, bool> lambda)
{
_expression = lambda;
}
public bool Equals(T x, T y)
{
return _expression(x, y);
}
public int GetHashCode(T obj)
{
/*
If you just return 0 for the hash the Equals comparer will kick in.
The underlying evaluation checks the hash and then short circuits the evaluation if it is false.
Otherwise, it checks the Equals. If you force the hash to be true (by assuming 0 for both objects),
you will always fall through to the Equals check which is what we are always going for.
*/
return 0;
}
}
you can then create an extension for the linq Except an Intersect that take in lambda's
/// <summary>
/// Returns all items in the first collection except the ones in the second collection that match the lambda condition
/// </summary>
/// <typeparam name="T">The type</typeparam>
/// <param name="listA">The first list</param>
/// <param name="listB">The second list</param>
/// <param name="lambda">The filter expression</param>
/// <returns>The filtered list</returns>
public static IEnumerable<T> Except<T>(this IEnumerable<T> listA, IEnumerable<T> listB, Func<T, T, bool> lambda)
{
return listA.Except(listB, new LambdaComparer<T>(lambda));
}
/// <summary>
/// Returns all items in the first collection that intersect the ones in the second collection that match the lambda condition
/// </summary>
/// <typeparam name="T">The type</typeparam>
/// <param name="listA">The first list</param>
/// <param name="listB">The second list</param>
/// <param name="lambda">The filter expression</param>
/// <returns>The filtered list</returns>
public static IEnumerable<T> Intersect<T>(this IEnumerable<T> listA, IEnumerable<T> listB, Func<T, T, bool> lambda)
{
return listA.Intersect(listB, new LambdaComparer<T>(lambda));
}
Usage:
var availableItems = allItems.Except(filterItems, (p, p1) => p.Id== p1.Id);
Can you not use a .Where with a lambda that filters out your required values?
Example as requested:
static void Main(string[] args)
{
var firstCustomers = new[] { new Customer { Id = 1, Name = "Bob" }, new Customer { Id = 2, Name = "Steve" } };
var secondCustomers = new[] { new Customer { Id = 2, Name = "Steve" }, new Customer { Id = 3, Name = "John" } };
var customers = secondCustomers.Where(c => !firstCustomers.Select(fc => fc.Id).Contains(c.Id));
}
public class Customer
{
public int Id { get; set; }
public string Name { get; set; }
}
I don't think you can directly with the basic LINQ interfaces, but I've seen people implement a LambdaComparer class with extension methods which will help you do it.
Here's an example
Here's something simple I whipped up:
public class CustomComparer<TSource, TCompareType> : IEqualityComparer<TSource> where TSource : class
{
private readonly Func<TSource, TCompareType> getComparisonObject;
public CustomComparer(Func<TSource,TCompareType> getComparisonObject)
{
if (getComparisonObject == null) throw new ArgumentNullException("getComparisonObject");
this.getComparisonObject = getComparisonObject;
}
/// <summary>
/// Determines whether the specified objects are equal.
/// </summary>
/// <returns>
/// true if the specified objects are equal; otherwise, false.
/// </returns>
/// <param name="x">The first object of type <paramref name="T"/> to compare.
/// </param><param name="y">The second object of type <paramref name="T"/> to compare.
/// </param>
public bool Equals(TSource x, TSource y)
{
if (x == null)
{
return (y == null);
}
else if (y == null)
{
return false;
}
return EqualityComparer<TCompareType>.Default.Equals(getComparisonObject(x), getComparisonObject(y));
}
/// <summary>
/// Returns a hash code for the specified object.
/// </summary>
/// <returns>
/// A hash code for the specified object.
/// </returns>
/// <param name="obj">The <see cref="T:System.Object"/> for which a hash code is to be returned.
/// </param><exception cref="T:System.ArgumentNullException">The type of <paramref name="obj"/> is a reference type and <paramref name="obj"/> is null.
/// </exception>
public int GetHashCode(TSource obj)
{
return EqualityComparer<TCompareType>.Default.GetHashCode(getComparisonObject(obj));
}
}
Usage:
var myItems = allItems.Except(theirItems, new CustomComparer(item => item.Name));
Use an extension!
public static IEnumerable<T> Except<T, TKey>(this IEnumerable<T> items, IEnumerable<T> other,
Func<T, TKey> getKey)
{
return from item in items
join otherItem in other on getKey(item)
equals getKey(otherItem) into tempItems
from temp in tempItems.DefaultIfEmpty()
where ReferenceEquals(null, temp) || temp.Equals(default(T))
select item;
}
Source
Here' is l except r solution, based on a LINQ outer join technique:
from l in new[] { 1, 2, 3 }
join r in new[] { 2, 4, 5 }
on l equals r
into rr
where !rr.Any()
select l
Will yield: 1, 3
But you may try this. This is so much easy, I think code contains errors. Of course, the code is for small amounts, without LINQ translates to db etc.
public static IEnumerable<TSource> ExceptPredicate<TSource>(this IEnumerable<TSource> first, IEnumerable<TSource> second, Func<TSource, TSource, bool> compare) {
foreach (var itmFirst in first) {
if (!second.Any(itmsecond => compare(itmFirst, itmsecond))) {
yield return itmFirst;
}
}
yield break;
}
For example given a Factory with a method
public static T Save<T>(T item) where T : Base, new()
{
/* item.Id == Guid.Empty therefore item is new */
if (item.Id == Guid.Empty && repository.GetAll<T>(t => t.Name == item.Name))
{
throw new Exception("Name is not unique");
}
}
how do I create a property of Base (say MustNotAlreadyExist) so that I can change the method above to
public static T Save<T>(T item) where T : Base, new()
{
/* item.Id == Guid.Empty therefore item is new */
if (item.Id == Guid.Empty && repository.GetAll<T>(t.MustNotAlreadyExist))
{
throw new Exception("Name is not unique");
}
}
public class Base
{
...
public virtual Expression<Func<T, bool>> MustNotAlreadyExist()
{
return (b => b.Name == name); /* <- this clearly doesn't work */
}
}
and then how can I override MustNotAlreadyExist in Account : Base
public class Account : Base
{
...
public override Expression<Func<T, bool>> MustNotAlreadyExist()
{
return (b => b.Name == name && b.AccountCode == accountCode); /* <- this doesn't work */
}
...
}
Try this:
public class Account : Base
{
...
public override Expression<Func<T, bool>> MustNotAlreadyExist()
{
return (b => b.Name == name && b.AccountCode == accountCode).Any();
}
...
}
The Any() method will return true if any record matches the predicate. It could be argued that it is outside the responsibility of the repository to check for presence of a record before saving.
UPDATE:
There is a great article on CodeProject that describes a generic Repository for Entity Framework:
http://www.codeproject.com/KB/database/ImplRepositoryPatternEF.aspx
This could be applied to a non-Entity Framework data context. Here is an excerpt that provides a very flexible method for checking for an existing value by accepting the name of a field, a value, and a key value. You can apply this to any Entity type and use it to check for the presence of an entity before attempting a save.
/// <summary>
/// Check if value of specific field is already exist
/// </summary>
/// <typeparam name="E"></typeparam>
/// <param name="fieldName">name of the Field</param>
/// <param name="fieldValue">Field value</param>
/// <param name="key">Primary key value</param>
/// <returns>True or False</returns>
public bool TrySameValueExist(string fieldName, object fieldValue, string key)
{
// First we define the parameter that we are going to use the clause.
var xParam = Expression.Parameter(typeof(E), typeof(E).Name);
MemberExpression leftExprFieldCheck =
MemberExpression.Property(xParam, fieldName);
Expression rightExprFieldCheck = Expression.Constant(fieldValue);
BinaryExpression binaryExprFieldCheck =
MemberExpression.Equal(leftExprFieldCheck, rightExprFieldCheck);
MemberExpression leftExprKeyCheck =
MemberExpression.Property(xParam, this._KeyProperty);
Expression rightExprKeyCheck = Expression.Constant(key);
BinaryExpression binaryExprKeyCheck =
MemberExpression.NotEqual(leftExprKeyCheck, rightExprKeyCheck);
BinaryExpression finalBinaryExpr =
Expression.And(binaryExprFieldCheck, binaryExprKeyCheck);
//Create Lambda Expression for the selection
Expression<Func<E, bool>> lambdaExpr =
Expression.Lambda<Func<E, bool>>(finalBinaryExpr,
new ParameterExpression[] { xParam });
//Searching ....
return ((IRepository<E, C>)this).TryEntity(new Specification<E>(lambdaExpr));
}
/// <summary>
/// Check if Entities exist with Condition
/// </summary>
/// <param name="selectExpression">Selection Condition</param>
/// <returns>True or False</returns>
public bool TryEntity(ISpecification<E> selectSpec)
{
return _ctx.CreateQuery<E>("[" + typeof(E).Name + "]").Any<E>
(selectSpec.EvalPredicate);
}
I am not shure if you problem is solvable since you need to access both the repository and the new item to be checked. The new item to be checked is not available in a seperate method.
However you can outsource the call to GetAll so that your code becomes something similar to
(not tested)
public static T Save<T>(T item) where T : Base, new()
{
if (item.Id == Guid.Empty && (Check(repository, item)))
{
throw new Exception("Name is not unique");
}
}
public class Base
{
...
public Func<Enumerable<T>, T, bool> Check { get; set;}
public Base()
{
Check = (col, newItem) => (null != col.FirstOrDefault<T>(
item => item.Name == newItem.Name));
}
}
OK, here is the answer, this is a combination of the code posted by Dave Swersky and a little but of common sense.
public interface IUniqueable<T>
{
Expression<Func<T, bool>> Unique { get; }
}
public class Base, IUniqueable<Base>
{
...
public Expression<Func<Base, bool>> Unique
{
get
{
var xParam = Expression.Parameter(typeof(Base), typeof(Base).Name);
MemberExpression leftExprFieldCheck = MemberExpression.Property(xParam, "Name");
Expression rightExprFieldCheck = Expression.Constant(this.Name);
BinaryExpression binaryExprFieldCheck = MemberExpression.Equal(leftExprFieldCheck, rightExprFieldCheck);
return Expression.Lambda<Func<Base, bool>>(binaryExprFieldCheck, new ParameterExpression[] { xParam });
}
}
...
}
public class Account : Base, IUniqueable<Account>
{
...
public new Expression<Func<Account, bool>> Unique
{
get
{
var xParam = Expression.Parameter(typeof(Account), typeof(Account).Name);
MemberExpression leftExprNameCheck = MemberExpression.Property(xParam, "Name");
Expression rightExprNameCheck = Expression.Constant(this.Name);
BinaryExpression binaryExprNameCheck = MemberExpression.Equal(leftExprNameCheck, rightExprNameCheck);
MemberExpression leftExprFieldCheck = MemberExpression.Property(xParam, "AccountCode");
Expression rightExprFieldCheck = Expression.Constant(this.AccountCode);
BinaryExpression binaryExprFieldCheck = MemberExpression.Equal(leftExprFieldCheck, rightExprFieldCheck);
BinaryExpression binaryExprAllCheck = Expression.OrElse(binaryExprNameCheck, binaryExprFieldCheck);
return Expression.Lambda<Func<Account, bool>>(binaryExprAllCheck, new ParameterExpression[] { xParam });
}
}
...
}
public static class Manager
{
...
public static T Save<T>(T item) where T : Base, new()
{
if (!item.IsValid)
{
throw new ValidationException("Unable to save item, item is not valid", item.GetRuleViolations());
}
if (item.Id == Guid.Empty && repository.GetAll<T>().Any(((IUniqueable<T>)item).Unique))
{
throw new Exception("Item is not unique");
}
return repository.Save<T>(item);
}
...
}
Essentially by implementing the IUniqueable interface for a specific type I can return a different Expression for each type. All good :-)
I have a list of integer values (List) and would like to generate a string of comma delimited values. That is all items in the list output to a single comma delimted list.
My thoughts...
1. pass the list to a method.
2. Use stringbuilder to iterate the list and append commas
3. Test the last character and if it's a comma, delete it.
What are your thoughts? Is this the best way?
How would my code change if I wanted to handle not only integers (my current plan) but strings, longs, doubles, bools, etc, etc. in the future? I guess make it accept a list of any type.
It's amazing what the Framework already does for us.
List<int> myValues;
string csv = String.Join(",", myValues.Select(x => x.ToString()).ToArray());
For the general case:
IEnumerable<T> myList;
string csv = String.Join(",", myList.Select(x => x.ToString()).ToArray());
As you can see, it's effectively no different. Beware that you might need to actually wrap x.ToString() in quotes (i.e., "\"" + x.ToString() + "\"") in case x.ToString() contains commas.
For an interesting read on a slight variant of this: see Comma Quibbling on Eric Lippert's blog.
Note: This was written before .NET 4.0 was officially released. Now we can just say
IEnumerable<T> sequence;
string csv = String.Join(",", sequence);
using the overload String.Join<T>(string, IEnumerable<T>). This method will automatically project each element x to x.ToString().
in 3.5, i was still able to do this. Its much more simpler and doesnt need lambda.
String.Join(",", myList.ToArray<string>());
I explain it in-depth in this post. I'll just paste the code here with brief descriptions.
Here's the method that creates the header row. It uses the property names as column names.
private static void CreateHeader<T>(List<T> list, StreamWriter sw)
{
PropertyInfo[] properties = typeof(T).GetProperties();
for (int i = 0; i < properties.Length - 1; i++)
{
sw.Write(properties[i].Name + ",");
}
var lastProp = properties[properties.Length - 1].Name;
sw.Write(lastProp + sw.NewLine);
}
This method creates all the value rows
private static void CreateRows<T>(List<T> list, StreamWriter sw)
{
foreach (var item in list)
{
PropertyInfo[] properties = typeof(T).GetProperties();
for (int i = 0; i < properties.Length - 1; i++)
{
var prop = properties[i];
sw.Write(prop.GetValue(item) + ",");
}
var lastProp = properties[properties.Length - 1];
sw.Write(lastProp.GetValue(item) + sw.NewLine);
}
}
And here's the method that brings them together and creates the actual file.
public static void CreateCSV<T>(List<T> list, string filePath)
{
using (StreamWriter sw = new StreamWriter(filePath))
{
CreateHeader(list, sw);
CreateRows(list, sw);
}
}
You can create an extension method that you can call on any IEnumerable:
public static string JoinStrings<T>(
this IEnumerable<T> values, string separator)
{
var stringValues = values.Select(item =>
(item == null ? string.Empty : item.ToString()));
return string.Join(separator, stringValues.ToArray());
}
Then you can just call the method on the original list:
string commaSeparated = myList.JoinStrings(", ");
If any body wants to convert list of custom class objects instead of list of string then override the ToString method of your class with csv row representation of your class.
Public Class MyClass{
public int Id{get;set;}
public String PropertyA{get;set;}
public override string ToString()
{
return this.Id+ "," + this.PropertyA;
}
}
Then following code can be used to convert this class list to CSV with header column
string csvHeaderRow = String.Join(",", typeof(MyClass).GetProperties(BindingFlags.Public | BindingFlags.Instance).Select(x => x.Name).ToArray<string>()) + Environment.NewLine;
string csv= csvHeaderRow + String.Join(Environment.NewLine, MyClass.Select(x => x.ToString()).ToArray());
You can use String.Join.
String.Join(
",",
Array.ConvertAll(
list.ToArray(),
element => element.ToString()
)
);
As the code in the link given by #Frank Create a CSV File from a .NET Generic List there was a little issue of ending every line with a , I modified the code to get rid of it.Hope it helps someone.
/// <summary>
/// Creates the CSV from a generic list.
/// </summary>;
/// <typeparam name="T"></typeparam>;
/// <param name="list">The list.</param>;
/// <param name="csvNameWithExt">Name of CSV (w/ path) w/ file ext.</param>;
public static void CreateCSVFromGenericList<T>(List<T> list, string csvCompletePath)
{
if (list == null || list.Count == 0) return;
//get type from 0th member
Type t = list[0].GetType();
string newLine = Environment.NewLine;
if (!Directory.Exists(Path.GetDirectoryName(csvCompletePath))) Directory.CreateDirectory(Path.GetDirectoryName(csvCompletePath));
if (!File.Exists(csvCompletePath)) File.Create(csvCompletePath);
using (var sw = new StreamWriter(csvCompletePath))
{
//make a new instance of the class name we figured out to get its props
object o = Activator.CreateInstance(t);
//gets all properties
PropertyInfo[] props = o.GetType().GetProperties();
//foreach of the properties in class above, write out properties
//this is the header row
sw.Write(string.Join(",", props.Select(d => d.Name).ToArray()) + newLine);
//this acts as datarow
foreach (T item in list)
{
//this acts as datacolumn
var row = string.Join(",", props.Select(d => item.GetType()
.GetProperty(d.Name)
.GetValue(item, null)
.ToString())
.ToArray());
sw.Write(row + newLine);
}
}
}
I like a nice simple extension method
public static string ToCsv(this List<string> itemList)
{
return string.Join(",", itemList);
}
Then you can just call the method on the original list:
string CsvString = myList.ToCsv();
Cleaner and easier to read than some of the other suggestions.
For whatever reason, #AliUmair reverted the edit to his answer that fixes his code that doesn't run as is, so here is the working version that doesn't have the file access error and properly handles null object property values:
/// <summary>
/// Creates the CSV from a generic list.
/// </summary>;
/// <typeparam name="T"></typeparam>;
/// <param name="list">The list.</param>;
/// <param name="csvNameWithExt">Name of CSV (w/ path) w/ file ext.</param>;
public static void CreateCSVFromGenericList<T>(List<T> list, string csvCompletePath)
{
if (list == null || list.Count == 0) return;
//get type from 0th member
Type t = list[0].GetType();
string newLine = Environment.NewLine;
if (!Directory.Exists(Path.GetDirectoryName(csvCompletePath))) Directory.CreateDirectory(Path.GetDirectoryName(csvCompletePath));
using (var sw = new StreamWriter(csvCompletePath))
{
//make a new instance of the class name we figured out to get its props
object o = Activator.CreateInstance(t);
//gets all properties
PropertyInfo[] props = o.GetType().GetProperties();
//foreach of the properties in class above, write out properties
//this is the header row
sw.Write(string.Join(",", props.Select(d => d.Name).ToArray()) + newLine);
//this acts as datarow
foreach (T item in list)
{
//this acts as datacolumn
var row = string.Join(",", props.Select(d => $"\"{item.GetType().GetProperty(d.Name).GetValue(item, null)?.ToString()}\"")
.ToArray());
sw.Write(row + newLine);
}
}
}
Any solution work only if List a list(of string)
If you have a generic list of your own Objects like list(of car) where car has n properties, you must loop the PropertiesInfo of each car object.
Look at: http://www.csharptocsharp.com/generate-csv-from-generic-list
CsvHelper library is very popular in the Nuget.You worth it,man!
https://github.com/JoshClose/CsvHelper/wiki/Basics
Using CsvHelper is really easy. It's default settings are setup for the most common scenarios.
Here is a little setup data.
Actors.csv:
Id,FirstName,LastName
1,Arnold,Schwarzenegger
2,Matt,Damon
3,Christian,Bale
Actor.cs (custom class object that represents an actor):
public class Actor
{
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
Reading the CSV file using CsvReader:
var csv = new CsvReader( new StreamReader( "Actors.csv" ) );
var actorsList = csv.GetRecords();
Writing to a CSV file.
using (var csv = new CsvWriter( new StreamWriter( "Actors.csv" ) ))
{
csv.WriteRecords( actorsList );
}
The problem with String.Join is that you are not handling the case of a comma already existing in the value. When a comma exists then you surround the value in Quotes and replace all existing Quotes with double Quotes.
String.Join(",",{"this value has a , in it","This one doesn't", "This one , does"});
See CSV Module
http://cc.davelozinski.com/c-sharp/the-fastest-way-to-read-and-process-text-files
This website did some extensive testing about how to write to a file using buffered writer, reading line by line seems to be the best way, using string builder was one of the slowest.
I use his techniques a great deal for writing stuff to file it works well.
A general purpose ToCsv() extension method:
Supports Int16/32/64, float, double, decimal, and anything supporting
ToString()
Optional custom join separator
Optional custom selector
Optional null/empty handling specification (*Opt() overloads)
Usage Examples:
"123".ToCsv() // "1,2,3"
"123".ToCsv(", ") // "1, 2, 3"
new List<int> { 1, 2, 3 }.ToCsv() // "1,2,3"
new List<Tuple<int, string>>
{
Tuple.Create(1, "One"),
Tuple.Create(2, "Two")
}
.ToCsv(t => t.Item2); // "One,Two"
((string)null).ToCsv() // throws exception
((string)null).ToCsvOpt() // ""
((string)null).ToCsvOpt(ReturnNullCsv.WhenNull) // null
Implementation
/// <summary>
/// Specifies when ToCsv() should return null. Refer to ToCsv() for IEnumerable[T]
/// </summary>
public enum ReturnNullCsv
{
/// <summary>
/// Return String.Empty when the input list is null or empty.
/// </summary>
Never,
/// <summary>
/// Return null only if input list is null. Return String.Empty if list is empty.
/// </summary>
WhenNull,
/// <summary>
/// Return null when the input list is null or empty
/// </summary>
WhenNullOrEmpty,
/// <summary>
/// Throw if the argument is null
/// </summary>
ThrowIfNull
}
/// <summary>
/// Converts IEnumerable list of values to a comma separated string values.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="values">The values.</param>
/// <param name="joinSeparator"></param>
/// <returns>System.String.</returns>
public static string ToCsv<T>(
this IEnumerable<T> values,
string joinSeparator = ",")
{
return ToCsvOpt<T>(values, null /*selector*/, ReturnNullCsv.ThrowIfNull, joinSeparator);
}
/// <summary>
/// Converts IEnumerable list of values to a comma separated string values.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="values">The values.</param>
/// <param name="selector">An optional selector</param>
/// <param name="joinSeparator"></param>
/// <returns>System.String.</returns>
public static string ToCsv<T>(
this IEnumerable<T> values,
Func<T, string> selector,
string joinSeparator = ",")
{
return ToCsvOpt<T>(values, selector, ReturnNullCsv.ThrowIfNull, joinSeparator);
}
/// <summary>
/// Converts IEnumerable list of values to a comma separated string values.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="values">The values.</param>
/// <param name="returnNullCsv">Return mode (refer to enum ReturnNullCsv).</param>
/// <param name="joinSeparator"></param>
/// <returns>System.String.</returns>
public static string ToCsvOpt<T>(
this IEnumerable<T> values,
ReturnNullCsv returnNullCsv = ReturnNullCsv.Never,
string joinSeparator = ",")
{
return ToCsvOpt<T>(values, null /*selector*/, returnNullCsv, joinSeparator);
}
/// <summary>
/// Converts IEnumerable list of values to a comma separated string values.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="values">The values.</param>
/// <param name="selector">An optional selector</param>
/// <param name="returnNullCsv">Return mode (refer to enum ReturnNullCsv).</param>
/// <param name="joinSeparator"></param>
/// <returns>System.String.</returns>
public static string ToCsvOpt<T>(
this IEnumerable<T> values,
Func<T, string> selector,
ReturnNullCsv returnNullCsv = ReturnNullCsv.Never,
string joinSeparator = ",")
{
switch (returnNullCsv)
{
case ReturnNullCsv.Never:
if (!values.AnyOpt())
return string.Empty;
break;
case ReturnNullCsv.WhenNull:
if (values == null)
return null;
break;
case ReturnNullCsv.WhenNullOrEmpty:
if (!values.AnyOpt())
return null;
break;
case ReturnNullCsv.ThrowIfNull:
if (values == null)
throw new ArgumentOutOfRangeException("ToCsvOpt was passed a null value with ReturnNullCsv = ThrowIfNull.");
break;
default:
throw new ArgumentOutOfRangeException("returnNullCsv", returnNullCsv, "Out of range.");
}
if (selector == null)
{
if (typeof(T) == typeof(Int16) ||
typeof(T) == typeof(Int32) ||
typeof(T) == typeof(Int64))
{
selector = (v) => Convert.ToInt64(v).ToStringInvariant();
}
else if (typeof(T) == typeof(decimal))
{
selector = (v) => Convert.ToDecimal(v).ToStringInvariant();
}
else if (typeof(T) == typeof(float) ||
typeof(T) == typeof(double))
{
selector = (v) => Convert.ToDouble(v).ToString(CultureInfo.InvariantCulture);
}
else
{
selector = (v) => v.ToString();
}
}
return String.Join(joinSeparator, values.Select(v => selector(v)));
}
public static string ToStringInvariantOpt(this Decimal? d)
{
return d.HasValue ? d.Value.ToStringInvariant() : null;
}
public static string ToStringInvariant(this Decimal d)
{
return d.ToString(CultureInfo.InvariantCulture);
}
public static string ToStringInvariantOpt(this Int64? l)
{
return l.HasValue ? l.Value.ToStringInvariant() : null;
}
public static string ToStringInvariant(this Int64 l)
{
return l.ToString(CultureInfo.InvariantCulture);
}
public static string ToStringInvariantOpt(this Int32? i)
{
return i.HasValue ? i.Value.ToStringInvariant() : null;
}
public static string ToStringInvariant(this Int32 i)
{
return i.ToString(CultureInfo.InvariantCulture);
}
public static string ToStringInvariantOpt(this Int16? i)
{
return i.HasValue ? i.Value.ToStringInvariant() : null;
}
public static string ToStringInvariant(this Int16 i)
{
return i.ToString(CultureInfo.InvariantCulture);
}
Here is my extension method, it returns a string for simplicity but my implementation writes the file to a data lake.
It provides for any delimiter, adds quotes to string (in case they contain the delimiter) and deals will nulls and blanks.
/// <summary>
/// A class to hold extension methods for C# Lists
/// </summary>
public static class ListExtensions
{
/// <summary>
/// Convert a list of Type T to a CSV
/// </summary>
/// <typeparam name="T">The type of the object held in the list</typeparam>
/// <param name="items">The list of items to process</param>
/// <param name="delimiter">Specify the delimiter, default is ,</param>
/// <returns></returns>
public static string ToCsv<T>(this List<T> items, string delimiter = ",")
{
Type itemType = typeof(T);
var props = itemType.GetProperties(BindingFlags.Public | BindingFlags.Instance).OrderBy(p => p.Name);
var csv = new StringBuilder();
// Write Headers
csv.AppendLine(string.Join(delimiter, props.Select(p => p.Name)));
// Write Rows
foreach (var item in items)
{
// Write Fields
csv.AppendLine(string.Join(delimiter, props.Select(p => GetCsvFieldasedOnValue(p, item))));
}
return csv.ToString();
}
/// <summary>
/// Provide generic and specific handling of fields
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="p"></param>
/// <param name="item"></param>
/// <returns></returns>
private static object GetCsvFieldasedOnValue<T>(PropertyInfo p, T item)
{
string value = "";
try
{
value = p.GetValue(item, null)?.ToString();
if (value == null) return "NULL"; // Deal with nulls
if (value.Trim().Length == 0) return ""; // Deal with spaces and blanks
// Guard strings with "s, they may contain the delimiter!
if (p.PropertyType == typeof(string))
{
value = string.Format("\"{0}\"", value);
}
}
catch (Exception ex)
{
throw ex;
}
return value;
}
}
Usage:
// Tab Delimited (TSV)
var csv = MyList.ToCsv<MyClass>("\t");
The other answers work, but my issue is loading unknown data from the database, so I needed something a bit more robust than what's already here.
I wanted something that fit the following requirements:
able to be opened in excel
had to be able to handle date time formats in an excel compatible way
had to automatically exclude linked entities (EF navigation properties)
had to support column contents containing " and the delimiter ,
had to support nullable columns
had to support a wide array of data types
numbers of every kind
guids
datetimes
custom type definitions (ie name from a linked entity)
I used month/day/year formats for the date exports for compatibility reasons
public static IReadOnlyDictionary<System.Type, Func<object, string>> CsvTypeFormats = new Dictionary<System.Type, Func<object, string>> {
// handles escaping column delimiter (',') and quote marks
{ typeof(string), x => string.IsNullOrWhiteSpace(x as string) ? null as string : $"\"{(x as string).Replace("\"", "\"\"")}\""},
{ typeof(DateTime), x => $"{x:M/d/yyyy H:m:s.fff}" },
{ typeof(DateTime?), x => x == null ? "" : $"{x:M/d/yyyy H:m:s.fff}" },
{ typeof(DateTimeOffset), x => $"{x:M/d/yyyy H:m:s.fff}" },
{ typeof(DateTimeOffset?), x => x == null ? "" : $"{x:M/d/yyyy H:m:s.fff}" },
};
public void WriteCsvContent<T>(ICollection<T> data, StringBuilder writer, IDictionary<System.Type, Func<object, string>> explicitMapping = null)
{
var typeMappings = CsvTypeFormats.ToDictionary(x=>x.Key, x=>x.Value);
if (explicitMapping != null) {
foreach(var mapping in explicitMapping) {
typeMappings[mapping.Key] = mapping.Value;
}
}
var props = typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance)
.Where(x => IsSimpleType(x.PropertyType))
.ToList();
// header row
writer.AppendJoin(',', props.Select(x => x.Name));
writer.AppendLine();
foreach (var item in data)
{
writer.AppendJoin(',',
props.Select(prop => typeMappings.ContainsKey(prop.PropertyType)
? typeMappings[prop.PropertyType](prop.GetValue(item))
: prop.GetValue(item)?.ToString() ?? ""
)
// escaping and special characters
.Select(x => x != null && x != "" ? $"\"{x.Replace("\"", "\"\"")}\"" : null)
);
writer.AppendLine();
}
}
private bool IsSimpleType(System.Type t)
{
return
t.IsPrimitive ||
t.IsValueType ||
t.IsEnum ||
(t == typeof(string)) ||
CsvTypeFormats.ContainsKey(t);
}
If your class uses fields instead of properties, change the GetProperties to GetFields and the PropertyType accessors to FieldType