I have a Generic function that contains an IQueryable<T> where each row contains an instance of a class with a set of properties.
I have another class (MyClass) that has some of the same properties as the class T above... i.e. same name and datatypes.
I also have a List of Strings containing the Property Names shared between the two classes.
I want to be able to create a new IQueryable<myClass> where the myClass instances properties are populated with the name-sake proprties in the original IQueryable<T>
Does that make sense? Please let me know if I can supply any more info or make anything more clear.
EDIT
I will try an add some code to illustrate better. I know there are countless faults here including Adding to an IQueryable - but this is for illustration:
IQueryable<T> qry = this.GetSomeDataIntoIQueryable();
// Just getting a list of the Shared Property Names between the two classes
List<string> sharedProprtyNames = new List<string>();
foreach (var item in ListofSharedPropertyNames)
{
sharedProprtyNames .Add(item.SharedPropertyName);
}
IQueryable<myClass> myClassIQ;
foreach(var classItem in qry)
{
myClass x = new myClass();
foreach(var sharedProperty in sharedProprtyNames )
{
myClass[sharedProperty] = classItem[sharedProperty];
}
myClassIQ.Add(myClass);
}
Something like:
static IQueryable<TTo> Select<TFrom, TTo>(
this IQueryable<TFrom> source,
params string[] members)
{
var p = Expression.Parameter(typeof(TFrom));
var body = Expression.MemberInit(
Expression.New(typeof(TTo)),
members.Select(member => Expression.Bind(
typeof(TTo).GetMember(member).Single(),
Expression.PropertyOrField(p, member))));
var lambda = Expression.Lambda<Func<TFrom, TTo>>(body, p);
return source.Select(lambda);
}
?
This creates an IQueryable<T>-friendly projection from TFrom to TTo, respecting all the members from members.
In your example, it would be:
IQueryable<myClass> myClassIQ = qry.Select<T, myClass>(ListofSharedPropertyNames);
(adjust between array / list for the members parameter to suit your convenience - because we use Select, it'll work with either)
Using System.Linq.Dynamic.Core you can:
IQueryable<Table1> query1 = ...
var res = query.Select<SubTable1>("new(" + string.Join(",", new[] { "Col1", "Col2" }) + ")").ToArray();
Where query1 is your query, Table1 is the T of your question, SubTable1 is MyClass, "Col1", "Col2" are the columsn that must be selected.
You could do everything without usig the System.Linq.Dynamic.Core library and simply building an expression tree... But it is a pain :-)
My variant of the code of Hans Passant:
public static IQueryable<TResult> Select<TSource, TResult>(this IQueryable<TSource> source, IEnumerable<string> columns)
{
// the x in x => ...
var par = Expression.Parameter(typeof(TSource), "x");
// "Bindings" (the Col1 = x.Col1 inside the x => new { Col1 = x.Col1 })
var binds = columns.Select(x => Expression.Bind((MemberInfo)typeof(TResult).GetProperty(x) ?? typeof(TResult).GetField(x), Expression.PropertyOrField(par, x)));
// new TResult
var new1 = Expression.New(typeof(TResult));
// new TResult { Bindings }
var mi = Expression.MemberInit(new1, binds);
// x => new TResult { Bindings }
var lambda = Expression.Lambda<Func<TSource, TResult>>(mi, par);
// Select(x => new TResult { Bindings })
return source.Select(lambda);
}
(nearly totally equivalent... Only difference is that he uses GetMember() while I used GetProperty() + GetField())
Related
I have been looking at many posts here and on the web but none of them seem to be helping.
I have a table with about 2 million records, it has over 200 columns.
A simple web service allow the user to pull a specific number of columns out of the table, the user has the option to choose which column to pull.
The result needs to be string of comma separated values, so my query needs to pull the requested columns and return a concatenate string.
I have done this using ADO.NET and pure SQL queries it works fine however I was asked to do it in Entity Framework.
Here is what I have and have done already.
I get the requested columns list as an array of strings.
The following is my query, not sure if it's the best solution or idea hence I'm asking for help here.
var valueList2 = ctx.mytable.Where(x => x.pcds == comValue).Select(x => new{temp = x.column1 +", "+ x.column2}).Select(x => x.temp).ToList();
The above gives me string of two columns separated by commas, I just need to somehow push my array of column names into the lambda part of it.
I did the following but then realised that it only works with a specific type of a class not anonymous, also I can't figure out how I can use it for a multiple columns and not make it so complex.
var createdType = typeof(mytable);
var Param = Expression.Parameter(typeof(string), "pr");
var obj = Expression.New(createdType);
var ValueProperty = createdType.GetProperty("long");
var ValueAssignment = Expression.Bind(ValueProperty, Param);
var memberInit = Expression.MemberInit(obj, ValueAssignment);
var lm = Expression.Lambda<Func<string, mytable>>(memberInit, Param);
Thank you
I'm using Dynamic Linq (source code). Sadly there is little documentation about how to use it :-) In a fun boomerang effect, there is an "evolved" version. The boomerang effect is because the code for generating the dynamic class is based on one of my responses :-) The remaining code seems to be very beautiful... And there is a full suit of unit tests with code samples!!! Note that this second library is a superset of the first library, so you can probably apply many examples to the first one! :-)
I'm adding some static methods to translate the result of a a Dynamic Linq query to a IEnumerable<object[]>.... Example code:
using (var ctx = new Model1())
{
var result = ctx.MyTable
.Take(100)
.SimpleSelect(new[] { "ID", "Col1", "Col2" })
.ToObjectArray();
foreach (var row in result)
{
Console.WriteLine(string.Join(", ", row));
}
}
More complex example:
var columnsNames = new[] { "SomeNullableInt32", "SomeNonNullableDateTimeColumn" };
// One for each column!
var formatters = new Func<object, string>[]
{
x => x != null ? x.ToString() : null,
x => ((DateTime)x).ToShortDateString()
};
var result = ctx.MyTable.Take(100).SimpleSelect(columnsNames).ToObjectArray();
foreach (var row in result)
{
var stringRow = new string[row.Length];
for (int i = 0; i < row.Length; i++)
{
stringRow[i] = formatters[i](row[i]);
}
Console.WriteLine(string.Join(", ", stringRow));
}
And the classes... One (SimpleSelect) produces the Dynamic SQL Select, and "anonymizes" the field names. I do this because for each type of return the Dynamic Linq will generate at runtime a class. This class won't be unloaded until the program ends. By anonymizing the columns (I rename them to Item1, Item2, Item3...) I increase the possibility that the same class will be reused. Note that different type of columns will generate different classes! (int Item1, string Item2 will be a different class from int Item1, DateTime Item2), the other (ToObjectArray) returns a IEnumerable<object[]>, something easier to parse.
public static class DynamicLinqTools
{
private static ConcurrentDictionary<Type, Func<object, object[]>> Converters = new ConcurrentDictionary<Type, Func<object, object[]>>();
public static IQueryable SimpleSelect(this IQueryable query, string[] fields)
{
// With a little luck, "anonymizing" the field names we should
// reduce the number of types created!
// new (field1 as Item1, field2 as Item2)
return query.Select(string.Format("new ({0})", string.Join(", ", fields.Select((x, ix) => string.Format("{0} as Item{1}", x, ix + 1)))));
}
public static IEnumerable<object[]> ToObjectArray(this IQueryable query)
{
Func<object, object[]> converter;
Converters.TryGetValue(query.ElementType, out converter);
if (converter == null)
{
var row = Expression.Parameter(typeof(object), "row");
// ElementType row2;
var row2 = Expression.Variable(query.ElementType, "row2");
// (ElementType)row;
var cast = Expression.Convert(row, query.ElementType);
// row2 = (ElementType)row;
var assign = Expression.Assign(row2, cast);
var properties = query.ElementType.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.Where(x => x.CanRead && x.GetIndexParameters().Length == 0)
.ToArray();
// (object)row2.Item1, (object)row2.Item2, ...
var properties2 = Array.ConvertAll(properties, x => Expression.Convert(Expression.Property(row2, x), typeof(object)));
// new object[] { row2.Item1, row2.Item2 }
var array = Expression.NewArrayInit(typeof(object), properties2);
// row2 = (ElementType)row; (return) new object[] { row2.Item1, row2.Item2 }
var body = Expression.Block(typeof(object[]), new[] { row2 }, assign, array);
var exp = Expression.Lambda<Func<object, object[]>>(body, row);
converter = exp.Compile();
Converters.TryAdd(query.ElementType, converter);
}
foreach (var row in query)
{
yield return converter(row);
}
}
}
This is a short and easy answer for whoever needs a different answer, but per our discussion with #xanatos, it's not the best as it also returns all the columns which need to be cut off before adding to a list of strings.
List<string> valueList = new List<string>();
using (var ctx = new DataEntities1())
{
var query = ctx.myTable.Where(x => x.pcds == scode).SingleOrDefault();
foreach (var item in columnsArray)
{
valueList.Add(typeof(myTable).GetProperty(onsColumns[Convert.ToInt32(item)]).GetValue(query).ToString());
}
}
Im pretty new to C#/Unity so forgive me.
Im writing a filtering system which will filter a list by any of the properties of the class at runtime.
Im planning on that building up some kind of whereclause to filter the lists (i know i could hit the server to get the list i need, but currently want to just filter the data i already have)
Say i have a list of class "MyClass" with 4 properties: "param1".."param4"
If i wanted to filter it normally by param1 and param2 i could do:
List<MyClass> myList = new List<MyClass>(existinglist);
myList = myList.Where(g => g.param1 == somevalue && g.param2 == someothervalue).ToList();
How could i generate the same where clause at runtime?
Thank you!
You can use a helper method which dynamically builds and compiles lambda based on the passed list of filters. I've used KeyValuePair<string, object> to represent the filter information (Key for the property name, Value - well, for the property value), but of course you can adjust it for another data structure (like custom class etc.)
public static class EnumerableExtensions
{
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, IEnumerable<KeyValuePair<string, object>> filters)
{
if (filters == null || !filters.Any()) return source;
var parameter = Expression.Parameter(typeof(T), "x");
var body = filters
.Select(filter => Expression.Equal(
Expression.PropertyOrField(parameter, filter.Key),
Expression.Constant(filter.Value)))
.Aggregate(Expression.AndAlso);
var predicate = Expression.Lambda<Func<T, bool>>(body, parameter);
return source.Where(predicate.Compile());
}
}
Sample usage:
var filters = new List<KeyValuePair<string, object>>
{
new KeyValuePair<string, object>("param1", somevalue),
new KeyValuePair<string, object>("param2", someothervalue),
};
var myList = existinglist.Where(filters).ToList();
You can write an extension method like this:
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, string propName, object value)
{
var type = typeof(T);
var propInfo = type.GetProperty(propName,BindingFlags.IgnoreCase | BindingFlags.Public | BindingFlags.Instance);
var parameterExpr = Expression.Parameter( type, "x" ); //x
var memberAccessExpr = Expression.MakeMemberAccess( parameterExpr, propInfo ); //x.Prop
var lambda = Expression.Lambda( Expression.Equal(memberAccessExpr, Expression.Constant(value)),
parameterExpr ); //x=>x.Prop==value
var mi = typeof(Enumerable)
.GetMethods()
.Where(m => m.Name == "Where")
.First(m => m.GetParameters().Count() == 2)
.MakeGenericMethod(type);
return (IEnumerable<T>)mi.Invoke(null, new object[] { source, lambda.Compile() });
}
You can now use it as
var test = new[] { new { a = 1 }, new { a = 2 } }.Where("a", 1).ToList();
A lambda expression is just shorthand for a function. So you can replace that lambda with any function which takes a Myclass and returns a bool. Then write in that method the code to evaluate what you need to dynamically - perhaps using reflection if necessary.
myList = myList.Where(myFunction).ToList();
I have several tables, for example: Cats, Dogs, and Rabbits. Each table has two fields: the integer primary key XId and the string XName, where X is the name of the table: Cat, Dog or Rabbit.
Then, I have a string[] for each table which contain names. I need to sync the table with the array: remove table records that do not have their names listed in the array, and add new records for names that exist in the array only.
The easiest approach would have been copying the code three times and changing CatName to DogName, etc. in all places. However, I'm looking for a more clean solution that does not duplicate code.
If it was not a table but a simple IList<T>, I would have written the following snippet:
void SyncData<T>(IList<T> source, string[] names, Func<T, string> nameGetter, Func<string, T> creator)
{
var toRemove = source.Where(x => !names.Contains(nameGetter(x))).ToArray();
foreach(var item in toRemove)
source.Remove(item);
var toCreate = names.Where(x => !source.Any(i => nameGetter(i) != x)).ToArray();
foreach(var item in toCreate)
source.Add(creator(item));
}
SyncData(cats, catNames, c => c.CatName, n => new Cat { CatName = n });
SyncData(dogs, dogNames, d => d.DogName, n => new Dog { DogName = n });
...
Can something similar be achieved for IQueryable<T>? Unluckily, it doesn't work as-is, because the LINQ provider cannot translate function calls to SQL queries.
Since the field names are different, I cannot make a common interface for my objects. Changing the POCO class definitions is also not an option.
With the help of Linq.Dynamic, it's possible to achieve that.
And IQueryable<T> doesn't have Add method, so we need to use IDbSet<T> or something similar.
You can try this.
public static class SyncExtension
{
public static void SyncData<T>(
this IDbSet<T> source,
string[] names,
Expression<Func<T, string>> name,
Func<string, T> creator) where T : class
{
var columnName = ((MemberExpression)name.Body).Member.Name;
var removePredicate = string.Format("!{0}.Contains(#0)", columnName);
var toRemove = source.Where(removePredicate, names).ToArray();
foreach (var item in toRemove)
source.Remove(item);
var addPredicate = string.Format("{0} = #0", columnName);
var toCreate = names.Where(x =>
!source.Where(addPredicate, x).Any()).ToArray();
foreach (var item in toCreate)
source.Add(creator(item));
}
}
Usage.
using (var db = new AppContext())
{
var names = new[] { "A", "B" };
db.Set<Cat>().SyncData(names, x => x.Name, x => new Cat { Name = x });
db.SaveChanges();
}
I have two lists I would like to compare them for updated/modified columns.
Compare 2 Lists of the same class and show the different values in a new list
I would like to do this using linq. The only problem is I am dealing with a lot of columns, over excess of 30 columns in each. Any suggestions would be of great help...
//In Dal
List<PartnerAndPartnerPositionEntity> GetAllPartnerAndPartnerPositionOldDB(int modelId);
List<PartnerAndPartnerPositionEntity> GetAllPartnerAndPartnerPosition(int modelId);
//BL
//get from new db
var list1= _partnerDAL.GetAllPartnerAndPartnerPosition(modelId);
//get from old db
var list2= _partnerDAL.GetAllPartnerAndPartnerPositionOldDB(modelId);
Let's assume that:
PartnerAndPartnerPositionEntity class contains a property named Id that represents the unique key of an item
Given the above you can:
Get all properties of your type
var properties = typeof(PartnerAndPartnerPositionEntity).GetProperties();
Join the two lists on the Id property and iterate through the properties to see which one has changed:
var list = list1.Join(list2,
x => x.Id,
y => y.Id,
(x, y) => Tuple.Create(x, y))
.ToList();
list.Foreach(tuple =>
{
foreach(var propertyInfo in properties)
{
var value1 = propertyInfo.GetValue(tuple.Item1, null);
var value2 = propertyInfo.GetValue(tuple.Item2, null);
if(value1 != value2)
Console.WriteLine("Item with id {0} has different values for property {1}.",
tuple.Item1,Id, propertyInfo.Name);
}
});
Well, if you want to avoid doing it the boring and tedious way, you need to use reflection to dynamically get the class members then get their values for each instance.
See C# Reflection - Get field values from a simple class for the code.
There...this will generate new IL!
public static class EqualityHelper
{
private ConcurrentDictionary<Type, object> _cache = new ConcurrentDictionary<Type, object>();
public bool AreEqual<T>(T left, T right)
{
var equality = (Func<T,T,bool>)_cache.GetOrAdd(typeof(T), CreateEquality<T>());
return equality(left, right);
}
private Func<T, T, bool> CreateEquality<T>()
{
var left = Expression.Parameter(typeof(T));
var right = Expression.Parameter(typeof(T));
var properties = from x in typeof(T).GetProperties()
where x.GetIndexParameters().Any() == false
select x;
var expressions = from p in properties
select Expression.Equal(
Expression.Property(left, p),
Expression.Property(right, p));
var body = expressions.Aggregate(Expression.AndAlso);
var lambda = Expression.Lambda<Func<T,T,bool>>(body, new [] {left, right});
return lambda.Compile();
}
}
I have IQueryable<> object.
I want to Convert it into List<> with selected columns like new { ID = s.ID, Name = s.Name }.
Edited
Marc you are absolutely right!
but I have only access to FindByAll() Method (because of my architecture).
And it gives me whole object in IQueryable<>
And I have strict requirement( for creating json object for select tag) to have only list<> type with two fields.
Then just Select:
var list = source.Select(s=>new { ID = s.ID, Name = s.Name }).ToList();
(edit) Actually - the names could be inferred in this case, so you could use:
var list = source.Select(s=>new { s.ID, s.Name }).ToList();
which saves a few electrons...
Add the following:
using System.Linq
...and call ToList() on the IQueryable<>.
The List class's constructor can convert an IQueryable for you:
public static List<TResult> ToList<TResult>(this IQueryable source)
{
return new List<TResult>(source);
}
or you can just convert it without the extension method, of course:
var list = new List<T>(queryable);
System.Linq has ToList() on IQueryable<> and IEnumerable<>. It will cause a full pass through the data to put it into a list, though. You loose your deferred invoke when you do this. Not a big deal if it is the consumer of the data.
Here's a couple of extension methods I've jury-rigged together to convert IQueryables and IEnumerables from one type to another (i.e. DTO). It's mainly used to convert from a larger type (i.e. the type of the row in the database that has unneeded fields) to a smaller one.
The positive sides of this approach are:
it requires almost no code to use - a simple call to .Transform<DtoType>() is all you need
it works just like .Select(s=>new{...}) i.e. when used with IQueryable it produces the optimal SQL code, excluding Type1 fields that DtoType doesn't have.
LinqHelper.cs:
public static IQueryable<TResult> Transform<TResult>(this IQueryable source)
{
var resultType = typeof(TResult);
var resultProperties = resultType.GetProperties().Where(p => p.CanWrite);
ParameterExpression s = Expression.Parameter(source.ElementType, "s");
var memberBindings =
resultProperties.Select(p =>
Expression.Bind(typeof(TResult).GetMember(p.Name)[0], Expression.Property(s, p.Name))).OfType<MemberBinding>();
Expression memberInit = Expression.MemberInit(
Expression.New(typeof(TResult)),
memberBindings
);
var memberInitLambda = Expression.Lambda(memberInit, s);
var typeArgs = new[]
{
source.ElementType,
memberInit.Type
};
var mc = Expression.Call(typeof(Queryable), "Select", typeArgs, source.Expression, memberInitLambda);
var query = source.Provider.CreateQuery<TResult>(mc);
return query;
}
public static IEnumerable<TResult> Transform<TResult>(this IEnumerable source)
{
return source.AsQueryable().Transform<TResult>();
}